FastMap: Revisiting Dense and Scalable Structure from Motion
Jiahao Li, Haochen Wang, @zubair-irshad.bsky.social, @ivasl.bsky.social, Matthew R. Walter, Vitor Campagnolo Guizilini, Greg Shakhnarovich
tl;dr: replace BA with epipolar error+IRLS; fully PyTorch implementation
arxiv.org/abs/2505.04612
Shoutout to the authors of the wonderful papers i.e. CtRNet-X, DUSt3R, Segment Anything, CLIP and Pytorch3D and for open-sourcing their codebase to advance science and make this effort happen!
Please check these works out if you haven’t already!
We have released our improved extrinsics. Try it out now at droid-dataset.github.io and read more details about it in the updated DROID paper at arxiv.org/abs/2403.12945
This was a fun collaboration with
@vitorguizilini, @SashaKhazatsky and @KarlPertsch!
There’s room to improve. Future work could explore:
• Extending to in-the-wild scenes via foundation models for robot segmentation & keypoints.
• Ensembling predictions over time for better temporal consistency.
• Fine-tuning pointmap models on real robot data to handle cluttered tabletops.
8/n
Large-scale auto calibration in robotics is challenging, and our pipeline has some limits:
• CtRNet-X is trained on Panda; generalization to other robots is untested.
• DUSt3R struggles with clutter or minimal view overlap.
• Steps 2️⃣ & 3️⃣ may yield false positives in tough lighting or geometry.
7/n
Similarly, we plot the distribution of number of matched points and cumulative curve after 3️⃣, helping to identify the top quantile of well-calibrated camera pairs within each lab.
6/n
Automatically calibrating a large-scale dataset is challenging. We provide quality assessment metrics across all three stages, with flexibility to narrow bounds for downstream tasks as needed.
1️⃣ and 2️⃣ quality metrics show IOU and Reprojection-error distributions post-calibration.
5/n
Below we show the Camera-to-Camera transformations, post-calibration improves the alignment of obtained pointclouds!
4/n
We provide:
🤖 ~36k calibrated episodes with good-quality extrinsic calibration
🦾 ~24k calibrated multi-view episodes with good-quality multi-view camera calibration
✅ Quality assessment metrics for all provided camera poses
3/n
To achieve this, we utilize:
1️⃣ Auto Segment Anything (SAM) based filtering (Camera-to-Base Calibration)
2️⃣ Tuned CtRNet-X for bringing in additional cams (Camera-to-Base Calibration)
3️⃣ Pretrained DUST3R with depth-based pose optimization (Camera-to-Camera Calibration)
2/n
Introducing ✨Posed DROID✨, results of our efforts at automatic post-hoc calibration of a large-scale robotics manipulation dataset.
Try it out at: droid-dataset.github.io
Learn more at:
🌐 arXiv: arxiv.org/pdf/2403.12945
📄 Blog: medium.com/p/4ddfc45361d3
🧵 1/n
🔗 Learn more & submit your work: robo-3dvlms.github.io
Join us in shaping the future of robotics, 3D vision, and language models! 🤖📚 #CVPR2025
🎤 We’re honored to host top experts in the field:
⭐ Angel Chang (Simon Fraser University)
⭐ Chelsea Finn (Stanford University)
⭐ Hao Su (UC San Diego)
⭐ Katerina Fragkiadaki (CMU)
⭐ Yunzhu Li (Columbia University)
⭐ Ranjay Krishna (University of Washington)
5/N
🎯 Key Topics:
✅ 3D Vision-Language Policy Learning
✅ Pretraining for 3D VLMs
✅ 3D Representations for Policy Learning
✅ 3D Benchmarks & Simulation Frameworks
✅ 3D Vision-Language Action Models
✅ 3D Instruction-Tuning & Pretraining Datasets for Robotics
4/N
📢 Call for Papers: Submission opens today!
📅 Deadline: April 15, 2024 (11:59 PM PST)
📜 Format: Up to 4 pages (excluding references/appendices), CVPR template, anonymized submissions
🏆 Accepted papers: Poster presentations, with selected papers receiving spotlight talks!
3/N
🔍 Explore how 3D perception and language models can enhance robotic manipulation in the era of foundation models. Engage with leading experts and be part of this new frontier in 3D-based VLMs/VLAs for robotics.
2/N
🚀Exciting News! Join us at the inaugural #CVPR2025 Workshop on 3D Vision Language Models (VLMs) for Robotics Manipulation on June 11, 2025, in Nashville, TN! 🦾
robo-3dvlms.github.io
1/N
@cvprconference.bsky.social
Welcome onboard!
Done, welcome aboard!
Welcome on board!
Congrats on the release, demos look cool and it's open source 👏
In a world full of AI, authenticity will be the most valuable thing in the universe.
Hello 👋
For my first post on Bluesky, this recent talk I did at the recent BMVA one day meeting on World Models is a good summary of my work on Computer Vision, Robotics and SLAM, and my thoughts on a bigger picture of #SpatialAI.
youtu.be/NLnPG95vNhQ?...
Just included :) Welcome @ajdavison.bsky.social!
go.bsky.app/HcQYMj
Check out this BEAUTIFUL interactive blog about cameras and lenses
ciechanow.ski/cameras-and-...
Hello 👋 Would love to join!
We are in the process of editing a SLAM handbook, to be published by Cambridge University Press, with many *stellar* contributors. Part 1 is available as an online draft for public comments. Help us find bugs/problems!
Link to release repo is here: lnkd.in/gZhTkaxb
Added you!
Welcome onbaord!
