We have released the Stereo4D dataset! Explore the real-world dynamic 3D tracks: github.com/Stereo4d/ste...

Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos Use stereo videos from the internet to create a dataset of over 100,000 real-world 4D scenes with metric scale and long-term 3D motion trajectories.

See more scenes & details of how it works on our website: stereo4d.github.io
Paper link: arxiv.org/abs/2412.09621
Thanks to the great team! Richard Tucker, @zhengqili.bsky.social , David Fouhey @snavely.bsky.social , @holynski.bsky.social

Please stay tuned for updates on data & code.

This type of data is ideal for learning the structure and dynamics of the real world.

We gave this a shot — by extending DUSt3R to model 3D motion, and training on our dataset. Given a pair of frames, our model predicts a 3D point cloud, and corresponding 3D motion trajectories.

Introducing 👀Stereo4D👀

A method for mining 4D from internet stereo videos. It enables large-scale, high-quality, dynamic, *metric* 3D reconstructions, with camera poses and long-term 3D motion trajectories.

We used Stereo4D to make a dataset of over 100k real-world 4D scenes.

A fast and accurate method to get camera poses, focal length, and consistent depth map from dynamic casual videos. Checkout this amazing work led by @zhengqili.bsky.social