Regularized self-play RL in grounded simulation effectively adapts driving policies to completely new cities. ๐ฝ -> ๐ผ
Really enjoyed collaborating on this work, led by Zilin and Saeed! Check out Zilin's post below for a great summary
๐งต: x.com/nirhso/statu...
๐: arxiv.org/abs/2602.15891
The most important finding from this analysis! See the post for more details
Several fast evals are included, too! Check out our release post:
emerge-lab.github.io/PufferDrive/...
Work done with Spencer Cheng* (co-first), Pragnay Mandavilli, Julian Hunt, Kevin Joseph, Waรซl Doulazmi, Valentin Charraut, Aditya Gupta, Joseph Suarez, and
@eugenevinitsky.bsky.social
What if you could train agents on a ๐ฑ๐ฒ๐ฐ๐ฎ๐ฑ๐ฒ of driving experience in ๐๐ป๐ฑ๐ฒ๐ฟ ๐ฎ๐ป ๐ต๐ผ๐๐ฟ, on a single GPU?
Excited to share ๐๐ช๐๐๐๐ง๐ฟ๐ง๐๐ซ๐ 2.0: A fast, friendly driving simulator with RL training via PufferLib at ๐ฏ๐ฌ๐ฌ๐ ๐๐๐ฒ๐ฝ๐/๐๐ฒ๐ฐ ๐ก + ๐
youtu.be/LfQ324R-cbE?...
Excited to share a new preprint, accepted as a spotlight at #NeurIPS2025!
Humans are imperfect decision-makers, and autonomous systems should understand how we deviate from idealized rationality
Our paper aims to address this! ๐๐ง โจ
arxiv.org/abs/2510.25951
a ๐งตโคต๏ธ
Rapid RL experimentation is great. But how do you catch silent errors before they slip by?
In this post, I share tools and habits that help me move quickly from idea to result without sacrificing reliability.
The single biggest epistemic challenge in the internet era is remaining calibrated about what "normal" people think while the internet throws up an infinite wall of crazy. Thousands of people sharing an absurd opinion on the internet tells you very little!
Overnight runs are the overnight oats of research โ prep, forget, and rewarding by morning
Building a "human-level" simulated driver that zero-shot generalizes to many benchmarks: a fun interview with @natolambert.bsky.social
www.youtube.com/watch?v=2Q66...
This was joint work with Aarav Pandya, Kevin Joseph, Joseph Suรกrez, and @eugenevinitsky.bsky.social
Links:
- Paper: arxiv.org/abs/2502.14706
- Project page: sites.google.com/view/reliabl...
- Codebase and pre-trained agents:
Results (2): Beyond in-distribution generalization, our agents show partial robustness to scenarios that rarely occur in the data.
More importantly, results show that agents can be fine-tuned in minutes to reach near-perfect performance in such cases.
Results (1): Self-play scales well with data. With 10,000 training scenarios, the model approaches nearly the ceiling of our benchmark, achieving a 99.81% goal-reaching rate, 0.44% collision rate, and 0.31% off-road rate on 10,000 held-out test scenarios.
We train sim agents using self-play PPO on 10K+ scenarios from the Waymo Open Dataset in GPUDrive, under a semi-realistic framework for human perception and control.
Agents learn goal-directed behavior, avoiding collisions and staying on the road.
SOTA generative models trained on large human datasets show unintended behaviors like crashes (5-6%) and off-road events (6-12%) in benchmarks for nominal driving.
Unpredictable deviations make it hard to separate signal from noise.
Sim agents are key for developing autonomous systems for safety-critical systems, like self-driving cars.
We're open-sourcing sim agents that achieve a 99.8% success rate with < 0.8% failures on the Waymo Dataset. These agents are built through scaling self-play.
Challenge accepted
Oh and, stay tuned for another big release tomorrow!
Huge thanks to my incredible collaborators for making this possible: Saman Kazemkhani, Aarav Pandya, @eugenevinitsky.bsky.social , Joseph Suarez for converting the sim to a package and optimizing the PPO loop, and Kevin Joseph for all his help with data processing, tutorials, and more! ๐
GPUDrive got accepted to ICLR 2025!
With that, we release GPUDrive v0.4.0! ๐จ You can now install the repo and run your first fast PPO experiment in under 10 minutes.
Iโm honestly so excited about the new opportunities and research the sim makes possible. ๐ 1/2
A large group of us (spearheaded by Denizalp Goktas) have put out a position paper on paths towards foundation models for strategic decision-making. Language models still lack these capabilities so we'll need to build them: hal.science/hal-04925309...
