Prof @Stanford, Distinguished Research Scientist and AV research lead @nvidia. PhD from @MITAeroAstro. Robotics, autonomous systems, AI. Opinions are my own.

Stanford, CA USA
System 1 models have generated significant excitement recently, particularly following the introduction of #Jev. We introduce Contrastive Language Model (CLM): an ultra-fast System 1 model that connects states and actions through contrastive learning. CLM-8B delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer use, gaming, and tool calling. With lightweight fine-tuning, it achieves 81.6% on DeepSWE and 87.6% on Terminal-Bench 2.1. We also introduce efficient training and serving infrastructure for CLMs and establish scaling laws across compute, model size, and dataset size. 📄 Blog: contrastive-lm.notion.site 💻 Code: github.com/Contrastive-LM/CL… 🗣️ Discord: discord.gg/5dAQEDJBs 🤗 Data & Models: huggingface.co/Contrastive-L… Work led by @jackyk02, in collaboration with @hangoo_kang @TarunSures41845 @JonSaadFalcon @Azaliamirh @HazyResearch More details in @jackyk02's post: nitter.net/jackyk02/status/210290… @StanfordASL @StanfordAILab @StanfordEng
Introducing Contrastive Language Model (CLM): an ultra-fast System One Model trained with a contrastive learning objective that connects states and actions. CLM-8B is pre-trained on internet-scale data and delivers up to 9× faster inference than Jev ⚡ while achieving comparable performance across computer-use, gaming, and tool-calling tasks. With lightweight fine-tuning, CLM-8B sets a new SOTA on challenging agentic coding benchmarks, such as DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%). In contrast, Jev fails to serve as an effective verifier for these long-horizon tasks. We also build an efficient training and serving infra for CLMs by disaggregating states and actions, allowing their embeddings to be cached and reused independently. This substantially reduces inference latency in settings where the state evolves continuously while the action set remains fixed. Finally, we establish scaling laws for CLMs and show that the test contrastive loss decreases predictably as a power law in training compute, model size, and dataset size. 📄 Blog: contrastive-lm.notion.site 💻 Code: github.com/Contrastive-LM/CL… 🗣️ Discord: discord.gg/5dAQEDJBs 🤗 Data & Models: huggingface.co/Contrastive-L… More details on CLM’s architecture, data recipe, and scaling laws in the thread below 🧵
9
12
141
16,131
Can we steer a robot foundation model without retraining it? Our new paper explores this question through control-inspired notions of feature observability and controllability in Vision-Language-Action (VLA) models. We find that state- and action-relevant information can often be linearly predicted from VLA internal representations—and that these representations can be steered at inference time to change robot behavior, without fine-tuning. As an example, on a real robot, our approach increased the preferred handle-grasp rate from 14% to 74%, while adding only ~1% inference-time overhead. The broader idea is that representation-level control could provide a lightweight interface for adapting robot foundation models to new preferences and constraints—without retraining. Joint work with Hugo Buurmeijer Carmen Amo Alonso @SwannAiden Paper: arxiv.org/abs/2603.05487 #PhysicalAI #Robotics #VLA #RobotLearning @StanfordEng @StanfordAILab
7
22
132
11,493
Following up on my previous post about PhD Research Intern opportunities, the Autonomous Systems and Physical AI Research (ASPIRE, research.nvidia.com/labs/asp…) group at @nvidia is also hiring for full-time positions! Priority will be given to candidates who can start by the end of January 2027. Open positions: - (Senior) Research Scientist — Apply here: nvidia.wd5.myworkdayjobs.com… - (Senior) Research Engineer — Apply here: nvidia.wd5.myworkdayjobs.com… We work across #reasoning models, #generative simulation, #agentic #AI workflows, and #Physical AI #safety, with applications spanning autonomous vehicles and a broad range of Physical AI systems. If you’re interested in joining us, we’d love to hear from you! @NVIDIADRIVE @NVIDIARobotics @NVIDIAAI
3
20
159
13,047
We are hiring! The Autonomous Systems and Physical AI Research (ASPIRE: research.nvidia.com/labs/asp…) group at @nvidia is looking for talented PhD Research Interns to join us in advancing the frontiers of #autonomous #systems and #Physical #AI. We work across a broad range of research areas, including #reasoning models, generative simulation, #agentic AI workflows, and Physical AI #safety, with applications spanning autonomous vehicles and a broad range of Physical AI systems. Interested in pushing the limits of what’s possible? Apply now: nvidia.wd5.myworkdayjobs.com…
12
25
231
65,389
Marco Pavone retweeted
From Robotaxis to L4 Autonomy: A Deep Dive Into NVIDIA Alpamayo 2 Super nitter.net/i/broadcasts/1oKMvNqQP…
2
7
25
2,357
Join us tomorrow for a deep dive into #Alpamayo 2 Super—our most capable model yet! We’ll unpack the key technologies behind the model, showcase hands-on notebooks to help you get started, and share the latest AlpaSim updates. We’ll also take a closer look at our two ongoing challenges, with prizes up for grabs! Save the date: addevent.com/event/mwwld54rm… @NVIDIADRIVE @NVIDIAAI
6
28
2,615
The @Stanford Department of Aeronautics & Astronautics has a faculty opening! This is an incredibly exciting time for #aerospace. Rapid advances in AI, autonomy, robotics, sustainable aviation, advanced propulsion, and space technologies are reshaping how we design, build, and operate the systems that will define the next generation of flight and exploration. Joining the Stanford AeroAstro Department is a unique opportunity to help shape these fields for decades to come — while educating and inspiring the next generation of aerospace leaders. We are looking for outstanding faculty who want to push the boundaries of aerospace research, collaborate across disciplines, and make a lasting impact through both research and education. 📅 Application review begins: October 15, 2026 📅 Full consideration deadline: October 31, 2026 Learn more and apply: facultypositions.stanford.ed… Please share widely with exceptional candidates who may be interested! @StanfordEng @StanfordAILab
3
3
26
7,035
Our latest finding: scaling self-verification can make open-weight models significantly more capable at a fraction of the cost. With DeepSeek V4 Flash, sampling just 5 candidate solutions and using the same model to rank them with LLM-as-a-Verifier improves Terminal-Bench 2.1 accuracy from 79% → 88%—outperforming Claude Fable 5 while costing 11× less. 💰 As open-weight models become more capable, we can generate many high-quality candidate solutions and verify them at very low cost. Try it out: github.com/llm-as-a-verifier… More on verification scaling in @jackyk02's previous post: nitter.net/jackyk02/status/207496… @StanfordAILab @StanfordEng
How can we extract richer signals from AI Feedback? Introducing LLM-as-a-Verifier✨— a simple verification scaling framework that achieves SOTA on agentic benchmarks 🚀 The key idea: - Use fine-grained scoring granularity (e.g., 1-20 instead of the standard 1-5 scale) - Take the expectation over the full logprob distribution of score tokens - Scale repeated evaluation and criteria decomposition You can use these fine-grained signals for more effective test-time scaling, RL, and agent monitoring! It achieves SOTA across Terminal-Bench V2, SWE-Bench Verified, RoboRewardBench, and MedAgentBench 👑 Advised by @Azaliamirh @istoica05 @drmapavone @chelseabfinn 🧵👇
10
14
128
11,036
We have just released #Alpamayo 2 Super — @nvidia’s frontier open reasoning model for #autonomous #vehicles. NVIDIA Alpamayo 2 Super is an open 34-billion-parameter #reasoning vision-language-action (#VLA) model designed to accelerate autonomous vehicle (AV) development. It combines the 32-billion-parameter NVIDIA #Cosmos 3 Super Reasoner with a 2-billion-parameter diffusion-based Action Expert and is post-trained with #reinforcement #learning. Two aspects make Alpamayo 2 Super particularly exciting: 1. Open and commercially deployable: Alpamayo 2 Super is available on Hugging Face under OpenMDW-1.1, the Linux Foundation’s permissive license for open AI model distribution. The OpenMDW license is now being applied across the entire Alpamayo model family, enabling developers to deploy these models commercially without requiring additional permissions. 2. A multi-task foundation model for autonomous driving: Alpamayo 2 Super produces five tightly coupled outputs: - A trajectory describing the vehicle’s planned path. - A chain-of-causation (CoC) trace explaining the reasoning behind each driving decision, achieving benchmark-leading reasoning performance at frontier scale. - A meta-action (e.g., yield, lane change, stop) capturing the model’s intent. - Reasoning auto-labels that generate CoC annotations for training and validation data. - Visual question answering responses with 2D visual grounding, linking answers to specific regions in camera images. These multi-task capabilities enable developers to leverage a single foundation model across more of the development process, simplifying tooling and accelerating iteration. Resources: 🔹 Interview: piped.video/watch?v=VeYZ-MrJ… 🔹 Video: piped.video/watch?v=xySgVPLY… 🔹 Blog: blogs.nvidia.com/blog/alpama… 🔹 Technical blog: developer.nvidia.com/blog/ge… 🔹 Hugging Face blog: huggingface.co/blog/nvidia/n… 🔹 Model weights: huggingface.co/nvidia/Alpama… 🔹 Inference code: github.com/NVlabs/alpamayo2 As @JensenHuang has emphasized, open models help advance safety and security. We hope Alpamayo 2 Super will contribute to this vision by enabling researchers and developers around the world to build, experiment, and innovate in autonomous driving. We are excited to see what the community builds with Alpamayo 2 Super. @NVIDIADRIVE @NVIDIAAI
1
15
50
10,089
#Alpamayo 2 Super, @nvidia's frontier open reasoning model for autonomous vehicles, is now available!
Today, we’re launching Alpamayo 2 Super, our frontier open reasoning model for autonomous vehicles. Beyond seeing, Alpamayo understands and reasons through the complex world - thinks before it acts. It’s a powerful backbone for robotaxis, trucks, shuttles, delivery vans, tractors and the long tail of mobile robots—billions of autonomous machines someday. We’re releasing it for commercial use under OpenMDW-1.1 so teams can inspect it, fine-tune it and deploy it—open models advance safety and security. The next wave of AI is robotics—and it starts with autonomous vehicles. Great work, Alpamayo team! blogs.nvidia.com/blog/alpama…
2
5
42
5,641
A thoughtful letter highlighting the critical importance of open frontier models — a must-read for researchers, developers, and anyone shaping the future of AI. We built the @nvidia #Alpamayo open platform (nvidia.com/en-us/solutions/a…) around the same philosophy: sharing state-of-the-art #Physical #AI models, data, and tools with the community to accelerate innovation and advance the development of safer, more capable #autonomous #vehicles and #Physical #AI systems.
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. images.nvidia.com/pdf/Open-W…
3
4
24
8,809
Verification is emerging as a new scaling axis for AI! Scaling pre-training, post-training, and test-time compute have driven much of the recent progress in large language models. Our new work explores a fourth scaling axis: #verification —the ability to determine whether a solution is actually correct. In LLM-as-a-Verifier, we introduce a general-purpose framework that provides fine-grained feedback across diverse modalities without additional training. We show that three simple ingredients—higher score granularity, repeated evaluation, and criteria decomposition—consistently improve verification performance. The approach achieves state-of-the-art results across robotics, coding, and medical AI, including RoboRewardBench, Terminal-Bench V2, SWE-Bench Verified, and MedAgentBench. I'm particularly optimistic about the implications for #Robotics and #PhysicalAI, where verification can serve as a dense reward signal for reinforcement learning, significantly improving the sample efficiency of SAC and GRPO and, in turn, enabling more capable and reliable autonomous systems. As AI continues to scale, I believe verification will become a foundational capability for building more capable and trustworthy autonomous AI agents. 🌐 Website: llm-as-a-verifier.com/ 📄 Paper: arxiv.org/abs/2607.05391 💻 Code: github.com/llm-as-a-verifier… Outstanding work led by @jackyk02, in collaboration with @shululi256, @pranav_atreya, @liu_yuejiang, @jyx_su, @chelseabfinn, @istoica05, and @Azaliamirh. #AI #LLMs #Verification #Reasoning #AgenticAI #Robotics #PhysicalAI #ReinforcementLearning @StanfordAILab @StanfordEng
9
18
76
16,108
How do we make robot policies robust to rare but high-impact failures? Video #World #Models (WMs) are rapidly becoming a powerful tool for robotics, enabling policy evaluation and improvement by "imagining" future outcomes. But there's a catch: these imagined futures are typically nominal samples, making it easy to overlook the rare yet safety-critical events that matter most. In our new paper, StressDream: Steering Video World Models for Robust Policy Evaluation and Improvement, we explore a simple but powerful idea: 💡 Instead of passively sampling futures, actively steer world model imaginations toward high-impact yet still plausible scenarios. StressDream optimizes the initial diffusion noise at inference time, allowing us to generate targeted stress-test scenarios without retraining the world model. This enables: - More robust policy evaluation by exposing failure modes that random sampling often misses. - Improved policy optimization by training against challenging but realistic imagined futures. As generative world models become a foundation for #Physical #AI, the ability to systematically probe their "long tail" of plausible futures will be increasingly important for building reliable and trustworthy autonomous systems. 📌 𝖯𝗋𝗈𝗃𝖾𝖼𝗍 𝖯𝖺𝗀𝖾: junwon.me/StressDream/ 📄 𝖯𝖺𝗉𝖾𝗋: arxiv.org/abs/2606.00267 Work led by Junwon Seo, with a great set of collaborators: Sushant Veer, Thomas Ran Tian, Wenhao Ding, Apoorva Sharma, Karen Leung, Edward Schmerling, Andrea Bajcsy. @NVIDIADRIVE @NVIDIAAI #Robotics #WorldModels #PhysicalAISafety #AISafety #AutonomousSystems #RobotLearnin
4
16
99
7,083
Introducing ConstrainedMimic (arxiv.org/pdf/2606.00374) — a control framework for #humanoid robot #safety that enables real-time constraint enforcement within #RL-based motion tracking policies by leveraging whole-body kinematics and dynamics. Recent advances in reinforcement learning have unlocked remarkable whole-body agility for humanoid robots. However, ensuring safety and satisfying constraints—especially those introduced after training—remains a significant challenge for deploying safe and reliable systems. ConstrainedMimic addresses this challenge by combining ideas from operational space control and control barrier functions (CBFs). The framework enables enforcement of arbitrary runtime constraints while preserving the ability of the policy to track complex motions. Importantly, constraints can be imposed on both the kinematic reference motion and the underlying robot dynamics, providing a principled approach to safer, more robust, and more controllable humanoid behavior. As #PhysicalAI, #humanoid #robotics, and #embodied #AI systems move from research environments into the real world, the ability to guarantee safety and respect operational constraints will become increasingly important — ConstrainedMimic is a step in this direction. 📄 Paper: arxiv.org/pdf/2606.00374 💻 Code: Coming soon Great work led by @danielpmorton . #PhysicalAI #AISafety #HumanoidRobotics #EmbodiedAI #ReinforcementLearning #Robotics @StanfordAILab @StanfordEng
4
16
67
7,239
Today we're releasing @nvidia AlpaGym, our new open-source reinforcement learning (RL) framework for end-to-end autonomous driving. A key challenge for #Physical #AI is enabling policies to learn from the consequences of their actions. While supervised learning can teach a model to imitate behavior, robust autonomy ultimately requires learning through interaction with the environment. AlpaGym enables exactly that. Built on top of: - AlpaSim: our high-fidelity closed-loop autonomous driving simulator - Cosmos-RL: NVIDIA's distributed RL training and rollout infrastructure AlpaGym provides the glue that connects simulation, training, and driving policies into a scalable framework for post-training autonomous vehicle models in closed loop. With AlpaGym, researchers and developers can: ✅ Train end-to-end driving policies using reinforcement learning ✅ Run large-scale closed-loop simulations ✅ Experiment with new reward functions, policy architectures, and training strategies ✅ Benchmark models on public leaderboards 📖 Learn how it works: developer.nvidia.com/blog/ho… 💻 GitHub: github.com/NVlabs/alpagym 🏆 Open Challenges: - AlpaSim Closed-Loop E2E Driving Challenge: huggingface.co/spaces/nvidia… - Physical AI AV Reasoning Challenge: huggingface.co/spaces/nvidia… Learn more about the #Alpamayo open platform: huggingface.co/blog/drmapavo… #PhysicalAI #AutonomousDriving #ReinforcementLearning #Robotics #OpenSource #NVIDIA #MachineLearning @NVIDIADRIVE @NVIDIAAI
1
16
51
3,631
On the power of data flywheels in Physical AI @nvidia recently introduced Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, images, video, audio, and actions within a unified mixture-of-transformers architecture. One aspect I find particularly exciting is the data flywheel emerging between #Cosmos 3 and #Alpamayo 2. Cosmos 3 was trained using data generated and curated in part through the Alpamayo ecosystem. In turn, the next generation of Alpamayo will build on Cosmos 3's capabilities. This creates a virtuous cycle in which better models enable better data generation, and better data leads to even stronger models. Much of the attention in #AI is naturally focused on model architectures and benchmark results. Yet, in robotics and autonomous systems, development processes matter just as much as models themselves. Robust data flywheels are increasingly becoming a defining characteristic of state-of-the-art robot autonomy stacks. Further reading: - Cosmos 3: arxiv.org/pdf/2606.02800 - Alpamayo 2: huggingface.co/blog/drmapavo… Join me on June 16 at 9:00 AM PT for a livestream on Alpamayo 2 Super: The Open Reasoning Model for Robotaxis: addevent.com/event/5djffx3cl…. We'll showcase brand-new elements of the open pipeline—from real-world fleet data to model training recipes to closed-loop development with simulation. If you're building toward L4 autonomy, I think you'll find it worthwhile. I'll be joined by @iamborisi @YurongYou @yan_wang_9 @MaxiIgl Looking forward to seeing you there. @NVIDIADRIVE @NVIDIAAI
2
15
51
5,068
How much time should robots spend thinking? Vision-Language Models are increasingly used as high-level planners for robots, and the prevailing strategy has been to scale test-time compute to boost capability. But more reasoning steps, bigger models, and longer memory all come with increased latency, tokens, and FLOPs—often with diminishing and uneven returns. So when, and where, is test-time compute actually worth its cost? 🧐 We study three dominant scaling axes and find that each unlocks a distinct capability, showing that test-time compute is not a uniform lever: - Chain-of-thought depth helps with tasks involving implicit semantic, physical, or spatial constraints, but its additional latency is not always necessary (on VLABench, a non-CoT model matches a CoT model on 44% of tasks). - Model size governs the breadth of skills a planner can reliably draw upon, but its benefits appear only when those additional skills are actually required. - Memory history improves performance on long-horizon, history-dependent tasks, but can actively hurt performance elsewhere. Across all three axes, a consistent pattern emerges: the gap between cheap and expensive configurations is large, but highly non-uniform and task-dependent. DIRECT (Dynamic Inference Router for Embodied Compute Tradeoffs) is a lightweight router that reads scene + instruction context and sends each task to the cheapest planner that can still solve it, allocating compute per task rather than committing to one fixed model. 👉 Takeaway: smart allocation of test-time compute can recover frontier-level planning at a fraction of the cost. 📄 Paper: arxiv.org/abs/2606.12402 🔗 Website: jadee-dao.github.io/direct/ Work led by @_jadelynn @milanganai With an outstanding team of collaborators: @ajaysridhar0 @Mozhgan_nasr @katielulula Clark Barrett @jiajunwu_cs @chelseabfinn #Robotics #VLM #EmbodiedAI #MachineLearning #TestTimeCompute
2
13
49
5,494
Marco Pavone retweeted
The Alpamayo Summit at CVPR brought together AV researchers and industry leaders together under one room. Hear from Marco Pavone (@drmapavone), Senior Director of Autonomous Vehicle Research, and other NVIDIA experts on how Alpamayo is accelerating AV development. 📺 Watch the on-demand replay: nvidia.com/en-us/on-demand/p…
3
6
1,147