Generalist robots need a generalist evaluator. But how do you test safety without breaking things? 💥
🌎 Introducing our new work from @GoogleDeepMind:
Evaluating Gemini Robotics Policies in a Veo World Simulator
veo-robotics.github.io
🧵👇
Dec 12, 2025 · 5:02 PM UTC
28
87
579
244,127
The challenge: hardware evaluation is slow and often dangerous for safety probing.
Traditional physics sims are great but struggle with visual realism, non-rigid objects, and the immense manual effort required for accuracy.
Can video models serve as a generalist simulator?
1
1
13
2,786
💡 We introduce a system that utilizes @GoogleDeepMind’s Veo to evaluate policies on:
✅ Nominal performance on real-world tasks
✅ Out-of-Distribution (OOD) generalization
✅ Semantic Safety
All without a single physics engine.
1
1
13
1,673
🛠️ We fine-tuned Veo for action conditioning & multi-view generation. This was achieved using a large-scale robotics dataset containing a wide range of different scenes and tasks.
1
2
11
1,445
We use the action-conditioned world model to roll out a given policy in an initial scene. We can score the policy rollout to predict real-world success or failure.
1
9
1,222
🤯 A surprise: we found that fine-tuning the video model didn't break its "common sense." We can edit scenes to include novel objects or distractors, and still simulate the physics and interactions, effectively preserving the base model's world knowledge.
1
1
9
1,121
📊 Validation: We ran 1600+ real-world evaluations across 8 Gemini Robotics policy checkpoints and 5 tasks. Result: the Veo (Robotics) model accurately predicts the relative performance and rankings of different policies in both nominal and OOD conditions.
1
4
1,179
🛡️ Predictive Red Teaming: We used Veo to probe semantic safety. We can simulate dangerous "long tail" scenarios without breaking things in real life. 🚫
We replicated scenarios using props and found that predicted behaviors were real.
2
2
7
1,508
This work represents a step toward scalable, safe, and generalist evaluation for the next generation of robot policies using world models.
📜Read the full report here:
arxiv.org/abs/2512.10675
💻Website:
veo-robotics.github.io
2
12
1,337
