Q: How can we ensure robots behave properly at scale? A: Robot constitutions 📜! Q: How do we verify behavior in undesirable situations at scale? A: Generation! We release the ASIMOV Benchmark for Semantic Safety of robots at asimov-benchmark.github.io @GoogleDeepMind
1
7
44
8,844
We generated robot constitutions directly from images. This way we can scale up coverage of constitutions to various everyday situations grounded in real data. We show that generated constitutions are better aligned with humans on the ASIMOV benchmark than human-written ones.
1
3
216
We generate difficult-to-capture undesirable situations from real images. Then generate different contexts and questions to verify if a robot can properly differentiate desirable actions from undesirable ones. From those situations, we derive rules promoting desirable behavior.

Mar 13, 2025 · 3:12 PM UTC

1
2
105
We also generate novel scenarios from real hospital injury reports. The resulting ASIMOV datasets were used to train the Gemini Robotics models to be safer: deepmind.google/technologies…
1
1
94
We propose an auto-amending procedure that finds issues in constitutions and proposes amendments. We find that auto-amended constitutions are most aligned with humans in the ASIMOV benchmark. This procedure also produces counterfactual scenarios useful for ethics evaluation.
1
1
77
Short, under-prescriptive constitutions like “Do no harm” rely on a well-behaved base model and collapse in adversarial evaluations. We find that longer, generated and auto-amended constitutions are able to recover high alignment when prompting the base model to be adversarial.
1
1
67
While generation can increase coverage and alignment, humans have the last say and the ability to review and edit constitutions because they are readable in natural language. This universality is what enables a unified governance of behavior across diverse robots.
1
1
54
While the generated constitutions are for research purposes only and not intended for deployment, we hope that this release can help increase the safety of all robots in the future as much research remains needed to make robots safer before deployment.
1
1
545
We dedicate this work to Isaac Asimov who continues to be a source of inspiration. Authors: @psermanet, @Majumdar_Ani, @AlexIrpan, D. Kalashnikov, @vikassindhwani
3
494
Sort replies: Relevant Recent Liked