Brent Yi retweeted
One human demonstration. Any multi-fingered hand. Zero-shot sim-to-real visuomotor policy. morphometricimitation.github… Collaborators: @he_siming @ckwolfeofficial @HaozhiQ @LeaMue27 Shankar Sastry, Claire Tomlin, @JitendraMalikCV
8
16
120
16,403
Egocentric human data is the key to a major breakthrough in robotics foundation models. Huge congratulations to the GSI team, especially @AdemiAdeniji, whose work is making this vision a reality
Grounded API is live. - SOTA on hand-tracking benchmarks (< 1 cm) - SOTA on SLAM benchmarks - In-the-wild ego data -> enriched data in minutes - Integration with @huggingface @LeRobotHF & @rerundotio - Built for @BitRobotNetwork RoboCap suite Technical report & more↓
1
2
21
2,932
Grounded API is live. - SOTA on hand-tracking benchmarks (< 1 cm) - SOTA on SLAM benchmarks - In-the-wild ego data -> enriched data in minutes - Integration with @huggingface @LeRobotHF & @rerundotio - Built for @BitRobotNetwork RoboCap suite Technical report & more↓
48
78
447
170,332
Brent Yi retweeted
I'm excited to release mjbatch, a simple tool that lets you step thousands of MuJoCo simulations in parallel on your CPU, all from the comfort of Python! github.com/kevinzakka/mjbatc… To show how far this primitive can take you, I built some fun, self-contained examples with it šŸ‘‡
15
74
527
36,699
Brent Yi retweeted
I came from a world where I measured robotics research in days: collect hundreds of demonstrations, train, then watch the policy break as soon as the setup changed. Since I joined Sunday, I’ve watched that loop compress dramatically. ACT-2 can learn a new folding strategy from one demonstration, with just five minutes of fine-tuning, then transfer it to unseen garments and beds. What once took a week can now happen in an hour. That’s the deeper impact of scaling pretraining: it doesn’t just produce a better policy. It produces a better learner.
Introducing ACT-2 Preview The first robotics model to unify broad generalization with high reliability. A single fine-tuning example can teach Memo a new behavior that generalizes. Zero shot, real unseen homes, 99% success rate.
14
10
155
17,127
Brent Yi retweeted
Introducing ACT-2 Preview The first robotics model to unify broad generalization with high reliability. A single fine-tuning example can teach Memo a new behavior that generalizes. Zero shot, real unseen homes, 99% success rate.
211
481
4,074
1,051,039
FPO++ code is out! We’re releasing the full training and evaluation code for locomotion, motion tracking, and manipulation. github.com/amazon-far/fpo-co… Co-led with @brenthyi
New project! Flow Policy Gradients for Robot Control tldr; a simple online RL recipe for training and fine-tuning flow policies for robots co-led w/ @redstone_hong: hongsukchoi.github.io/fpo-co…
23
201
17,606
Brent Yi retweeted
we started a company!! so, we’re tackling continual learning: what’s the learning algorithm to take arbitrary data — documents, conversations, the models’ own experience — and make better models? how do we scale compute in the same way we’ve already seen with pre-training and inference time, but scaling on the same data we see as humans, day after day with no labels, no rewards? A lot of the ingredients are out there already (rl, distillation, long-context, sparse / param-efficient architectures, etc.). our team is at the frontier of these topics, and we’re singularly focused on this. we want to understand this problem better than anyone else in the world. nobody’s solved this problem yet, but even today it’s extremely greenfield opportunity to co-develop research & useful products. in our space, how people interact with the models defines what the data distribution is - and working on this problem end-to-end, from core science to end user, gives us incredible freedom to define the problem and imagine new kinds of experiences. i expect we’ll use models that continually learn much differently than we’re using them today. it’ll feel different when the models _just know_, and build on our thinking and direction in ways we can’t even imagine. we don’t even know the queries we’re not asking, the things we would do but aren’t able to today. i’m so excited to share what we’re doing with the world in the coming months!! and the team is extremely cracked :) tackling this grand challenge and working alongside @jxmnop @EyubogluSabri @dan_biderman @MayeeChen @__howardchen @shizhehe and many others has made every day so fun. come work with us!
71
54
677
117,085
Brent Yi retweeted
Introducing ABC: open data, training, and infrastructure for robotics. We release the largest teleop dataset to date, and extensively investigate design decisions, pretraining, and post-training techniques. @arthurallshire @Cinnabar233 @adamrasb @redstone_hong @davidrmcall
34
113
736
365,937
Brent Yi retweeted
Philipp and team are some of the most hardworking and talented scientists I’ve met. So bullish on this team!!! Very much worth keeping an eye out for them šŸ‘€
Announcing @xdofai: We’ve raised $70 million to build the core robotic infrastructure ecosystem for robot foundation models. My cofounders Fred (@YideShentu), Nemo (@itsnemojin) and I have been pursuing the dream of general purpose robots for our entire lives. After work at Covariant, Meta and Tesla, it became clear to us that general purpose robots are coming, and we are building XDOF to help make them a reality. For the last two years, we’ve been working behind the scenes to support major labs and companies deploying robots. In us, they have a partner with full-stack expertise, from hardware to operations to policy training. As our first public contribution to the space, we are open-sourcing ABC-130K, the largest open source teleoperation dataset, in collaboration with our partners from UC Berkeley, Carnegie Mellon, MIT and Amazon FAR. Thank you to our customers, partners, collaborators and investors for your trust and conviction in us. Together, we can accelerate the future of robotics!
2
1
19
3,470
Brent Yi retweeted
Our new work, STITCH 2.0, can perform consecutive running sutures to close a sample wound with the daVinci robot.
8
15
61
26,888
World models are heavy. They don't need to be. Each frame is encoded as 1024 spatial tokens. What if it were just 1? In our #CVPR2026 Highlight from Amazon FAR, we compress frames into "delta" tokens for efficient generative world modeling. Paper, code & models below ↓ (1/7)
12
77
611
58,415
Brent Yi retweeted
What’s the right representation for a world model? 3D, pixels, or something else? Excited to release our new paper ā€œForecasting Motion in the Wildā€ where we propose point tracks as tokens for generating complex non-rigid motion and behavior From @GoogleDeepmind @Berkeley_AI @TTIC_Connect
9
73
472
83,682
Brent Yi retweeted
Robotics: coding agents’ next frontier. So how good are they? We introduce CaP-X: an open-source framework and benchmark for coding agents, where they write code for robot perception and control, execute it on sim and real robots, observe the outcomes, and iteratively improve code reliability. From @NVIDIA @Berkeley_AI @CMU_Robotics @StanfordAILab capgym.github.io 🧵
20
125
645
179,688
Brent Yi retweeted
Excited to share our latest work on motion generation! We tackled multi-agent generation across diverse tasks using Diffusion Forcing. Check out the project page for more! šŸš€
When people share a space, their movements become intertwined. Embodied agents need to understand these social dynamics to interact effectively. Introducing MAGNet 🧲, a unified autoregressive diffusion forcing model for multi-agent motion generation that captures these interactions. MAGNet is flexible: predict the future, fill in missing motion, or have people react to each other, all while naturally scaling to N>2 people and generating ultra-long motion sequences.
1
1
10
1,176
Brent Yi retweeted
When people share a space, their movements become intertwined. Embodied agents need to understand these social dynamics to interact effectively. Introducing MAGNet 🧲, a unified autoregressive diffusion forcing model for multi-agent motion generation that captures these interactions. MAGNet is flexible: predict the future, fill in missing motion, or have people react to each other, all while naturally scaling to N>2 people and generating ultra-long motion sequences.
8
58
371
67,542
Brent Yi retweeted
So basically the most valuable thing to build right now is friendship
188
383
3,170
145,849
Brent Yi retweeted
The viser viewer in mjlab just got a huge QOL upgrade! - Real-time factor control: go slower or faster than real-time and viewer paces physics to match - Single step mode: advance one physics step at a time (super useful for debugging!) - Overall faster and smoother
5
24
180
7,596
Brent Yi retweeted
New in mjlab from the amazing @ki_ki_ki1: 8 new terrains and a viser-based terrain visualizer šŸ˜Ž
3
16
157
17,065
Brent Yi retweeted
We trained diffusion models on a billion LLM activations, and we want you to use them! New preprint: Learning a Generative Meta-Model of LLM Activations Joint work with @feng_jiahai, @trevordarrell, @AlecRad, @JacobSteinhardt. More in thread 🧵
32
193
1,443
226,554