Co-Founder and CEO at Walden Robotics. Professor at MIT.
- Tweets 36
- Following 99
- Followers 6,638
- Likes 12
Thanks for hosting me, Brad.
The best case for wheels I've heard came from the guy who has taught legged locomotion at MIT for 20 years.
AMR safety cases assume you slow proportional to distance and come to rest. A balancing biped may need to take steps to stop. The safety zone explodes and the robot ends up behind a fence.
A robot behind a fence isn't deployed.
Deployed EP12 is live: cobot-deployed.com/ep12
4
16
4,894
When we first started working on Diffusion Policy, which felt like an "unlock" for fitting human demonstration data, the explanation I gave myself was that "human demonstration data was subtly much more multimodal than we thought, so we need the modeling power of denoising diffusion to fit it properly."
What Michael's paper brings to light is that some of this multimodality was probably exacerbated (more than I realized) by the fact that we've only been giving our models incredibly short observation context lengths.
For me, this is pretty compelling evidence that we should be pushing for policies with long-context lengths and short action prediction horizons.
Action chunking — especially executing long action sequences open-loop — is widely used in imitation learning for robotic manipulation. Why is it so effective and do we really need it? We find a key reason:
Long open-loop execution helps short-context policies imitate non-Markovian experts.
With this insight, we show how to move beyond open-loop execution: extending policy context restores reactivity while achieving even higher task performance.
🧵(1/5)
8
25
237
23,077
Russ Tedrake retweeted
Building great robots takes great partners.
Proud to be collaborating with @SamsungSDSA on a proof of concept focused on manufacturing use cases, combining Walden's robotics platform with Samsung SDS' manufacturing expertise.
More: bit.ly/4fZJwoJ
ALT Walden Robotics and Samsung representatives shake hands as colleagues applaud, with two Walden robots on either side.
3
9
42
8,967
In January, I started "building something new" with an incredible team. Today I finally get to share some first details about what we've been building. We've called it Walden Robotics (waldenrobotics.com).
I thought long and hard about my own reasons for starting this company. It's not only about the robots. It's also about people. I've tried to capture those thoughts in my first Walden blog post: waldenrobotics.com/news/why-…
It's been an incredible ride so far. Within just a few months of forming the company, we were already operating a general-purpose robot with an end-to-end policy in production in one of the most important factories in North America. It's amazing at how much I've already learned from that experience. There is a lot of work to do, but the mission has never been so clear.
Please help me welcome Walden Robotics into the world. And stay tuned for more updates!
piped.video/watch?v=fewvZrck…
68
115
1,171
203,613
When it comes to diffusion policies, VLAs, and LBMs, we all know that data quality matters. Many of us have spent considerable time filtering data, and discarding large amounts of data that might not be considered to be sufficiently high quality. This feels very inefficient.
We’ve seen several proposals of late for how we might learn from suboptimal data. pi0.7, for instance, suggested that we learn models conditioned on the data quality.
Adam’s new paper exposes a different (complementary) angle. When we’re training denoising diffusion models, each data point is potentially used several times — at different noise levels. **The value of a data point might be different depending on the noise level.**
Adam’s work explores this nicely and shows that he can extract value even out of significant portions of the OXE dataset that many of us had previously discarded as suboptimal.
Definitely worth having a look.
🤖 We introduce Ambient Diffusion Policy, a simple and principled method for training policies with suboptimal data in robotics.
Suboptimal data is everywhere in robotics…
❌ Data filtering is wasteful
❌ Co-training learns both good and bad features
✅ Ambient Diffusion Policy selectively learns useful features via noise-dependent data usage
👇🧵(1/5)
3
20
159
24,737
I've been saying for years that the biggest challenge for simulation in robotics is not actually the physics engine (although you do have to get that right). The real challenge is capturing the *diversity* of the real world. There was no doubt that generative AI had the potential to change that, but it's still amazing to see it take shape.
Watching Nick's incredibly fast progress has convinced me that content generation might not actually be a bottleneck anymore. This is a beautiful combination of hardened tools for e.g. low-level mesh processing with the latest tools for generative asset creation, wrapped in a powerful agentic workflow. Please do give it a try and share your feedback.
Meet SceneSmith: An agentic system that generates entire simulation-ready environments from a single text prompt.
VLM agents collaborate to build scenes with dozens of objects per room, articulated furniture, and full physics properties.
We believe environment generation is no longer the bottleneck for scalable robot training and evaluation in simulation.
Website: scenesmith.github.io/
👇🧵(1/8)
12
33
335
40,994
Russ Tedrake retweeted
Introducing Large Video Planner (LVP-14B) — a robot foundation model that actually generalizes. LVP is built on video gen, not VLA. As my final work at @MIT, LVP has all its eval tasks proposed by third parties as a maximum stress test, but it excels!🤗
boyuan.space/large-video-pla…
23
94
574
101,304
Russ Tedrake retweeted
Boston Dynamics and Toyota Research Institute are advancing humanoid robotics with Large Behavior Models (LBMs) for Atlas.
64
235
1,073
124,056
TRI's latest Large Behavior Model (LBM) paper landed on arxiv last night! Check out our project website: toyotaresearchinstitute.gith…
One of our main goals for this paper was to put out a very careful and thorough study on the topic to help people understand the state of the technology, and to share a lot of details for how we're achieving it.
piped.video/watch?v=BEXFnru5…
8
107
492
89,712
Probably my favorite plot from the paper, which sums it all up, is this one.
The plot compares performance using different amounts of pretraining data used before training a new task: 0% (aka single task), 25, 50, or 100% of TRI’s data, then 100% of TRI’s data + all of the open-source robot data (the red line) that we’ve curated. It’s just awesome that the distributions over task completion are so tight and that trends as we increase data are so consistent. The results show clearly that with pretraining, we can train a novel skill with substantially less data or use the same amount of data and get much better task performance. And the benefits appear to continue with more data.
2
3
37
6,129
Very proud of Nicholas, who recently shared scalable-real2sim.github.io/ (for physics-quality assets from a small amount of interaction with a robot) and is now following up with his work on scene-level generation.
Want to scale robot data with simulation, but don’t know how to get large numbers of realistic, diverse, and task-relevant scenes?
Our solution:
➊ Pretrain on broad procedural scene data
➋ Steer generation toward downstream objectives
🌐 steerable-scene-generation.g…
🧵1/8
1
8
68
10,232
This work really sharpened my thinking about sim+real cotraining.
Learning from both sim+real data could scale robot imitation learning. But what are the scaling laws & principles of sim+real cotraining?
We study this in the first focused analysis of sim+real cotraining spanning 250+ policies & 40k+ evals arxiv.org/abs/2503.22634 (1/6)
3
4
56
7,125
Side note: I'm proud of the title of this paper, which we intentionally made pretty narrow/specific. I think that some of the most important work that we have to do as a field right now is careful empirical work to interrogate the properties of these models that we're creating.
15
1,557
One of the most interesting take-aways for me is that "high-performing policies need to know whether they are executing in sim or in real." A number of implications flow from that, including that sim+real cotraining can decrease performance if the visual gap is too small.
2
2,117
Russ Tedrake retweeted
New Paper: "Scalable Real2Sim: Physics-Aware Asset Generation via Robotic Pick-and-Place Setups"! 🤖
We introduce a fully automated pipeline that generates simulation-ready assets for real-world objects—no manual intervention needed!
🌐 Website: scalable-real2sim.github.io/
(1/5)⬇️
4
23
120
13,380