PhD Candidate at NYU & LAAS-CNRS - Robotics, Reinforcement Learning, Deep Learning
New York
- Tweets 22
- Following 82
- Followers 38
- Likes 71
Joseph Amigo retweeted
Unitree Breakthrough: The World’s First Real-Time World Model-Driven Fully Autonomous Humanoid Robot Combat🥊
UnifoLM-X2-1.0 breaks through world-action foundation models' bottlenecks in instant planning, decision-making, and dynamic interactive execution, achieve high dynamics, strong interaction, real-time prediction and planning of the future, achieve fully autonomous combat for humanoid robots. This validates the fundamental feasibility of large-scale deployment of world model-driven humanoid robots.
142
319
1,831
265,261
Joseph Amigo retweeted
Come checkout my lab mates great work on using world model for RL policy! They trained their interactive video model completely from scratch - the best I have ever tried.
coupled-global-local-wm-rl.p…
Reinforcement learning has unlocked tremendous gains in broader AI and humanoid robot whole body control, but still has yet to deliver strong results for robot manipulation due to the difficulty of capturing tasks in an accurate simulation; one way to fix this is to create complex environments with learned world models. But learning fully within world models is often intractable as powerful world models are computationally too expensive.
Instead, @Jsphamigo and @Rk4342R propose to break the problem down: into a large-scale world model that can generate forward trajectories, and a lightweight, low-dimensional latent-space model which can approximate local dynamics of a problem, without needing to back-propagate through a heavy global model. What this means: you can do reinforcement learning in this coupled local-global world model, and learn contact rich skills for a robot.
To learn more, watch Episode #101 of RoboPapers, with @micoolcho and @DJiafei!
1
3
345
Joseph Amigo retweeted
Clever split, but the local model still needs feeding. Contact dynamics come from real contact hours, so the bottleneck moves from simulator fidelity to collecting messy physical data.
1
2
37
Joseph Amigo retweeted
Super cool! Agree this is the tractable path. Contact dynamics are high frequency and short range, so paying for resolution only where the fingers touch is where the compute should go.
1
1
38
Joseph Amigo retweeted
Reinforcement learning has unlocked tremendous gains in broader AI and humanoid robot whole body control, but still has yet to deliver strong results for robot manipulation due to the difficulty of capturing tasks in an accurate simulation; one way to fix this is to create complex environments with learned world models. But learning fully within world models is often intractable as powerful world models are computationally too expensive.
Instead, @Jsphamigo and @Rk4342R propose to break the problem down: into a large-scale world model that can generate forward trajectories, and a lightweight, low-dimensional latent-space model which can approximate local dynamics of a problem, without needing to back-propagate through a heavy global model. What this means: you can do reinforcement learning in this coupled local-global world model, and learn contact rich skills for a robot.
To learn more, watch Episode #101 of RoboPapers, with @micoolcho and @DJiafei!
5
13
57
16,773
Joseph Amigo retweeted
Learn more
Project page: coupled-global-local-wm-rl.p…
ArXiV: arxiv.org/abs/2602.06219
This on substack: robopapers.substack.com/p/ep…
1
1
4
785
Joseph Amigo retweeted
Learn contact-rich humanoid manipulation policies, purely within a world model. This is usually not tractable, because you need a very large model to get the contact dynamics right -- instead, @Jsphamigo @Rk4342R couple a large global model with a local model. Very cool work on @RoboPapers ->
Reinforcement learning has unlocked tremendous gains in broader AI and humanoid robot whole body control, but still has yet to deliver strong results for robot manipulation due to the difficulty of capturing tasks in an accurate simulation; one way to fix this is to create complex environments with learned world models. But learning fully within world models is often intractable as powerful world models are computationally too expensive.
Instead, @Jsphamigo and @Rk4342R propose to break the problem down: into a large-scale world model that can generate forward trajectories, and a lightweight, low-dimensional latent-space model which can approximate local dynamics of a problem, without needing to back-propagate through a heavy global model. What this means: you can do reinforcement learning in this coupled local-global world model, and learn contact rich skills for a robot.
To learn more, watch Episode #101 of RoboPapers, with @micoolcho and @DJiafei!
9
19
132
12,920
Joseph Amigo retweeted
Ten grasp-and-lifts in a row on a G1, zero-shot, from a single head-mounted camera. THAT is the demo.
1
1
102
Joseph Amigo retweeted
Full episode dropping soon!
Geeking out with @Jsphamigo @Rk4342R on Coupled local and global world models coupled-global-local-wm-rl.p…
Co-hosted by @micoolcho @DJiafei
1
9
23
2,171
Joseph Amigo retweeted
Full episode dropping soon!
Geeking out with @Jsphamigo @Rk4342R on Coupled local and global world models coupled-global-local-wm-rl.p…
Co-hosted by @micoolcho @DJiafei
8
19
1,753
Introducing our new work DMO: Decoupled Model-based policy Optimization! First-order gradient RL that unrolls trajectories with high-fidelity sims & computes gradients via learned models.
Paper & demos: machines-in-motion.github.io…
#CoRL2025 w/ @Rk4342R
2
5
28
2,280
Joseph Amigo retweeted
First Order Model-Based RL through Decoupled Backpropagation (DMO)
machines-in-motion.github.io…
Simulation rollouts, learned model for first order optimization
2
17
108
6,557