We've hit 100 episodes! Here's a look back on our journey so far (website with some stats): robopapers100.com/ Some highlights in the thread 🧵:

Achieving human-like dexterity is the next frontier for robotics, and yet dexterity data is often subtly hard to scale. Real-world dexterity data, including things like finger-pose estimates, is often slightly off, making it physically invalid and hard to execute on real hardware and hard to learn from. DO AS I DO is an algorithm for reconstructing and retargeting monocular RGB videos to robot hands, outperforming the state of the art and even working from generated videos. @bhawna_paliwal_ @HarithejaE @willjhliang and @notmahi join us to tell us more. Watch Episode 107 of RoboPapers, with @chris_j_paxton and @DJiafei, now!

Full episode dropping soon! Geeking out with @bhawna_paliwal_ @HarithejaE @willjhliang @notmahi on Do as I Do: Dexterous Manipulation Data from Everyday Human Videos do-as-i-do.com/ Co-hosted by @chris_j_paxton @DJiafei

Full episode dropping soon! Geeking out with @bhawna_paliwal_ @HarithejaE @willjhliang @notmahi on Do as I Do: Dexterous Manipulation Data from Everyday Human Videos do-as-i-do.com/ Co-hosted by @chris_j_paxton @DJiafei

World Action Models are becoming more popular in robotics, as they learn to predict the world jointly with learning how to act on it. However, these world predictions are usually purely based on reconstructing color images from video. This is limiting, because color is far from the most important quality for a robot moving around in the world — more important are qualities like 3D geometry and object semantics. In Flex-π, @GeYan_21, @Jesse_Y_Zhang, and team train a 6 billion parameter world action model to do exactly this, predicting 3D pointmaps and DINO features along with color. This results in a policy which is much more demonstration-efficient, generalizes well, and can perform complex long-horizon tasks. Learn more in Episode 106 of RoboPapers, hosted by @DJiafei and @ruijie_sg!

Full episode dropping soon! Geeking out with @GeYan_21 @Jesse_Y_Zhang on Flex-π: A Multi-Stream World-Action Model with Compute Flexibility flex-pi.github.io/ Co-hosted by @DJiafei @ruijie_sg

Full episode dropping soon! Geeking out with @GeYan_21 @Jesse_Y_Zhang on Flex-π: A Multi-Stream World-Action Model with Compute Flexibility flex-pi.github.io/ Co-hosted by @DJiafei @ruijie_sg

Imitation learning, especially with interventions, has driven so much recent robotics progress. However, improving a policy via targeted interventions until it reaches a useful and deployable success rate is a time and labor intensive process. Instead, wouldn’t it be great if policies could improve on their own? That’s what @VarunGiridhar3 and @animesh_garg join us to talk about. In Q-Planning, they start with a large policy like pi-0.5, and add a Q-function estimator to predict value instead of just actions, then use both successful and failed rollouts to update this Q-function online, then use it to guide sampling and trajectory selection. With just a few rollouts they can dramatically improve policy performance online. This provides a way to do really difficult tasks like inserting a credit card into a wallet, increasing success rate from 25% to 80% in just a few iterations. Learn more in Episode 105 of RoboPapers, hosted by @micoolcho, @chris_j_paxton, and @ruijie_sg.

Full episode dropping soon! Geeking out with @VarunGiridhar3 @animesh_garg on Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning q-planning.github.io/ Co-hosted by @micoolcho @chris_j_paxton @ruijie_sg

Full episode dropping soon! Geeking out with @VarunGiridhar3 @animesh_garg on Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning q-planning.github.io/ Co-hosted by @micoolcho @chris_j_paxton @ruijie_sg

One of the key advantages of legged robots like humanoids should be how effectively they can move across a wide variety of terrain types to accomplish their task. But Light-Loco-Parkour from the team at Light Origins aims to change that: using only onboard sensing, they show a policy which can decide when to walk, vault, climb, or otherwise traverse as it moves through a complex environment. Unlike many others, it uses sparse seeds instead of relying on a large motion corpus, learning when to use its skills to move around without specific sub-task labels. @ChemXiaodao and @Yuntao144 join us to go into the details. Watch Episode 104 of RoboPapers now, with @micoolcho and @chris_j_paxton, to learn more!

Full episode dropping soon! Geeking out with @ChemXiaodao @Yuntao144 on Light-Loco-Parkour: Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation light-loco-parkour.github.io… Co-hosted by @micoolcho @chris_j_paxton

Full episode dropping soon! Geeking out with @ChemXiaodao @Yuntao144 on Light-Loco-Parkour: Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation light-loco-parkour.github.io… Co-hosted by @micoolcho @chris_j_paxton

Specifying reward functions for robots is one of the hardest things about reinforcement learning. Robot rewards often need to be very detailed; metrics like progress can be ill-defined and hard to estimate. This leads to most robot learning defaulting to sparse rewards or simple preference learning. But naive preference learning (having human annotators choose one trajectory over another) is an easy solution, but obscures a lot of the signal in complex tasks and can make learning a lot less efficient. @marceltornev, @anubhamahajan01, and @AbhijnyaBhat join us to talk about their solution: freeform preference learning, which lets annotators define natural-language axes to compare trajectories over. This improves real-world performance on long-horizon manipulation tasks over sparse rewards and simple binary preference learning. Watch Epsiode 103 of RoboPapers, with @micoolcho and @DJiafei, today to learn more!

Full episode dropping soon! Geeking out with @marceltornev @anubhamahajan01 @AbhijnyaBhat on Freeform Preference Learning for Robotic Manipulation freeform-pl.github.io/fpl.we… Co-hosted by @micoolcho @DJiafei

Full episode dropping soon! Geeking out with @bgxc on EgoSuite-Open100K egosuite-oepn-100k-test.ligh… @LightwheelAI Co-hosted by @micoolcho @DJiafei

Full episode dropping soon! Geeking out with @bgxc on EgoSuite-Open100K egosuite-oepn-100k-test.ligh… @LightwheelAI Co-hosted by @micoolcho @DJiafei

Reinforcement learning has unlocked tremendous gains in broader AI and humanoid robot whole body control, but still has yet to deliver strong results for robot manipulation due to the difficulty of capturing tasks in an accurate simulation; one way to fix this is to create complex environments with learned world models. But learning fully within world models is often intractable as powerful world models are computationally too expensive. Instead, @Jsphamigo and @Rk4342R propose to break the problem down: into a large-scale world model that can generate forward trajectories, and a lightweight, low-dimensional latent-space model which can approximate local dynamics of a problem, without needing to back-propagate through a heavy global model. What this means: you can do reinforcement learning in this coupled local-global world model, and learn contact rich skills for a robot. To learn more, watch Episode #101 of RoboPapers, with @micoolcho and @DJiafei!