Introduce HumanPlus - Autonomous Skills part Humanoids are born for using human data. Imitating humans, our humanoid learns: - fold sweatshirts - unload objects from warehouse racks - diverse locomotion skills (squatting, jumping, standing) - greet another robot Open-sourced!
11
119
561
158,567
We build our customized 33-DoF humanoid, and a data collection pipeline through real-time shadowing in the real world.
Zipeng Fu
@zipengfu
13 Jun 2024
Introduce HumanPlus - Shadowing part Humanoids are born for using human data. We build a real-time shadowing system using a single RGB camera and a whole-body policy for cloning human motion. Examples: - boxing🥊 - playing the piano🎹/ping pong - tossing - typing Open-sourced!
1
3
24
5,106
Using the data collected through shadowing, we then perform supervised behavior cloning to train skill policies using egocentric vision. We introduce Humanoid Imitation Transformer. Based on ACT, HIT adds forward dynamics prediction on image feature space as a regularization.
1
3
15
2,740
Compared to baselines, HIT uses - binocular vision, thus having implicit stereos for depth information - visual feedback better, avoiding overfitting to proprioception given small-sized demos

Jun 13, 2024 · 4:06 PM UTC

1
2
8
1,919
Besides vision-based whole-body manipulation skills, our humanoid has strong locomotion skills: - outperforming H1 default standing controller under strong perturbation forces - enabling more whole-body skills like squatting and jumping
1
2
6
2,459
This project is not possible without our team of experts, covering from computer graphics to robot learning to robot hardware: - co-leads: @qingqing_zhao_ @Qi_Wu577 - advisors: @chelseabfinn @GordonWetzstein project website: humanoid-ai.github.io hardware: tinyurl.com/humanplus-hardwa… code: github.com/MarkFzp/humanplus
2
7
31
2,882
Sort replies: Relevant Recent Liked