Synthetic Data | World Models | Radiance Fields | Computer Vision | GenAI | Chief Evangelist at @LightwheelAI
Salem, OR
- Tweets 11,656
- Following 1,749
- Followers 13,539
- Likes 7,097
What if making a cup of coffee could help train a humanoid robot?
It sounds like one simple task, but a robot sees dozens of smaller actions.
At @HumanoidsSummit this week, I tried on one of @LightwheelAI's egocentric data capture systems: head-mounted cameras plus wrist cameras recording how I interact with the world from a human perspective.
Even something as simple as making coffee becomes a sequence of actions: grab the cup, place it, pick up the pod, open the machine, insert the pod, close it, press the button.
Capturing and annotating these human demonstrations is one way we can create training data for robots learning longer-horizon tasks in the real world.
And yes, making coffee while wearing all of this is about as natural as it looks. 😅
#Robotics #PhysicalAI #EgocentricData #HumanoidRobots
1
1
162
I’ll be at IROS next week. DM me!
Lightwheel is heading to #IROS2026. Booth 714, Pittsburgh, Sep 27 to Oct 1.
🔹 Learn: EgoSuite-Open100K, 100K hours data of humans doing diverse and real tasks
🔹 Train: SimFoundry, simulation worlds where robots practice
🔹 Evaluate: RoboFinals, a standard exam for robot policies
🔹 Deploy: RoboStack, from sim to real robot deployment
1
1
400
For people of a certain age, this video has way more meaning.
Interesting robot, intriguing price point.
Feather Robotics
2
7
1,118
I had a great time at @HumanoidsSummit Seoul! If you didn’t get to come learn about EgoSuite or our continuous learning cycle for robotics, you can come see me next week at IROS in Pittsburgh!
#robotics #physicalai
3
1
19
1,113
Sim2Real is so back baby!!!!
We trained a robot to play football.
How? Self-play for 140 years in a virtual World Cup.
Meet the #Messinator:
2
2
14
2,118
My biggest AI unlock this year: it now reviews all of my kids’ school emails and tells me what I need to know. I swear I get 20 emails a week from their schools, most not worth me reading.
Today it called out one teacher for listing a concert date and it was incorrect. My AI noticed it and emailed the teacher about the mistake 😆😆😆
1
2
699
Jonathan Stephens retweeted
Happy to announce the full ABC release! We’re also excited that ABC was accepted to CoRL 2026!
Check out our website for code, 400+ hours of sim data on 24 tasks, and 5,850 labeled policy-evaluation episodes.
@arthurallshire @Cinnabar233 @ritvik_singh9 @redstone_hong @davidrmcall
5
32
248
74,300
A video started circulating yesterday of a robot unplugging itself from a box and standing up. Obviously AI. This is actually how it’s done 😎
This is the @AGIBOT_US A3 if you’re curious.
1
20
5,174
There's a lot of comments on how these home robots are really slow and a human would be much faster.
That's bad reasoning.
It is doing chores passively. I go to work and it has all day to clean up. Speed is irrelevant. It can also work while I'm asleep. It does not have to complete chores in the timescale I have to work in.
This is also a fascinating video to watch. You can see the robot occasionally make small mistakes, reason on how to correct, and then fix it's action. That process is slow, but again, it has all day to folds laundry, load the dishwasher, make beds, clean the bathroom, etc.
1
2
20
3,035
Scaling laws and next robot action prediction was my favorite part of this video. As we scale data, the better these zero shot models will get. However, I still have questions. Will Index inherently have enough data diversity to keep scaling?
I love this progress.
The holy grail for robotics is being able to generalize: doing work in unseen places
We rented 30 homes in the Bay Area and are doing tasks without any new training
1
9
1,810
This is insane. The NVIDA campus models from only 32 images. I have been here, it's full of unique architecture, occlusions, lighting differences. Looks like Atlas NAILED IT!
World Labs
3
3
56
3,228
I finally got my hands on @SpAItial_AI’s Echo. Here are my first 3 tests with just a single reference image. An outside urban area, a hotel room, and an expansive lake. I also toggled on the point view. None of these use Echo 2 high quality. What do you think?
#worldmodel #3dgs
5
14
118
5,498
I got roughly 100,000 waking hours of experience by the time I was 18. I was pretty self-sufficient by then. How many hours of egocentric data does a robot need before we can match the intelligence I had at 18?
516
The market is being flooded by purpose built egocentric data capture sensors. Just remember, hardware is only part of the equation that makes data high quality.
Grounded API is live.
- SOTA on hand-tracking benchmarks (< 1 cm)
- SOTA on SLAM benchmarks
- In-the-wild ego data -> enriched data in minutes
- Integration with @huggingface @LeRobotHF & @rerundotio
- Built for @BitRobotNetwork RoboCap suite
Technical report & more↓
2
1
9
1,476
Really cool what @jeremyparkphd is doing with computer vision and rock climbing. This could be turned into a full product for people who want to get better at it.
He’s got other experiments just like this. I suggest following him if you want to see interesting CV experiments.
I made a computer vision tool for rock climbing holds analysis!
As someone who is newer to climbing and wants to improve, I've noticed more and more recently how everyone solves the route differently. As I said to my friends the other day, "Everyone has to find their own way." 🤣
So I extended my last rock climbing demo with new analytics that I'm really excited about. I added what percentage of the total time your hands and feet are on a hold, as well as which holds each hand and foot is using.
This data can be used in a few ways. For example, I know that I should use my feet and legs more when climbing. This tool can show me how much I use my feet, so I can use that as a baseline to improve.
What I'm most excited about is how it enables comparing climbing sequences on the same route. You can compare against your friends' sequences: maybe you use 1, 3, 5, 7, and another friend uses 1, 2, 6, 7. Or you can compare against your own historical data, like I show here at the end.
I used ViTPose+ Large for pose estimation and SAM 3.1 to segment the holds, both models accessed through the @vlmrun Gateway. I also use SAM 3.1 to segment the gray floor, so we know when the feet leave the ground to help with starting the route timer.
Would love to hear what you think!
The code is open-source on GitHub: github.com/jeremyipark/visio…
1
3
9
652
Everyone is having fun training their MicroDuck in sim. But let’s not forget you can have fun with @pollenrobotics’ Reachy Mini today!!!
I’ve had a blast with this thing over the last couple of months. My kids love it too. My nephew even helped build it!
7
753