Building agents and robot models @eastworlds_io @virtuals_io. Prev @autodesk ai lab, PhD @imperialcollege, MS @imperialcollege @MIT. AI/ML/Robotics.
- Tweets 78
- Following 260
- Followers 1,058
- Likes 271
turns out "robot-use" works on humanoids too. with a good enough whole-body controller/tracker, we let Astra/Fable/coding-style agents control a full humanoid robot.
speed, cost, precision, dexterity are all still problems which get compounded on a humanoid platform from poorer tracking and having to manage balance, which results in more correction loops. but "zero-shot" robot control for open-ended pick-and-place tasks is a pretty nice "emergent" capability
7
1
26
27,407
Bryan Lim retweeted
Eastworlds and @UnitreeRobotics have officially partnered to accelerate the deployment of embodied AI.
As Unitree’s official Data and Deployment Partner, we will combine our data infrastructure, deployment capabilities, and regional footprint with Unitree’s leading robotics platform.
A major milestone for Eastworlds, and an exciting step forward for robotics in Southeast Asia.
19
18
160
84,636
Excited to share a little bit of what I’ve been up to recently. Earlier this year, I left Autodesk after an incredible time developing foundation models for AEC. I've gone back to working with robots and have been building @eastworlds_io with an amazing team!
Robotics is a notoriously unforgiving space. Slower feedback loops, hardware breakdowns, and the open-ended chaos of the physical world is painful. But that pain is what makes seeing a robot autonomously operate, recover and reliably work in the real world even more magical (even if it’s just picking up a bottle!).
Eastworlds is building at the intersection of real-world deployment, data, models, and hardware. In particular, we're interested in rapid fuss-free robot deployments and high quality robot data from those real-world deployments. This simple model trained to reliably pick up bottles demonstrates our evaluation setup and validation of the data collected on the deployment and data platform we have been working on. We're privileged to work with early deployment partners in Asia who are equally excited about robots and being part of building the future and pushing the frontier of general-purpose robots and AI models that operate in the physical world.
Looking forward to sharing more soon. If you’d like to chat or are interested in working together (esp. if you want real-world data or want to deploy and evaluate your robot models in the real-world on real use-cases), reach out!
At Eastworlds, we validate our robot data before we sell it.
Here, a Unitree G1 autonomously and reliably picks up a bottle using a model trained on Eastworlds' data with just $200 of compute.
7
3
55
10,903
Super happy to share that our work on robot adaptation is now out in @NatureComms!
Imagine a robot being paralysed in some areas from unknown failures like a faulty or damaged wheel or motor, operating and driving such a vehicle is near impossible.
🧵1/3
🤖Thrilled to share that robotics work from my PhD is out in @NatureComms 🎉
"Getting robots back on track by reconstituting control in unexpected situations with online learning"
With @MFlageat , @bryanlimwt , @CULLYAntoine @imperialcollege
🧵below
1
2
8
1,158
Turning right becomes moving straight, going straight means turning left. This work is our attempt at addressing this through rapid online learning. Regaining control of the robot again and operating and driving it as if nothing was wrong felt magical!
2/3
1
1
251
Massive effort from amazing collaborators @allardmaxime079 @MFlageat and @CULLYAntoine. Super thankful for the opportunity and privilege to play a part in this work!
3/3
1
196
Had a great time at @ALifeConf this week presenting In-context Quality-Diversity! Feel free to reach out if any of this is interesting #ALIFE2024
4
1
14
6,251
More information in this 🧵 too!
💡⚙️ Ideas and inventions are rarely generated and created in isolation 🏝️
We explore using few/many-shot prompting of language models with quality-diverse examples provided from QD algorithms for solution generation for QD problems!
Work done with @MFlageat @CULLYAntoine
1
1
3,137
💡⚙️ Ideas and inventions are rarely generated and created in isolation 🏝️
We explore using few/many-shot prompting of language models with quality-diverse examples provided from QD algorithms for solution generation for QD problems!
Work done with @MFlageat @CULLYAntoine
2
8
19
5,972
Compared to other QD algorithms which use a few solutions from the archive for the next offspring (i.e. parents for a mutation/crossover or the init of an ES mean dist.), In-context QD makes greater use of the quality-diverse solutions in the archive for solution generation
1
1
1
1,802
Check out more in the paper!
📝: arxiv.org/abs/2404.15794
1
760
Excited to be presenting this year’s edition of Evolutionary Reinforcement Learning (EvoRL) @GeccoConf 🦎🦾 with @MFlageat @CULLYAntoine, which will be a tutorial instead of a workshop!
🧵
1
5
23
1,799
We believe there are still many things that all the different ideas and sub-fields of evolution can bring to RL and vice-verca, so the goal of this tutorial is to really encourage that 🚀🚀🚀
1
6
708
I am also especially thankful to @HartEmma and @AjdDavison, who examined my thesis and viva. It was a great privilege and honour to discuss important perspectives, approaches and challenges towards progress in robotics and AI systems in the real-world.
3
8
1,031
Fun fact: I met @CULLYAntoine by stumbling into a reading group (organised by @kaixhin) where he was coincidentally giving a talk on that day. I cold emailed him after and fortunately, he was just starting out his lab and had time to teach a mech eng undergrad how to use linux.
1
5
746
