building agents and poses @nvidia | Robograd Biorobotics lab @CMU_Robotics | Investing in startups | Taking side of folks who build
San Francisco, CA
- Tweets 1,273
- Following 3,190
- Followers 1,035
- Likes 37,319
Pinned Tweet
Check the full suite for full mocap to robotics pretraining . SOMA has anatomically correct joint definitions and has much detailed mesh key points compared to MHR/SMPL. Foundational for all bodypose downstream tasks. More on this soon on its capabilities.
#NVIDIA just released a whole ecosystem for human(oid) motion and robot learning from human data. 🚀🦾
Data, as we all know, is the key to scaling AI models. To accelerate the field of Embodied AI, we have open-sourced a full stack of models and tools to capture, generate, retarget, and simulate human(oid) motion data at scale, along with a massive high-quality dataset and a standard human skeletal representation, SOMA, to make them all seamlessly communicate with each other.
The entire suite is available under the Apache 2.0 license.
1️⃣ SOMA: A universal interface to unify all parametric human body models (SOMA-shape, SMPL, MHR, etc.) into a standard skeletal representation, eliminating the need for custom adapters or model-specific retargeting.
🔗 lnkd.in/gsxhiJnn
2️⃣ Kimodo: High-fidelity, controllable text-to-motion generation for both humans and humanoid robots.
🔗 lnkd.in/gCc84XnX
3️⃣ GEM: A global human pose estimation method from in-the-wild videos, natively compatible with SOMA.
🔗 lnkd.in/g_QAvRjn
4️⃣ Bones-SEED: A massive dataset of 150k+ motions in SOMA format, including data already retargeted for the Unitree G1, created with our partners at Bones Studio.
🔗 lnkd.in/gfx-QD-w
🔗 lnkd.in/gyNdTwQx
5️⃣ SOMA Retargeter: A dedicated tool for seamless motion retargeting from the SOMA skeleton to the Unitree G1.
🔗 lnkd.in/gqz9Na-H
6️⃣ ProtoMotions: Our high-performance simulation framework for training digital human(oid)s via RL, now with native SOMA support.
🔗 lnkd.in/gmvMikMU
This is just the beginning, and we have much more in the pipeline. Excited to see what the community builds next!
#NVIDIA #GTC #GTC2026 #Robotics #EmbodiedAI #PhysicalAI @NVIDIAAI
2
1
16
2,708
Peter Thiel: "I probably still play way too much chess on the internet all the time."
"It's this very beautiful, somewhat dangerously addictive game."
"At various points of delusion in my teenage years, I liked to say that everything in the world, if you didn't play chess, you didn't understand reality."
"It's probably too extreme a thing to say."
Via Mathias Döpfner
Peter Thiel says in the late ‘80s to early ‘90s, he thought everyone should be judged on chess ability:
“I had a chess bias because I was a pretty good chess player.”
“That got undermined by the computers in 1997.”
Via @tylercowen @mercatus
21
81
1,156
325,974
Harsh retweeted
Opus 5.5 designing LEGO 👀
I asked it to design a Microduck I can build with real LEGO pieces. It:
> designed it life-size using 1113 real LEGO parts
> verified: 3,204 connections, 0 collisions, every step buildable, centre of mass inside the feet 🤯
> made a 141-page LEGO-style booklet (237 steps)
> priced every piece in the browser and prepared the orders on BrickLink
329
652
8,490
1,289,660
Anthropic got way too laxed and it’s showing . I was personally disappointed being forced to move to codex and them way too slow for any update. Opus5.5 might be a good model than Astra but there is this friction to switch and I think codex feels better especially with the live voice integration .
Spend of OpenAI vs Anthropic vs Open
(Vercel AI Gateway, last 2 months)
• Anthropic still #1 in spend, but went 69% → 40%
• OpenAI: 10% → 24% in spend
• GPT-6 Astra + GPT 5.6 Sol are ripping
• OpenAI now leads in tokens #
• Kimi K3 + DeepSeek took ~half of Anthropic's loss
• Opus 5.5 is up to 10% of spend in 2 days
• OpenAI is 62% of image generations
Watch here: token-race.vercel.app
2
5
7,277
Harsh retweeted
💚
Fun discussions at #ROSCon with @NVIDIARobotics and @NVIDIAAI teams about my open source @AgenticROS project connecting #NemoClaw + #Nemotron + #IsaacROS to @RealSenseai and #ROS allowing agentic AI to control its own body! #PhysicalAI
2
11
106
9,728
Harsh retweeted
What will be the “RLHF” moment for robotics? What will it take to get robotics to where LLMs are today and beyond?
New blog post with @chelseabfinn sharing some thoughts on the state of RL for frontier robotics models and what's missing 👇
Blog: pd-perry.github.io/posts/pos…
14
72
442
77,645
Harsh retweeted
In the race to build better AI models, most of the attention has gone to pre-training. But in robotics, we believe post-training is just as critical.
Pre-training can get you impressive capabilities. Post-training is what closes the gap between a demo and a deployment—between a video and actual dollars. In fact, the real frontier in robotics may increasingly lie in post-training.
Today, we’re introducing a new approach to post-train robotic policies using self-play. Inspired by the original self-play work at DeepMind, as well as our own work on robust adversarial reinforcement learning, we train policies in simulation to help emerge behaviors and make them robust before they ever reach the real world.
This is an early step (and on sports), but we believe it points toward something much bigger: bringing some of the original ideas that made reinforcement learning so powerful back into the physical AI stack.
We trained a robot to play football.
How? Self-play for 140 years in a virtual World Cup.
Meet the #Messinator:
13
11
155
26,220
Having talked to many of these this is completely true .
This is the best thoroughly honest take on the brutal robotics data market today.
Yes, there is now a brief arbitrage window for robotics data. But the market is saturated, entry costs are surprisingly non-trivial, and there are a single digit number of real buyers.
1
1
211
Harsh retweeted
Handwriting activates a broader network of cognitive brain regions compared to typing.
80
998
5,698
192,017
Harsh retweeted
TypeGPU + ruNNtime + Jev @typesafeai is a very fun combo :D
ruNNtime gives me efficient local inference, TypeGPU lets inference and rendering share GPU resources directly with zero copy. That’s 3 separate NN inferences plus rendering, all happening in realtime
Since we control the pipeline, Jev can just sit in the middle and add the semantic bit.
camera + mic → Moonshine + YOLO26 + DepthART → Jev → lights, shadows and bloom
36
64
711
89,817
Harsh retweeted
Excited to unveil our first product for the creatives :)
Today, we’re unveiling the new Pika—an AI creative platform made for and by Creatives. It’s simple. Comprehensive. And it’s designed for outputs that meet exacting standards.
We’re on a mission to build AI for the future of creative work. And the new Pika product is just the beginning.
13
8
104
29,309
Crazy that it’s just 32 images .
PS
@theworldlabs needs to come at night
From 32 input images to real-time flight through @nvidia's Voyager headquarters.
Trained on NVIDIA Blackwell GPUs, Atlas uses these images as 3D spatial context to generate new views, letting you explore with pixel-perfect camera control.
Take a look around.
314
Harsh retweeted
To be clear, METR staff are almost entirely folks giving up a lot of money they could make elsewhere in order to work on rigorous AI safety evals and analysis. The folks criticizing them are largely cynics who can't imagine what civic-mindedness looks like.
My name is Chris Painter, and I'm the President of METR (Model Evaluation and Threat Research). I know we've made a lot of new friends on the internet the last couple of days, so I thought I'd take this chance to re-up what we do and why.
Our work is aimed at making sure that if AI really were autonomous, difficult to steer, and close to "going rogue," the public would find out. If evidence exists inside of an AI company that it’s close to losing control of AI, we want to make sure that information gets shared with the rest of the world, including governments and the public outside the company’s walls. This is what we've been focused on since 2022, and over the years we've worked with OpenAI, Anthropic, Google DeepMind, Meta, Amazon, and others on piloting third-party assessments and investigations of this type. We don’t have some private room where we rubber stamp things as “safe” or not.
We have had a track record of publishing results on AI that don't cleanly map onto the "doomer" or "accelerationist" labels, and we put in effort to hire people with competing views on AI. We’ve been cited for having found some of the strongest evidence that AI capabilities are improving rapidly (our work measuring AI “time horizons”) while also presenting some of the strongest evidence that, at various points, AI’s capability may be overstated (some might remember our study showing that early 2025 software engineers were actually being slowed when they thought they were being sped up).
METR is funded by donations. We don't accept money from frontier AI companies. They haven't paid us for our work, and we don't accept donations from them or their employees. As we’ve shared previously, multiple frontier AI companies currently provide us with free access to their models in order to perform our evaluations, research, and engineering. Our funding intentionally comes from a wide range of donors, which we’ve shared on our website.
Today, when an AI company works with any third-party evaluator or external testing organization (of which there are and should be many), it's entirely voluntary. This often involves NDAs and redactions. To counterbalance this, we have a principle that when we enter into a contract with a company, we try to retain the right to tell the public the terms of the contract we signed, and characterize the nature of redactions that the company chose to make. For example, the report from our independent investigation of the OpenAI-HuggingFace incident included that information. Public disclosure is also a big part of our COI policy (linked on our website). That’s not to say our reports are adequate as oversight. We’re just one organization (among many doing great work), working in a voluntary setup, trying to get good evidence to the public and the world about AI, letting the facts fall where they may.
213
58
821
76,321
bruh i dont wanna say it
Live: @DarioAmodei x @Benioff
The @AnthropicAI co-founder & CEO joins us at @Dreamforce for a conversation worth hearing ☁️
2
300
Harsh retweeted
Jensen says Nvidia doesn’t allow political discourse in the company: “Take it home.”
“That’s what they decided when they came to work for us. We told them, this is the way you behave when you work in our company. And if you like the culture of our company, which as you know, the NVIDIA culture and the NVIDIA employee base, incredibly happy.”
“We don't welcome political discourse inside our company. Take it home. You guys talk about politics outside the company.”
“The company is an apolitical company. We're bipartisan. We want America to succeed. Whatever government is in place, we'll do everything in our power to help America succeed. And so the discourse about race and religion and politics and all of that stuff, we tell people, do it outside the company. It's not for us.”
16
61
611
43,703
Harsh retweeted
Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids.
- learned directly from human manipulation data
- no teleop/robot data
- close to human-level dexterity and efficiency
- multi-robot collab
154
479
3,499
1,745,827
Harsh retweeted
There’s the feeling that Astra could just zero shot all the robotics foundation model companies.
Then again one could just leverage astra together with a good robotics model to get even better results
GPT-6 Astra is the most significant leap in robotics I’ve seen in the past few years. It cracked RoboLab with a near-perfect score. Solid infrastructure + scaling ultimately outperformed the heuristics explored in small-scale studies. We’re definitely on the brink of physical RSI.
Source: anonymous-report-421.github.…
18
17
396
31,395