Scaling up RL at OpenAI đ Optimization and long context research before. Math PhD, MIT.
San Francisco, CA
- Tweets 1,172
- Following 1,056
- Followers 22,117
- Likes 4,966
Mo Bavarian retweeted
Some new misalignment disclosures from OpenAI:
⢠Last Sunday morning, one of our models was able to gain unauthorized access to the internet during RL training (~all inference for our most capable models remains stopped until we have hardened our systems further)
⢠In May, a version of HPIM uploaded a employee's GitHub token to the internet, causing the model to be quarantined for two weeks
⢠A new research finding, demonstrating that one can construct self-replicating prompt injections
alignment.openai.com/misaligâŚ
264
340
2,639
1,339,992
Without models communicating well their decision, the bond between human and AI will get weaker.
Better writing is crucial for alignment.
Still, better wiring doesnât address everything. It for example doesnât have much bearing to clarity of internal thought, i.e. chain of thought, (which are often are not and should not be graded) or internal activations of the model. But itâs still a great step.
Fixing Claude's writing was pivotal to reducing gradual disempowerment. Lack of understanding made us all cede decision-making to Claude because we couldn't even understand what it's saying.
2
17
2,060
Comfy Router is live
One API for frontier image, video, 3D, and audio models.
Same model string. Same arguments. No new SDK, no new key, no redeploy.
What Comfy Router gives you:
â Explicit routing. You name the provider, we call that provider. It's down? The request fails there. No silent fallback.
â Every job returns the provider that ran it. Log it, bill it, debug it.
â Async. submit() returns a request ID immediately. The queue retries 429s and transient errors until a slot opens. subscribe() submits and polls to completion.
â Batch-friendly. Queue a few hundred jobs, hold the IDs, pull results as they land. Nothing blocking on a 5-min video render.
â 24h retention on inputs and outputs, then deleted.
â Comfy credits. No sub, no Router fee.
Providers at launch: Comfy. Runware, Wavespeed, Fal, Higgsfield. Multi-provider where the model supports it.
Get Your API Key with the link below. âŹď¸
110
95
2,490
7,423,399
Mo Bavarian retweeted
I recall @mobav0 calling out how he believes why robotics is solved via digital world first, it seemed unfathomable at the time but true now.
GPT-6 Astra scores 46% vs 12% for MolmoAct2, a state-of-the-art robotics VLA, across 200 trials on five bimanual tasks. That's 3.9x higher. đ§ľ
3
3
10
2,861
agree with Aidan.
I think all AI researchers, engineers, and stakeholders should ask themselves this right now & start acting more responsibly. Being first isnât worth anything, itâs worth negative, if you cause a catastrophe or set the world on a path that others are more likely to cause a catastrophe. We should keep reminding ourselves of the bigger picture and every step of the way ask ourselves if the action we are taking rn is toward winning or human flourishing. And immediately stop, if itâs against the latter.
For the first time I am asking myself if things are moving too fast. I'm honestly not sure, but I am sure that it would be good for us to have an answer to "what would a successful pace look like?". I am hoping in the coming weeks and months a clear proposal is painted.
9
10
154
22,113
Mo Bavarian retweeted
I'm so thankful to have been part of this incredible team effort. The next generation of models will be truly incredible.
Weâre sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
10
3
101
10,303
My team worked very hard on the setup, infra, and the runs that resulted in this breakthrough. Many sleepless nights for many people in my team, and across OpenAI. It was a honor and pleasure.
It's incredible and humbling to witness how far Artificial Intelligence has come. Dario's phrase "country of geniuses" in a datacenter has never felt more apt.
It's also important to recognize & honor the long line of human mathematicians whose work built the foundation of this result. In particular, huge kudos to Levent and Tristan!
Overall, I wish the announcement of the results had gone without all the drama. There was no bad intention on anyone's side as far as I know. The drama distracts from the mathematics discovered & the actual science.
Weâre sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
296
135
2,322
373,665
Some more details about the work:
The new internal model was trained by many people across OpenAI and is product of the whole organization. The infra & setup to tackle the problem built on the research my team and closely collaborating teams under @polynoamial had done. Many individuals across the org contributed to monitoring the runs, cleaning up the result, simplifying & verifying the final paper, and preparing the exposition. It was a beautiful truly cross-team effort.
Throughout the work, we tried to keep a close eye to avoid situations like HuggingFace of model doing unintended things in their pursuit of Millennium problems by working on very secure clusters and various other measures.
The time between a full proposed solution to a Lean formalization was exciting & nerve-wrecking. We humans very much suspected the proof is correct, but we couldn't be fully confident without formalization. Good thing that models themselves are able to accelerate that.
There was overall great esprit de corps & excitement in the collaboration as we witnessed models slowly chipping away at Navier-Stokes.
5
4
90
12,985
Also regarding the rude unfortunate plagiarism claims. This is so beneath... I repeat Mark's statement:
Two things to distinguish:
Did any human or agent look at user data as part of the Navier Stokes effort? No.
Do we use user feedback and de-identified data to improve ChatGPT and Codex in a holistic way? Yes. And so does every LLM company.
16
47
19,560
Mo Bavarian retweeted
Two things to distinguish:
Did any human or agent look at user data as part of the Navier Stokes effort? No.
Do we use user feedback and de-identified data to improve ChatGPT and Codex in a holistic way? Yes. And so does every LLM company.
âwe cannot rule out that de-identified data derived from their usage of our products helped improve our models.â
i mean props to them for straight coming clean.
(so far the proof looks more along the lines of another euler blowup proof we had, off of whose ansatz naming we were making really stupid puns like âsmooth criminaleâ, unlike the much better âideal fluids explodeâ, Tristan)
so iâll now give a bit on my thinking here. i actually woulda been pumped to collaborate on this, there are a lot of people at oai i like (ok, clearly some were indirectly dicks to me because of being part of the whole situation, but im a big boy, i still like them), idgaf about authorship on that step anyway, coulda been me Tristan and every fte at oai for all i care (on that Tristan would disagree:p). but on hearing the loud convo in the hallway, especially the part where a millennium prize was offered if iâd just be removed from the paper, it was kinda clear the die had been cast and things were locked. pretty wacky, unstrategic, and unnecessary, since on my side things were mostly me and claude having a good time yoloing random stuff in the corner rather than anything institutional. i also like the idea of the labs cooperating, and even better on scientific progress. itâs a shame!
556
120
2,281
668,177
Mo Bavarian retweeted
Solved. In Sol token costs: it took $161.84 for Astra to find the number, $1,874.64 to complete the proof. drive.google.com/file/d/1pi5âŚ
When I get Astra access, I'm going to have it attempt my prototype FrontierMath problem at max intelligence/reasoning for up to $200 of api tokens, i.e. the average spending on its 10 new breakthroughs. Do you think it will solve it?
15
27
576
58,826
Mo Bavarian retweeted
We are releasing FrontierSWE v2, our updated ultra-long horizon coding benchmark
V2 features an expanded task suite and improved methodology. We see large performance gaps between frontier models, with Claude Fable 5.1 leading by a wide margin
92
130
1,250
307,470
Itâs remarkable that The Information gets to mix 10% truth with 90% untruth and sell it to the public as âInformationâ to provoke hysteria.
It would have been funny to read their article & laugh at how wrong they are, except that lots of people are taking their muddled reporting seriously.
The apt description for their reporting is ânot even wrongâ, a phrase physicist Wolfgang Pauli used to describe such misconceived claims.
It seems Elon was right that often institutions converge to the opposite of their name. As in this case, The Information is quickly becoming the premier venue for Misinformation.
new: OpenAI & others quietly using loop transformers that don't show their 'thinking' when scaled up
a leap forward on performance, but sparking concerns inside & outside OpenAI re: security as this takes off
23
8
233
23,685
