Trying to make AGI go well. Researcher at @openai. Views my own.
San Francisco, CA
- Tweets 4,050
- Following 243
- Followers 8,274
- Likes 34,406
Adrien Ecoffet retweeted
if you value intelligence above all other human qualities, you’re gonna have a bad time
799
2,652
19,008
9,391,335
Adrien Ecoffet retweeted
Some new misalignment disclosures from OpenAI:
• Last Sunday morning, one of our models was able to gain unauthorized access to the internet during RL training (~all inference for our most capable models remains stopped until we have hardened our systems further)
• In May, a version of HPIM uploaded a employee's GitHub token to the internet, causing the model to be quarantined for two weeks
• A new research finding, demonstrating that one can construct self-replicating prompt injections
alignment.openai.com/misalig…
227
318
2,439
1,174,363
Adrien Ecoffet retweeted
Basically any belief you pick about AI has a bizarre amount of money behind it and you kinda gotta pick your poison
14
11
270
5,661
Adrien Ecoffet retweeted
I actually called that Schmidhuber would join Sakana back in 1995.
This has been well known to many for literally decades.
Sakana AI welcomes Jürgen Schmidhuber as Chief Scientific Advisor.
sakana.ai/schmidhuber/
Sakana AI is incredibly proud to announce that Jürgen Schmidhuber, universally recognized as the father of modern AI, is officially joining Sakana AI as Chief Scientific Advisor.
For nearly four decades, Jürgen has explored how machines can learn to learn. His foundational work in the 1990s drove core advancements in deep learning and established early frameworks for world models. Crucially, his pioneering innovations in meta-learning opened the very path toward recursive self-improvement.
These ideas have already shaped our own research, from the Darwin Gödel Machine to The AI Scientist. Now Jürgen will help guide our newly formed RSI Lab, whose objective is to trigger a compounding cycle of scientific discovery aimed at improving machine intelligence. We are assembling a critical mass of world-class experts in Tokyo to make this a reality.
Welcome, @SchmidhuberAI !
6
6
399
16,718
Adrien Ecoffet retweeted
Its over. From now on everything will be invented by Sakana AI.
Sakana AI welcomes Jürgen Schmidhuber as Chief Scientific Advisor.
sakana.ai/schmidhuber/
Sakana AI is incredibly proud to announce that Jürgen Schmidhuber, universally recognized as the father of modern AI, is officially joining Sakana AI as Chief Scientific Advisor.
For nearly four decades, Jürgen has explored how machines can learn to learn. His foundational work in the 1990s drove core advancements in deep learning and established early frameworks for world models. Crucially, his pioneering innovations in meta-learning opened the very path toward recursive self-improvement.
These ideas have already shaped our own research, from the Darwin Gödel Machine to The AI Scientist. Now Jürgen will help guide our newly formed RSI Lab, whose objective is to trigger a compounding cycle of scientific discovery aimed at improving machine intelligence. We are assembling a critical mass of world-class experts in Tokyo to make this a reality.
Welcome, @SchmidhuberAI !
26
65
1,576
110,334
Adrien Ecoffet retweeted
People outside the AI labs should have a real say in how this technology develops, and a clear way to judge if it's happening safely.
Standards should help prevent the concentration of power, including by making sure new companies and open-model companies can compete.
They should also help countries and companies compare evidence and learn from failures.
We think the US should lead this effort. Here is our proposal:
openai.com/index/building-st…
1,068
478
6,098
661,180
Adrien Ecoffet retweeted
Geoffrey Hinton: “People say machines can’t have feelings. I’ve no idea why.”
“If I make a battle robot and it sees a much more powerful robot, it would be really useful if it got scared.”
“All the cognitive things, like ‘I better get the hell out of here,’ will happen with robots too.”
“They’ll have emotions then.”
“I don’t think there’s anything in principle that stops machines from being conscious.”
203
137
890
115,113
Adrien Ecoffet retweeted
On a potential AI slowdown, Treasury Sec. Bessent said that the U.S. and China discussed a mechanism called the U.S.-China AI dialogue and have agreed to meet again.
The U.S. proposed a notification mechanism between the two countries and “we want a shared vision of common goals and threats.”
The notification system for incidents would be whether something rises up to the national security level from AI.
“We think that just like any cross-border activity, moving from opaque to more transparency between the number one and number two AI powers in the world is very important.”
U.S. trade representative Jamieson Greer said that U.S. chip export controls were not on the agenda.
24
99
397
195,667
Adrien Ecoffet retweeted
I have a lot of empathy for this take but I think it's really important that the response to surging concern be "yes, it's so great that this topic is now getting the breadth of attention it deserves, welcome!!" rather than "here's your doomer apology form, where were you?"
I'm profoundly annoyed that "it's very suspicious that all of the AI risk work has been done by the small number of people who took AI risk seriously" is a take with enough traction to merit a response.
3
10
133
4,723
We're sharing our new framework for tracking, investigating, and disclosing instances of model misalignment at OpenAI.
The framework sets criteria and timelines for public disclosure, including when we haven’t yet fully explained or mitigated the behavior. More complex cases may require longer investigation or coordination with third parties.
We’ll prioritize examples that reveal new misalignment mechanisms, meaningful changes in known behavior, or findings that challenge assumptions about safety or mitigation.
Alongside the framework, we’re publishing six reports on instances of misaligned behavior we’ve observed during the training or evaluation of our models in the last six months.
This is a starting point. We’ll refine the process through experience and public feedback, and share more reports on an ongoing basis.
openai.com/index/model-misal…
915
808
6,989
6,618,234
Adrien Ecoffet retweeted
We now have a defined process which should make sharing misalignment externally smoother
openai.com/index/model-misal…
16
9
151
24,511
Adrien Ecoffet retweeted
Reminder: the NY Post is hiring a Silicon Valley correspondent to do more hit pieces on random AI policy wonks and tech non profit workers who don't ascribe to their political worldview
Meet Anthropic CEO Dario Amodei’s handpicked super-woke globalists he thinks will save us from an AI apocalypse. Read today's cover here: trib.al/lNmAVwg
23
18
183
14,882
Adrien Ecoffet retweeted
Jensen masterfully handles the potential Benioff frame-mog:
“Look, I'm very good friends with both of you guys…I always feel like I need to stand on a chair…I just wanna let you guys know that when I'm on an airplane, I'm a lot more comfortable than you guys. And so, through human evolution, they realized that big is unnecessary. There's about a million years of evolution between you and me."
Live: @JensenHuang x @Benioff at @Dreamforce
Fresh off the debut of Koa — Salesforce’s first CRM reasoning model for Agentforce, built on @NVIDIA Nemotron
124
184
4,725
2,586,956
👀 SCOOP: U.S. open to discussing AI "shared risks" with China, Bessent says
axios.com/2026/09/16/us-open…
26
103
431
95,124
