New: from 404 Media. The catastrophic FBI hack also exposed the FBI's own hacking unit. The Remote Operations Unit is a highly secretive part of the FBI, responsible for making tools to break into peoples' devices. Some of their names are in the data 404media.co/fbi-hack-exposed…

ceo of anthropic dario amodei publishes letter calling for an AI development slowdown

JUST IN: Yemen's Houthis used Anthropic's Claude AI to build ballistic missiles.

EXCLUSIVE: Sam Altman met with top power utilities to discuss ways to secure the electrical grid. He also offered one possible solution to the cybersecurity risks: OpenAI’s own cyber services. politico.com/news/2026/09/10…

I'm very worried about changes to AI architectures that result in AIs thinking in opaque activations instead of in chain of thought (aka "neuralese" architectures). Based on limited public evidence, it seems like Astra was a concerning step in this direction. Unfortunately, there was insufficient public information to have a well-informed public scientific discussion about whether the changes to architectures and training methods that went into Astra were a good trade-off between monitorability and performance. We also don't know how AI companies will make these trade-offs going forward. I think companies should release the evidence needed for a reasonably informed public conversation about how these trade-offs should be made. They should also make and publish their policies around this. We've written up a proposal for how this could work. I don't know if this proposal will be sufficient to avoid the most concerning architectures, but it seems like a relatively robust step in the right direction. AI companies should be very cautious about pursuing architectures that could eliminate or greatly reduce dependence on chain of thought, and certainly shouldn't do this before the rest of the world has a chance to discuss the evidence and their policies.

[Writing this in a personal capacity, not on behalf of my employer (Anthropic).] Jacob’s thread is very worth reading. Here’s my birds-eye view of the situation with risks from AI: 1. AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are. 2. Why do AI developers continue despite the risk? Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely. 3. Unlike traditional software, we can’t “program” AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this. 4. We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them. Insofar as there is a plan, it’s to make sure that AIs are good enough at alignment training that they can align their successors better than we can align current AIs. 5. Many AI developer staff desperately want to slow down to figure out how to build AI more safely. That was the intent of this open letter (which I signed): pacingthefrontier.com/ I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.