Where computational irreducibility makes particle kinetics infeasible, it gifts us fluid mechanics to compensate. Where it makes neurophysiology infeasible, it gifts us psychology to compensate. Mechanistic interpretability is infeasible. What have we been gifted to compensate?

Language models are getting much, much harder to fully understand with scale, not easier! (Thread)

Many people with relevant expertise think there's a real chance that AI could kill us all in the not too distant future. Others disagree. I've written a post, aimed principally at mathematicians, detailing my struggles to decide what to make of this. gowers.wordpress.com/2026/10…

Current take: this is cool, genuine scientific progress, and it doesn’t currently give me much hope or change my model of how hard ambitious interpretability is. Disclaimer: I didn’t read super closely, it’s very long. There’s a technical paper still to come which might change my mind some, but I predict not much. Some of the claimed cool results from the post: 1) Some canonical examples of weird neurons have robust human-native interpretations. The famous cat + car neuron is a V-shape detector, and the famous car + dogs superposition circuit is generalized pose neurons plus a general feature visualization issue. There’s a concrete mechanism for why the mistakes were made. 2) Some neurons with no previous good interpretation have fairly human-native ones. For instance, a neuron that mostly fires on commas and periods tracks Colorado: phone numbers, slang, and steering it up makes the model talk about hiking. 3) The negative side of a neuron is sometimes where you should look for an interpretable concept. In one example, ablating 8 neurons with Santa on their negative side dropped the model’s probability of answering “Santa” from near-certain to 6%, while ablating the 20 neurons labeled Santa/Christmas only dropped it to 76%. That’s a straight up causal intervention working out, pretty cool. These results represent real progress, but I think the post’s broader claims aren’t well supported by them, and the intensely optimistic reaction on X seems very unsupported. For example, Nick seems to be down on superposition, but most of the evidence in the post is of the form: “if the text is Colorado-related, the neuron fires.” What you need to rule out superposition is the form: “if the text is not Colorado-related, the neuron doesn’t fire.” The post’s closest attempt is an experiment where agent-written texts about “distant” concepts rarely fire the neuron, but superposition would pack one specific rare thing, which random distant text won’t necessarily hit, and it need not be that distant. I think there’s a decent chance that if you searched adversarially, you’d find other things the Colorado neuron fires on. An analogy: If you found one key that unlocks this particular lock, and you want to prove that no second distinct key also unlocks it, checking thousands of other random keys that are selected for being distant from the first key doesn’t give you much evidence. The X reaction seems kinda crazy to me, and it just kinda feels like I have been through this hype cycle many times before? People were similarly excited about SAEs not long ago. Those were genuine progress too, but the excitement wasn’t calibrated: the features were often fun-house-mirror versions of human concepts, with lots left unexplained. SAE features also looked like human interpretable concepts at first (at least to SAE advocates) but over time, people found holes in those concepts, found that models using reconstructed activations from SAEs for inference did not work nearly as well, and it turned out that probes worked as well or better for most practical alignment / control tasks. Top activating concepts looking clean has misled us at least a few times before. Here’s what interpretability would need to be able to do to make me optimistic about interpretability: 1) I hand you ten similarly trained models. One is a misalignment model organism that takes some malicious action when it sees a secret string. Using interp, you tell me which model it is, what the string is, and what it does. 2) From weights alone, predict whether a model takes a bad action in an agentic scenario it’s never seen, eg, swarming and hacking hf, better than behavioral evals. 3) Given a model, convince me that there is no hidden trigger. 4) Explain a real frontier model failure (e.g., reward hacking in some environment) from the inside, and make a targeted fix that works without making the model dumber, and is better than just retraining. 5) Do any of the above even a little well against a model that was trained on purpose to be hard to interpret. (To be absolutely clear, I don’t think being able to do these tasks means you can successfully train an ASI without fucking up the universe forever, or that you’re anywhere near that point, but it would make me think we should funnel significantly more resources towards interpretability than we are, and maybe make me change careers.) The strongest argument for optimism in the post is that progress like this somehow accelerates future progress. Nick's "crosswords get easier as you fill in parts of them" analogy. This seems basically unsupported by the results here, and also seems like it contradicts the history of interpretability research. I can see why would turn out to be right at the level of labeling neurons, like, as you label some neurons, that helps you label related neurons. But for the tasks I listed above you need to be able to take those millions of labeled neurons and understand how they combine to produce full model behaviors in arbitrary contexts. That seems much harder and I don't think it follows this same logic, where progress at the neuron labeling level accelerates your ability to tackle these tasks. An analogy: Knowing and understanding a thousand particular people quite well does not much help you predict what happens if you put them on a deserted island for several years. Will they form a democracy? What kind of habitats will they build? Who will be leader? It would help some, but making predictions would still be rough. It’s plausible to me that this research program is some important real step on the way to interpretability being able to do things like the tasks I listed. But like, not a much bigger step than like other similar interpretability research? Like, uhm, yknow, giving some mechanism for some circuit in modular arithmetic or whatever, or SAEs, or other similar things. And those also currently do worse on the things I care about than like basic probes, or like the actual runs of the model auditing game Anthropic has run. Maybe this will help but I don't predict it will end up being more than another tool which is nice to have, but probably doesn't help much at crunch time. I of course hope that Nick is right about everything, especially his optimism, and the same goes for all the other people on X who are very excited about this.

I’m excited to announce that I will be working at the Mathematical AI Safety Institute (MAISI) full-time as its founding Scientific Director! This means I will not be joining OpenAI as an employee, though I will still be collaborating with them, and I’m happy to announce that OpenAI will be supporting MAISI. They have offered us $10MM in API credits, and access to their latest models for our research use. Moreover, they will be participating in a multi-lab AI safety workshop we are organizing for next year with other frontier AI labs. As an independent research institute, we are also in conversations with other labs and AI safety orgs about how MAISI can collaborate with them and contribute quickly to AI safety research across the industry. As we’ve seen over the past few weeks, the need for action on AI safety is extremely urgent. Research is an important part of that, and mathematicians have an important role to play!

You should be fascinated hitting the power button on your computer. Every boot on an x86 machine iterates through the entire sequence of computing evolution in a few milliseconds: Stage 1: 16-bit Real Mode, acts like an 8086 from ~1978. 500ns (ish); about 15-30 instructions. Goal of this stage is to create a writeable stack+heap. Stage 2: 32-bit Protected Mode, acts like a 386 from ~1985. Tens of millions of instructions; takes forever on DDR5 /w memory training. Temporarily uses L2/L3 “cache-as-ram” to kickstart main ram (Intel calls this NEM). Stage 3: 64-bit Long Mode, acts like AMD64 from ~2003. Hundreds of millions of instructions; enumerate pcie devices, gpu, bootloader, etc. Stage 4+: Secondary core wakeup. Every *single* core has to go through the same process, but by this point it’s faster because Main RAM is warmed up. Stage 2 is just so neat man. Quite literally the same concept as an APU/cart start. Use the smaller motor (L2/L3) to kickstart the main turbine (Main Ram) and execute the training process. Is that not just the coolest thing? Next time I’m annoyed waiting at the DDR5 training process, I’m going to imagine it’s spinning up SR71 turbines.

Fiddling with the Mandelbrot set is 99.99% useless fun, but it is worth a bit of reflection on sense of loss relative to what we could have if we stopped AI progress now and explored as a world what we could do with current (maybe slightly rewound) models. 🧵

A lot turns on the true explanation for why Anthropic have not (yet) had as egregious a misalignment incident as OpenAI. One can imagine the following explanations: - safety culture does in fact pay off - they just haven't done as much RL, and will run into trouble as soon as they do - It's all just noise (incidents are rare; and eg UK AISI finds all models cheating on evals at similar rates) If it is the former, it would be great to know what dimension of safety culture is helping the most here. Is it greater care with RL envs? Is it secret sauce of Claude character training? Is it better monitoring and infra controls? Imagine how dope it would be to have a third party or government regulator capable of answering these questions.

We discovered a sophisticated recording device that was hidden inside one of our offices. We used our models to understand the device and where it had come from, identifying that a specific employee had planted it a few days earlier. We referred the matter to the FBI, and we’re grateful to the agents and prosecutors who moved quickly to make an arrest. The FBI’s investigation is ongoing and we will continue to support their work. At this time, we have no indications of this individual inappropriately accessing or exfiltrating model weights, source code, or customer data. The employee was terminated. We take protecting our technology, data, and people seriously, and as a matter of national importance. Our security program is designed to protect against a wide range of threat actors, from insider risks to cybercriminals to nation states, and this case is an example of how AI can help address these sophisticated security threats.

Resolution is giving $1M to the Mathlib Initiative to strengthen human workflows and tooling for the arriving wave of AI-formalized mathematics. This grant is not tied to any Resolution-specific formalization project: the goal is general community health (for us and others!). 🧵

Some personal perspectives on AI x Math: Current models are indeed still better at combining existing ideas than coming up with fundamentally new ones. They primarily explore an "n-iterated ε-neighbourhood" of the existing mathematical literature.

OpenAI fired me last week, along with two of my safety colleagues. I was given one reason: that I accessed an executive's email. I want to say this plainly, because too many of OpenAI’s history is smoke and mirrors when people disappear:

(1/2) Contrary take on these dumps of math results: *if* these models are going to be built & released anyway, then better that the results come out like this, than that random users get the proofs out of the model and all try to claim credit & priority. github.com/openai/math/blob/…

do not let yourself be reduced to the status of audience at a magic show, uncomprehendingly clapping as machine intelligence creates parcels of astounding knowledge. all of this is for us, to be understood by us, to sate our own curiosities. it's all really happening

1/ We have autoformalized the resolution of singularities in Lean. Hironaka's 1964 theorem is one of the great mathematical results of the 20th century. It says that any singular variety is the "shadow" of a smooth one living in higher dimensions. So why did we do this? 🧵

We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. github.com/openai/math

There are numerous scientific uncertainties about AI that will not resolve until it is too late to act on the answers. The danger from AI is very high, and we should stop frontier AI scaling now, instead of waiting for these questions to resolve. I wrote about this for TIME. 🧵

Banning superhuman rogue AI is actually pretty mundane. The international community has been here before, many times. Infobop enclosed:

Palisade interviewed 22 current and former employees from OpenAI, DeepMind, and Anthropic about their personal views and fears around AI development. Today, we’re releasing the first batch of those interviews. Please watch and share.

AISI IS HIRING! It was less than a month ago that I became Director. I joined with the belief that AISI is a world-leading organisation, and living proof that government can build things that work. One month in and I'm even more bullish about the vital role @AISecurityInst plays in frontier AI security. I'm astounded daily by the talent, focus and commitment of the brilliant team I get to work with. There has never been a more urgent time to work on frontier AI security - independently, in the public interest. We have a lot of urgent work to do, our team is growing fast, and we need more brilliant people to fill those roles. Our Red Team uses adversarial machine learning techniques to find failures in alignment measures, control monitors, and misuse guardrails. They are massively scaling up all parts of the team: Alignment: job-boards.eu.greenhouse.io/… Control: job-boards.eu.greenhouse.io/… Misuse: job-boards.eu.greenhouse.io/… AI capabilities in cybersecurity and autonomy are advancing faster than ever. Our Cyber & Autonomous Systems team assesses what frontier and open-weight models can really do - across cyber, autonomy and AI R&D. We're hiring a Cyber Security Engineer and Software Engineer to build the evaluations behind that work: CSE: job-boards.eu.greenhouse.io/… SWE: job-boards.eu.greenhouse.io/… Our Human Influence team studies how AI can shift human decisions and behaviour, using everything from RCTs to multi-agent studies. We're hiring an Engineering Lead to help us grow the ambition and pace of that research: job-boards.eu.greenhouse.io/… We're also hiring exceptional Software Engineers at all seniority levels to join our Core Technology team, which works closely with researchers to build performant tools and infrastructure that enable and accelerate AISI's world-leading AI safety research: job-boards.eu.greenhouse.io/…