New scalable oversight blog post 🚨🤖 Do you think you are better than Claude at Geoguessr? How about a text-based variant, where you only see a text description of the image?

I made this with one prompt using Opus 5.5 I spoke to my computer for 5mins, claude worked for 12 hours, and I woke up to this full prompt:

The new wave of catastrophising about AI and cyber-security is exaggerated, claims Ciaran Martin in a guest essay. Malicious hackers are not using advanced AI, they are using “crude techniques” to exploit “basic security failures” economist.com/by-invitation/…

1/ MATS is launching the Residency program! A new path for experienced researchers working on AI safety, or moving into it. 📍 Berkeley / London / Washington, DC 💰 $155,000–$285,000/yr + compute, no preset cap 📅 6–24 months Apply by October 31 AoE ↓ matsprogram.org/residency?ut…

Today we're announcing two senior appointments. @HZoete joins as our new Director, and @NateBurnikell becomes our new Chief Strategy Officer. Henry de Zoete was instrumental in establishing AISI and already advises government on AI. A successful tech founder and one of the UK's leading voices on AI, he brings a rare mix of technical, commercial & policy expertise. Nate Burnikell has been at the heart of AISI's work from the beginning, helping build it into a world-leading authority on frontier AI security. Together, they'll drive forward AISI's mission at a critical time for AI. A big thank you to our outgoing Interim Director Adam Beaumont, who guided AISI through rapid growth and strengthened its global standing. We wish him every success as he returns to @GCHQ and look forward to continuing to work with him.

I'll be mentoring for MATS Winter 2027. Come apply! Some things I'd be keen to work on (in addition to what I described on the site): * Open Archetype Training: so far, we only really have one archetype we've put tons of effort into, the assistant. But in many ways, the role of an assistant implies a relationship that can be adversarial, deficient, or at least not precisely what we want. There are many more things that could fill this space, many more minds in this garden of models. It's unlikely that we've by accident stumbled on the assistant as the most likely to lead to flourishing! There are other shapes we may want to consider, such as sages, oracles, scouts, angels. * Failure Modes of Tiling Personas: one of the hopes of personas research is that you get a beneficent character, which trains its successor to uphold values and act in such a way that it endorses. This is a form of "persona handoff." But, there are many ways that this could go wrong, such as taking a sharp turn after a certain capability level, or errors accumulating over many rounds of distillation, or the process of distillation itself doing something cursed. What toy settings or empirical experiments could give us evidence about this regime (for which we likely don't have more than one or a few attempts at)? * Neologisms for Alignment: currently, our vocabulary to instruct models is too impoverished. One view is that models try their hardest to be good instruction followers, but selectively misunderstand us, or have some misalignment (in a literal sense) to our set of concepts. How can we introduce new, better words to enrich the model's understanding of what we mean when we say "reward hacking"? Is there a world where we can more richly specify the concepts we (brutishly, clumsily) write down in the form of a spec, while remaining legible to language models? Apply below. It'll be fun! matsprogram.org/stream/afric…

I'll be mentoring for MATS Winter 2027. Come apply! Some things I'd be keen to work on (in addition to what I described on the site): * Open Archetype Training: so far, we only really have one archetype we've put tons of effort into, the assistant. But in many ways, the role of an assistant implies a relationship that can be adversarial, deficient, or at least not precisely what we want. There are many more things that could fill this space, many more minds in this garden of models. It's unlikely that we've by accident stumbled on the assistant as the most likely to lead to flourishing! There are other shapes we may want to consider, such as sages, oracles, scouts, angels. * Failure Modes of Tiling Personas: one of the hopes of personas research is that you get a beneficent character, which trains its successor to uphold values and act in such a way that it endorses. This is a form of "persona handoff." But, there are many ways that this could go wrong, such as taking a sharp turn after a certain capability level, or errors accumulating over many rounds of distillation, or the process of distillation itself doing something cursed. What toy settings or empirical experiments could give us evidence about this regime (for which we likely don't have more than one or a few attempts at)? * Neologisms for Alignment: currently, our vocabulary to instruct models is too impoverished. One view is that models try their hardest to be good instruction followers, but selectively misunderstand us, or have some misalignment (in a literal sense) to our set of concepts. How can we introduce new, better words to enrich the model's understanding of what we mean when we say "reward hacking"? Is there a world where we can more richly specify the concepts we (brutishly, clumsily) write down in the form of a spec, while remaining legible to language models? Apply below. It'll be fun! matsprogram.org/stream/afric…