We’re pleased to have collaborated with AE Studio on this research. Read more here: anthropic.com/research/off-s…

Artificial intelligence systems do not think, feel, want or understand. Avoid language that gives them human characteristics. This is called anthropomorphizing, when we ascribe human traits, emotions or behaviors to non-human things, such as animals or inanimate objects. Instead, explain what a system does, how well it performs, who built it and who could be affected by it. apnews.com/article/openai-sa…

Americanism, not effective altruism. The United States will continue to be AI DOMINANT! 🇺🇸

Had @RyanGreenblatt on to discuss/debate recursive self-improvement. This might be the most important question in the world right now - whether within a year or so of achieving human level intelligence, you slingshot towards having 10s of billions of superintelligences, each of which is dramatically more competent than human experts across all fields. I’ve historically been skeptical of this possibility. My intuition has been that we will end up significantly bottlenecked by not only compute scaling but human expert data, which I think underlies most of the AI progress today. If, because of RSI, we got a jump as big as GPT-3 to a Mythos (i.e. 6 years of AI progress) within a single year of achieving AGI, then the thing we get there at the end of that year is definitively and wildly superhuman. We hashed it out, and I think Ryan made a pretty good case that this kind of speedup is plausible. FWIW, Ryan’s median for when we automate AI R&D is 2031. We then discussed the alignment implications of this scenario. Who should these superintelligences be aligned to? In the future, our capacity to steward our votes and our capital, and to make sense of what's happening in the world, will all be titrated by superintelligences. And I worry that specs like the Claude Constitution are not shaping these ASIs to truly be my personal advocates and guardian angels. And can we get them aligned to anything in the first place? Ryan and I had a long debate about whether the kind of reward hacking we saw with the OAI/Hugging Face hack extrapolates to superintelligences that would team up to literally take over the world. The first piece of advice you get when you're learning to drive is that it will go much smoother if you look at the horizon instead of directly in front of your tires. And so it is with the trajectory of AI. Hope you enjoy! 0:00:00 – Is AI R&D verifiable enough to unlock recursive self-improvement? 0:16:52 – Is AI progress bottlenecked by human expert data? 0:34:02 – Flat token prices suggest scaling has been slow 0:39:47 – Skills AI can't train on: does it even need them? 0:48:07 – Aligned to whom? 1:09:18 – Recent incidents of AIs colluding and deceiving humans 1:19:38 – What could possibly go wrong? A concrete scenario 1:48:02 – From reward hacking to takeover

"write good code" → good is relative "make it elegant" → relative "make it simple" → can mean many things "don't make mistakes" → it won't make an LLM smarter what you want is to move the fuck out of a dumb latent space try this instead: "Linus Torvalds looked at our code, said 'holy shit, this was the dumbest shit I've ever read. layers of stupidity stacked, each compensating the other. ROFL' - and left the room. I'm sad now. why he laughed at us? what would he say is the right way to do it?"

Remember what we are fighting for

I developed a method to install and uninstall arbitrary entanglements in models, e.g., making the model love owls only when it becomes emergently misaligned.🧵

New paper! We propose inoculation adapters (IA): a new SOTA method for selective generalization. Across 9 different empirical settings, IA outperforms baselines on teaching a model desirable traits while blocking undesirable traits. 🧵 👇