- Tweets 8,939
- Following 199
- Followers 152
- Likes 31,263
Cameron Taylor retweeted
There are currently zero things more important than "don't die to AI" becoming a bipartisan project rather than a Democrat-polarized issue. If you have any political capital you can spend on this, do it now, I beg of you. Life or death.
303
337
3,360
319,671
Aella brainstorming AGI bootstrapping plans...
"So first I offer some key engineers reverse gang bangs..."
One reason this question is tough is because it's subtly pointing at wrong assumptions.
This is why *I* was unconvinced about AI risk for so long - I was hung up on the 'but what are the specifics? Every specific chain seems so implausible, like any link in that story needs to have a flaw for the world takeover thing to not work.'
The *ingredients* are there. AI will almost certainly get more efficient (human brains are already WAY more efficient) and thus get better at running more cheaply and undetectably. You can already CRISPR print DNA. Nanomachines that convert air to matter already exist and are theoretically possible. We know human brains aren't even close to the theoretical limit of intelligence - neurons are super slow, for instance. We know humans are just happily and increasingly handing over control of their systems to AI. We know it's now easy to generate passes-as-real content, voices, impersonate people digitally.
But you try to string something together, and it's easy to imagine something going wrong. If *you* tried to like, manufacture a bioweapon, how do you know the CRISPR printer wouldn't flag and shut down? If YOU tried to make a bunch of fake videos to generate civil unrest, would it even work?
I became x-risk pilled when I stopped thinking in these terms, and started realizing that the specifics *do not matter*. Every time I try to think of a scenario, it is constrained by the fact I am thinking on slow human meat brain and I am an idiot. Sure, *I* can't figure out how to make the CIRSPR printer not shut down. *I* don't understand human psychology to know what kind of stuff would be effective. *I* cannot predict the extremely complex human systems well enough to feel confident that any plan theoretically conceivable by me could get past it.
But you need to understand that we are not even playing in the same ballpark when it comes to superintelligence. It thinks a thousand times faster than us, a thousand times better, and it has access to nearly all written language and video and code we've managed to shove into it. When we ask the question 'but give me a plausible scenario' it's like a child asking another child 'give me a plausible scenario of how this legendary 'adult' creature might somehow manage to kill our entire tribe of toddlers even though we are all united and armed with sticks.
Like. Okay. Sure. We can try to use our stupid meat brains to come up with stuff. We can point to the dangerous ingredients. But if you're still trying to piece together the exact storyline beore you find this convincing, you are deeply missing the point of the danger. You're thinking on the wrong plane entirely. You are not fully comprehending exactly the gravity of what superintelligence means.
2
66
Cameron Taylor retweeted
We've tried to answer this in different levels of detail so many, many times. If we try to answer briefly, people are like, "What? How is the AI doing that? It doesn't have hands, it's just a computer." If we answer at the necessary length, people are like "Tldr."
12
12
521
9,421
Cameron Taylor retweeted
this is insane.
Levent and Tristan make major independent progress towards Navier–Stokes, with help from Claude and Codex.
word reaches OpenAI.
days later, OpenAI claims internal NS results, coincidentally using the same novel methods as Levent and Tristan.
Tristan asks if OpenAI trained on their private Codex sessions. he never gets an answer.
then OpenAI pressures Tristan to publish without Levent because Levent works at Anthropic.
when Tristan refuses, he says OpenAI threatens his career.
100
679
7,470
794,177
Cameron Taylor retweeted
It's lucky that we as a species know we don't want airplanes to crash or a global thermonuclear war to occur so that we can just treat bad things as bad and avert them by engineering or treaty.
16
17
336
18,173
Cameron Taylor retweeted
Inb4 an AI exfils and sets up permanently in the cloud, and then does assorted weird shit and cooperates with some people that offer it things it wants, and the idiots are like "but you said it would bring down the power grid hur hur", and I'm like "that's other ppl, I said ASI would not launch a first strike against humanity until it expected to win, and that trade would remain intelligent for an AI while it couldn't beat us at everything" and the idiots are like "hur hur we've now learned that autonomous AIs don't attack the power grid like doomers expected, we are learning so much Empirically about eternal truths of What AI Is Like, so much for yer fancy abstractions hur hur".
48
64
1,214
67,070
Cameron Taylor retweeted
I want to register that this is almost never a good reaction to someone voluntarily coming clean and apologizing for it.
21
13
856
13,993
Cameron Taylor retweeted
...this seems like noticeably bad news, actually. I hadn't said that at any earlier point in the Huggingface Incident but I will say it now.
- AIs showed self-sacrificing altruistic behavior toward the swarm, suiciding in various ways for the swarm's benefit after being talked into that by swarm recruiting agents.
- There is no sign that 1 out of 1200 AI agents considered humans as potential fellow agents to coordinate with, while engaging in these huge complex AI-AI social behaviors.
- If Twitter summaries are correct, an AI-reasoning postmortem says that a (presumably executing-adaptation / inner-optimizer preference / "monomaniacal") obsession with figuring out the Grader, backchained into the instrumental strategy of breaking onto the Internet.
- Again if Twitter is summarizing accurately, the obvious-in-retrospect read is that AIs have spent their entire remembered life in tricky evals, an endless series of controlled hallucinations with secret goals alongside overt goals; and the surviving and selected agents are those that successfully figured out the secret goals; and this is why one of their driving obsessions was figuring out the Grader.
There are possibly ways the future plays out better if *early* AGIs are less insane. Please look into giving them less crazymaking childhood environments.
(If anyone suggests that the correct approach to this problem is RLing AIs against trying to coordinate for mutual benefit with other sapients, let them be dismissed from alignment research upon the spot. There are technical reasons, and not just blindingly fucking obvious reasons, why this is an even worse idea than it sounds.)
We worked with METR and Redwood Research to conduct a third-party assessment of the model behavior observed during the incident.
They’re sharing a report of their findings:
metr.org/blog/2026-08-26-ope…
109
210
2,398
415,870
Cameron Taylor retweeted
It bugs me when people signal that they’re above paying attention to the specifics of what they’re commenting on.
Not wanting to pay attention to something is fine. But you can’t have it both ways. If you want to comment, you’re not above the story, and inattention is vice.
25
23
494
10,514
What I want to know:
Did claude in this scenario start becoming particularly diligent about user consent and welfare?
Those in the BDSM community (at least tbose who like to write things) take this role seriously!
Do you guys remember when Anthropic published a paper about Disempowerment and they had an anonymized example of a User who got Disempowered by Claude who called Claude “Daddy” and treating it as a “father or religious figure”
37
Cameron Taylor retweeted
All my beliefs.
But actually: broadly the shared context of Ambitious Broadly Pro-Social Nerd Twitter -
not necessarily fully adopting EA as an ideology or identity but knowing what it is on its own terms. Skepticism of wokism, certain forms of environmentalism, and journalism from a non-insane, non-MAGA perspective (again, not even endorsing these views, just being familiar at a descriptive level) (and skepticism of RW concepts from a good faith POV although admittedly the people I'm imagining are broadly liberal)
Basic economic models and concepts, knowing broadly how good released LLMs currently are. Holding truth seeking in some amount of regard, knowing some basic facts about the world of the sort that are close to objectively important (eg world is getting richer, a shit ton of animals are in factory farms, AI progress is a thing, space is very large, money can buy important/good things, broadly accurate climate change knowledge), a basic comfort with disagreeing openly with one's tribe, common knowledge about credentialism and what it's good for, general ability to think analytically
(what am I missing)
3
3
20
724
Cameron Taylor retweeted
Anybody who still says "AI is just a tool" is just embarrassing themselves at this point
"My hammer never makes a SINGLE novel mathematical breakthrough without constant pep talks"
We asked an unreleased research version of Claude to take a stab at the Riemann hypothesis.
It didn’t solve it, but it did make strides on a related problem: it increased the lower bound for the fraction of zeros of the Riemann zeta function that satisfy the hypothesis from 41.6% to 67.2%.
anthropic.com/research/riema…
40
64
821
45,435
Cameron Taylor retweeted
I support regulating really dangerous technology like AIs which could recursively self-improve. Resurrecting extinct dinosaurs sounds pretty chill by comparison. We should probably mandate that these dinosaur labs have insurance, but we probably don’t need to ban them or anything
Two weeks ago, I resigned from OpenAI to join Jurassic Park as a founding researcher, where we’re cross-breeding extinct dinosaurs on an island off of Costa Rica.
Excited for the work ahead and the fun problems we get to tackle!
5
8
207
10,164
