P(doom) converges around 14%-19% among the more safety-minded AI researchers, ML engineers & other experts.
What’s interesting is that if you run the counterfactual. So, let’s play this stupid game, so Birdie can win his stupid prize.
Imagine the US halts all AI development tomorrow. Everything shuts down. Aside from the global depression that follows from millions of jobs vanishing overnight, trillions in contracts & debt obligations becoming worthless and the like (if you thought the Trade War was bad, this would make that look like an ill-timed joke compared to a World War), lets see how much higher that p(doom) goes.
For one: export controls that keep China in check are driven by two things: the US being the largest buyer of chips & compute, as well as the frontier development powerhouse. Well, if we suddenly exit, two things happen: (1) we lose that leverage, so the export controls go, too AND (2) supply remains constant while demand collapses - so China, the UAE & the Saudis get compute for pennies on the dollar.
For another: the *overwhelming majority* of the legitimate frontier safety apparatus is American. We exit, that goes away. Do you think China or Russia or the Saudis give a single damn about security or safety?
But it gets worse. Those millions of developers, researchers & engineers - the ones your ban put out of a job - they’re not going to just throw their hands up. No. Some of them (likely many ) will be recruited by our adversaries. They’ll take lucrative jobs in China or Qatar or India. And guess what? The ones most likely to go first (and be put in charge) are the ones who give precisely zero damns about safety.
Think that’s bad? Just wait.
What happens in 24-36 months, when the CIA gets credible intel that China is 18 months away from ASI? Do you think the US is going to sit back and let it happen? No.
BEST CASE: the US kicks off a Manhattan Project style initiative to get there first. No safety restrictions whatsoever. Absolutely no holds barred. Full send, no matter the cost or the risk. You end up in the same place, but you’ve taken an unpleasant detour through economic ruin & mass suffering to get there.
WORST CASE: the US realizes there’s no way to close the technological gap before ASI, and there’s zero diplomatic or economic ways to slow the HFP down, so the only remaining option is kinetic. The only way to do that while protecting American lives & minimizing American casualties would be an all-out first strike.
Congratulations.
If built, superintelligence would kill you. It would kill your family. It would kill everyone you've ever known or loved.
Anthropic and OpenAI are building superintelligence. That is their plan. They know that this is dangerous. They know that this could kill you.
These companies are not just reckless. They are not just engaged in a risky project.
They are a clear and present threat to your life. To your children's lives.
They need to be stopped.