Doomerism.
What does it even mean?
And why does almost every major technology eventually create its own version of it?
➪ Fair warning:
This is probably the longest piece I’ve written on here in a while.
But there’s a TL;DR at the end if you want a cheat.
Anyways;
I’ve seen this word get thrown around so much in AI now that basically anyone worried about where this might go can get called a doomer.
I don’t think that’s particularly useful.
Because thinking AI could create a serious problem doesn’t mean you expect that problem to happen.
There’s a massive difference between saying something is possible, assigning some probability to it, and becoming convinced it is inevitable.
A lot of the AI conversation these days keep collapsing those things into one.
And tbh, I can see why.
Because the Ai race is ridiculously fast.
➥ Capabilities change.
➥ Timelines change.
➥ Researchers start to disagree.
➥ The labs building the systems disagree.
And we’re all lowkey just trying to think about the consequences while the technology itself changes underneath us.
That kind of uncertainty gives doom a lot of room to grow.
Some of it is reasonable.
Some of it is people filling unknowns with their worst possible outcome.
Some of it is probably just wrong.
Still, I don’t think doom itself is useless.
I actually think an industry this powerful needs people constantly asking how things could go horribly wrong.
There’s an old example I think about when I see people talk about Ai doom:
In the 1970s, scientists working on recombinant DNA became concerned that some experiments could create biological risks they didn’t properly understand.
Researchers paused some experiments, met at Asilomar in 1975, then developed safety and containment rules before continuing the research.
Some of the feared risks later turned out to have been exaggerated.
But the precautions still helped shape safer research practices.
And I think that distinction matters here.
A warning can overestimate the eventual danger of a thing and yet still produce something useful.
➪ ➪ Now bring all of that to AI.
Back in 2023, over 2k researchers who had published at major AI venues were surveyed about where advanced AI could eventually lead.
Most were optimistic overall.
68% thought good outcomes from superhuman AI were more likely than bad ones.
Yet depending on how the question was phrased, around 38% to 51% still assigned at least a 10% probability to outcomes as bad as human extinction.
Even among the optimists, almost half still gave at least a 5% chance to extremely bad outcomes.
Which I think is such a weird result.
Because it does not mean science has calculated a 10% chance that AI kills humanity.
There is no dataset that can give us that number.
These are subjective forecasts made under ridiculous amounts of uncertainty.
But I also wouldn’t throw them away.
The fact that catastrophic AI risk has gone from a fringe thought experiment to something a meaningful share of people working in AI assign non-zero probability to is worth paying attention to.
It doesn’t prove the doomers right.
It just makes the question harder to dismiss.
And jobs are probably the cleanest example of why I think both sides need to chill on the certainty.
Jacob Coxon, ex-Anthropic whose AI-doom-post went viral (143 million impressions after 24 hours) said on Fox News:
"We know how to control nuclear weapons... We don't yet know how to control AI."
and
"possibly the most dangerous technology that humanity has ever created."
That's one of the most dooming statements I've heard in a long time. And as a former Anthropic employee, it's obviously a huge burden for their IPO.
As I said yesterday: without good and valid arguments for the fear-mongering, I cannot take it seriously.