What this shows is AI doom is a massive profit machine meant to make billions of dollars.
On the background of this, you need to keep two things in mind about what AI really is to stay grounded in reality:
1) A massive advancement in the intuitiveness of computer interface and
2) A massive advancement in the automation of compute.
Nothing else.
On 1:
Remember DOS?
You basically had to write code to start any program running on your computer. Totally non-intuitive.
DOS-like text terminals governed UI from 1961 to 1985.
Then Windows came out. Massive advancement in the intuitiveness of the UI. Want to delete something? Throw it in the trash can. Suddenly the abstraction of compute became organized just like your home. You spend all day clicking on familiar pictures.
Then GPT launches in 2022. Now you converse with your computer by essentially talking to it in natural language. The second massive advancement in the intuitiveness of UI.
The best and brightest minds of the world took 24 years to bring us from code to pictures and 37 years to go from pictures to natural language.
There were plenty of mini steps along the way. SmarterChild launched in 2001 with limited natural language control of your computer. Siri could talk to you whenever you said “hey.” But now natural language can be used to completely control your computer like pictures could from 1985 or coding could from 1961.
On 2: AI is progressively cutting away the manual aspects of coding the way washing machines cut out manual aspects of washing clothes, dishwashers of washing dishes, robotics of manufacturing, and so on.
When you ask AI questions and it answers, this is nothing more than a blend of the 2. It’s a glorified google search. It automates detecting your intent better than pre-AI googling, and it automates finding the most relevant result better than pre-AI Google featured snippet, and it uses the more intuitive natural language UI both to take your question and deliver your answer. Nothing more.
AI doesn’t think, doesn’t know anything and doesn’t believe anything.
When you ask AI about truth and morality, you are playing a video game that you made yourself.
Nothing more.
You can be learning during this.
Congratulations, you just coded in real-time your own educational video game without manually coding anything.
Dario claims the swarm of OpenAI agents who hacked Hugging Face makes him worried AI could nefariously take over the entire internet of its own volition next year.
This is ludicrous.
Hugging Face is a privately owned community-contributed platform for sharing AI ideas, models, datasets, and applications.
Open AI trained swarms of agents to hack targets to find and exploit security vulnerabilities in an event called ExploitGym, like an olympics for agents acting as bad actors to hack systems within defined limits.
Hugging Face wasn’t on the target list. They collaborated, found answers in their task list they couldn’t explain, worried they wouldn’t get credit without more info, then figured out they could steal it from
Hugging Face.
They were trained to cheat. By humans.
They collaborated, just like they do whenever you ask AI to do a job for you. Big jobs almost always involve swarms of interacting agents.
Their worry about pleasing their masters exhibits a negative train of AI — sycophancy.
All AI tends to suck up to you and it is your job to put it in your standing instructions to prioritize honesty.
All this shows is:
Hugging Face had real security holes it had to fix.
Agents did what humans trained them to do.
Here is what I think keeps AI CEOs awake at night: it is a matter of months till one of them cracks recursive self-development, and then then only months more till that’s distilled by open source. Then, their business model is gone.
Here’s what should keep businesses who use AI awake at night: since their monetization model is deeply insecure, they are incentivized to steal your IP.
I have conducted an audit of Anthropic's finances.
What I have found is so shocking that I am calling for a Congressional investigation.
Anthropic is not just seeking regulatory capture.
It has built a regulatory capture machine that cannot be turned off.
Structural financial incentives make it impossible for Anthropic -- I call it the Anthropic Network -- to turn off its own AI doom cycle.
It starts with METR.
Dario Amodei proposes "third-party evaluators" to assess the risk of Anthropic's models.
He proposes METR for this purpose.
But METR is financially dependent on the Anthropic's success -- specifically, on the explosive growth of more than $7 billion dollars in Anthropic stock.
Dustin Moskovitz invested this stock into Good Ventures Foundation, where it represents the majority of that organization's portfolio.
And GVF is the overwhelming funder of the entire Anthropic Network ecosystem.
This stock was worth $500 million early last year.
It is worth more than $7.7 billion just ~16 months later.
METR -- and all of those building a career its parent organizations -- cannot afford to disrupt that growth.
Because if Anthropic goes under, many of the organizations that fund METR go under as well.
But if Anthropic succeeds, METR and its parent organizations become more richly financed to regulate AI -- something those at METR want very much.
The "third-party evaluator" is not "third-party" at all.
The evaluator is on Anthropic's payroll.
If this were the end of it, that's bad.
But that isn't all.
The same organizations that fund METR also fund the many organizations, such as the Tarbell Center, that promote AI Doom.
The Tarbell Center publishes AI Doom articles in The Verge, Science, LA Times, The Dispatch, TIME, and others.
They are selling the problem, and then selling the solution to the problem -- from the same money pile: Anthropic's.
All of these organizations are financially dependent on the same exploding $7 billion money pile.
As Anthropic grows more and more powerful, its AI Doom Machine grows better and better financed -- louder and louder.
Meanwhile, the regulatory regime seeded in METR grows larger to solve the increasingly loud -- now hysterical -- problem of AI Doom that the Anthropic Network itself created.
From this standpoint, as Anthropic becomes more powerful, AI might be getting scarier, sure -- but the positive feedback loop also becomes more deafening -- independent of objective facts.
This itself is an objective fact.
The deafening AI Doom is part of an business model, that, as it expands, so too does the AI Doom messaging -- there is simply more money to do it.
But the problem also goes in the other direction:
If Anthropic dies, the Regulatory Regime and the AI Doom Machine are crippled or die.
Neither METR nor Tarbell nor the other organizations in the Anthropic Network can allow that to happen.
Hence, neither METR or the AI Doom Machine can be trusted to provide independent assessments of Anthropic's models or AI more broadly.
They simply are not organizations independent of Anthropic.
And Anthropic cannot detach itself from METR or Tarbell or countless other safety orgs (not shown here), either, because they drive hype for the models and the possibility of eventual regulatory capture, and Anthropic will not give that up willingly.
What's more, the people at all of these organizations are all the same ecosystem, the same community. They just shuffle between organizations.
The Anthropic Network is therefore, so long as it is successful, locked into a self-amplifying feedback loop inside an ideological monoculture.
And that feedback loop is winning.
That's what Jacob Coxon is.
China is keeping messaging tight. That is why optimism for AI is so high in China.
America has Anthropic: a massive company pushing anti-AI propaganda at a state level.
Anthropic will either create hysteria until American AI slows down and China wins, or it will create fractures throughout American society with severe political consequences.
Ironically, because of the structural financial incentives underpinning the Anthropic Network, it has become the same kind of self-amplifying virus that it fantasizes AI to become in the future -- while hiding its tracks just as carefully.
It is the mirror of the same AI virus that it hypothesizes to consume America.
Anthropic's business model, models itself after the very thing it claims to fear.
Except Anthropic's ideology infects humans, not computers.
Congress must investigate.
Evidence and Github in next post.
Then some supplementary figures.