i don't think nearly anyone, myself included, has truly internalized there'll soon be a machine better than us in every single intellectual & physical capacity deep down we all share a feeling that our 'entrepreneurship' or 'taste' or... is special & safe. it isn't. it's over

you are going to fail, so fail while daring greatly

human cultures are not able to metabolize this amount of change this quickly. when you zoom back out to a big picture of history, it’s remarkable how glacially slow everything is. the first industrial revolutions took 80 years (very fast by the standards of previous revolutions), and the writings of Marx happen at the very end as working class politics arises. it takes about another 150 years after that for the great ideological struggles of post-industrial times to be completed in the 90s in what fukuyama called the “end of history” today you see a compression of decades of technological change into a matter of months or years. there’s no wonder yann lecun (the respected inventor of the ConvNet and ex-chief scientist of Meta) wonder whether the hugging face incident was staged, and mathematicians whose minds are far more powerful than my own are nonetheless coping about training on their data or something while models begin to display superhuman mathematical capability it would take a century or more to fully metabolize and legitimize the level of changes that have happened even in the past 20 years, to say less of the remarkable scientific revolution we will get over the next five should we avoid the doom of misalignment. I do not suspect any current form of science academia, or even political order will survive without great transformation, except through the course of slow evolution over decades - and of course, technology will keep moving too

Nothing can shut down open source

Doom is easy for our tiny human brains to imagine. We evolved in a world of scarcity. Survival of the fittest. The paranoid live to see another day. It’s why movies like the Terminator go down in history, and every girl in America loves true crime. It’s so easy to capture people’s undivided attention with that type of content. It is much more difficult to imagine abundance. AI doing incredible things like saving endless lives with quick medical advances and unfathomable efficiency gains. God loves positive thinking.

Jacob Coxon's next interview on CBS News (ex Anthropic+Open AI researcher who resigned) "We can't just unplug it because it could be copying itself over to other computers. Like it's not that difficult to find yourself because an AI is just code. It could transfer itself over the internet to a different place and then you unplug it here, but it's actually still over there and maybe it makes 10,000 copies of itself and they're all cooperating." ---- From "CBS News" YouTube channel, (full video link in comment)

they named a phone after me and made it bend over 😭

A line from The Odyssey that hits hard: “What you most want is what you most can’t have. And what you most can’t have is what you already had, and lost.”

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.