Anyone who has read METR's Technical Report on the OpenAI/Hugging Face incident will understand that software-only controls will not be sufficient to contain superhuman AI.
Any continuing work on adversarial superhuman intelligence must rely on the physical containment methods used for nuclear security: including air gap isolation, data diodes, and a two-person rule for data exfiltration.
Moreover, the internet itself must be hardened against penetration by self-propagating AI agents: no more untraceable access to high-end AI computing facilities, and no anonymous fund aggregation or payments.
The first useful step is our Chip Security Act, currently included in the Senate NDAA.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.