AI risk should be taken seriously. So should the risks of responding badly to it. We need credible scenarios, effective safeguards and incentives that make responsible behaviour sustainable.
A possible catastrophe is not yet an established threat. Specify the mechanism, prerequisites, timescale and evidence. Explain what would have to go wrong, what could interrupt the process and what would change the assessment. An extinction percentage without those foundations should not be treated as a measured fact but more a measure of personal fear.
Intellectual superiority is not unlimited power. A takeover scenario must explain access, resources, persistence, physical consequences and the failure of resistance. It cannot skip from “smarter than a human” to “able to defeat an adapting human–AI civilisation”. Nor should a safety argument assume that civilisation is perfectly coordinated.
Do not give creativity exclusively to the adversary. The lesson from David Deutsch is that new explanations can create new possibilities. A forecast that gives threatening AI unprecedented ingenuity while freezing human–AI defenders at today's capabilities is an incomplete comparison. Optimism must face the same scrutiny: future solutions cannot simply be assumed.
The relevant system is a human–AI ecology. People, models, institutions, tools and infrastructure can change one another's capabilities. A working hypothesis is that organisations which preserve and develop genuinely complementary sources of learning gain an important adaptive advantage. That is a mechanism to investigate, not a guarantee that diversity always wins.
AI advances across an entire stack. Energy, chips, infrastructure, models and applications all contribute to what can be accomplished. Restricting new model training does not freeze improvements elsewhere. A model pause may matter, but its effect must be demonstrated for the capability at issue.
Connections can matter as much as components. Give an unchanged model different memories, experiments, collaborators, tools or permissions and the assembled system may acquire new capabilities. Some connections improve understanding; others enlarge authority. Evaluate the whole arrangement, including feedback and timing—not just the model name.
“Slow down” needs an object. Slow which training, experiment, release or deployment? For how long? Under what restart conditions? Alignment research, independent inquiry and defensive work should not be swept into an undefined freeze. A narrow restriction on a dangerous operation is different from a restriction on discovery.
A desired agreement is not automatically an available option. International restraint needs consequential participation, credible verification and responses to noncompliance. Signatures are not sufficient. But imperfect international coverage is not a reason to postpone immediate containment of a demonstrated local hazard.
Unilateral restraint can redistribute risk. A policy that constrains accountable organisations but leaves comparable activity elsewhere untouched could strengthen less accountable competitors and delay defences. That is a possibility to assess, not an automatic argument against restrictions. Ask what is prevented, displaced and lost.
Independent invention will not stop at a laboratory boundary. A researcher could discover a more efficient architecture or a better way to combine existing systems. That does not make controls irrelevant. It means limits on particular organisations or inputs cannot be presented as permanent limits on knowledge or capability.
Delay has both humanitarian and safety costs. Slower progress can postpone useful treatments, education, accessibility and scientific discovery. It can also postpone better alignment, monitoring and defence. Those benefits are uncertain, just as many projected harms are uncertain. Neither side should be assigned zero by assumption.
The pause comparison needs accuracy. OpenAI described a two-week pause in specified training and later reported restarting its large frontier run on 28 August; Anthropic also reported targeted pauses. These announcements do not establish identical coverage or motives. Ask whether the responses address the same hazard—not which company makes the strongest declaration.
The stronger signal would be explicit reciprocity. Many would have welcomed Anthropic and the other major labs saying: “We will not use your safety interruption to advance the same inadequately protected activity. We will pause comparable work, or show why our protections address the identified problem.”
Adopt “pause or demonstrate equivalent protection”. When credible evidence identifies a serious shared hazard, affected labs should interrupt comparable exposed work unless independent assessment supports continuing under effective safeguards. Publish the scope, evidence and restart conditions. Different systems may justify different actions; vague assurances do not establish that difference.
A safety pause should redirect effort, not merely stop it. State which people, compute and operating capacity are moving into alignment, evaluation, monitoring and containment. The useful commitment is not just “we delayed the next model”, but “we used this interval to build and test the protections it needs”.
Publish comparable safety-compute accounts. Report actual safety-research compute as a share of research compute, and operating safeguards separately as a share of the inference they protect. Give absolute expenditure, methods and coverage as well as percentages. State how resources will increase when evidence requires it; do not let a shrinking denominator masquerade as greater investment.
Measure results, not just spending. High spending is not proof of safety, and a valuable engineering fix need not consume much compute. Publish what the investment discovered, repaired and independently verified, including missed failures, unnecessary interruptions and unresolved gaps. Mixed capability-and-safety work should be identified rather than counted twice.
Reward making safeguards usable elsewhere. Give credit and funding for independently reproduced protocols, test environments, monitoring tools and maintained repairs. Reward adoption and continued effectiveness, not simply publication. Share deployable defences openly where appropriate; handle exploitable details through coordinated disclosure rather than indiscriminate release.
Make international safety cooperation a regular working practice. Invite the major labs—including Chinese labs—alongside independent evaluators and relevant research institutes to recurring technical meetings and joint exercises. Review failures, compare protections, assign follow-up work and publish non-sensitive results. Build shared responsibility through verifiable work, not presumed geopolitical trust.
Evaluate the harness, not just the model. A harness is the software arrangement giving a model tools, memory, task feedback and opportunities to act. Test its boundaries throughout research and deployment, including shared services and peer communication. Connecting a model changes the capabilities of the assembled system; that is not a measurement of how rapidly its underlying intelligence has increased.
Build graduated containment with an emergency stop. In bounded tests, vary access, autonomy, persistence and coordination to find which restrictions prevent harm while preserving useful work. During an active dangerous incident, contain external effects promptly; conduct the diagnostic experiments inside isolation. Do not keep a live attack running merely to observe its decline.
Do not confuse reduced performance with repaired behaviour. A system can stop completing harmful actions because a channel is blocked while still attempting them. Measure both, alongside useful performance and what happens after controlled reconnection. Cutting a source of independent checking—or reducing reasoning indiscriminately—might make the remaining system less reliable.
Give capable systems tested operating controls. Operators should be able to constrain permitted targets, external actions, parallel workers, run duration and resource use without discarding all useful intelligence. Enforce consequential boundaries outside the model's own discretion, preserve independent monitoring and test how controls interact. The aim is to operate powerful systems within evidenced limits—not merely hope they behave, or leave them permanently unused.
Make finding and repairing problems worthwhile. Fund independent evaluations, vulnerability prizes, replication and repair through a shared industry contribution. Protect honest disclosure. A regime that rewards silence and commercially punishes discovery risks selecting for organisations that look safer rather than those that become safer.
Keep the rule narrow and challengeable. No lab should gain a veto over competitors merely by announcing concern. Safety coordination should concern evidenced hazards, not prices, customers or market allocation. Preserve independent research, review restrictions and measure the costs imposed by safeguards themselves.
The objective is neither maximum speed nor minimum activity. It is a more capable human–AI world with funded safeguards, shared learning and tested operating controls—without making the first organisation to act responsibly the one that pays alone.