I think recently there is a glaring disconnect between the apocalyptic calls to "slow down" and the empirical reality of software engineering and in general an overvaluation of AI skills.
Strictly speaking, frontier models are not "intelligent" until proven otherwise. They are high-density statistical compute engines executing massive matrix multiplications to interpolate known human distributions. What we have built is an unprecedented supercomputing retrieval-and-synthesis layer, not a reasoning entity.
The absence of an actual causal world model becomes undeniable the moment you move past basic boilerplate:
Failure on long-horizon planning: As repos scale in depth, context drift sets in, regression loops trigger, and latent edge-case hallucinations compound.
Zero architectural intuition: A model can optimize a localized function, but it cannot evaluate holistic system trade-offs (e.g., distributed state consistency vs. latency, memory layout vs. cache coherency).
Supercomputer scale\neq cognitive agency: Scaling compute compresses and organizes knowledge with incredible throughput, but interpolating tokens is fundamentally distinct from deterministic deduction.
Frontier models, including Claude, are elite syntax accelerators and scaffolding tools. But framing advanced pattern-matching supercomputers as autonomous minds on the brink of escaping human control leans far closer to safety theater and regulatory capture than technical reality.
Until machines demonstrate true causal deduction rather than high-probability token prediction, the human engineer remains the baseline, the architect, and the only real safeguard.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here:
darioamodei.com/post/we-must…