The Machine That Cannot Know
Most of what people call “AI intelligence” is a grammatical parlour trick.
The system is fluent. It is often persuasive. It can sound like it “knows.” But it doesn’t. It produces syntax—tokens arranged into statistically likely sequences—without semantics—without understanding, reference, or accountability to reality.
That distinction isn’t philosophy-club trivia. It is now an institutional fault line.
The essay argues that large language models are the modern Chinese Room: fast, huge, and still empty of comprehension. They can generate legal memos, medical summaries, policy briefs, and “research” that looks correct. But they cannot verify truth, because they have no access to truth—only to patterns in training data. That is why “hallucinations” aren’t a bug; they’re a predictable outcome of a system that optimises plausible text, not grounded knowledge.
And that brings us straight to Hayek.
Hayek’s knowledge problem was never “we need more computing.” It was always: the relevant knowledge for coordination is dispersed, tacit, contextual, and often not articulable—knowledge of time and place, of lived practice, of embodied judgement. The essay connects this directly to what AI systems cannot do: they cannot acquire tacit knowledge, they struggle with edge cases and thin data, and they cannot generate the kind of abductive leaps humans use when the map is incomplete.
So the question is not: “Can AI write?” It can.
The question is: “Can AI know?” It cannot. Not in the way institutions require when decisions have consequences.
That’s the pivot: if AI-generated content floods the information environment, the real problem becomes provenance—traceability, integrity, auditability. Not prophecy. Not vibes. Not “trust the model.”
If you care about law, science, markets, governance, or journalism, you should care about this: a world of syntactically perfect text without semantic grounding is a world that demands verification infrastructure—systems that can show where claims came from and what process checked them.
Essay: “The Machine That Cannot Know: Why AI Will Never Solve the Knowledge Problem.”
open.substack.com/pub/singul…