AI Truth...
A language model is a machine that has read an enormous amount of human writing & learned which words tend to follow which other words. When you ask it something, it does not look out at the world. It produces the kind of answer that, based on all that reading, usually comes next. The model has no eyes, no instruments & no private line to the facts. It only knows what people have already written down.
On the other hand, people write a lot of true things, especially when being wrong has a cost. Engineers describe bridges that have to stand. Doctors describe treatments that have to work. Scientists describe measurements that other scientists will check. Those careful descriptions show up again & again in the writing the model learned from. So the “most typical continuation” is often the accurate one, not because the model checked reality, but because reality already shaped what people wrote.
Asking AI a question is like asking someone who has read every cookbook ever printed how to boil an egg. They have never boiled an egg themselves. They are repeating the pattern of what cooks wrote. Most of the time you still get a boiled egg, because cooks were describing a real process. Sometimes you get a confused or outdated answer, because some of the books were wrong, joking, or arguing with each other.
The model is not limited to copying old sentences. It can stitch pieces together into an answer nobody wrote before. That new answer can be right, or it can be smooth nonsense.
Newer systems can do more than guess from old writing. They can look something up, run a calculation, or compare an answer against a source. That is still not the same as knowing the world directly. It is an extra check bolted on afterward.
So the simple version is this: the model does not know what is true. It knows what people tend to say. When people were careful & reality was pushing back on them, what they tend to say is often true. When they were guessing, arguing, or performing, the model follows that instead & it cannot always tell the difference.
This means most of today’s AI is a very good reader, not a witness.
AI has no direct contact with the world. It learned by absorbing what people wrote, said, coded & argued. So its answers are shaped by that record. Where the record is thick & reality kept correcting the writers such as medicine with outcome data, engineering with things that break, arithmetic, well-tested code , the AI is often right, sometimes remarkably so. Where the record is thin, disputed, fashionable, or full of confident guesses, the AI still answers in the same smooth voice. It is following the pattern of how people talk, not checking whether the talk was true.
A model of how humans describe the world is not a model of the world. AI can recombine descriptions into something new & sometimes that new thing is right. It can also recombine them into something nobody meant & stated with the confidence of a textbook. Without a way to touch the thing being described, it cannot tell those apart on its own.