When a language model states something false with complete confidence, the instinct is to treat it as a defect awaiting a patch. The more useful framing, well supported by recent research, is that hallucination is the predictable output of a system trained and graded the way these systems are. It is not mysterious, and that is precisely why it is hard to eliminate.
The pretraining argument
The influential treatment here is a 2025 paper by Kalai, Nachum, Vempala and Zhang, Why Language Models Hallucinate, which makes the case formally.
The core move is to reduce generation to classification. Producing a valid statement is strictly harder than judging whether a given statement is valid — so if a model would make errors distinguishing true from false statements, it must make errors generating them. A lower bound on hallucination follows from the error rate on the easier task. Generation cannot be more reliable than discrimination.
The second argument is the genuinely counterintuitive one, and concerns calibration. Standard pretraining with cross-entropy loss produces models whose output probabilities roughly match real-world frequencies — which is a desirable property. But a calibrated model assigning ninety per cent to the right answer and ten per cent spread across wrong ones will emit a wrong answer roughly one time in ten. As the authors put it, calibration — and hence errors — is a natural consequence of the standard objective. A model that never hallucinated would have to be systematically overconfident in a way that breaks calibration.
And some facts are irreducible. For arbitrary information with no learnable pattern — a specific private individual’s birthday, say — there is nothing in the structure of language to generalise from. The model can only guess.
Why it persists: the grading problem
Pretraining explains why hallucination arises. The more actionable finding explains why it survives post-training, and it is not about architecture at all.
Virtually every benchmark used to evaluate models grades in binary: right or wrong. Under binary grading, saying “I don’t know” scores exactly the same as a wrong answer — zero. A guess with any chance of being correct has positive expected value. Abstention is therefore strictly suboptimal, and models are optimised against these benchmarks.
We are, in effect, training models to bluff, because bluffing scores better than honesty on the tests we use. The paper’s framing is explicitly socio-technical: adding a dedicated hallucination benchmark cannot outweigh the dozens of mainstream leaderboards rewarding confident guessing. The proposed remedy is not a new eval but rescoring the existing ones — penalise confident errors more than abstentions, and give credit for calibrated uncertainty.
Reasoning models hallucinate more
This surprises people who expect more deliberation to mean more accuracy. The evidence across model families points the other way on grounded tasks: enabling extended reasoning commonly raises hallucination rates on document summarisation, in reported cases by a factor of two to three relative to the same model’s non-reasoning mode. The precedent was visible in vendor system cards for early reasoning models, which showed higher rates on factual-recall evaluations than their predecessors.
The plausible mechanism: a model reasoning at length about a source document generates many intermediate statements, each an opportunity to drift from what the document says, and later steps condition on earlier ones. Verification against the model’s own reasoning is not verification. See reasoning models explained.
Reading hallucination statistics carefully
Published rates vary enormously and are frequently compared when they should not be. Two distinct things get measured under one word.
Grounding failure asks whether a summary stays faithful to a supplied document. Leading models typically land in the single digits to low teens on these. Open-domain factual error asks whether the model knows something unaided, and rates are far higher — evaluations designed to reward both knowing and abstaining put every frontier model well under two-thirds accuracy.
A model described as having a “five per cent hallucination rate” is almost certainly being described on a grounding benchmark. That number says nothing about its reliability on unaided factual questions. Treat any cross-benchmark comparison as invalid unless the task is identical.
What actually helps
- Retrieval and grounding. The highest-leverage single intervention — supply the facts rather than relying on recall. It reduces rather than eliminates the problem, and a bad retrieval makes things worse than no retrieval. See RAG explained.
- Detect-then-rewrite. A second pass that identifies unsupported spans and regenerates them. Measurably effective, and cheap relative to human review.
- External verification. Checking claims against a source works. Sampling several answers and comparing them catches inconsistency but not confident consistent error.
- Abstention training and rescored evaluation. The only intervention addressing the root incentive rather than patching output. Early, and not yet reflected in most public leaderboards.
- Citations — with a caveat. Citations make claims checkable by a human; they do not make them correct. A citation produced without an actually retrieved document can itself be fabricated. Treat citations as a verifiability feature, not a reliability one.
The honest conclusion
Survey literature converges on the same answer: no single technique eliminates hallucination, and combinations of complementary methods are the best available approach. Hallucination is a managed risk, not a solved problem.
That has a direct design implication. Any system where a confident falsehood carries real cost needs verification in the architecture — not a hope that the next model will be better. The mechanism that produces hallucination is the same mechanism that makes these models useful, and the research consensus is that it is reducible rather than removable.
Last reviewed: September 2026.
