A hallucination is output from a language model that reads as fluent and confident but is false or not supported by any source, for example an invented court case, a fabricated citation, or a wrong dosage stated as fact. The term covers both claims that contradict the provided context and claims that are simply untrue about the world.
Types
- Intrinsic: the output contradicts the source it was given, such as a summary that reverses a finding. This is a faithfulness failure.
- Extrinsic: the output adds content that cannot be verified from the source, which may or may not be true.
- Factual errors in open-ended answers: no source was given, and the model states something false from its parameters.
Why models hallucinate
A language model is trained to produce likely continuations, not to track what it knows. When the training data contains a fact rarely or not at all, the model can still produce a plausible-looking answer. Training and evaluation that reward a confident guess over "I don't know" make this worse: if abstaining scores zero and guessing sometimes scores one, guessing is the score-maximising policy. Decoding at high temperature and long generations add further chances to drift.
Measuring hallucination
The usual unit is the atomic claim. Split answers into claims, verify each against sources or a knowledge base, and report the share of unsupported claims and the share of answers containing at least one. For example, if 200 answers contain 1,600 claims and 40 are unsupported, the claim-level rate is 2.5%; if those 40 fall in 22 answers, the answer-level rate is 11%. Both numbers matter: the second is what a user experiences.
Mitigations
- Retrieval with citation and a faithfulness check before display.
- Training and grading that give credit for calibrated abstention, measured with calibration error.
- Self-consistency: sample several answers and flag claims that do not recur.
- Human review for high-stakes outputs.
Why it matters
Hallucinations are the failure users notice most, and in regulated settings a single fabricated figure or citation can cause real harm. Because rates depend heavily on domain and prompt, a general benchmark number tells you little; measure it on your own data in the eval suite and keep measuring in online evaluation.