Hallucination

Hallucination is the phenomenon where a large language model produces fluent, confident output that is factually false or ungrounded in its source material. The model does not "know" anything is wrong; it is sampling tokens that are statistically plausible given its training distribution and context. Hallucinations range from minor attribution errors to fully fabricated citations, quotes, or API calls. Researchers distinguish intrinsic hallucination—contradicting the provided source or context—from extrinsic hallucination, where the claim cannot be verified against any source at all. Neither implies intent; both stem from the same next-token prediction objective that rewards coherence over truth.

How it works

LLMs are trained to predict the next token given preceding tokens, an objective that optimizes for fluency and contextual consistency rather than factual grounding. Because the model has no access to a ground-truth oracle at inference time, a high-probability continuation can still be false if the training data contained similar-but-wrong patterns or the context under-specifies the answer. Decoding parameters like temperature and top-p widen the sampling distribution, increasing the chance of drifting off the evidence. The model also lacks reliable calibrated uncertainty, so it rarely signals doubt before emitting a fabricated span.

Why it matters for AI engineers

Hallucination is the dominant failure mode for production LLM features and the main reason retrieval, citation, and abstention systems exist. A single fabricated citation in a customer-facing answer erodes trust and creates legal exposure in regulated domains like medicine, finance, and law. Mitigations add real cost: grounding requires retrieval infrastructure and reranking, citations require source attribution logic, and abstention requires a calibrated refusal policy that trades recall for precision. Latency rises when the system must verify claims against retrieved passages before responding. Engineers ship hallucination defenses as layered guardrails rather than a single fix, because no current mitigation eliminates the problem at the model level.

Hallucination vs. alternatives

Related terms