Hallucination

What Is an AI Hallucination

A hallucination is output from a large language model that is factually incorrect, fabricated, or unsupported by the input it was given, but that is presented with the same fluency and confidence as accurate output. Hallucinations can range from small factual errors, such as an incorrect date, to entirely invented citations, code that calls functions which do not exist, or confidently stated claims that have no basis in reality.

Why It Happens

A language model generates text by predicting the most statistically likely next token given its training and the current context, not by checking claims against a verified knowledge base. When a model lacks reliable information about a topic, or when a prompt is ambiguous or leads it outside its training distribution, it can still produce a fluent, plausible-sounding answer because fluency and factual accuracy are not the same thing to the underlying prediction process. Hallucination risk tends to increase with longer, open-ended generation, with questions about obscure or recent topics, and with tasks that require precise recall, such as citing sources or quoting exact figures.

Why It Matters

Hallucinations are a central limitation to account for whenever LLM output feeds into a decision, a published document, or an automated system. In agent systems that take actions, such as running commands or writing code, an unnoticed hallucination can propagate into a broken build, an incorrect report, or an invalid data write, especially when the agent operates with minimal human review.

Mitigation Strategies

• Retrieval-augmented generation: grounding responses in retrieved documents rather than relying solely on the model's internal knowledge.
• Lower temperature: reducing randomness for tasks where accuracy matters more than variety.
• Output contracts: requiring structured, schema-validated output that is easier to check programmatically than free text.
• Human or automated review: verifying critical claims, code, or actions before they are acted upon, particularly for irreversible operations.
• Citations and tool use: asking the model to reference specific sources or use tools that return verifiable data instead of recalling facts from memory.

No single technique eliminates hallucination entirely, so systems that rely on LLM output for consequential decisions typically combine several of these approaches rather than trusting the model's output at face value.