LLM Reliability: Hallucination, Consistency, and Reasoning Statistics

Last updated: July 2026

Large language models are increasingly trusted to produce factual answers, run calculations, and generate work that people ship. How reliable are they, really? Below is what the peer-reviewed research and the recognized benchmarks actually measure, across hallucination, run-to-run consistency, numerical reasoning, and output diversity, with the primary source for every figure.

How often do LLMs hallucinate

1.8% to 3.3% hallucination rate for the best models, on grounded summarization. On Vectara's Hallucination Leaderboard (snapshot of 11 May 2026, scored by the HHEM-2.3 evaluation model), the top models introduced unsupported information in 1.8% to 3.3% of summaries. Critical caveat: this measures factual consistency with a provided source document (the summarization/RAG setting), not the truthfulness of a model's general knowledge. It is a best case, and the figures move as the leaderboard is re-run. (Vectara Hallucination Leaderboard)

38.2% of short factual questions answered correctly by GPT-4o, when it has no source to lean on. On OpenAI's SimpleQA benchmark of 4,326 short fact-seeking questions, GPT-4o scored 38.2% correct, o1-preview 42.7%, Claude-3.5-Sonnet 28.9%. Every frontier model tested scored below 50%. This is open-book-less factuality, the opposite setting to grounded summarization above. (Wei et al., OpenAI, 2024, "Measuring short-form factuality in large language models")

Systematic overconfidence. Models state more certainty than their accuracy warrants. The same SimpleQA study found that "models consistently overstate their confidence," concluding there is "a lot of room to improve the calibration of large language models in terms of stated confidence." A model that sounds sure is not, on that basis, more likely to be right. (Wei et al., OpenAI, 2024)

Hallucination in professional, high-stakes settings

58% to 88% hallucination rate for general-purpose models on legal questions. Stanford RegLab tested models against more than 800,000 verifiable legal questions and found hallucination rates of 58% for GPT-4, 69% for GPT-3.5, and 88% for Llama 2 (2023-era models). (Dahl, Magesh, Suzgun & Ho, Stanford, 2024, "Large Legal Fictions", Journal of Legal Analysis)

17% to 33% hallucination rate for commercial "hallucination-free" legal RAG tools. Testing purpose-built retrieval-augmented tools on 202 expert-graded legal queries, Stanford researchers measured 17% for Lexis+ AI and 33% for Westlaw AI-Assisted Research, and concluded providers' hallucination-free claims are "overstated." Grounding a model in verified content reduces hallucination; it does not eliminate it. (Magesh, Suzgun, Dahl, Ho et al., Stanford, 2024, "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools")

The same prompt does not give the same answer

Up to 15% accuracy variation across identical runs, with a best-to-worst gap up to ~70%, even at temperature 0. Testing five models over eight reasoning tasks with ten runs each under "deterministic" settings, researchers found accuracy varied by up to 15 percentage points across naturally occurring runs, with the gap between a model's best and worst possible performance reaching up to 70 points. Raw exact-match agreement across runs was often below 50%. A single prompt is not a single answer. (Atil et al., 2024, "Non-Determinism of 'Deterministic' LLM Settings")

LLMs degrade on numerical and multi-step reasoning

Up to 65% drop in reasoning accuracy when one irrelevant clause is added to a math problem. Apple's GSM-Symbolic benchmark added a plausible-but-inconsequential sentence ("NoOp") to grade-school math problems. Accuracy fell up to 65%: GPT-4o dropped from 94.9% to 63.1%, and smaller models fell far more (Phi-3-mini 80.7% to 18.0%). The models were not robustly reasoning; they were pattern-matching. (Mirzadeh, Alizadeh, Bengio, Farajtabar et al., Apple, 2024, "GSM-Symbolic")

12% to 15% best-to-worst accuracy spread when only the numbers in a question change. In the same study, generating 50 versions of a question with different numeric values (structure unchanged) moved accuracy by more than 12 points on some models. Swapping the digits is enough to shift the result. (Mirzadeh et al., Apple, 2024)

Outputs converge (mode collapse)

11 words appear in 88.3% of generated stories, across four different models. Sampling 20,000 stories from four current models with five prompts, researchers found 11 words (recurring names like Elias and Elara, settings like lighthouses, occupations like clockmaker) appeared in 88.3% of outputs, with little difference between models. Left to their defaults, models regress to a narrow set of patterns; the diversity has to be supplied from outside. (Hamilton & Mimno, 2026, "Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories")

Why this happens

Evaluations reward guessing. Hallucination is incentivized, not incidental. OpenAI researchers argue that "standard training and evaluation procedures reward guessing over acknowledging uncertainty," so a model that confidently guesses scores better than one that admits it does not know. The behavior is a predictable product of how models are graded, not a bug that a bigger model removes. (Kalai, Nachum, Vempala & Zhang, OpenAI, 2025, "Why Language Models Hallucinate")

The measurement itself is unsettled. Older truthfulness benchmarks lost adoption. Stanford HAI's 2025 AI Index notes that earlier benchmarks like TruthfulQA and HaluEval "have failed to gain widespread adoption," with the field shifting to newer evaluations (HHEM, FACTS, SimpleQA). The report does not publish a single headline hallucination rate; it points to the third-party benchmarks above. (Stanford HAI, 2025 AI Index Report, Responsible AI)

Sources

Changelog

2026-07: Initial version. Ten figures across hallucination, professional-setting reliability, run-to-run consistency, numerical reasoning, and mode collapse. The isolated "0.7% Gemini" hallucination figure was excluded (tied to an older HHEM version, only meaningful with its version and date); a single universal "LLMs hallucinate X% of the time" figure was excluded (no primary source, entirely task-dependent); blog aggregators were not used as sources.

LLM Reliability: Hallucination, Consistency, and Reasoning Statistics (2026)