So, will hallucinations ever go away? No, not entirely, and that's a proof, not a guess. The monofact-rate bound from page 6 doesn't shrink with scale, because a fact seen once in training stays seen once no matter how large the model gets.[6] What does shrink is narrower, and it's the part actually moving: how often a model has to face that gap unaided. Retrieval, verification, and training a model to abstain instead of guess all lower how often the floor gets reached, without erasing it.[46] That's the honest version of the question, not whether hallucination stops happening, but whether the system around it stops being surprised when it does. It's the same shift that turned packet loss and memory errors into routine, engineered-around non-events.
What actually reduces it
Given that hallucinations are never going to go away, what needs to happen to reduce how often they do? Split it into the three moments where a model can be shaped: while it first learns from raw text (pre-training), while it's fine-tuned afterward to be more helpful and accurate (post-training), and in the moment it's actually answering your question (inference).
Pre-training
Building the base model
Post-training
Fine-tuning it afterward
- Reward honest uncertainty over confident wrong guesses.[46]
- Train the model to recognize its own knowledge boundary.[45]
- Correct for sycophancy in reward and preference data.[26][48]
- Calibrate stated confidence to match actual accuracy.[33]
- Fine-tune directly on verified factuality, not just helpfulness.[52]
- Add a loss term that penalizes ungrounded generations.[53]
- Train the retriever and generator together, end to end.[36]
Inference
Answering your question
- Retrieve relevant documents before generating an answer.[36]
- Verify each claim against a source the draft never used.[43]
- Sample multiple times and flag inconsistent answers.[35][44]
- Screen answers with rule-based guardrails before they ship.[47]
- Weight decoding toward retrieved evidence, not memorized text.[54]
- Have independent models cross-check or debate the answer.[55]
- Extract exact quotes from the source before answering.[56]
- Explicitly allow "I don't know" as an answer in the prompt.[56]
A language model predicts text, not truth. Every hallucination comes from that same mechanism, just pointed at a question it can't answer. The reasons repeat throughout this explorable: training data has a long tail of facts the model saw once or never, and compression turns those into plausible guesses instead of visible gaps; scoring rewards a confident guess over an honest "I don't know"; and once a wrong token lands, staying consistent with it looks more fluent than backtracking, so one error snowballs into a fabricated paragraph.
None of this is visible unless something checks for it, which is what benchmarks, resampling, and confidence checks do. Fixing it works the same way: hand the model an independent signal its own fluency can't produce, whether that's retrieved evidence, a second model, a resampled answer, or a human check. That signal can enter at whichever stage you actually have leverage, the training data, the fine-tuning objective, or the moment it answers. Each layer has its own blind spot, so no single fix was ever enough. Stacking several imperfect, independent checks is what actually lowers the odds of a bad answer reaching a reader unchecked.
There is a real mathematical limit here, since a fact seen once in training stays a coin flip no matter how large the model gets. Hallucination will not be engineered away completely. It is a permanent property of how these models work, which is why the judgment this explorable has built matters: knowing what an answer is actually worth, and when to check it, isn't a stopgap. It's the skill required to use a confident machine well.