ANDRAE WILLIAMSRisk · Software · AI

SIGNAL · RAG / DATA

Why RAG Fails on Complex Documents

Real document sets contradict themselves and change over time, so many questions have more than one correct answer — a RAG that always returns one answer is misdesigned, not "hallucinating."

4 MINAI

TLDR

  • Real document sets contradict themselves and change over time, so many questions have more than one correct answer — a RAG that always returns one answer is misdesigned, not “hallucinating.”
  • Most RAG failures are unforced: superseded or conflicting documents sitting in the vector store that should have been removed by document management.
  • The core rule is “AI ≤ your data”: if the underlying documents support X, Y, or Z, the system must surface that ambiguity (and label opinions as opinions), not collapse it into a single confident fact.

Caveman

Messy docs, many answers. Don’t fake one.

ELI5

Real paperwork disagrees with itself because different people wrote it at different times, so some questions honestly have more than one right answer. If you build an AI that always picks just one, it will sound sure while being wrong. A good system either cleans up the paperwork first, asks you what you actually mean, or admits when there are several answers.

Trunk → Branches

Trunk: A RAG system can only be as trustworthy as the document set under it, so design must respect contradiction and ambiguity instead of forcing a single answer. Branches:

  • Complex language and human-written archives are inherently contradictory: a policy from 2019 and its 2024 replacement, or laws from 1912 vs 2012, coexist; the “2020 NCAA champion” question is genuinely LSU or Alabama depending on intent [0:27–1:32].
  • Baseline architecture is thin: user question → vector database → search results + original question → LLM → answer back to user [1:55–2:46]. Everything that follows is augmentation of this loop.
  • Kill unforced errors first: don’t let a superseded document and its replacement both live in the vector store — that duplication is what confuses retrieval, so you need real document management, not just ingestion [3:04–3:44].
  • Add a clarification loop: catch nonsense questions (“why are banana”) and under-specified ones (“who won the championship in 2010?” — which sport?) and prompt the user to rephrase before answering [3:49–5:02].
  • Enforce “AI ≤ your data”: if a human reading everything would conclude the answer could be X, Y, or Z, the system must return all of them, not just one [5:08–5:55].
  • Label epistemics: if the store holds legal opinions, present them as opinions in context, not as facts — treating opinion-data as fact is what people later call hallucination when it’s really a design defect [6:04–7:27].

Key Patterns

  • Pattern: Fix the corpus before the model — pruning superseded/conflicting documents removes errors at the source because retrieval can only return what’s indexed; transfers to any pipeline (search, analytics, training data) where garbage-in dominates model quality.
  • Pattern: Match output confidence to input evidence — when sources genuinely disagree, returning the set of answers with their context beats picking one, because faked certainty is the actual failure; transfers to summarization, forecasting, and any decision-support UI.
  • Pattern: Insert a clarification step before retrieval so an ambiguous query is disambiguated by the user rather than guessed by the model — transfers to search UX and agent tool-calling where a wrong assumption compounds downstream.

Stress Test

  • Hidden assumption: You actually can curate the corpus — that someone can authoritatively decide which document is current/superseded. In many real archives (legal, regulatory, multi-team) that determination is contested or unknown, which is exactly why both docs are there.
  • Best counterargument: For many production RAG use cases the goal is one accurate answer from clean docs; over-engineering for contradiction adds clarification-loop friction and “it could be X, Y, or Z” hedging that frustrates users who wanted a straight answer.
  • Fails when: Applied to a well-scoped, single-source-of-truth corpus — the extra document-management overhead and ambiguity handling buy nothing and slow the build.
  • Fact vs opinion: “It’s not a hallucination, it’s just not designed correctly” is a useful reframing but is the presenter’s opinion, not a settled definition — genuine retrieval/generation hallucination exists independent of corpus quality. “AI has to be ≤ your data” is a memorable heuristic, not a measured law.

In Practice

  • Audit any RAG vector store for superseded/contradictory document pairs and remove the stale ones before blaming the model ¶3:34
  • Add a pre-retrieval clarification loop that rejects nonsense queries and prompts for specificity on ambiguous ones, then measure whether answer quality improves ¶3:49
  • Design the answer path to return multiple candidate answers with their context when the retrieved documents genuinely disagree, instead of forcing one ¶5:45
  • Present opinion-type sources (legal opinions, policy interpretations) labeled as opinion-in-context, never as fact ¶6:35
  • Profile the underlying document set (contradictions, dates, one-answer vs many-answer questions) before designing the RAG solution ¶6:48

Connections

↑↓ navigate · ↵ open · esc close