← AI

How LLMs Predict Text

Verified Sep 7, 2026

TL;DR

An LLM doesn't know facts the way a database does — it has no lookup table of "true things." What it actually does, at every single step, is predict the most statistically likely next wordgiven everything written so far, based on patterns learned from a huge amount of text. There's no separate "understanding" module — prediction is the whole mechanism. Scale that simple trick up with enough data and compute, and it starts producing essays, code, and conversation that read as if something understood the question.

ELI5

You've finished a friend's sentence before — not by reading their mind, but because you've heard them talk a thousand times and picked up the pattern. Now imagine doing that, but:
  • you'd "listened" to a meaningful slice of everything ever written, not just one friend
  • you could recall patterns from millions of conversations at once
  • you did the guessing in a fraction of a second, one word at a time, for as long as needed
That's an LLM — the exact same trick, guess the next word from patterns, done at a scale no single human could match. It's also why it can sound completely confident while being completely wrong: confidence and correctness were never the same thing when the entire mechanism is a guess.

WHY IT MATTERS

  • This is why LLMs sound confident even when they're wrong — fluent-sounding text and true text are produced by the same mechanism.
  • It's why more context in your prompt changes the output — you're changing what the model conditions its guess on.
  • It's why the same question worded differently can get a different answer — different words, different prediction.

HOW IT WORKS

Two things decide how good the prediction is:

  1. Context window — how much prior text the model looks back at.
  2. Training data + model size — how nuanced the learned patterns are.

The lab below uses a context window of exactly one wordand a training set of a few hundred words. Real models use context windows of thousands to hundreds of thousands of tokens, trained on a meaningful fraction of all text on the internet. Same idea, wildly different scale.

Try it yourself: a real (tiny) predictor

Runs entirely in your browser. This model was trained — just now, on your machine — on a ~230-word toy story about a cat and a dog. That's the whole "model." Everything below is real math on real (tiny) data, not a canned demo.

Training words—
Vocabulary—
Learned contexts—
Context window1 word
Next word candidateSeen this oftenModel's confidence

FAILURE MODES

  • "It looked that up." It didn't. It predicted plausible-sounding text. Sometimes plausible and true line up; sometimes they don't.
  • "It's just autocomplete, so it's not impressive." The mechanism is genuinely that simple. What's impressive is what that mechanism can do at a large enough scale — the toy above literally cannot hold a coherent thought past a few words; a real model can, from the same core idea plus orders of magnitude more scale.
  • "It sounded confident, so it's probably right." Confidence and correctness come from unrelated places in this architecture. Fluency is cheap; verify anything that matters.

DEEP DIVE

Real models differ from this toy in three specific ways — same core idea, different execution:

  • Subword tokens, not whole words. Real models split text into subword pieces ("tokens") so they can represent any word — including ones never seen in training — as a sequence of familiar fragments.
  • Attention, not a fixed window. Our toy only ever looks at the single previous word. A transformer's attention mechanism lets every token look back at every other token in the context, weighting each one dynamically — not a hard cutoff.
  • Learned parameters, not a lookup table. Our predictions come from literally counting word pairs. Real models encode patterns across billions of parameters learned from training — there's no table to point at, but the underlying question being answered at every step is the same one this toy answers: given everything so far, what comes next?

WHAT I LEARNED

Fluent text is not the same thing as correct text — they come from the same prediction mechanism, so fluency tells you nothing about accuracy. Treat an unverified claim from an LLM the way you'd treat one from a stranger: plausible until checked, not true because it sounded sure.

Go deeper