← AI

Context Engineering

Verified Sep 7, 2026

TL;DR

A model can only reason about what's actually in its context window — everything else might as well not exist. Prompt engineering asked "how do I word this?" Context engineering asks a bigger question:what actually goes in the window, and what gets left out?Every context window has a token budget, and every piece of context — system prompt, tools, memory, retrieved docs, conversation history — competes for a slot in it. Choosing badly wastes the budget on low-value filler and starves the model of what it actually needed to see.

ELI5

Imagine packing one carry-on bag for a trip, and everything you own is begging to come with you. You can't fit it all, so you have two options:
  • pack in whatever order things are lying around, and stop packing the second something doesn't fit
  • decide what actually matters for this trip, and pack the most useful things first
The first approach means a lucky-placed bulky item can block everything packed after it — you end up with half a bag of stuff you barely needed. The second gets you a full bag of things you'll actually use. Context engineering is packing the bag on purpose instead of by luck.

WHY IT MATTERS

  • Every context window has a hard token limit — there's no such thing as "just include everything."
  • An agent that naively appends its entire conversation history can burn its whole budget on stale small-talk before it ever gets to the document it actually needs to read.
  • This is why two systems using the same model can perform completely differently — the model didn't get worse, its context got worse.

HOW IT WORKS

Two decisions define a context-engineering strategy:

  1. What competes for the budget — system prompt, tool schemas, memory, retrieved documents, conversation history.
  2. How ties get broken when it doesn't all fit — by relevance, recency, an explicit priority order, or (the naive default) just by whatever order things happened to be assembled in.

The lab below runs both strategies against the same eight candidate context pieces and the same budget, side by side.

Try it yourself: the same budget, two strategies

Runs entirely in your browser on a fixed, honest example — nothing here calls a real model.

Naive (assembly order)

— tokens used

    Engineered (by relevance)

    — tokens used

      FAILURE MODES

      • "Just raise the context window size." A bigger window doesn't fix a bad packing strategy — it just delays when the problem shows up. The lab above uses the same broken naive strategy at every budget size.
      • "More context is always better." Irrelevant context doesn't just waste budget — it can actively distract the model from what matters, the same way a cluttered desk makes the one important paper harder to find.
      • "Retrieval solves this automatically." Retrieval decides what's available to include. Context engineering still has to decide what actually earns a spot once everything's competing for the same budget.

      DEEP DIVE

      • Prompt caching rewards a stable prefix. Several model APIs charge less (and respond faster) for context that stays identical between calls. That means a good context-engineering strategy also orders context by stability — put things that never change (system prompt, tool schemas) first, and things that change every turn (the latest message) last.
      • Structured context beats one long blob. Real systems don't just concatenate strings — they separate context by role and type (system instructions, tool definitions, retrieved documents, conversation turns) so the model can weight each kind appropriately instead of treating everything as equally-important prose.
      • The strategy only matters when context is scarce.Slide the budget above all the way up — naive and engineered converge, because when everything fits, order stops mattering. For a real agent working a long task, it's almost never that generous.

      WHAT I LEARNED

      A context window isn't a bottomless bag — it's a budget, and something always loses when it's spent badly. Before asking "how do I word this prompt," ask "what's actually earning a place in the context, and what got in by accident."

      Go deeper