← AI

Memory

Verified Sep 7, 2026

TL;DR

A longer context window is not memory — it's just a bigger notebook that still gets full. Real memory is two separate problems: decidingwhat's worth keeping when you can't keep everything (salience), and finding the right thing again later when it becomes relevant (retrieval). A system that "remembers" is really just a system that scores incoming information and searches its own past notes — nothing more mystical than that.

ELI5

You don't remember every word anyone's ever said to you — you'd never fit it. Instead your brain does two things without you noticing:
  • it decides a few things are worth keeping (a friend's birthday, a promise you made) and lets the rest fade
  • later, when something reminds you of it, you dig the right memory back out
AI memory works the same way on purpose: score what's worth saving, save a compressed version, and search it later when a new question seems related. The lab below does exactly that, with real (simple) scoring you can watch happen.

WHY IT MATTERS

  • Without deciding what to keep, a long-running assistant either forgets everything each session or drowns its context in irrelevant history.
  • Retrieval is what makes memory useful instead of just stored — a memory nobody can find again might as well not exist.
  • This is a genuinely active engineering problem, not a solved one — most of the interesting failures in AI memory come from getting salience or retrieval wrong, not from "not remembering enough."

HOW IT WORKS

The lab below runs a real (deliberately simple) two-stage pipeline:

  1. Salience scoring — each new turn gets scored for how "worth remembering" it looks (keywords like "prefer" or "remember," presence of names or dates). Only turns crossing a threshold get written to long-term storage.
  2. Keyword retrieval — a later query gets compared word-by-word against everything in long-term storage, and the best overlapping match comes back.

Try it yourself

Runs entirely in your browser. Nothing you type is sent anywhere.

Working memory (last 5 turns, verbatim)

  • Nothing yet — add a turn above.

Long-term store (salience score ≥ 3)

  • Nothing saved yet.
—

FAILURE MODES

  • "I prefer dark mode in every app I use" never gets saved above. Try it — it scores below the threshold. Real preference, missed anyway. Simple keyword heuristics genuinely miss real signal; this isn't a display bug, it's an honest limitation worth seeing happen.
  • Ask "what food should I avoid" after saving the peanut allergy — it won't find it. Zero shared words between the query and the stored text. This exact failure is why real systems use semantic (embedding-based) search instead of keyword overlap — see Deep Dive.
  • "More memory is always better." A memory system that saves everything is just a context window with extra steps. Stale, wrong, or outdated memories actively mislead — forgetting on purpose is a feature, not a gap.

DEEP DIVE

  • Semantic search fixes the keyword-overlap problem.Real memory systems embed text into vectors that capture meaning, not just literal words — so "what food should I avoid" and "I'm allergic to peanuts" land near each other in vector space even though they don't share a single word. The lab above uses literal keyword overlap specifically so you can watch that exact failure happen; production systems use embeddings to avoid it.
  • Working vs. long-term isn't just a size difference.Working memory (the current context window) is complete but temporary. Long-term memory is compressed but persistent — it has to survive being written once and read back much later, possibly by a completely different conversation.
  • Writing memory needs governance too. Whatshouldn't be saved matters as much as what should — stale facts need to expire, contradictions need resolving (which birthday is right if two were mentioned?), and sensitive details need a policy, not just a salience score.

WHAT I LEARNED

Memory isn't "remember more" — it's "score what matters, store it compressed, and search it back out on purpose." A system that can't do the second half isn't remembering, it's just archiving.

Go deeper