← Systems

Architecture

Verified Sep 10, 2026

The big picture — click a step to jump to it

1

Where a decision lives

A change's "blast radius" is every file it forces you to touch — not a metaphor, a countable thing. Clean boundaries keep that count low and predictable; leaked logic doesn't. Pick a task, flip the boundary quality, watch which layers light up.

Boundary quality

As layers

As real files

An AI coding agent works from whatever files it can see — small blast radius means small context, means a safer edit. A tangled boundary inflates exactly the window an agent has to reason over.

2

Wiring in AI

An "AI pipeline" is a normal request path with an LLM somewhere in it. Client and API are fixed; everything behind the gateway is a choice. Toggle what's wired in, run a request — an invalid combination fails with a real reason, not a canned error.

Add when anything downstream needs routing -- required before retrieval, an LLM, or tools can do anything.

Add when the model needs facts it wasn't trained on -- your docs, your data, anything that changes after training.

Add when the task needs judgment or language generation the model already knows how to do.

Add when the model needs to do something, not just say something -- call an API, run code, change a record.

As a diagram

Retrieval
Tools

As code

Wiring in an LLM is still a systems-design problem first — where the call lives changes latency, cost, and who can reuse it. Run with no orchestrator above: still a valid, ordinary endpoint.

3

Choosing between real options

There's rarely one right architecture — only real tradeoffs given real constraints. Pick a scenario, then weigh the options side by side.

All three are viable. The right one depends on constraints a diagram won't show you — traffic, team size, latency tolerance.

4

Shaping a new app

One level earlier than the rest of this page: before there's a codebase to fix or an AI pipeline to wire in, there's the shape of the app itself. Pick a scenario, weigh the options.

Monolith-first isn't a compromise — it's usually correct. Microservices pay off once a team or a hot path actually needs independent scaling, not before.

When to use what

  • Knowing the pieces isn't the skill — knowing which one a given task actually needs is. That judgment call is what the four steps above are for.
  • Plain endpoint, no LLM: the task is deterministic. Don't reach for a model because you can.
  • + Orchestrator + LLM: the task needs judgment or language generation the model already knows how to do.
  • + Retrieval: the model needs facts it wasn't trained on — your docs, your data, anything that changes after training.
  • + Tools: the model needs to do something, not just say something.
  • Invest in clean layering exactly where more than one kind of change — or reader, human or agent — will touch that code again.
  • Start monolith, split later: a service boundary you don't need yet is pure overhead — extract only once something specific actually needs independent scaling.

Failure modes

  • "More layers is always better." Step 1's phone-number task shows a case where wide blast radius is correct, not a smell.
  • "Adding an LLM call makes it an AI pipeline." Step 2 with no orchestrator is a valid, ordinary endpoint that happens to exist.
  • "There's one right architecture." Step 3's options are all viable — the right one depends on constraints the diagram alone won't show you.
  • "Microservices are the mature choice." Step 4's straining-monolith scenario shows staying monolith and investing in boundaries is often the better call.

Go deeper