Course resource

AI Stack Architecture Template

How the pieces of your AI setup fit together, and how to document it so someone else can maintain it.

The four layers

INTERFACE   where you and your team interact with it
ORCHESTRATION  what decides the sequence and connects the parts
INTELLIGENCE   the models doing the judgement
DATA           what it knows and where that lives

Most stacks fail at the data layer — not because the models are weak, but because the model cannot see what it needs, or sees a stale version.

Your stack

Interface

Where work arrives	Who uses it	Notes
Chat assistant
Email
Chat platform bot
Forms
Scheduled, no interface

Orchestration

Tool	What runs there	Why this tool

If you have more than two orchestration tools, ask why. Splitting workflows across platforms is the main reason nobody can debug them later.

Intelligence

Job	Model tier	Temperature
Classification / routing	cheap, fast	0–0.2
Extraction	cheap, fast	0–0.2
Drafting for humans	capable	0.5–0.7
Final customer-facing output	capable	0.5–0.7
Bulk / high volume	cheapest that works	0–0.3

The rule: use the cheapest model that passes your quality bar for each step, not the best model for everything. A classification step does not need a frontier model, and running one there is most people's largest avoidable cost.

Data

What it knows	Where it lives	Who updates it	Last reviewed

Connections

Draw or list every connection. For each, the two questions that matter:

From	To	What flows	Contains personal data?	What happens if it fails

The "what happens if it fails" column is what turns a diagram into a working system.

MCP, in one paragraph

Model Context Protocol is the emerging standard for how an AI connects to tools and data. Instead of a bespoke integration per tool per platform, a tool exposes an MCP server once and any MCP-capable client can use it. The major automation platforms now support it on both sides — exposing your workflows as tools an agent can call, and letting your workflows call external MCP servers.

Practically: it means the connection layer of your stack is becoming standardised, and a tool you wire up once is reusable across assistants. Worth designing toward rather than building more one-off integrations.

The principles

Own the portable parts. Your prompts, your documents, your processes. If those live somewhere you control, swapping the model or the platform underneath is an afternoon. If your work only exists inside one vendor's format, you are locked in.

Put the human where the risk is. Not everywhere — that defeats the point — but before anything that reaches a customer, moves money, or writes to a system of record.

Log the judgement steps. Every AI step should record its input and output. Without this you cannot debug anything, because the same input may not reproduce the same failure.

Design for the model changing. It will. Keep prompts in one place rather than scattered through workflow steps, so a model swap is one edit.

Fail loudly. Silent failure is the default in automation and the worst property a stack can have.

The single-point-of-failure audit

If this breaks	What stops	How I find out	Fallback
The orchestration platform
The model provider
One specific integration
The person who built it

That last row is the one people never fill in, and it is the most common failure in practice.

Documentation

Someone else should be able to run this without you. At minimum:

A diagram or list of what connects to what
Every workflow named, with one sentence on what it does
Where the prompts live
Where the credentials live, and who can rotate them
What to do when each thing breaks
Monthly cost, and what drives it
Named owner, and a review date

Review

Date	What changed	Cost	Anything retired

Retiring things matters. Stacks accumulate workflows nobody uses, still running, still costing money.

Back to dashboard