Course resource
Verifying AI Output
A practical procedure for catching wrong answers, and for knowing which answers need checking at all.
Why confidence tells you nothing
A model produces its most likely continuation. When it has good information, that is the right answer. When it does not, it is still the most likely sounding answer — delivered in identical prose, with identical confidence.
There is no tone change. This is the single most important thing to internalise: you cannot tell by reading.
What to check, by risk
Not everything needs verification. Verifying everything is how people give up on verifying anything.
| Type of output | Check |
|---|---|
| Brainstorming, first drafts you will rewrite | Nothing |
| Explanations of things you know | Your own judgement |
| Explanations of things you do not know | Spot-check the load-bearing claims |
| Any specific number, date or name | Always |
| Any citation, case, paper, quote | Always, by opening it |
| Anything legal, medical, financial, regulatory | A qualified human |
| Anything going to a customer | Every factual claim |
| Anything a decision rests on | Every factual claim, plus a second opinion |
The five checks
1. The citation test. Open every reference. Not "search for the title" — open it and confirm it says what was claimed. Fabricated citations are realistic: plausible authors, plausible journals, plausible DOIs.
2. The number trace. For every figure, ask where it came from. If the answer is "the model", it is not a figure, it is a guess wearing a number's clothes.
3. The quote-back. For anything derived from a document you supplied:
For each claim in your summary, quote the exact sentence from the source
that supports it. If no sentence supports it, mark the claim NOT SUPPORTED.
This catches drift, which is the most common and least visible failure.
4. The fresh-context check. Ask the same question in a new conversation, phrased differently. Two independent wrong answers that agree are rare; a discrepancy is a flag.
5. The inversion.
What would have to be true for this answer to be wrong?
What is the strongest argument against it?
Useful because it moves the model off defending its previous output.
Prompts that reduce invention
Build these into your standing instructions:
Never invent a fact, figure, date, citation or quote. If you do not know,
write [UNKNOWN] and continue. I would rather have gaps than guesses.
Distinguish clearly between: what the source says, what is widely believed,
and what you are inferring. Label each.
For anything time-sensitive, state the date of your information.
The escape hatch measurably reduces fabrication, because it makes "I don't know" an acceptable output.
Domain-specific traps
| Domain | Trap |
|---|---|
| Legal | Invented cases and statutes, correct-looking citation format |
| Medical | Plausible dosages and interactions; confidently out of date |
| Finance | Arithmetic errors; stale rates and regulations |
| Academic | Fabricated papers; real papers misattributed |
| Technical | APIs and flags that no longer exist, or never did |
| Historical | Dates and attributions confidently wrong |
| Local/regional | Weakest coverage; confident answers about the wrong jurisdiction |
The team habits that actually work
Name the reviewer. "Someone should check this" means nobody does. A named person per output.
Verify before formatting. Once something is in a nice deck, people stop questioning it. Check the numbers while it still looks like a draft.
Log what you caught. A shared list of errors found builds a real sense of the failure rate — and stops the drift into rubber-stamping.
Sample automated output permanently. One in ten, forever. Automation bias sets in within weeks otherwise.
Automation bias
The failure mode of verification itself: the human reviewer stops reviewing, because it has been right for three weeks.
Counters that work:
- Review a random sample rather than everything — smaller volume, more attention
- Occasionally seed a known error and see whether it is caught
- Rotate reviewers
- Record the catch rate; when it hits zero for a month, ask whether anyone is really looking
Your standard
Write down your own rule and stick to it:
| I always verify | |
| I never publish without | |
| For client work, additionally | |
| My reviewer for high-stakes output | |
| I re-test my automations | monthly / quarterly |
The one-sentence version
Check every number, open every citation, and never let AI be the only source for something that matters.