How do you check an AI answer before you use it?
Written lesson · about 2 minutes to read
The short answer
Match the check to the stakes: a quick look for a brainstorm, a careful read for anything going to a customer, and a strict check against the original source for money, law, safety and health. Check every number, name, date, quote and link, and have a second person look at anything important.
Three things to remember
- 01Check lightly for ideas, carefully for customers, strictly for money, law, safety and health.
- 02Check numbers, names, dates, quotes and links against the original source.
- 03Confident detail that appears from nowhere is a warning sign, so ask where each fact came from.
Why the check is the job
An AI tool writes a wrong answer as smoothly as a right one. At a glance a plausible answer looks the same as a correct one, which is why mistakes get missed. Checking is part of using the tool, in the same way that proofreading is part of writing a letter.
It does not need to take long. The costly mistakes cluster in a few places: the numbers, the names, the dates and the promises. Look there, skim the rest, and the check takes seconds.
Match the check to the stakes
A brainstorm, such as ideas for a slogan or names for a promotion, needs only a glance, because nothing depends on it yet. Anything sent to a customer or posted in public needs a careful read: every figure, name, date and promise, and the tone as the customer will hear it.
Money, law, safety and health need a strict check against the original source, such as the price list, the contract, the manufacturer's manual or a qualified professional. For important work, have a second person check it too. The tool can help you get oriented, but it does not get the last word.
What to check and how
Check numbers, names, dates, quotes and links against the source. Does the link open? Does the page say what the tool says it says? Is the quote really in the document? Ask the tool where each fact came from, then open that source. If it cannot point to one, treat the answer as a draft.
Watch for confident detail that appears from nowhere, such as an exact figure, a rule with a number or a name you never mentioned. For a factual question, ask again in a fresh conversation. If two fluent answers disagree, at least one is invented. Before anything goes out, read it once for tone, as if you were the customer opening it.
Where to start this week
Adopt one rule for everyone: nothing with a number or a promise in it leaves the building unread. Then write a half-page note saying which kinds of work get which level of check, and who the second person is for the important ones.
For the next week, keep a short list of every mistake you catch. It shows which tasks need the most care, and it is the fastest way to learn where your tool is weak.
More in part 9
- 1How do you check an AI answer before you use it?WrittenYou are here
- 2Where should humans stay in the loop?
- 3What are guardrails?
- 4How do you keep AI on-brand in front of customers?Written
- 5What is AI bias, and can it hurt your business?Written
- 6What happens when an agent gets it wrong?
- 7What is an audit trail, and why will you need one?
- 8What happens when the model updates and your results change?
- 9What is a reward model?
- 10What are AI evals?
- 11What is a golden set?
- 12What is LLM-as-a-judge?
- 13What is calibration (and why confidence scores matter)?
- 14What is shadow testing?
- 15What is red teaming?
- 16What is AI observability?
Next step
Where does this fit your business?
Six quick questions about your business, then your AI plan: what to start with, what a person still checks, and the lessons that go with it. Or bring it straight to a free thirty-minute call.