AI output review is the habit of checking what a model produced before anyone relies on it. A summary can read well and still contain a wrong total, a made-up source or a quiet change of meaning. Fluent text is not the same as correct text.
This guide gives analysts and small teams a short routine they can run in about ten minutes per document.
Why AI output review needs a routine
A model writes with the same confidence whether it is right or wrong. That means tone tells you nothing. Only a check against the original material does.
Without a routine, review depends on who is tired, who is rushed and who trusts the tool. A fixed list removes that luck.
Step 1: Sort the claims by type
Read the output and mark each claim as one of three kinds.
- Numbers: totals, percentages, dates, counts.
- Facts about the world: named people, laws, products, studies.
- Judgements: "sales are healthy", "risk is low".
Each kind needs a different check, so sorting first saves time.
Check every number against the source file
Pick the figures that drive a decision and recalculate them from the raw data. If the model says revenue grew 12%, open the spreadsheet and work it out yourself. A data validation pass on the input file also catches blank cells and duplicate rows that a model may have skipped over.
Do not trust a number just because it appears twice in the text. A model can repeat its own mistake.
Check named facts at the original source
If the output names a study, a regulation or a quote, find the original. Search for the exact title. If you cannot find it, treat it as invented and remove it.
Check the link as well as the name. A real publication with a wrong page is still a wrong citation.
Step 2: Look for what the model left out
Errors of omission are harder to see. Ask three questions.
- Did the summary drop a caveat that was in the source?
- Did it merge two groups that should stay apart?
- Did it ignore rows that did not fit the pattern?
Imagine a model that summarises customer complaints and says "most complaints are about delivery". Your file may show that the largest single group is billing, and delivery only looks big because three categories were merged. The sentence is fluent and the file contradicts it.
Step 3: Test the judgements
Judgements are where a model drifts furthest from the data. For each one, ask what number would prove it wrong. If you cannot name one, the claim is too vague to keep. Rewrite it so it points at a figure, or cut it.
Step 4: Record who reviewed what
Review only counts if it leaves a trail. Keep a short note with each report:
- the source files used and their dates
- which claims were checked and by whom
- what was changed or removed
This is the core of data verification. It also lets the next person see what has been checked and what has not, which matters when a figure is questioned months later.
Step 5: Use a second reader for high-stakes work
Anything that goes to a client, a regulator or a board should be read by someone who did not write the prompt. The author knows what they meant to ask for and tends to read the answer kindly. A second reader reads only what is on the page.
Our post on tracing every number in your report shows how to document where each figure came from, which makes the second reader's job much quicker.
A short checklist to keep
- Sort claims into numbers, facts and judgements.
- Recalculate the numbers that matter.
- Find the original of every named source.
- Look for caveats and groups that disappeared.
- Tie each judgement to a figure.
- Write down what was checked.
- Get a second reader when the stakes are high.
Keep the human in the loop where it counts
You do not need to check every sentence with the same effort. Spend the time on figures that change a decision and on claims a reader will repeat. Skim the rest.
A model is a fast first drafter. The review is what turns a draft into something your team can stand behind.