« What if it makes things up? » — getting an agent to cite its sources
It is the first question in every workshop, and it is the right one. A language model produces a plausible sentence; nothing in its mechanics guarantees it is true. The craft is in making the error visible.
The single rule: cite
Every claim an agent produces points to the document and version it came from. Not « according to our procedures » — the exact reference, clickable, checkable in three seconds.
The rule looks modest. It changes everything: it turns review from an act of trust into an act of verification. Someone who must believe reviews badly; someone who can check reviews fast.
If a sentence has no source, it has no business in the output.
Four designs that work
Answer from the documents, not from memory
The agent searches your base first, then writes from what it found. If it finds nothing, it must say so — not fill the gap with what it « knows ».
Allow « I don’t know »
An agent that is not allowed to fail will invent. The circuit must provide a « not in the base » exit that escalates to a person — and that exit should be frequent at the start.
Separate extraction from writing
Extracting a date from a document is verifiable. Writing a paragraph is not. When both are mixed, nobody knows what they are reviewing.
Log the inputs
The log keeps what the agent read, not only what it produced. That is what lets you fix the cause instead of the output.
Two false fixes
« We’ll use a more powerful model. » A better model reduces how often errors happen, not their nature. On a question whose answer is nowhere in your documents, the best model on the market will also invent — just more convincingly.
« We’ll review everything. » Nobody reviews three hundred outputs a day. After two weeks, review becomes a glance. Better to review ten per cent properly than one hundred per cent in passing.
How we check that it holds
When a system opens, we build a set of real cases — forty to a hundred — whose correct answer is known. We measure three things: the share of correct answers, the share of justified « don’t knows », and the share of silent errors. Only the third really matters.
That case set is not only for go-live: we replay it on every model or prompt change. It is what lets you say, with numbers, whether an update improved or degraded the system.
- AI for banking — answer by quoting the file, never from memory
- AI for legal teams — the finding of a departure, not the advice
- AI for healthcare — cite the version of a procedure, nothing else
- The survey of your ten systems — free, delivered within 72 hours
Other notes
Your document base, or why the agent answers beside the point
An agent is only worth what it can read. The four defects that produce answers beside the point — duplicates, undated versions, scanned PDFs, implicit context — and the preparation that fixes them.
Responsible AI · 8 September 2026The EU AI Act: what it means if you sell into Europe
Application dates for Regulation (EU) 2024/1689, what it asks of a company that uses AI without building it, and why it reaches UK businesses that serve EU customers.
Systems · 10 July 2026Getting cited by ChatGPT and Perplexity: what actually works
Generative engines cite what they can read, structure and verify. Five concrete moves — including serving your pages as Markdown — and what is a waste of time.