A language model that invents plausible-sounding answers is, in day-to-day business use, worse than one that stays silent. The usual worry is that the fix is worse than the fault: teach a model to hold back and it goes quiet on everything. It turns out you can have both — the curve below shows how much reliability costs, and how little it takes to buy.
For a test I put 74 questions to a locally-run model about real German public-sector documents — including, deliberately, questions whose answer is not in the documents at all. A usable system has to do both: find the answer that is there and honestly say “not in the text” for the one that is not.
The biggest lever is a single rule
With no instruction at all, even a well-chosen model answers most unsupported questions anyway — from world knowledge, and often wrongly: 79% of them. One clear rule — “answer only from the context; otherwise say it is not in the text” — cuts that to 4%. And it costs nothing where it counts: the grey line does not move, correct answers holding at 88%. The largest drop on the chart is also the cheapest thing on it — a single carefully worded sentence.
The last few percent are a dial
The remaining 4% are stubborn. These are questions whose topic matches, but whose specific answer is missing from the text, and which the model answers from world knowledge. You can drive them to zero — but not for free.
Two mechanisms take care of the rest: a retrieval gate that turns away questions for which no matching passage is found, before the model even runs; and an independent verifier — a second model that checks every answer for support. The verifier has to be a different model: let a model check its own answer and it waves its own mistake through. Together they reach zero inventions — but drop coverage from 88% to 56% in the process. A verifier strict enough to catch the last invention also rejects some correct answers.
That is not a bug, it is a choice: a public authority would rather have the hard zero and an honest “don’t know”, an internal helper tool would rather keep the coverage. This curve makes the choice visible instead of hiding it.
And if coverage does collapse?
Then it is almost always the model, not the layers. A model that is too small cannot quote its answers verbatim, and gets swept up by the very same safeguards — there, coverage falls much further. The lesson is not “guard less” but “choose the right model.” The second factor is retrieval itself: what is never found cannot be answered by even the best answering logic — good recall lifts the whole curve.
The core holds: reliability does not mean the model goes quiet. With the right model and a single rule the inventions fall while coverage stays put — and all of it runs on your own hardware, measured rather than asserted.