The problem
A language model predicts words. Usually that produces a good answer, but sometimes it makes up a fact, a source or an amount, with the same conviction. This is called hallucinating. It happens everywhere: courts around the world have already dealt hundreds of times with filings that contained invented rulings or sources.
Why it keeps coming back
- A model does not know what it does not know.
- Too much or too little context increases the chance of errors.
- An answer sounds just as certain whether it is right or not, so the error goes unnoticed.
Our approach
- Provide the right information. Not the whole file, but the few relevant facts, with their source. That reduces the chance of fabrication.
- Calculation and lookup by code. Amounts, dates and facts come from the source, not from the model's memory.
- Have answers checked. A model can give different answers to the same question. We have another model check those answers. That gives us a cross-section. For numerical answers, their average can give a picture.
- Escalate when needed. If the check is not certain, the system looks for more evidence or has a heavier model review the answer.
- Admit doubt honestly. If doubt remains, the system says so instead of guessing.
What to watch out for
- Always ask for sources, and check them for decisions that matter.
- Let a person decide where errors cost money or reputation.
- Measure quality with a fixed set of test questions, so you can see whether a change makes things better or worse.
Would you like advice on reliable AI? Request a free advice conversation.
Sources
Updated on