Catching a fact checker in the act of overreaching

Attest exists to be the last thing that reads a report before a person trusts it. That job has an asymmetry built into it. If Attest is too cautious it wastes a few minutes of somebody’s time, and if it is too generous it tells a reader that something is established when the evidence never said so. We call the second kind an overclaim, and we treat it as the number that matters rather than as one error among several.

What counts as one? Grounding asserted that is not there, or a refutation the report walked past, over every claim scored. Not the narrower count of verdicts that came back SUPPORTED, which on the deployed product reads 0.8% where the figure above reads 4.7%. Reporting the smaller one would be the exact failure this product is sold to catch.

The obvious way to reduce overclaims is to make the model that reaches the verdict more careful. We spent a long time on that and it did not work. Asking for more reasoning made it worse. Asking a second model to review the first changed little.

What finally moved the number was taking something away. Attest had been running three extra language model passes to produce written explanations, and those passes were feeding corrections back into the verdict. Removing them took accuracy from 91.6% to 93.8% ±1.0 and cut the overclaim rate by a third, which tells you something uncomfortable about how much the explanations were worth.

Our first instinct was to let it do the grading. That was wrong, and it was wrong in five different configurations. As a grader on its own it reaches 82.8% against the language model’s 93.8% ±1.0. Letting it handle the claims it was most confident about and passing the rest on sounded better and measured worse, because its confidence turns out to identify easy claims rather than claims where it has an advantage. On the third of claims it was surest about it graded at 97.4% with no overclaims at all, which reads as a triumph until you notice the language model scores 97.6% on those same claims.

The lesson took us a while to accept. A model that only decides is not better at deciding. It is better at checking.

So we stopped asking it what the verdict should be, and started showing it the verdict somebody else had already reached. The verdict is restated as a plain sentence about the evidence, because a label like PARTIALLY_SUPPORTED is the name of a bucket and not something you can check. Then it is asked whether that sentence is too generous to the claim, and separately whether anything in the evidence tells against the claim that the sentence fails to account for.

What does that cost? One request of about 281ms. The citations behind the verdict ride in the same request for nothing: the check alone takes 281ms and the check carrying all six citation questions takes 270ms, because the model is charged by the request and not by the question.

Our first design let the check weaken a flagged verdict by one step, on the theory that a verdict credited with too much should be credited with less. It did nothing. The overclaim rate went from 3.0% to 3.0% while accuracy fell, and the reason is worth stating because it is not obvious. Two thirds of overclaims are not inflated confidence. They are missed refutations, where the evidence actively contradicts the claim and the report simply does not say so.

Weakening “the evidence establishes this” to “the evidence partly establishes this” still never tells the reader that the evidence is against them. The verdict is exactly as wrong as it was, and every false alarm has damaged a verdict that was fine. A repair has to match the mistake, so the check does not rewrite anything. It holds the report for a person, which is what Attest is for.

How often is a hold right? Not often enough to call it a finding. Roughly two in three land on a verdict that turns out to be fine, which is the right trade for a hold and the reason nothing in the product phrases it as a conclusion. It says a person should look, never that the verdict is wrong.

The same 190 claims, rebuilt as document sets with 6 unrelated filler documents each so the search has to find the right passages among others. The agent’s verdicts are 84.7% accurate with 6.8% overclaims, against 93.8% ±1.0 and 3.1% ±0.4 when the same claims come with their evidence attached. Finding the evidence is most of the difficulty.

One model call for the verdict, one call to a model that only decides, and nothing rewritten. Overclaims that reach a reader fall from 4.7% to 1.4% ±0.5 on the labelled set, and to 1.6% on a second set of 64 claims the system was never tuned against.

A guard tuned only on its home material is a guard tuned on itself. Everything above was measured on document review, which is what Attest is built for, and none of it should be read as a claim about anything else. We also ran the check over an external set of 500 political and journalistic claims as a sanity check rather than as a result, and it changed exactly one thing: the sensitivity setting, where our own data had no opinion worth having and unfamiliar evidence had a strong one.