Evidence beats confidence

The most common thing an AI inspection tool shows a user is a number between zero and one. It is the wrong artefact. Not useless, but wrong as the primary output, and I think the habit of shipping it is holding the category back.

Here is the test I use. Take an answer the system produced, hand it to an inspector, and ask them to disagree with it. If the only thing they can do is disagree with the whole answer, the system is not correctable. If they can point at the specific claim that is wrong, it is.

A confidence score fails that test by construction.

What "evidence" has to mean concretely

Vague versions of this argument are easy to agree with and impossible to act on, so here is the specific list. For an answer to be evidence-bearing it has to carry:

The structure it is about. Not "a pole" but which pole, resolved deliberately, with the reasoning available. Most images of distribution infrastructure contain more than one candidate.

The component, localised. A box or a mask that points at real pixels, produced by something whose job is localisation. If the coordinates came out of a language model describing where it thinks the damage probably is, they are a guess wearing the costume of a measurement.

The condition, stated in the form's own language. Inspection programs have fixed vocabularies, and an answer that paraphrases is an answer somebody has to translate before they can file it.

Provenance. Which model, which version, which prompt, which code. Six months later somebody will ask why the system said what it said, and "the model thought so" is not an answer you want to give a regulator or a jury.

An explicit abstention when the evidence is not there. A system that always produces an answer has quietly decided that a fabricated answer is better than a gap. In inspection work it is not.

Why this changes the engineering, not just the UI

It would be convenient if evidence were a presentation layer over the same model. It is not, and the difference shows up in two places.

The first is that you have to actually produce the localisation rather than ask a model to describe it. We went through this. Early on, evidence boxes in one of our pipelines came from a vision language model's own description of where a defect was, which is fast to build and reads convincingly in a demo. It is also unverifiable, and unverifiable evidence is worse than none, because it invites trust it has not earned. Replacing those with boxes from real detectors, routed by what is being looked for, was more work and produced answers you can argue with.

The second is that provenance has to be captured at the moment of inference, not reconstructed later. That means a run carries the versions of everything that produced it, including prompts, because a prompt is code. If you cannot reproduce an answer from its record, you cannot investigate a complaint about it.

The correction loop is the actual product

Once answers carry evidence, something more useful becomes possible than better accuracy. Corrections become attributable.

An inspector rejecting an answer is a low-information event. An inspector rejecting an answer and the record showing that the located component was on the adjacent structure is a specific defect with an owner. Aggregate a few hundred of those and you no longer have a backlog of complaints, you have a ranked list of what to fix, per step, with counts.

This is the part I would push any team building in this space on. It is tempting to treat review as a cost to be minimised, a tax on the automation. It is the opposite. Review is the only source of labelled data that arrives at the same rate as your product is used, and it is free if you built the answer so that a correction lands somewhere specific.

What this looks like from the buyer's side

There is a version of this argument that is really about procurement, and it is worth stating plainly because it is where the commercial and the technical arguments meet.

Utility inspection outputs get audited. They inform work orders, spend, and regulatory filings. When somebody asks why a structure was flagged, or why one was not, the acceptable answers are all of the form "here is the image, here is what was identified, here is where, here is what produced it, and here is who reviewed it." An accuracy figure on a slide is not an answer to that question, and a confidence score attached to a specific finding is not either.

So the requirement is not really "explainability" in the academic sense. Nobody is asking for a saliency map. They are asking for a record. That is a much more tractable engineering problem, and it happens to be the same work that makes the system improvable.

The short version

Confidence tells you how the model feels. Evidence tells an inspector what to look at, and tells you what to fix. Only one of those compounds.

If you want the argument for why a single model cannot produce that chain in the first place, that is the other half of this.