The verify layer — how you know the bot was right

The question every bookkeeper asks about AI, usually about ninety seconds in, is the right one: *how do you know it did it correctly?*

It got asked on a live call I ran recently, first question out of the gate, and it's the question I'd ask too. Here's the honest answer: you don't know because the bot says so. A bot reporting "done" is making a claim. What makes a fleet trustworthy isn't better claims — it's a layer that sits between the work and your trust and tests the claim independently.

That layer is the whole product. The bots are the cheap part.

1. Tie to something the bot can't move

The strongest check is arithmetic against an outside source. A reconcile ties to the bank's statement, not to the ledger's own opinion of itself. My standing fleet rule: a bot may complete a reconciliation only at a $0.00 difference. Anything else it leaves alone, with the difference computed and a note explaining where it stopped.

That one rule does more work than any amount of prompt engineering, because it makes the failure mode *visible instead of plausible*. A bot that cannot force a tie cannot quietly hide a $340 problem inside a finished-looking close.

2. Re-derive, don't re-read

When a bot reports a number, the verify step must recompute it from source, not read the bot's own summary back. This sounds obvious and it is violated constantly — including by me, which is how I learned to say it this bluntly.

The trap: a bot writes a summary, a second step reads the summary, and everything agrees beautifully. Of course it does. They're the same claim wearing two hats. Real verification goes back to the statement, the invoice, the register — the thing that existed before any bot touched it.

3. A check that cannot fail is not a check

This is the law I'd tattoo on the whole industry. Every verification needs two controls:

• a positive control — plant the thing you're searching for and confirm your check actually finds it, and
• a dirty control — run the check against something you know is broken and confirm it fails.

Without those, a green result means nothing at all. I once ran a content scan that came back perfectly clean and felt great about it — the scan was searching a copy of the file that didn't contain the change I was testing. It passed because it was looking at the wrong thing. A test that cannot fail is a lie wearing a checkmark, and you will not notice, because everything looks fine.

Ask any AI tool that reports a clean result one question: *what would have made this come back dirty?* If nobody can answer, you don't have a check.

4. Verify the artifact, not the change

A fix that's confirmed in the source doesn't mean the thing you hand over carries it. I've watched a correction land properly in a document and *not* reach the exported PDF built from it — same content, one stale export, and the export was what would have gone out the door.

So the last check is always on the thing that actually ships: open the PDF, load the page, read the report the client receives. Not the diff, not the log line, not the bot's summary. The artifact gets its own eyes.

5. A dropped check must fail loud, not fail invisible

Here's one from this week, still warm. My review dashboard chips every completed job with the proof behind it — screenshots, signed-off findings, changed tools, the receipts that let me trust a "done" without redoing the work myself. Checking the same forty completed jobs before and after a fix: the proof-chip count went from sixteen to thirty-five, and seven jobs that had shown zero proof suddenly had real evidence attached. Nothing that already had proof lost any.

The sixteen weren't fake, and the missing ones weren't actually missing. The proof existed — filed, real, sitting on disk. My renderer just had a narrower idea of "evidence" than the work did: it only knew how to resolve a reference inside one folder, for five specific file types. A signed-off finding, a changed tool, a properly filed document one folder over — all real, all invisible, because the *display* layer's definition was smaller than the *work* layer's.

The fix wasn't teaching the renderer every possible file shape. It was smaller than that: when a named piece of evidence can't be resolved, show it anyway — as a dashed, unclickable outline, not a working link, and not nothing either. A missing proof now looks like a missing proof. Before the fix, it looked exactly like a job that never bothered to attach any, and those are two different failures that should never wear the same face.

That's the test-that-cannot-fail law, pointed at the other end of the pipe. A verify layer that can silently swallow real evidence is not more trustworthy than one that never collected it — it's worse, because it looks identical to success until somebody counts.

What this buys you

A fleet with a real verify layer fails *loudly and early* instead of quietly and late. That's the entire trade. You are not trying to build bots that never make mistakes — you're building a system where a mistake announces itself before it reaches a client, which is a solvable engineering problem rather than an impossible one.

It also changes what your day is. When verification is systematic, your attention moves off the transactions and onto the exceptions — the ambiguous deposit, the judgment call, the client conversation. Which is the promise everyone makes about AI and bookkeeping, and this is the unglamorous plumbing that actually delivers it.

The bots do the work. The verify layer is why you can sleep.