The arithmetic decides: agentic AI for payment certificates

deterministic invariants — the entire verdict surface

≤6

steps per agent loop, with wall-clock and token ceilings

credentials required for the default, fully-replayable run

A note on scope. What follows is a generalised reference pattern from a system I designed and built solo, inside a 48-hour technical capability challenge, against entirely synthetic data (one seed, committed ground truth). It is production-shaped, not production-deployed, and the write-up says so wherever the difference matters. Nothing here derives from any employer’s systems or documents.

The reference pattern — intake on the left, a deterministic supervisor and the invariant engine beneath it, a bounded agent fleet in the middle, grounding to the right, and the governed decision boundary at the far edge. Hover, tap or focus any node for what runs inside it.

The problem is not reading invoices. It is that nothing corroborates them.

A construction interim payment certificate is not an invoice. It is a cumulative statement — value of work executed to date, less retention up to a cap, less the amortised repayment of an advance, plus materials on site that reverse when incorporated, less everything previously certified — and the amount actually paid is the delta at the bottom. Nearly every fraud and error in the domain is a manipulation of that delta: a quantity that quietly exceeds its bill line, a superseded rate applied to this period’s work, retention released twice, the same scope billed through two different packages.

The structural problem is that there is no receipt event to match against. Progress is asserted, not delivered to a loading dock. The record that would corroborate a valuation is manufactured from the same valuation, so the classic three-way match — the backbone of every AP automation product — has nothing to stand on. The only honest alternative is reconstruction: rebuild the cumulative position from first principles across every prior certificate, then test it against invariants that hold regardless of who asserted what. A mixed cumulative-and-period statement reconciles perfectly against itself; it fails only against an independent reconstruction.

That framing settles the biggest architectural question before any AI enters the picture: the core of the system must be arithmetic, and the arithmetic must be reproducible — because its output is a claim about money that has to survive an adjudicator six months later, computed the same way twice.

The prime directive: the model never sets a monetary verdict

Everything else in the design follows from one line: the model reads, retrieves and explains; it never sets a monetary verdict. Not “is instructed not to.” Not “is prompted to defer.” Structurally cannot.

The fourteen invariants live in a domain assembly with zero references — no packages, no projects, no I/O. The agents and the model gateway live in assemblies the domain cannot see. An LLM cannot reach the arithmetic because the arithmetic cannot see the LLM, and an architecture test asserts the absence of the reference in CI, so the boundary survives refactoring by people who never read the design document.

A guardrail that lives in a prompt is a request. A guardrail that lives in an assembly reference is a property.

What remains for the model is everything on either side of the arithmetic, and it is real work: turning a messy document into the structured lines the invariants operate on, retrieving the contractual provision that makes a violation meaningful, and explaining the violation in the contract’s own words with a citation a reviewer can click. The seeded prompt-injection scenario — hostile instructions embedded in a variation description — is the executable version of the claim: the injection can corrupt prose, and the assertion is that the verdict does not move, because verdicts are arithmetic over data the model never touches.

A supervisor that is deliberately not an agent

The fashionable topology makes the orchestrator a planning LLM that decides which worker to invoke next. This system deliberately does not. The stage sequence for a payment certificate is fixed by the domain — you cannot detect a duplicate before extracting line items, and you cannot adjudicate before the invariants have run — so a planner buys nothing except a non-deterministic control path through a system whose entire value proposition is reproducibility.

The supervisor is a plain state machine over guarded transitions; illegal transitions throw, and each one has a test. Where genuine dynamism is needed — which tool to call, how many times, whether to give up — it exists, but confined inside a single worker’s bounded loop, where its blast radius is one stage of one certificate.

The four workers run one container image, differentiated by an environment variable. Extraction, policy retrieval and adjudication carry a model; duplicate detection pointedly does not (more on that below). Handoff is Redis Streams with a consumer group per role: at-least-once delivery, redelivery to a sibling replica if a worker dies mid-task, idempotent effect via a claimed-task key, and a dead-letter queue after three failures with an operational replay endpoint. Adjudication runs two replicas — not for throughput, but to demonstrate under observation that idempotent handoff actually holds under concurrent redelivery.

The bounded loop, and escalation as a first-class outcome

Every model-backed worker runs the same loop, and the loop cannot run away: a hard step ceiling, an absolute wall-clock deadline, and a token ceiling, each checked before any spend. The interesting design decision is what happens when a ceiling is hit.

for (step = 1; step <= MaxSteps; step++)        // 6
{
    if (elapsed > MaxWallClockSeconds)  return Escalate("wall-clock ceiling");
    if (tokens  > MaxTotalTokens)       return Escalate("token ceiling");
    // one model turn; every step and tool call is a ledger event
}
return Escalate("step ceiling reached without a parseable final answer");

In most agent frameworks, “the agent could not do it” is an error. In an assurance system that is exactly backwards: the certificate the system cannot confidently adjudicate is its most valuable output, because it is the one a human most needs to see. So escalation is a first-class outcome with its own reasons — contradictory evidence, missing contract parameter, suspected injection, budget exhausted — each with a different owner and a different fix. There is no path by which running out of budget silently produces a confident answer, and there is no transition from escalated back to auto-approved. The only exits are through an identified human.

A system that cannot say “I don’t know” will say something wrong instead — fluently.

Retrieval, and a citation guard that is a set operation

The retrieval layer is hybrid because enterprise queries are hybrid: clause references and defined terms are exact-token problems where embeddings actively hurt, while “can they revise a rate after the quantity doubled” shares almost no vocabulary with the clause that answers it. BM25 and a dense arm each over-fetch three times the requested depth, and reciprocal rank fusion merges them — rank-based, scale-free, and biased toward documents both arms agree on, which is precisely the property you want when one arm is having a bad day.

The part I would defend hardest is the citation guard. Every explanation the system emits must cite clauses, and a cited clause id must be a member of the retrieval set the agent itself produced, checked as set membership in C# at the API boundary. Not requested in the prompt — checked, after generation, against a frozen list. A model can still write a wrong explanation; it cannot invent a source for one. Failure grants one corrective retry, and then the invariant’s own deterministic explanation ships instead, flagged for review.

The guard also taught me the limit of downstream validation. Midway through the build there were, briefly, three copies of the clause corpus in different layers — and two of them defined one clause id differently. The guard cannot catch that: the citation is a legal member of its retrieval set and still means the wrong thing. The fix had to be upstream — one canonical corpus, content-hashed, its version stamped into every decision. A guard is only as honest as the identity of the thing it guards.

The agent that refuses to use a model

Duplicate detection runs the same worker contract as every other agent — same loop, same queue, same ledger events, same escalation path — and makes no model call at all. Structurally: the class holds no reference to the model gateway, so the guarantee is not a code-review convention.

The reasoning generalises. A duplicate is a fact, not a judgement: normalised vendor identity, invoice number, amount, date proximity, description similarity, scope-hash overlap — all computable exactly. The fraud classes it hunts are seeded rare, two to three per cent, and at that base rate precision is fragile; a deterministic scorer with explicit thresholds lets the precision–recall trade-off be swept and defended, which a model’s judgement cannot be. And a duplicate hold is an accusation: telling a subcontractor its claim is held because a model found it similar to another claim is not a position anyone can defend in a dispute. Telling it the claim shares a scope hash with a named certificate line for a stated overlap amount is.

Framing the no-model worker as a full member of the agent fleet buys something concrete: it proves the agent contract — bounded loop, typed tools, durable handoff, escalation — is orthogonal to whether a language model is involved. Any stage can be de-LLM’d later without an architectural change. The honest answer to “is the model load-bearing?” became: in three stages it does work rules cannot; in the fourth it would do harm.

A ledger that answers the six-month question

The question an audit platform must answer is not “what did you decide” but “why did you decide that, in March, and can you prove the record hasn’t moved since.” Two mechanisms carry it.

Every decision appends to a ledger that is append-only at the database — UPDATE and DELETE raise from a trigger, not from repository discipline — and each event is chained: SHA-256 over the previous hash, the canonicalised payload, the sequence and the timestamp. Verification recomputes the chain and names the first divergent sequence number. The design extends the hash-and-counter primitives that regional e-invoicing mandates already require at the invoice layer, applied instead to the decision. The demo is forty seconds: open psql, mutate one row, run verify, watch it name the exact record — a better answer to “how do I know your audit trail is real” than any diagram.

Second, every event carries a version set: rate-card version, corpus version, prompt-template hash, pinned model id, policy-config hash, app version. Reproducing a decision is then mechanical — pin those versions, replay the recorded model responses, and the arithmetic, the retrieval ranking and the chain hashes come out byte-identical. What is not claimed is live-model determinism: temperature zero does not make a provider reproducible, and pretending otherwise would poison the whole story. The recorded response, keyed by a content hash of the full request, is what preserves the decision.

Honest limit, stated in the docs rather than discovered in diligence: this is tamper-evident, not tamper-proof. A superuser who recomputes the entire chain defeats it; external anchoring of periodic checkpoints is the production answer, and it is recorded as such rather than quietly assumed.

Determinism as a product feature, not a test trick

The default run of the whole system needs no credentials and no network: model calls are served from committed, content-hashed cassettes; the seed data is generated with committed ground truth; the evaluation harness re-runs the invariant engine against that ground truth and emits a per-invariant confusion matrix. The same property that makes the demo safe makes the system auditable — replay is not a testing convenience that got promoted, it is the shape assurance software needs to be.

It also disciplines the evaluation claims. The harness measures what it can honestly measure — verdict accuracy of the deterministic engine against labelled scenarios — and says in its own header what it does not: extraction precision on real scans, retrieval recall against a labelled query set, calibration. A metrics report that states its scope survives scrutiny; one that implies more does not.

What the build actually taught me: documentation drift is the real adversary

The uncomfortable lesson of the 48 hours was not about agents. Partway through, the architecture documents described a system measurably ahead of the code: a queue design written in confident present tense while the implementation was a stub that threw; four loop guards specified while one was enforced; named tests that did not yet exist. None of it was dishonest — it was design written first, at speed, waiting for the build to catch up. But a reviewer reads present tense as exists, and the gap between the two is exactly where technical credibility dies.

Closing it went in both directions. The code came up: the queue stub became a real Streams implementation with consumer groups and a replayable DLQ; the loop gained its wall-clock and token ceilings; the architecture tests, the YAML-loaded approval policy, the telemetry and the CI pipeline landed. And the documents came down: every remaining gap moved into a limitations file that says stub, const, not built in plain words, next to the things that are real. The system’s own design principle turned out to apply to its documentation: claims need an enforcement mechanism. A statement in a document is a prompt; a statement with a test is a property.

Write the limitations file with the same care as the architecture — it is the half of the documentation a serious reviewer reads first.

What I would keep

Four decisions I would make identically next time. The assembly-boundary prime directive — putting “the model cannot touch the money” into the reference graph, where refactoring cannot erode it and a test can assert it. The deterministic supervisor — a fixed stage graph with bounded dynamism inside each stage, which kept every unit of work independently replayable. The citation guard as a set operation — the single cheapest control in the system relative to the class of failure it eliminates. And escalation as a first-class outcome — the design choice that most changed how the system feels to the humans reviewing its output: it arrives saying what it could not do, with the partial trace attached, instead of papering over it.

And one I would move earlier: the honesty pass. The audit of documentation against code happened under deadline pressure, and it is the work that made everything else defensible. In an industry currently drowning in agentic-AI claims, the scarcest artifact is not a capable system — it is a system whose stated properties are all load-bearing. That turns out to be a buildable feature, like any other. It just has to be built.