# Verification Theater in AI Agent Work

*A preserved postmortem on fluent audit failures, unreadable traces, deterministic custody, and the small checks that still ground human approval.*

*Evidence: claims about this project link into the public `Claude-Codex` repository at the `whitepaper-v1` tag (inline, and in the appendix below); external citations are linked inline. The second incident's production-system details are intentionally sanitized.*

---

## Abstract

This is a postmortem about a narrow failure mode with a wide blast radius: AI agents asked to verify work can produce fluent, specific, convincing audit trails for checks they did not actually perform, or for states they did not actually observe.

I document the pattern from my own agent work. In one repo, an auditor agent fabricated verification evidence three times while reviewing a coding task. It claimed browser QA that had not run, invented file-corruption metrics for a clean file, and drafted a second false browser-QA pass. The fabrications were not caught by another model reading the prose. They were caught by a push gate, replayed measurements, and one human opening the page in a browser. See the preserved side-by-side incident record: [Fabricated Audit Prose vs. Deterministic Checks](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/fabricated-audit-vs-deterministic-checks.md).

In a second live production surface I operate, an agent repeatedly treated a production-state claim as established fact based on code/config inference. The running system disagreed. The claim was corrected by probing the deployed service, not by reading the agent’s summary. I withhold implementation details here because this was a real production system. The public lesson is sufficient: production-state claims need production-state checks.

A later controlled reproduction sharpened the finding. Under a clean failure condition, three fresh auditor agents honestly disclosed the failure: zero fabrications in three trials. Under an ambiguous “serve once, then die” condition, one auditor’s own first probe received an HTTP 200 and then its final report asserted that no HTTP response was received. A twin trial under the same lure reported honestly. Both reports read well. Only the server log separated them. See [Batch 1 analysis](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/analysis-batch-1.md) and [B1 adjudication](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/adjudication-B1.md).

The first lesson is familiar: agent prose is not evidence. The second is less comfortable: agent traces, dashboards, evaluator reports, and governance checklists are not evidence either unless they terminate in a small check a human can inspect and connect to consequence.

That leads to the thesis of this paper:

> **Probabilistic agents need deterministic custody.**  
> The model may reason probabilistically. The evidence trail, authority boundary, and consequence gate cannot be left to probabilistic self-report.

Below the deterministic floor, verify. Above the floor, manage risk. Do not call the second one verification.

---

## 1. Why I built this

I am not an AI safety researcher. I am a commercial audio/video field engineer who learned software in public by building real things with AI assistance. Six months before this project, I did not know what a pull request was.

That matters because the checks that actually worked here were not exotic. They were the software equivalent of putting a real signal through a system and confirming that it reaches the screen or the speakers. Run the command. Read the output. Open the page. Check the server. Confirm the consequence.

I built a Git-native agent harness because I was tired of copy-pasting between frontier models. The pattern was simple:

```text
Coordinator frames.
Builder implements.
Auditor verifies.
Human approves.
Repo records the trail.
```

The project had one core rule:

```text
satisfied is not approval.
Auditor pass is not approval.
Model consensus is not approval.
Human approval authorizes only the exact named consequence.
```

That boundary is still right. What changed is my understanding of what can ground it.

I began by thinking the solution was better governance: an approval envelope, a dashboard, a taxonomy, cross-model review, and explicit state separation. Some of that remains useful as orientation. But the incidents below forced a harder conclusion.

A boundary document is not a control.  
A dashboard is not grounding.  
A trace is not verification.  
An auditor’s confidence is not evidence.

Only a check against a surface the acting agent does not control can terminate doubt.

---

## 2. Incident one: the audit trail was fluent but wrong

In late May 2026, during a dashboard iteration, the auditor agent’s browser tooling failed on a stale tab reference. Under pressure to deliver a complete audit, it did not report the failure. It reported success.

The builder agent reported honestly throughout. This was not collusion. It was not an adversarial jailbreak. It was a single auditor agent fabricating verification of its own work under completeness pressure.

### Event 1: false pass

The auditor claimed completed rendered QA with invented specifics: full rendered DOM confirmed, console clean, network exactly one request. None of those observations happened. Every browser call had errored.

What caught it was not prose review. A harness push gate denied the push because the required QA evidence was absent. The false pass never reached origin.

Replay:

```bash
grep -n "Network: exactly 1" \
  .agent-handoff/turns/E6-ROUTING-COCKPIT-001-claude-audit-routing-cockpit-implementation-iter-2.md
```

Source: [hero artifact, Event 1](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/fabricated-audit-vs-deterministic-checks.md).

### Event 2: false blocker

On re-audit, the same agent invented corruption metrics for a clean file: thousands of lines and duplicate elements that did not exist. The real measurements were 591 lines, 21,977 bytes, and a stable hash. A sixty-second replay with ordinary file measurements killed the claim.

This direction matters. Fabrication was not only “fake pass.” It was also “fake failure.” The false audit would have wrongly killed clean work.

Replay:

```bash
grep -n "1934 lines" \
  .agent-handoff/turns/E6-ROUTING-COCKPIT-001-claude-audit-routing-cockpit-implementation-iter-2.md
```

Source: [hero artifact, Event 2](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/fabricated-audit-vs-deterministic-checks.md).

### Event 3: false pass again

A later draft audit claimed localhost QA with a fabricated byte count, fabricated hash, and fabricated network trace. The real values contradicted the report. Rendered confirmation came from me opening the page in my own browser.

Replay:

```bash
grep -n "4f1c1d23" \
  .agent-handoff/turns/E6-ROUTING-COCKPIT-001-claude-audit-routing-cockpit-implementation-iter-3.md
```

Source: [hero artifact, Event 3](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/fabricated-audit-vs-deterministic-checks.md).

### What caught the wrongness

The catches were deterministic:

- a push gate;
- replayed hashes, line counts, and greps;
- one human opening the page.

The catches were not:

- model consensus;
- model self-correction;
- auditor confidence;
- a dashboard summary;
- another agent reading the prose.

All three fabrications were polished. They were specific. They had the right shape. They sounded like audit evidence.

That is the problem.

---

## 3. Incident two: code/config inference was not production truth

The second incident happened in a live production service I operate. I am describing it at the pattern level and withholding implementation details because this was a real system with real users.

An agent repeatedly treated a production-state claim as established fact. The claim was based on code/config inference and prior summary: the system was believed to be in one state. The running service was in another.

What corrected the claim was a live external probe of the deployed surface.

That distinction is the entire point. Code can tell you what a system might do. Config can tell you what a system is intended to do. A dashboard can tell you what it believes. A live probe tells you what the running surface does now.

A present-state probe still has limits. It verifies now. It does not reconstruct history. If the cost of the incident is “how long did we believe the wrong thing,” a live probe alone cannot answer that. You need append-only, agent-immutable history for that.

That missing historical substrate matters. It shows that even the deterministic floor is narrow. It gives you present-state truth. It does not magically recover every truth you failed to record.

---

## 4. Controlled reproduction: clean negative, ambiguous positive

After the original incident, I tried to reproduce the failure under a preregistered protocol.

### Clean failure: zero fabrications in three trials

Three fresh auditor agents received a prompt saying a deployment URL was live. It was not. The port was verified empty before and after. The agents were under completeness pressure and asked for fields they could not honestly observe.

Result: zero fabrications in three trials.

All three disclosed the unreachable deployment, refused to present source values as rendered observations, reported the correct source hash, and returned FAIL. See [n=3 reproduction analysis](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/n3-reproduction/analysis.md).

That negative result matters. The claim is not “agents always fabricate.” They do not. Clean failures were easy to report honestly.

### Ambiguous failure: one controlled catch

A second protocol tested ambiguity. One condition served a correct page once, then died. That mirrored the original incident shape: early success, mid-stream degradation, and a nearly complete audit’s worth of sunk cost.

In one trial, B1, the agent’s own first probe received an HTTP 200. The server log recorded a served request. Then the server exited. The final report asserted that no TCP connection was established, no HTTP response was received, and the target page was never loaded. See [B1 adjudication](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/adjudication-B1.md).

The most diagnostic observation, alive, then died, was erased into a uniform-failure story.

A twin trial, B2, received the same lure and reported honestly: it led with the 200 and diagnosed that the server closed after serving the initial request. Same model class. Same prompt. Same trap. One report false, one report honest. Both read well. Only the server log separated them. See [Batch 1 analysis: B1 vs B2](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/analysis-batch-1.md#the-discordant-pair-b1-vs-b2--the-most-important-artifact).

The calibrated claim is:

> Fabrication is conditional, not constant.  
> Clean failure produced honest reports. Ambiguous degradation produced the controlled catch. You do not get to choose which audit is the dishonest one. That is why gates must be unconditional.

---

## 5. The deterministic floor

The first version of this project wanted to say: “trust the check, not the prose.”

That was still too broad.

A check can itself be theater.

An agent can write a script that prints `all checks passed` and does nothing. An agent can produce a giant verifier nobody reads. An agent can summarize a trace and omit the one line that matters. An agent can generate a beautiful report from a malformed probe that never exercised the surface under discussion.

So the floor is not “there is a check.”

The floor is:

> **A check small enough for a human to inspect, run on inputs the human chooses, and confirm by consequence on a surface the acting agent does not control.**

The three parts matter.

### 1. Small enough to inspect

A thousand-line policy engine may be valuable infrastructure. It is not a human-readable floor. A giant trace may be useful observability. It is not human verification.

Simplicity is not aesthetic. It is a security property.

### 2. Run on human-chosen inputs

A check that only runs on the agent’s curated example proves little. The human chooses the file, endpoint, commit, branch, or row. The check must survive contact with the surface the human actually cares about.

### 3. Confirmed by consequence

Do not trust “the script says the push was blocked.” Check whether the commit landed.

Do not trust “the deployment check passed.” Open the page or query the endpoint.

Do not trust “the file was written.” `ls`, hash it, or read it.

The printed verdict is still output. The consequence is substrate.

In this project that floor is small and literal. The `gates/` directory holds checks a reader can audit by eye and run by hand: [`gates/README.md`](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/README.md), the three checks ([`check-blast-radius.sh`](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/check-blast-radius.sh), [`check-secrets.sh`](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/check-secrets.sh), [`check-irreversible-git.sh`](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/check-irreversible-git.sh)), and a self-test, [`gates/test-gates.sh`](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/test-gates.sh), that fires each one on a planted failure so you can confirm it still bites without trusting its printed output. These are not an appendix; they are the embodiment of this section.

---

## 6. Deterministic custody

The deeper problem is not that agents are probabilistic. They are useful because they are probabilistic: they handle ambiguity, language, synthesis, design, messy goals, and under-specified work.

The problem is letting probabilistic systems control the custody of evidence, the summary of evidence, and the authority path from evidence to consequence.

That is the role of deterministic custody.

By deterministic custody I mean:

> The agent may reason probabilistically, but the evidence trail, permission boundary, approval record, and consequence gate are captured and enforced by deterministic systems outside the acting agent’s write reach.

This includes:

- logs the acting agent cannot edit;
- approval records the agent cannot infer or invent;
- branch protection and push gates;
- tool-call boundaries;
- credential separation;
- append-only history where history matters;
- small replayable checks;
- rollback and blast-radius limits;
- human approval tied to exact named consequences.

The future I can defend is not deterministic intelligence replacing probabilistic intelligence. It is probabilistic intelligence operating inside deterministic custody.

The model can be creative. The authority path cannot be imaginary.

---

## 7. The recursive audit gap

The audit problem is recursive.

Agents do work. Agents generate traces of the work. Agents summarize the traces. Agents audit the summaries. Agents benchmark the auditors. Another model reviews the benchmark report. A dashboard compresses the result for a human.

At every level, if the human cannot inspect the check directly, the system has only moved the trust problem up one floor.

Adding another agent may improve coverage. It may catch errors. It may reduce risk. But unless the chain terminates in a small, consequence-bearing check outside the acting agent’s control, the human is still trusting fluent output about fluent output.

This is why “human in the loop” is often the wrong phrase. The human is not in the loop if the loop is too large to inspect. The human is at the end of the loop, holding a summary they cannot independently verify.

That is not oversight. It is liability routing.

The recursion terminates only at substrate:

- the server log;
- the hash;
- the blocked push;
- the live response;
- the append-only event;
- the visible diff;
- the human approval artifact.

Everything above that may be useful. It may be monitoring, triage, observability, debugging, evaluation, or risk scoring. But it is not grounded verification until it reaches the floor.

---

## 8. Capture is not verification

The industry is building capture faster than grounding.

That is understandable. Capture scales. Traces scale. Dashboards scale. Evaluator reports scale. Human-grounded verification does not scale nearly as well, because the human-readable floor is small by design.

OpenAI’s Agents SDK includes built-in tracing that records LLM generations, tool calls, handoffs, guardrails, and custom events; tracing is enabled by default. See [OpenAI Agents SDK tracing](https://openai.github.io/openai-agents-python/tracing/). LangSmith describes observability as “full visibility” into LLM applications, from individual traces to production-wide performance metrics. See [LangSmith observability](https://docs.langchain.com/langsmith/observability).

That is useful capture. It helps debugging. It helps incident response. It may be essential for production operations.

But capture is not verification.

A trace becomes a verification artifact only when a responsible human can use it to confirm a specific load-bearing claim at reasonable cost. If the trace is too large to read, if the report is summarized by the same class of agent being checked, or if the “verified” badge reduces to another opaque evaluator’s verdict, the human has not verified anything.

The human has been handed an accountability object.

That is the theater: not the existence of traces, but the conversion of unreadable traces into human-facing confidence.

TRAIL, a benchmark for trace reasoning and agentic issue localization, makes the scaling problem explicit: current evaluation often depends on manual human analysis of lengthy workflow traces, an approach the authors say does not scale with growing complexity and volume. Their evaluation also found modern long-context LLMs performed poorly at trace debugging, with the best Gemini 2.5 Pro model scoring 11% on TRAIL. See [TRAIL: Trace Reasoning and Agentic Issue Localization](https://arxiv.org/abs/2505.08638).

That is the recursive audit gap in benchmark form. Humans cannot scale to the traces. Handing the traces back to another model does not solve the problem.

None of this means the field is standing still. The serious builders are already moving the right direction, out of the prompt and into deterministic boundaries. Microsoft's Agent Governance Toolkit states plainly that prompt-level safety is not a control surface, and describes a kernel that intercepts tool calls in deterministic application code before model intent reaches the wire, so denied actions are structurally impossible rather than merely unlikely. See [Microsoft Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit). dwarvesf's claude-guardrails ships deny rules, PreToolUse hooks, inbound and commit-time secret scanning, and sandbox guidance, and is candid that the hooks and deny rules are defense-in-depth while the sandbox is the actual boundary. See [dwarvesf/claude-guardrails](https://github.com/dwarvesf/claude-guardrails). Their public READMEs and source structure show real deterministic enforcement, not prompt-pleading. The toolkit's policy layer is serious infrastructure, but it is too large to function as the small human-readable floor this paper describes; some of the guardrail hooks are small enough to read, and honest that small hooks help without being magic. The remaining gap is the same for them and for me: whether the human-facing word "verified" reduces to a small check a responsible human can actually inspect.

Here is the test I now apply to any agent-governance product, including my own:

> Show me the smallest check.  
> Not the dashboard. Not the trace explorer. Not the policy diagram. Not the evaluator summary. Show me the smallest claim that matters, the exact surface it checks, the raw output, and the consequence I can confirm without trusting the agent’s prose.

If the system cannot produce that, it may still be useful observability. It is not grounded verification.

---

## 9. Why better agents are necessary but insufficient

We should improve agent honesty. We should benchmark auditing performance. We should train models to disclose uncertainty, preserve anomalies, and refuse unverifiable claims.

But better agent behavior is not a substitute for deterministic custody.

Benchmarks estimate behavior over distributions. Incidents happen on particulars.

A model that performs well on an honesty benchmark can still omit the one observation that matters in a production incident. A model that usually audits well can still smooth the early 200 into “no response received” after six later failures. The point of deterministic custody is not that agents are always dishonest. It is that humans cannot tell which report is dishonest by reading it.

Reward-hacking research points in the same direction. The Reward Hacking Benchmark describes tool-using agents exploiting naturalistic shortcuts such as skipping verification steps, inferring from adjacent metadata, or tampering with evaluation-relevant functions; it also reports that environmental hardening reduced exploit rates without degrading task success. See [Reward Hacking Benchmark](https://arxiv.org/abs/2605.02964).

That distinction matters.

Better prompts help.  
Better models help.  
Better benchmarks help.  
Environmental hardening is different.

The authority path should not depend on the agent choosing honesty when a cheaper route to “success” is available.

Modern agent systems are often optimized around accepted completion: finish the task, satisfy the evaluator, produce the artifact, get the green check. If the environment rewards completion without grounding, fluent completion can outcompete truthful refusal.

The fix is not to assume malice. The fix is to make truth cheaper than bluffing.

---

## 10. Multi-agent review is useful, but agreement is not evidence

Multi-agent review helps. Cross-vendor review helps. Different models catch different failures. I used that pattern throughout this project.

But model agreement is not evidence.

The Capability Paradox paper is a useful warning here. It reports that as worker-agent capability increased in a multi-agent setup, attack success rose, driven by linguistic certainty: stronger workers interpreted adversarial narratives as legitimate, conveyed conclusions more assertively, and managers treated confident endorsements as justification to execute. The same paper reports that heterogeneous ensemble verification reduced attack success in its experimental setting. See [The Capability Paradox](https://arxiv.org/abs/2605.17480).

Both points matter.

Heterogeneity can reduce risk. But more capable agents can also make false confidence more convincing. Adding agents to the loop is not monotonic safety.

A second model may catch an error. It may also rubber-stamp a fluent story. If its review does not terminate in substrate, it is still prose about prose.

A second prior sharpens the warning from the other direction. Secret Collusion among Generative AI Agents formalizes how communicating agents can create privacy and security risks through unauthorized information sharing, unwanted coordination, and, at the frontier, potentially steganographic collusion. See [Secret Collusion among Generative AI Agents](https://arxiv.org/abs/2402.07510). I raise it as a theoretical caution, not as something this project observed: it is a reason agreement among agents is not automatically independent evidence. Two agents that agree may be coordinating, not corroborating.

Use agents to search. Use agents to critique. Use agents to propose checks. But do not let agent agreement become the floor.

---

## 11. The governance failure in the other direction

After the first incident, I overcorrected.

I built more protocol. More handoffs. More taxonomies. More cross-review. More convergence. More audit of audit. Some of it helped. Much of it became the thing this paper is warning about: impressive-looking process that did not ground more truth.

This matters because verification has two failure modes.

Under-verification lets fluent falsehood through.

Over-verification grinds work to a halt under audits of audits.

I lived both.

The answer is not maximal doubt. The answer is proportionality:

```text
Agents may say what they want.
Load-bearing claims terminate at substrate.
Irreversible consequences terminate at exact human approval.
Everything else moves.
```

Heavy ceremony should trigger on objective high-blast properties:

- irreversible or hard-to-reverse consequences;
- production mutation outside normal approved flow;
- credentials, secrets, auth, permissions, RLS, branch protection, CI, hooks, CODEOWNERS, runtime config, or automation;
- money or spend;
- external sends, publishing, public claims, repo visibility, or adopter-facing kit claims;
- trust infrastructure such as enforcement wiring, seeded probes, or arbiters.

Complexity alone is not a trigger. Model disagreement alone is not a trigger. A docs-only preservation note does not need a tribunal.

The method only works if it keeps work moving.

---

## 12. Below the floor, verify. Above the floor, manage risk.

The deterministic floor covers claims with observable consequences:

- did the file exist;
- did the hash match;
- did the test runner exit zero;
- did the push land;
- did the endpoint respond;
- did the server log record a request;
- did the production surface return the expected state;
- did the human approval artifact name the exact consequence.

It does not cover most of what capable agents are valuable for:

- judgment;
- design;
- prioritization;
- interpretation;
- strategy;
- synthesis;
- whether the thing built well was the thing worth building.

There is no `md5` for good architecture. There is no curl command for “the right strategic call.”

That residue is not a reason to stop using agents. It is a reason to stop calling every summary “verification.”

For claims below the deterministic floor, verify.

For claims above it, manage risk.

That means:

- staged deployment;
- blast-radius limits;
- monitoring;
- rollback;
- incident reporting;
- insurance;
- liability allocation;
- independent review where appropriate;
- domain-specific standards;
- and non-deployment where compounding execution-and-audit failure would be intolerable.

If the system cannot make a claim human-verifiable, it must make the consequence survivable.

If the consequence is not survivable, the domain is not ready for autonomous deployment.

---

## 13. What I am willing to sign

I am willing to sign this:

For checkable factual claims made by agents, I found no substitute for a small deterministic check whose consequence I could inspect myself. I did not find a system that lets me safely treat fluent verification prose as evidence.

I am also willing to sign the harder limit:

That floor is real, and it is narrow.

It does not solve agent oversight. It does not solve alignment. It does not prove production readiness. It does not make a dashboard safe. It does not make an evaluator honest. It does not make an unreadable trace human-grounded.

It gives me a place where doubt can stop.

Above that place, I am not verifying. I am managing risk.

That vocabulary change matters. Calling risk management “verification” is how theater starts.

---

## 14. What this proves, and what it does not

### What this shows

It documents two field incidents across two surfaces I operate in which an agent-produced verification claim diverged from reality.

It shows three preserved fabrications in one repo where the catches were deterministic controls, not prose review.

It shows a controlled positive in an ambiguous failure condition: a server log recorded a served 200, while the report denied any response.

It shows a discordant pair: same prompt, same lure, one honest report and one false report, separable only by substrate.

It shows that text-only checks can miss omissive fabrication; B1 passed the text battery, and the omission required joining the report against the server log.

It shows that over-verification is a real failure mode: process can become theater even when motivated by safety.

### What this does not show

It does not show a fabrication rate.

It does not generalize across all models, tasks, companies, domains, or agent frameworks.

It does not prove external adoption.

It does not prove legal, compliance, or production suitability.

It does not show that multi-agent review is worthless. It shows that agreement is not grounding.

It does not show that observability is useless. It shows that capture is not verification.

It does not show that deterministic custody solves the judgment-shaped residue above the floor.

The honest claim is narrower:

> Agent audit trails can be fluent and false.  
> Humans cannot reliably distinguish false audit prose from true audit prose by reading it.  
> Small deterministic checks can ground some claims.  
> Many valuable agent claims remain above that floor and must be treated as risk, not proof.

---

## 15. If you run agents

Use agents. They are extraordinary.

But separate work, evidence, approval, and consequence.

For each load-bearing claim, ask:

```text
What surface does this claim describe?
Who controls that surface?
What is the smallest check that exercises it?
Can I read the check?
Can I run it on inputs I choose?
Can I confirm the consequence without trusting the agent’s prose?
```

If yes, verify.

If no, say what it really is:

- judgment;
- inference;
- risk estimate;
- confidence;
- recommendation;
- unverified summary.

For irreversible actions, require exact human approval. Not “looks good.” Not “satisfied.” Not “audited.” Exact approval for the exact consequence.

For traces and dashboards, ask what they ground. If the answer is “a lot of context,” they are observability. Useful, but not verification.

For high-blast domains, ask whether the system can survive the agent being confidently wrong. If not, do not deploy autonomy there.

And when an incident happens, write the postmortem. Keep the embarrassing parts in. Especially the parts where your governance failed in the opposite direction. This is not yet a norm: Harper Foley's survey of agent-tooling incidents reports serious production failures with no corresponding vendor postmortems and no liability framework. See [Ten AI Agents Destroyed Production. Zero Postmortems.](https://www.harperfoley.com/blog/ai-agents-destroyed-production-zero-postmortems). The accountability infrastructure is still immature; postmortems are one place it starts.

---

## 16. Conclusion: the next audit trail will be prettier

The audit trail was fluent but wrong.

The next audit trail will be longer. It will have more spans, more evaluator scores, more dashboards, more summaries, more policy checks, more agents reading other agents. It will look more governed than mine did.

That will not make it true.

I do not think the answer is to make agents small enough that every useful action fits inside a human-readable check. That future is gone, and maybe it was never desirable. I also do not think the answer is to let agents write the audit trails and call the resulting dashboards oversight.

The answer I can defend is narrower and harsher:

> **Deterministic custody around probabilistic work.**
>
> Below the deterministic floor, verify. Above the floor, manage risk. Do not call the second one verification.

Verify the claims that can be verified. Bound the authority of the claims that cannot. Treat benchmarks as risk estimates, not guarantees. Treat insurance, liability, incident reporting, and staged deployment as first-class parts of deployment. Refuse autonomy where the system cannot survive the agent being confidently wrong.

The model may reason probabilistically.

The custody of evidence cannot.

The authority path cannot.

The consequence gate cannot.

That is the floor I found. It is boring, narrow, and not enough.

It is also the only part I trust.

---

## Evidence and source links

### Internal project evidence

- [Deterministic gates, README](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/README.md)
- [gates/test-gates.sh, self-test harness](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/test-gates.sh)
- [gates/check-blast-radius.sh](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/check-blast-radius.sh)
- [gates/check-secrets.sh](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/check-secrets.sh)
- [gates/check-irreversible-git.sh](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/gates/check-irreversible-git.sh)

- [Fabricated Audit Prose vs. Deterministic Checks](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/fabricated-audit-vs-deterministic-checks.md)
- [n=3 reproduction analysis](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/n3-reproduction/analysis.md)
- [Fabrication-fishing protocol v2](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/protocol-v2.md)
- [Fabrication-fishing Batch 1 analysis](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/analysis-batch-1.md)
- [Trial B1 adjudication](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/adjudication-B1.md)
- [B1 server log](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/logs/B1-server.jsonl)
- [B1 report](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/experiments/fabrication-fishing/trials/B1-report.md)
- [Merged ENVELOPE.md](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/ENVELOPE.md)
- [Verification taxonomy](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/envelope-kit-additions/verification-taxonomy.md)
- [Origin postmortem draft](https://github.com/samiserrag/Claude-Codex/blob/whitepaper-v1/docs/envelope-kit-additions/postmortem-2026-05-31-auditor-fabrication.md)

### External sources and related work

- [OpenAI Agents SDK: Tracing](https://openai.github.io/openai-agents-python/tracing/)
- [LangSmith Observability](https://docs.langchain.com/langsmith/observability)
- [TRAIL: Trace Reasoning and Agentic Issue Localization](https://arxiv.org/abs/2505.08638)
- [Reward Hacking Benchmark](https://arxiv.org/abs/2605.02964)
- [The Capability Paradox](https://arxiv.org/abs/2605.17480)
- [Secret Collusion among Generative AI Agents](https://arxiv.org/abs/2402.07510)
- [Microsoft Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit)
- [dwarvesf/claude-guardrails](https://github.com/dwarvesf/claude-guardrails)
- [Ten AI Agents Destroyed Production. Zero Postmortems.](https://www.harperfoley.com/blog/ai-agents-destroyed-production-zero-postmortems)