AUDIT TRAIL / run 8f2c41d9 / BILL-4471 add refund endpoint merged
Ian Johnson · Harness Engineering · tacoda.dev
a worked example, rendered as the thing itself
What this page is

An audit trail exists to answer one question, asked by someone who was not there: why does this line exist? Below is a single agent run, recorded end to end, including the moment a gate refused its first attempt. Click any step to see what was written down at that point, and why that field is in the schema.

Everything here is fabricated to be representative. The schema in the right-hand column is the part to steal.

9steps
14tool calls
4gates fired
1blocked
1self-corrected
3m42swall clock
6files read
2files written

Run pipeline · click a step

01 · INTAKE ok

run_id8f2c41d9
parent_run_idnull (top-level)
initiated_byu/priya · human · web
ticketBILL-4471
modelclaude-sonnet-5
harness_version2.4.1
charter_shaa91f3c2
repo_sha4d7e0b8
The field everyone forgets

charter_sha. Without it you cannot answer "which rules were in force when this ran", and a year from now the charter will have moved thirty times. It also makes every run joinable against the eval history, so a regression in output quality can be traced to the exact rule edit that preceded it.

02 · CONTEXT ok

files_read6
at_sha4d7e0b8
bytes_in41,880
retrievalgrep + explicit paths, no vector store
billing/refund.py          sha 6c1a…  read
billing/order.py           sha 91be…  read
billing/money.py           sha 0f4d…  read
tests/test_refund.py       sha aa20…  read
CHARTER.md                 sha a91f…  read
.claude/rules/money.md     sha 77c9…  read
Record references, not contents

Store path plus content hash plus the repo sha. Never the file body. Bodies make the trail enormous, duplicate your source of truth, and turn a log store into somewhere secrets go to live forever. The hash is enough to prove what was read, and the repo already has the text.

03 · PLAN ok

prompt_shad3f9a11
tokens_in / out12,404 / 806
alternatives2 considered, 1 rejected
CHOSEN   extend refund_for_order(), branch on order.status
REJECTED new refund_cancelled_order() entry point
         reason: duplicates the discount reconciliation
                 block; charter §consistency
Log the road not taken

The rejected alternative is the highest-value row in the whole trail and the one almost no system records. It is what makes a review possible six months later, and it is the same artifact an engineer needs for evidence of judgment. Capture it here and it is produced automatically instead of remembered.

04 · EDIT ok

tooledit_file
targetbilling/refund.py
pre_sha6c1a2e0
post_shab30f9c4
lines+22 −3
Pre and post, always both

Recording only the result makes concurrent edits unresolvable. With the pre-image hash you can prove the agent edited the file it thought it was editing, which is the failure that appears the moment two runs touch one repo.

05 · GATE money_is_integer blocked

rule_idmoney.integer-cents
defined_in.claude/rules/money.md:12
altitudegate (refuses, does not warn)
verdictBLOCK
matchedbilling/refund.py:48
surfaced_to_agenttrue
✗ money.integer-cents
  billing/refund.py:48
      - refund_total = order.total * (1 - discount_pct / 100)
                                              ^^^^^^^^^^^^^^^^^
  float division on a money path. Charter §money:
  "money is integer cents everywhere; percentages resolve
  via Decimal, then round half-up to cents."
This is the row the whole trail exists for

A gate that blocks and is never recorded looks, from the outside, exactly like a gate that never fired. Six months later nobody can tell whether the rule is load-bearing or dead weight, and dead rules are what R4 is supposed to delete.

Record every firing, including the ones that pass. The pass count is what tells you a rule still earns its place; the block count is what tells you it is doing work today.

06 · REVISE retry 1 of 2

triggergate block · money.integer-cents
retry_index1
retry_budget2 (then escalate to human)
post_shae77a015
agent received the gate output verbatim and revised:

  + pct = Decimal(discount_pct) / Decimal(100)
  + refund_total = round_half_up(order.total_cents * (1 - pct))
Bound the retries, and record the bound

An unbounded correction loop burns budget and eventually talks itself into satisfying the gate rather than solving the problem. Two attempts, then a human. The budget belongs in the trail so that "why did this escalate" has an answer that is not somebody's memory.

07 · GATE re-run pass

money.integer-centsPASS
scope.allowed-pathsPASS 2 files, both in billing/
test.must-fail-when-revertedPASS
secrets.no-literalsPASS
Passes are data too

Four rules ran, four are written down. A trail that records only failures cannot answer the most useful maintenance question there is: which of our rules has not blocked anything in twelve months?

08 · VERIFY ok

suitepytest · 412 passed · 0 failed
new_testtests/test_refund.py::test_cancelled_with_discount
revert_checktest goes red when refund.py is reverted
duration64.2s
Record the falsifier, not just the green

"412 passed" is the non-claim. The line that matters is the revert check: proof the new test can fail. Without it the trail records a green pipeline, which has never meant the code is correct.

09 · PROPOSE merged

pr#2291
diff_shae77a015
human_revieweru/dev-marcus
merged_at2026-08-14T11:22:07Z
trail_url/audit/8f2c41d9
Put the trail link in the pull request

A trail nobody can find is a trail nobody uses. One line in the PR body, pointing at this view, is what converts the store from compliance furniture into the thing a reviewer opens when a diff looks odd.

THE RECORD · one row per step

fieldwhy it is there
run_idJoins everything. Generate before the first token.
parent_run_idSubagents. Without it a fan-out is unreconstructable.
initiated_byA human is accountable for every run. Name them.
modelExact id. "the model" is not a value.
charter_shaWhich rules were in force. The most-forgotten field.
repo_shaWhat the world looked like going in.
prompt_shaHash, not text. Proves sameness without storing it.
context_refspath + content hash. Never bodies.
tool_calls[]name, args (redacted), result hash, ms.
gates[]rule_id, verdict, matched location. Passes included.
alternatives[]What was rejected and why. The rarest, most valuable.
retry_indexWith the budget, so escalations explain themselves.
pre_sha / post_shaBoth. One of them is useless alone.
outcomepr number, merge sha, or the reason it stopped.
Never write these

Raw prompts containing customer data, tool arguments carrying credentials or personal information, whole file bodies, and full model outputs. Hash or redact at the point of capture, not in a cleanup job later. An audit store is append-only by design, which means a mistake in it is permanent by design.

Three readers, three needs
  • Tomorrow · the debugger. Wants tool args and error strings. Short retention, high detail.
  • Next week · the reviewer. Wants alternatives and gate verdicts. This is the one most systems fail.
  • Next year · the auditor. Wants actor, charter version, and immutability. Low detail, long retention.

Different retention per tier is what keeps the store affordable: 30 days for tool detail, forever for the step summary.

Changes produced by this run

billing/refund.py+22 −3 · post e77a015
@@ -44,9 +44,12 @@ def refund_for_order(order, reason):
if order.status == Status.CANCELLED:
- refund_total = order.total * (1 - discount_pct / 100)
+ pct = Decimal(discount_pct) / Decimal(100)
+ refund_total = round_half_up(order.total_cents * (1 - pct))
return Refund(cents=refund_total, reason=reason)
tests/test_refund.py+18 −0 · post 9a1c447
@@ -0,0 +1,18 @@
+def test_cancelled_with_discount_refunds_full_paid_amount():
+ order = make_order(total_cents=10_000, discount_pct=15)
+ order.cancel()
+ assert refund_for_order(order, "cancelled").cents == 8_500

Event log

11:18:02.4run.start8f2c41d9actor=u/priya ticket=BILL-4471 charter=a91f3c2
11:18:02.5context.read8f2c41d96 files, 41880 bytes, at repo_sha=4d7e0b8
11:18:04.6plan.alternative8f2c41d9rejected=new_entry_point reason=duplicates_reconciliation
11:18:15.9tool.edit_file8f2c41d9billing/refund.py pre=6c1a2e0 post=b30f9c4 +22-3
11:19:03.8gate.block8f2c41d9rule=money.integer-cents at=billing/refund.py:48
11:19:03.9agent.notified8f2c41d9gate output surfaced verbatim, retry 1/2
11:19:35.1tool.edit_file8f2c41d9billing/refund.py pre=b30f9c4 post=e77a015 +3-2
11:19:35.2gate.pass8f2c41d9money.integer-cents scope.allowed-paths secrets.no-literals
11:20:39.4verify.revert8f2c41d9new test goes red on revert — falsifier confirmed
11:20:42.2run.propose8f2c41d9pr=#2291 diff_sha=e77a015
11:22:07.0human.merge8f2c41d9reviewer=u/dev-marcus

The replay test

One question, run quarterly

Pick a merged change from three months ago at random. Using only the trail, answer: why does this line look like this? If you can name the alternative that was rejected, the rule that constrained it, and the human who is accountable, you have an audit trail. If you can only see what happened and in what order, you have logs.

Fails the testlogs only

Timestamps, tool names, token counts. Answers "what happened". Cannot answer "why this and not the other thing".

Partialno charter version

Gates recorded but not which charter produced them. You know a rule fired; you cannot reconstruct the ruleset it belonged to.

Passesaudit trail

Alternatives, gate verdicts including passes, charter sha, accountable human. Reconstructable by a stranger.

WHERE TO START

Do not build a store. Append one JSON line per run to a file, with eight fields:

{"run_id","initiated_by","model","charter_sha",
 "repo_sha","gates","alternatives","outcome"}

That is an afternoon, it fits in the repo, and it passes the replay test. Everything on this page is what you add after the cheap version has proved somebody actually opens it.

The trap

Teams build the dashboard first and the schema second, and end up with a beautiful view over data that cannot answer the only question that matters. The schema is the product. This page is just a way of looking at it.