AUDIT TRAIL / run 8f2c41d9 / BILL-4471 add refund endpoint merged
Ian Johnson · Harness Engineering · tacoda.dev
a worked example, rendered as the thing itself
What this page is

An audit trail answers one question from a stranger: why does this line exist? Below is a single agent run, recorded end to end, including the moment a gate refused its first attempt. Click any step to see the record it produced, and why each field is in the schema.

Nothing here comes from a real run: every name, hash, and timestamp is a stand-in. The schema in the right-hand column is the part to steal.

9steps
14tool calls
4gates fired
1blocked
1self-corrected
3m42swall clock
6files read
2files written

Run pipeline · click a step

01 · INTAKE ok

run_id8f2c41d9
parent_run_idnull (top-level)
initiated_byu/priya · human · web
ticketBILL-4471
modelclaude-sonnet-5
harness_version2.4.1
charter_shaa91f3c2
repo_sha4d7e0b8
The field everyone forgets

charter_sha. Without it, nobody can say which rules were in force, because the charter moves thirty times a year. It also joins every run to the eval history. Trace a quality regression back to the exact rule edit that preceded it.

02 · CONTEXT ok

files_read6
at_sha4d7e0b8
bytes_in41,880
retrievalgrep + explicit paths, no vector store
billing/refund.py          sha 6c1a…  read
billing/order.py           sha 91be…  read
billing/money.py           sha 0f4d…  read
tests/test_refund.py       sha aa20…  read
CHARTER.md                 sha a91f…  read
.claude/rules/money.md     sha 77c9…  read
Record references, not contents

Store path plus content hash plus the repo sha. Never the file body. Bodies bloat the trail, duplicate your source of truth, and turn a log store into a place secrets live forever. The hash proves what the agent read, and the repo already has the text.

03 · PLAN ok

prompt_shad3f9a11
tokens_in / out12,404 / 806
alternatives2 considered, 1 rejected
CHOSEN   extend refund_for_order(), branch on order.status
REJECTED new refund_cancelled_order() entry point
         reason: duplicates the discount reconciliation
                 block; charter §consistency
Log the road not taken

The rejected alternative is the highest-value row in the trail, and almost no system records it. That row is what makes review possible six months later, and an engineer needs it as evidence of judgment. Capture it at plan time, and the run produces that evidence instead of an engineer remembering it.

04 · EDIT ok

tooledit_file
targetbilling/refund.py
pre_sha6c1a2e0
post_shab30f9c4
lines+22 −3
Pre and post, always both

Recording only the result makes concurrent edits unresolvable. With the pre-image hash you can prove the agent edited the file it thought it was editing. That failure appears the moment two runs touch one repo.

05 · GATE money_is_integer blocked

rule_idmoney.integer-cents
defined_in.claude/rules/money.md:12
altitudegate (refuses, does not warn)
verdictBLOCK
matchedbilling/refund.py:48
surfaced_to_agenttrue
✗ money.integer-cents
  billing/refund.py:48
      - refund_total = order.total * (1 - discount_pct / 100)
                                              ^^^^^^^^^^^^^^^^^
  float division on a money path. Charter §money:
  "money is integer cents everywhere; percentages resolve
  via Decimal, then round half-up to cents."
The row the trail exists for

A gate that blocks without a record looks, from the outside, exactly like a gate that never fired. Six months later nobody can tell whether that rule is load-bearing or dead weight. Deleting dead rules is R4's job, and the record of every firing is how R4 finds them.

Record every firing, including the ones that pass. The pass count tells you a rule still earns its place. Its block count tells you the rule is doing work today.

06 · REVISE retry 1 of 2

triggergate block · money.integer-cents
retry_index1
retry_budget2 (then escalate to human)
post_shae77a015
agent received the gate output verbatim and revised:

  + pct = Decimal(discount_pct) / Decimal(100)
  + refund_total = round_half_up(order.total_cents * (1 - pct))
Bound the retries, and record the bound

Two attempts, then a human. An unbounded correction loop burns budget and talks itself into satisfying the gate, not the problem. Record the budget in the trail, so "why did this escalate" has an answer that is not somebody's memory.

07 · GATE re-run pass

money.integer-centsPASS
scope.allowed-pathsPASS 2 files, both in billing/
test.must-fail-when-revertedPASS
secrets.no-literalsPASS
Passes are data too

Four rules ran, and the trail records all four. A trail that records only failures cannot answer the most useful maintenance question. Which of our rules has not blocked anything in twelve months?

08 · VERIFY ok

suitepytest · 412 passed · 0 failed
new_testtests/test_refund.py::test_cancelled_with_discount
revert_checktest goes red when refund.py is reverted
duration64.2s
Record the falsifier, not just the green

"412 passed" is the non-claim. The line that matters is the revert check: proof the new test can fail. Without it the trail records a green pipeline, which has never meant the code is correct.

09 · PROPOSE merged

pr#2291
diff_shae77a015
human_revieweru/dev-marcus
merged_at2026-08-14T11:22:07Z
trail_url/audit/8f2c41d9
Put the trail link in the pull request

A trail nobody can find is a trail nobody uses. One line in the PR body, pointing at this view, is enough. That link turns the store from compliance furniture into something a reviewer opens when a diff looks odd.

THE RECORD · one row per step

fieldwhy it is there
run_idJoins everything. Generate before the first token.
parent_run_idSubagents. Without it a fan-out is unreconstructable.
initiated_byA human is accountable for every run. Name them.
modelExact id. "the model" is not a value.
charter_shaWhich rules were in force. The most-forgotten field.
repo_shaWhat the world looked like going in.
prompt_shaHash, not text. Proves sameness without storing it.
context_refspath + content hash. Never bodies.
tool_calls[]name, args (redacted), result hash, ms.
gates[]rule_id, verdict, matched location. Passes included.
alternatives[]Which option the agent rejected, and why. The rarest, most valuable.
retry_indexWith the budget, so escalations explain themselves.
pre_sha / post_shaBoth. One of them is useless alone.
outcomepr number, merge sha, or the reason it stopped.
Never write these

Raw prompts with customer data, and tool arguments carrying credentials or personal data. Never whole file bodies, and never full model outputs. Hash or redact at the point of capture, not in a cleanup job later. An audit store is append-only by design, which means a mistake in it is permanent by design.

Three readers, three needs
  • Tomorrow · the debugger. Wants tool args and error strings. Short retention, high detail.
  • Next week · the reviewer. Wants alternatives and gate verdicts. This is the one most systems fail.
  • Next year · the auditor. Wants actor, charter version, and immutability. Low detail, long retention.

Different retention per tier is what keeps the store affordable: 30 days for tool detail, forever for the step summary.

Changes produced by this run

billing/refund.py+22 −3 · post e77a015
@@ -44,9 +44,12 @@ def refund_for_order(order, reason):
if order.status == Status.CANCELLED:
- refund_total = order.total * (1 - discount_pct / 100)
+ pct = Decimal(discount_pct) / Decimal(100)
+ refund_total = round_half_up(order.total_cents * (1 - pct))
return Refund(cents=refund_total, reason=reason)
tests/test_refund.py+18 −0 · post 9a1c447
@@ -0,0 +1,18 @@
+def test_cancelled_with_discount_refunds_full_paid_amount():
+ order = make_order(total_cents=10_000, discount_pct=15)
+ order.cancel()
+ assert refund_for_order(order, "cancelled").cents == 8_500

Event log

11:18:02.4run.start8f2c41d9actor=u/priya ticket=BILL-4471 charter=a91f3c2
11:18:02.5context.read8f2c41d96 files, 41880 bytes, at repo_sha=4d7e0b8
11:18:04.6plan.alternative8f2c41d9rejected=new_entry_point reason=duplicates_reconciliation
11:18:15.9tool.edit_file8f2c41d9billing/refund.py pre=6c1a2e0 post=b30f9c4 +22-3
11:19:03.8gate.block8f2c41d9rule=money.integer-cents at=billing/refund.py:48
11:19:03.9agent.notified8f2c41d9gate output surfaced verbatim, retry 1/2
11:19:35.1tool.edit_file8f2c41d9billing/refund.py pre=b30f9c4 post=e77a015 +3-2
11:19:35.2gate.pass8f2c41d9money.integer-cents scope.allowed-paths secrets.no-literals
11:20:39.4verify.revert8f2c41d9new test goes red on revert : falsifier confirmed
11:20:42.2run.propose8f2c41d9pr=#2291 diff_sha=e77a015
11:22:07.0human.merge8f2c41d9reviewer=u/dev-marcus

The replay test

One question, run quarterly

Pick a merged change from three months ago at random. Using only the trail, answer: why does this line look like this? If you name the rejected alternative, the rule that constrained it, and the accountable human, you have an audit trail. If you can only see what happened and in what order, you have logs.

Fails the testlogs only

Timestamps, tool names, token counts. Answers "what happened". Cannot answer "why this and not the other thing".

Partialno charter version

Gates recorded but not which charter produced them. You know a rule fired; you cannot reconstruct the ruleset it belonged to.

Passesaudit trail

Alternatives, gate verdicts including passes, charter sha, accountable human. Reconstructable by a stranger.

WHERE TO START

Do not build a store. Append one JSON line per run to a file, with eight fields:

{"run_id","initiated_by","model","charter_sha",
 "repo_sha","gates","alternatives","outcome"}

That is an afternoon, it fits in the repo, and it passes the replay test. Everything on this page is what you add after the cheap version has proved somebody actually opens it.

The trap

Teams build the dashboard first and the schema second. The view is beautiful, and the data under it cannot answer the only question that matters. The schema is the product, and this page is only a way of looking at it.