Harness Engineering / One workflow, three architectures / 3 of 3 · Event sourcing

Your agent transcript is already an event log

Event sourcing says state is not stored, it is derived: the authoritative record is an append-only log of what happened, and every read model is a fold over it. Agent systems produce that log whether or not they take it seriously — which means the choice is not whether to have one, but whether to get replay, projections and audit out of it, or keep treating it as a chat history you occasionally grep.

companion repo · github.com/tacoda/iii-event-sourcing
built twice · once as iii configuration, once from scratch in stdlib Python
the workload · the same incident analysis all three repos run — extract claims, propose an explanation, score it, accept
verification · every iii command shown was executed against a live engine (0.22.0) before it was written down
tests · 8 passed, offline and deterministic
The log
Append-only, ordered by an explicit parent chain. The only authoritative record.
Projection
A read model: a pure fold over the log. Disposable, rebuildable, never authoritative.
Notification
At-least-once and unordered. A doorbell, not data — and the source of the hazard below.
0 The shape

State as a fold, not a cell

"What is the state" and "how did it get that way" stop being two questions with two different levels of support.

Nothing is updated in place. A write appends HypothesisScored{confidence: 0.86} to a stream; the current confidence is whatever you get by folding the stream. Read models are derived, disposable, and rebuildable — if a projection has a bug you fix the fold and replay, rather than migrating corrupted rows.

what changes when the log is authoritative
 State-in-placeEvent-sourced
Current valueRead a rowFold a stream (or read a projection)
HistoryWhatever you remembered to logThe system itself
Fixing a bad read modelData migrationFix the fold, replay
"Why is it like this?"ForensicsA query
StorageBounded by state sizeGrows forever
Cost of a schema changeOne migrationOld events are immutable — the fold carries every version, forever
Why this suits agents specifically

With deterministic code, replay is a debugging convenience — you could always just run it again. With a model in the loop you cannot: re-running gives you a different run. The log is the only record of what the agent actually saw and decided. That moves replay from convenience to the only reliable debugging surface you have, and it is why the unbounded-growth row above is a price worth paying here when it often isn't elsewhere.

1 On iii

An event store wearing a chat-shaped label

session-manager is documented as a conversation store. It is also, without modification, a correct event log.

A stream is a session. An event is a custom entry whose custom_type is the event type — a real discriminator slot, not something you smuggle into the payload. Ordering is an explicit parent_id chain returned on every append, so the log is a linked list rather than a timestamp-sorted bag.

appending a domain eventcustom_type is the event type
iii trigger session::append session_id=$SID \
  message='{"role":"custom",
            "custom_type":"ClaimRecorded",
            "content":[{"type":"text","text":"{\"claim\":\"one consumer attached\"}"}],
            "timestamp":1717800000000}'

{ "entry_id": "e_45d958d6…", "parent_id": "e_5e3d1086…", "timestamp": … }
event sourcing, mapped and verified
ConceptiiiStatus
Streama session; aggregate identity in metadataconfig
Event typecustom_type on the entryconfig
Orderingexplicit parent_id chainconfig
Read the logsession::messages, already in chain orderconfig
Projectionsession::message-added binding → state scopeconfig
Per-aggregate filtermetadata subset match on the bindingconfig
Fork / what-if replaysession::forkconfig
Aggregate version guardstate::compare-and-setnot in the logconfig
Optimistic concurrency on appendabsent
2 The hazard

Two writers, two success responses, one surviving event

The dangerous kind of missing feature is not the one that errors. It is the one with a convincing near-miss.

session::append has no conditional write. Pass expected_revision and it is accepted, ignored, and the append succeeds anyway — no error tells you your concurrency control does nothing:

the near-missaccepted. ignored.
$ iii trigger session::append session_id=$SID expected_revision=3 message=…
{"entry_id": "e_399c8b87…", "parent_id": "e_20ec1d48…"}   ← appended regardless

parent_id, meanwhile, is real and is validated — an unknown parent returns session/parent_not_found. So it looks exactly like the conditional-append primitive. It isn't. Two writers appending at the same parent both succeed:

the corruptionboth writers got a 200
writer A -> e_fd3ce7e1ffe6   (parent e_9da6…)   ok
writer B -> e_cad909ed44b3   (parent e_9da6…)   ok

$ iii trigger session::messages session_id=$SID
  … e_9da67bbfd6c1  {"confidence":0.86}
  … e_cad909ed44b3  {"claim":"writer B"}
                                       ← A is not here

$ iii trigger session::get-message entry_id=e_fd3ce7e1ffe6
  {"id":"e_fd3ce7e1ffe6", "message":{…"writer A"…}, "revision": 0}

A's event is durable — you can fetch it by id — but it sits on a sibling branch that session::messages does not walk. Any projection reading the ordinary way will never see it. A's writer received a success response. Nothing, anywhere, reports a problem.

The operational consequence

The version guard has to live in state, and the ordering is not negotiable: compare-and-set first, append second. Append first and you have written an event you cannot un-append — the log is append-only, which is the point, and which means a concurrency mistake is permanent. There is also no branch enumeration (session::leaves does not exist; include_branches is not honored), so recovering an orphaned entry requires already knowing its entry_id.

3 From scratch

Four lines that upstream doesn't have

Writing the missing feature by hand is the fastest way to judge how missing it really is.

scratch/eventstore.pyreal optimistic concurrency
def append(self, stream, type, payload, *, expected_version=None) -> Event:
    log = self._streams.setdefault(stream, [])
    if expected_version is not None and expected_version != len(log):
        raise Conflict(f"{stream}: expected {expected_version}, found {len(log)}")
    event = Event(seq=len(log) + 1, type=type, payload=payload, stream=stream)
    log.append(event)
    self._notify(event)
    return event

Four lines. Which is what makes the absence notable — this is not a hard feature, and its convincing near-miss is what makes it dangerous rather than merely missing.

The more valuable half of the from-scratch build is the UnreliableBus: a notification channel that is at-least-once and unordered on purpose, because that is the contract session-manager documents for itself. It exists so a test can prove the following, which is the single most transferable finding in this series:

the headline testsame events, same fold, different answers
_seed(store)                       # 4 events, appended in order
bus.drain(lambda e: (naive_handler(e), doorbell_handler(e)))

truth = Projection("truth", incident_fold).rebuild(store.read(STREAM))

assert doorbell.state == truth    # re-reads the ordered log. correct.
assert naive.state    != truth    # folds in arrival order. corrupted.
assert len(naive.state["claims"]) != len(truth["claims"])

Identical events. Identical fold function. One projection is right and one is silently wrong, and the only difference is whether it trusted the delivery order. No exception is raised in either case — the wrong one just reports a wrong number.

4 What it taught

Four findings, one of which applies everywhere

The ordering one is not about iii. It is about every event-sourced system with an asynchronous notification channel.

  • The log being ordered does not make your projections ordered. iii's log is ordered by parent chain. Its notifications are explicitly "at-least-once and unordered". Those two facts live in different documents, and the gap between them is where wrong dashboards come from. Check this in whatever you are using — the two guarantees are almost never stated in the same place.
  • Treat the notification as a doorbell, not as data. Re-read the ordered log on every notification and you are immune to reordering, duplication and gaps, for the cost of one read per event. Fold the delivered event directly and you are fast and wrong. Both implementations arrived at this independently.
  • A near-miss is worse than an absence. A missing feature makes you build something. A feature that accepts your parameter and ignores it, next to a similar parameter that validates convincingly, makes you believe you are protected. Test that your concurrency control actually rejects something.
  • Replay is worth more with a model in the loop. Forking a stream just before a model call and replaying it with a different prompt is a genuinely better debugging loop than re-running and hoping — because re-running does not reproduce the run.
5 Takeaway

You already have the log. Decide whether it counts.

The work is not producing events. It is committing to the log being authoritative, and getting the ordering discipline right.

Agent systems are event-sourced whether or not they mean to be. The transcript is an append-only record of what happened. Event sourcing is the decision to treat it as the source of truth rather than as a log file, and to derive read models from it instead of maintaining them alongside.

On iii, the store is already there and the concurrency control is not. session-manager gives you streams, typed events, chain ordering, projections via trigger bindings, and fork-for-replay. It does not give you conditional append, it accepts expected_revision and ignores it, and concurrent appends branch silently. Put the version in state, CAS before you append, and never the other way round.

Write the ordering test before you write the projection. A projection that folds in arrival order is correct in development, correct under light load, and wrong the first time anything retries. It never raises. The only way you find out is by deliberately delivering events out of order and asserting that it still matches the log.

That is the end of the series: the same incident-analysis workflow implemented over actors and supervision, a blackboard, and an event log. Three architectures, one task, two implementations each. The pattern across all three is that iii supplies more of each model than expected, and the parts it doesn't supply are consistently the ones that fail quietly rather than loudly.