← Resources / Harness Engineering / Factories / The functions

What a factory must do, no matter what it runs on

Five factories taught one lesson: the machinery changes and the functions do not. A queue, a worker engine, an orchestrator library, or a bare while loop can run the machinery. This page names fourteen functions a factory must do, the reason each one matters, and the smell that says the function is missing. Three of the five ship in public (ghola, bastion, open-refinery). Two stay private and appear here unnamed.

5 factories · 14 functions · evidence: decision logs, plan files, limits files, commit history
companion anatomy: /factory — one of these factories, mechanism by mechanism
The function
What a factory must do. True of a queue, a worker engine, or one loop in one process.
The mechanism
How this stack carries the function. Swappable, priced, replaceable.
The smell
What you notice when the function is missing. The smell is the design review.
1 Keep a body

The work keeps a body outside any process

A crash is a normal event, not an incident. The job exists in a durable account. Decisions derive from that account. A crash costs one step, never one job. Three functions carry this part: one account, one derivation, one retry that resumes.

file one JSON per job row one run row msg queue payload rule state is written, not remembered

1 · One durable account of the job

One record per job holds the stage, the owner, the settings, and the history. The factory writes through it. No process holds state that matters in memory. Any process that reads the record can continue the job. The record is also the debug surface.

The hosted factory kept state in a stock store. That store restarted mid-run. A live job died with it. State then moved to one JSON file per job, and cat state/jobs/<id>.json became the debug tool.

The delivery factory made files canonical and kept the database as a rebuildable read-model. The open control plane holds one run row as the truth and keeps nothing in memory between ticks. The Rust control plane stamps a revision on each row and refuses stale writes.

queue payload carries the state db row with a revision debug read the record, not a debugger

Smell. A restart loses jobs. Debug needs an attached debugger. State lived in a process that died.

2 · Derive, do not remember

The next step is a pure function of state. One function takes the job, the graph, and the world, and returns one move. Inference sits outside it. The model call is the only non-determinism in the whole machine, and this seam is where it stays out.

In the open control plane, advance takes three dictionaries and returns a move. Tests use literals. The starter kit keeps its decisions in a package with no I/O, so 675 tests run in seconds with no engine and no spend. The delivery factory names the same shape: one pure state machine, with the model call as the one step outside it.

rule pure decision, no I/O agent one call, inside a step test literals in, move out

Smell. Stage logic smeared across handlers. Tests that need live keys. Two handlers that disagree about the next step.

3 · Retry, do not restart

Every transition is a durable message, and every step guards on the stage it expects. Delivery is at-least-once, so the guard makes the work happen once. The difference between resume and restart is the difference between one step of cost and one job of cost.

The hosted factory lost a full pipeline to one mismatch. Git pushed under one account. The pull request opened under another. The job paid for everything and failed at the last step.

The starter kit copies resolved settings onto the job at submit. A rework months later rebuilds the environment the job was born in, not today's environment. The delivery factory carries the full kit: idempotency keys, bounded retries, backoff, checkpoints, and a dead-letter path.

queue idempotency key + backoff guard step checks the stage settings copied at submit

Smell. A crash costs the whole job. Retries open duplicate pull requests. Rework runs on today's settings.

2 One owner

One owner at a time, and no immortal runs

Two writers corrupt state silently, and a stuck run spends while it waits. Ownership is claimed by construction. Dead runs are buried on a schedule, not when someone notices.

lease expiry + takeover rev stale write refused reap reaper at startup

4 · Claim before work

A unit of work has exactly one owner, and the claim says who. A lease carries an expiry. A lease that dies is taken over, not mourned. Where two writers could meet a shared record, the record itself refuses the stale one.

The starter kit gives each job one worktree plus a claim. The delivery factory reclaimed orphaned leases at startup after crashes wedged cards. Two processes once appended to one audit chain. The chain then failed its own verification with no tampering anywhere. One writer is now construction, not discipline, and sub-agents stay read-only.

worktree one per job + claim lease reclaimed at startup audit one writer, hash chain

Smell. Duplicate runs. An audit chain that fails its own checks. Cards wedged after a crash.

5 · Bury the dead runs

An unattended loop needs bounds and a reaper. Every run carries a progress bound, a budget, a wall clock, and a path out. Something reaps what stalled. Silence is the dangerous state, so the system makes it visible.

The delivery factory locks loop bounds in policy: progress, budget, wall clock, kill switch. The hosted factory met the opposite problem. A long turn emits nothing, so its log shows an elapsed time beside the words still working. The Rust control plane bounds every recovery loop by revision count. Past the bound, a person decides.

policy bounds + kill switch, locked queue dead-letter path log elapsed time on silence

Smell. A run that never ends. Spend that never stops. A zombie worktree nobody dares delete.

3 Judgment costs

Judgment is the expensive part

Inference is the only non-determinism and the only spend. Software frames it. The strong model spends once. No check reads the worker's own account of itself.

rule plain code owns order split plan once, execute cheap prove blind to the summary

6 · Sequencing is software

Plain code owns the order of steps. The agent judges inside a step. It never chooses the next step. This one boundary buys low cost, reproducibility, audit, and parallel work, because steps without hidden order partition cleanly.

The open control plane defends the boundary on exactly those grounds and holds it as a rule: never let an agent decide sequencing. The hosted factory states the split as a test. Its test reads: a number or a name is settings, a judgement is a callback, the rest is the framework.

rule the agent never picks the next step settings numbers and names callback judgement, one seam

Smell. A prompt that asks the model what to do next. Failure paths nobody can test. A run that costs what it wants.

7 · Spend the strong model once

The strong model runs one read-only plan turn. The cheap model spends the tokens. Spend ceilings sit where cost becomes countable, at stage boundaries, because a turn's cost is unknown until the turn ends.

The hosted factory plans after reading the repo, so the plan names real files. The delivery factory runs the same split. The open control plane refuses the next stage at the ceiling, not mid-call. The hosted factory also omitted the cost budget while models carried no prices. A budget that silently fails a turn is worse than none.

phase model + thinking level budget checked between stages plan a durable document on the job

Smell. One model for everything. Cost visible only on the invoice. Plans that name files that do not exist.

8 · Verify blind

Checks stack, and none of them reads the worker's account of itself. Tests gate the diff. Review reads the diff. One check drives the result like a user. Evidence must exist before any done.

The delivery factory's prove step never sees the executor's summary. A self-graded check is worthless. The hosted factory downgrades a claim with no command to unproven. The starter kit refuses model graders outright: a model grading a model is the weakest evidence available. The open control plane treats unparseable output as a fail.

The delivery factory moved a decision from prose into harness code. Evals arbitrated the move: green, strip the prose, green again. The harness then owned the decision.

tests gate the pull request prove contract only, no summary evals arbiter of safe migration

Smell. The worker grades its own work. A check that cannot fail. A done that flips on retry.

4 Rules have rungs

A rule is only as real as the mechanism that carries it

The same sentence can sit at six places: prose, grant, hook, callback, diff gate, CI. Rank rules by place, not by strictness. Gates raise decisions. Records keep them. Uncertainty becomes a question.

rung 0 prose in a charter rung 1 tool grant rung 2 repo hook rung 3 in-turn callback rung 4 diff gate rung 5 CI

9 · A rule is a mechanism, not a mood

A rule carries a rung. The rung is a place, not a strictness, and the rung is data, so promotion edits one field. Each rung sees one thing: the in-turn rung sees a call and never a diff. The diff rung sees a diff and never the call. That asymmetry is the safety property.

The starter kit calls the ladder its one idea and ships it as an extractable worker. The hosted factory mounted a rule inside its editing tools. A shell heredoc walked past it twice in testing, so the rule moved in front of every call.

An old harness version once ignored a denial, so the version is pinned and a live test asserts the denial is honored. The delivery factory's image build asserts the hook landed. One predicate per rule mounts at every rung it needs. Two implementations of one rule will disagree, and the agent finds the seam first.

test consequential, recurrent, decidable promotion edits one number live test the deny is honored

Smell. A rule everybody believes is enforced. Two implementations of one rule. A hook that silently vanished.

10 · Gates refuse, records remember

Gates raise a decision to a person before the action. Records follow the action and name the actor. A refusal is a readable result, never an exception, because a refusal the model can read is one it can work around. And nothing merges itself.

All five factories refuse to self-merge. The starter kit's dial never turns ask into allow. An unattended factory that reads ask as yes answers a question nobody asked. The hosted factory feeds the gate's own words back as the next brief, bounds revisions, and stops on a repeated identical refusal. An agent that cannot satisfy a gate twice will not satisfy it on the ninth try. The audit trails are append-only and hash-chained, and every entry names its actor.

gate refusal as a tool result merge a person, always audit append-only, hash-chained

Smell. An exception ends a turn. No record of who did what. A merge nobody approved.

11 · Ambiguity becomes a question

Uncertainty parks as a question with an inbox, not as a guess. The dial sets who answers: a person always, a person per step, or nobody on a bounded path. An answer amends the contract, so downstream checks grade the work against the requirement that now holds.

The starter kit's first live turn parked on a file read with no human to answer it. The dial now runs from manual through dark. The Rust control plane holds a tool call and parks the turn alive there. A decisions inbox lists everything held.

The hosted factory accepts only an opening interrupt line, and a resumed turn loses the hatch. An interrupt that stops a whole job discards a turn that was mid-thought.

dial manual → attended → supervised → dark inbox everything held, in one list answer amends the contract

Smell. A stalled turn nobody sees. A guess that reached the diff. ask treated as yes.

5 Stay legible

Entropy is the real failure mode

Prose rots because nothing fails because of a paragraph. Synonyms multiply. Vendors thread into the core. Lessons pile up in side docs. Legibility is maintenance: one name per thing, one seam per vendor, and lessons that use their own gate.

rule one name per thing seam adapters behind ports lane proposals, never self-apply

12 · One name per thing

Two mechanisms for one job is a defect, not a feature. So is two words for one idea. Subtract once the feature set is complete, and let the written list audit you.

The Rust control plane cut twenty-plus synonyms of rule to one word per thing. Then it deleted the built-in stages. A workflow is only its steps now. The open control plane's 3.0 program was deletion: two execution paths, two workflow engines, two authorization models.

The open control plane states it plainly: a complete feature set nobody can find is not complete. The written feature list found a route that called a function that does not exist, plus five tables no code read. The starter kit went through three framings: factory, then framework, then starter kit. The ladder and the stage graph survived every rewrite. Everything else was a worker that already existed.

glossary one word per concept audit write the feature list, read it 3.0 subtraction as a program

Smell. A setting parsed three times and read zero times. A page nobody opens. Two systems that must agree.

13 · The seam is the contract

Stock first. Own code only where no stock piece exists. One place in the tree imports the vendor, and the core knows no vendor name. The seam is a boundary you can point at, so an upgrade touches a directory, not a repository.

The starter kit composes 26 workers and writes 2. The seam is a function id, not an import, so its ladder worker swaps for an upstream one. The Rust control plane lets one directory import the SDK. A CI script fails the build on a crossed seam.

The hosted factory deleted its own turn loop, about 5,600 lines, because the framework already answered those questions. The open control plane refuses any other database engine outright. It refuses to half-work.

port protocol per connector ci seams script fails the build limit state the refusal, ship it

Smell. Vendor types in the core. A second call site for the vendor. An upgrade that forks the repository.

14 · Lessons walk the same gate

A factory that fixes itself must use its own door. Failures become proposals. Proposals travel the gated path as ordinary jobs. Nothing self-applies, and a proposal with no evidence is dropped, not repaired.

The delivery factory's charter lane once applied itself and escaped its own gate. Proposals are staged now. The starter kit's improve lane found a bug in itself on its first run, and it may propose and never apply.

The Rust control plane turns what a job teaches into a charter change or a plugin part, never a shadow copy. The hosted factory turns accepted proposals into specs that travel the same pipeline as any other work.

lane stages specs, applies nothing rule evidence or drop gate the same one as real work

Smell. Lessons in a side document. A lane that edits itself. A factory that outvotes its owner.

What matters most

Fourteen functions is a lot to hold. The five factories each answered the question differently, and their answers agree on the short list below. The mechanisms underneath differ. The functions do not.

  1. Day one A durable account, one owner, and a gate at the exit. A factory without these three is a script with a budget.
  2. The functions The mechanisms are replaceable. Pick the stack per constraint. The functions are the design.
  3. The first refusal Your first real refusal and your first parked question are features. Silence there means a function is missing.
  4. The first migration Evals arbitrate. Green, strip, green again. The harness then owns the decision.
  5. Always A report carries no more confidence than the mechanism behind it. And nothing merges itself.

The one-sentence version: build the functions, borrow the mechanisms, and never let the report outrun the rung.