Harness Engineering / Building a harness with pi / A coding agent with the coding removed

The harness was already there. Take the coding out.

A terminal coding agent is a loop, a tool registry, a context assembler, a permission layer, and a renderer. Only one of those has anything to do with code. Strip the builtin tools, replace the system prompt, register tools for a different domain, and you have a harness for something else entirely. This is that move carried all the way to a URL: a read-only SQL analytics service for people who cannot write SQL, built on pi's extension surfaces with no fork and no from-scratch agent loop.

companion repo · github.com/tacoda/pi-harness-sql-analytics
substrate · pi · six extension surfaces, zero forks · ANTHROPIC_API_KEY
the harness · plain-English question in, one sentence and a number out, every query audited
the belts · Docker (read-only rootfs, dropped caps) · --no-builtin-tools · read-only SQLite · SELECT-only guard
substitutions · SQLite = your read replica · analytics.example.com = wherever you put it
Instruction
What the model reads: the replacement system prompt, the skill, the prompt template.
Refused
What the harness will not do: dropped builtin tools, the SELECT guard, the container's dropped capabilities.
Recorded
What survives the session: the audit log and the findings file, both outside the container's control.
0 The premise

What is actually coding-specific about a coding agent?

Almost nothing. Take the inventory honestly and the coding is a thin layer of tools and a system prompt sitting on top of machinery that does not care what domain it is in.

a terminal coding agent, part by part
PartCoding-specific?What it actually is
The agent loopNoSample, call tools, feed results back, repeat until done
Context assemblyNoSystem prompt, project files, skills, history, compaction
Permission layerNoWhich tools may run, and what has to be confirmed
Session and rendererNoA terminal UI, a transcript, a resumable tree
Builtin toolsYesread, write, edit, bash, glob, grep
System promptYes"You are a coding agent working in a repository"

Two rows are domain-specific and the rest is general infrastructure that took real work to build. The interesting question is not "how do I write an agent loop" but "why would I write a second one". If a coding agent exposes its extension surfaces properly, the fastest path to a non-coding harness is to remove the two coding-specific parts and supply your own.

The harness this builds

The user is a support lead, an ops person, or a PM. They have a question in plain English, "how many accounts churned in June?", and no SQL. The harness inspects the schema, writes one guarded read-only SELECT, runs it against a replica, sanity-checks the result, and answers in a sentence with the number in it. Every question and every query that ran is written to an audit log.

the whole user experienceno SQL involved
$ ./bin/pi-sql -- "how many accounts churned in June 2026?"

→ list_tables()          accounts, subscriptions, plans, events
→ describe_table(events)  id, account_id, kind, occurred_at
→ run_query(SELECT COUNT(DISTINCT account_id) FROM events
             WHERE kind='churn' AND occurred_at >= '2026-06-01' …)

142 accounts churned in June 2026.
Why this shape is worth knowing

Every organization has a queue of "could an AI just answer this" questions that are not coding questions: analytics, triage, log spelunking, document lookup. The instinct is to build a bespoke agent for each. A coding agent you have already paid for, adopted, and learned to operate is usually the better substrate, and the work is subtraction rather than construction.

1 Strip & replace

Remove the coding, then say what the job is instead

One flag deletes the coding half. One file supplies the replacement. These two moves are the entire conversion, and everything after them is refinement.

flag --no-builtin-tools strategy A .pi/SYSTEM.md context AGENTS.md

--no-builtin-tools drops read, write, edit, and bash at the pi layer. This is not cosmetic. It means the agent has no general-purpose way to touch the filesystem or shell out, so the only verbs it has are the ones you register. An agent whose entire vocabulary is four domain tools cannot wander, because there is nowhere to wander to.

The replacement system prompt lives in .pi/SYSTEM.md and pi loads it from the project. Note what the launcher therefore does not need to do:

bin/pi-sqlthe prompt comes from the repo
# No --system-prompt here: pi loads the replacement prompt from
# .pi/SYSTEM.md on the mounted repo (strategy A).

That comment is doing more work than it looks like. Because the prompt is a file in the project rather than a flag in a launcher, it is reviewable, diffable, and versioned with the tools it describes. A system prompt passed as a command-line argument is a system prompt nobody reviews.

The general move

Subtract the domain, then declare the new one. If your substrate has no equivalent of --no-builtin-tools, that is a strong signal it will fight you: a harness you cannot narrow is a harness whose blast radius you do not control.

2 Domain tools

Four verbs, and nothing else

With the builtins gone, the tool registry is the harness's capability surface. Four tools is the whole vocabulary, and each one is narrow on purpose.

strategy E .pi/extensions/sql-analytics.ts command /ask
  1. list_tables Tables plus row counts. The agent has no ls, so this is how it finds out what exists at all.
  2. describe_table Columns and types for one table, and only for a name that appears in the live schema. Introspection without a path to arbitrary interpolation.
  3. run_query A single SELECT, capped in rows and bytes, audited on every call. The one tool with real power, and therefore the one with the guard on it (rung 3).
  4. save_finding Append a record to sql-output/findings.jsonl. The only write in the whole harness, and it writes to a mounted output directory rather than anywhere the data lives.

Registration is ordinary and that is the point. pi.registerTool five times and pi.registerCommand("ask", …) once, in a single TypeScript file. There is no agent loop in that file, no retry logic, no context management, no renderer. All of that is inherited.

The design test for a tool set

Ask what the agent could do that you have not thought about. With four narrow tools the answer is genuinely short, and you can enumerate it. With bash in the registry the answer is "anything the container can do", and you are back to relying on isolation for everything. Narrow tools are cheaper than sandboxes and easier to reason about.

3 The guard

Three belts, because network egress is not optional

A file-only harness can be air-gapped with --network none. This one cannot: the model API is on the network. So the containment has to live at the data layer instead, and it is layered so that no single mistake exposes a write.

strategy D tool gate belt 1 readOnly: true belt 2 checkReadOnly() belt 3 schema whitelist
  1. Belt 1 The connection. node:sqlite opened with readOnly: true. SQLite rejects every write at the engine level. This is the belt that holds even if the other two have a bug.
  2. Belt 2 The statement guard. A single SELECT or WITH only. Defense in depth, and it gives the model a clean reason string instead of a raw SQLite error it will flail against.
  3. Belt 3 The whitelist. Table names are checked against the live schema before they are ever interpolated, so schema introspection cannot be turned into a string-building primitive.

The guard is worth reading closely, because the interesting part is not the keyword list. It is the scrubbing that happens first.

.pi/extensions/sql-analytics.tsdefense in depth
function scrub(sql: string): string {
	// Neutralize comments and string/identifier literals so keywords or
	// semicolons *inside* them don't trip the checks below.
	return sql
		.replace(/--[^\n]*/g, " ")
		.replace(/\/\*[\s\S]*?\*\//g, " ")
		.replace(/'(?:''|[^'])*'/g, "''")
		.replace(/"(?:""|[^"])*"/g, '""');
}

function checkReadOnly(sql: string): { ok: boolean; reason?: string } {
	const raw = sql.trim().replace(/;+\s*$/, "");   // allow one trailing ';'
	if (!raw) return { ok: false, reason: "empty query" };
	const scrubbed = scrub(raw);
	if (scrubbed.includes(";"))
		return { ok: false, reason: "multiple statements are not allowed" };
	const head = scrubbed.replace(/^[\s(]+/, "").toUpperCase();
	if (!/^(SELECT|WITH)\b/.test(head))
		return { ok: false, reason: "only SELECT / WITH queries are allowed" };
	if (FORBIDDEN.test(scrubbed.toUpperCase()))
		return { ok: false, reason: "query contains a forbidden keyword" };
	return { ok: true };
}

Without scrub, a legitimate query containing the word "update" inside a string literal gets rejected, and a semicolon inside a literal reads as statement stacking. A guard that produces false refusals on valid work is a guard the operator eventually removes, so the scrubbing is not a nicety. It is what keeps the belt in place.

The guard has a runnable spec

The rules are duplicated as plain JavaScript in docker/sql-guard.test.mjs. The source carries a comment saying so, and saying that changing one means changing the other. That duplication is a deliberate trade: a security rule with an executable spec you can run in the container is worth more than a rule with a single definition and no test.

Match the belt to the threat

The reflex answer for agent isolation is the network sandbox. It does not apply here, because the harness must reach the model API. Naming the belts honestly, and knowing which one is load-bearing (the read-only connection) rather than assuming the container is doing the work, is the difference between security and the feeling of it.

4 Teach the loop

A skill for the method, a template for the kickoff

Tools give the agent verbs. They do not give it a method. The answer loop is a skill the model loads when the question arrives, and /ask is the one-line entry point a non-engineer actually types.

strategy C skills/analytics-playbook/SKILL.md strategy B prompts/ask.md

Skills load on demand, so the method costs nothing on turns that do not need it. The description is the trigger, and it is written as a set of conditions rather than a topic:

.pi/skills/analytics-playbook/SKILL.mdloaded on demand
name: analytics-playbook
description: Plain-English-to-SQL answer loop (understand the question →
  inspect schema with list_tables/describe_table → write one guarded SELECT →
  run_query capped → sanity-check → answer in one sentence and save_finding).
  Load when the user asks a data/business question, asks for a
  count/total/breakdown, or invokes `/ask`.

Two details in that description are worth stealing. The loop is spelled out as an arrow chain, so the model has the whole method before it reads the body. And the load condition names three concrete triggers rather than saying "load when relevant", which is the instruction that produces skills that never load or always do.

Where the method lives, and why not in the system prompt

  1. System prompt Who the agent is and what it must never do. Always resident, so it stays short.
  2. Skill How to do the job, in detail, including the sanity-check step. Loaded only when a data question arrives.
  3. Prompt template /ask. The user-facing surface. It exists so a non-engineer has one thing to type and does not have to phrase a good agent prompt.
The step people leave out

The playbook's loop includes sanity-check before answer. A query that runs is not a query that answers the question asked, and the failure mode of this harness is not a crash: it is a confident sentence containing the wrong number. That step is cheap, and it is the only thing standing between a fluent answer and a wrong one.

5 Containerize

One command, and an outer belt

A harness a colleague cannot start is a demo. The launcher collapses build, seed, and run into one command, and the container is the belt that sits outside everything the extension does.

strategy F bin/pi-sql outer belt docker/
bin/pi-sql · the container flagsouter belt
docker_args=(
	run --rm -i
	--read-only                       # rootfs is immutable
	--tmpfs /tmp:rw,size=64m           # the only scratch space
	--cap-drop ALL
	--security-opt no-new-privileges
	-v "$here":/work
	-v "$db_path":/data/analytics.db:ro   # the one host artifact, read-only
	-e ANTHROPIC_API_KEY
	…runs as a non-root user
)

Read the mount list rather than the flags. Exactly one host artifact is visible to the harness, mounted read-only, and it is a database file. Not a home directory, not a source tree, not an SSH agent socket. Most of what makes a container safe is what you decline to mount into it.

Ergonomics that decide whether it gets used

  • Builds on first run. No separate build step to forget.
  • Seeds the sample database on first run, as the host user, so the resulting file is owned by you and can then be mounted read-only.
  • Allocates a TTY only when stdout is one, which is what lets the same launcher work interactively and in CI.
  • Takes a different database as an argument, so pointing it at a real replica is a path, not a rebuild.
The launcher is part of the harness

Every one of those bullets is a reason someone does not fall back to running the thing unguarded on their laptop "just this once". Convenience is a security property when the alternative to your safe path is a fast unsafe one.

6 Front door

Put a URL in front of it

The user is not an engineer, so a terminal on their machine was never the delivery mechanism. Two adapters over the same harness process, behind TLS and basic auth.

deploy compose.yml ws ttyd rest shell2http edge nginx · TLS · basic auth
  1. WebSocket A live terminal in the browser. For the person who wants to ask follow-up questions and watch the agent work. ttyd in front of the same launcher.
  2. REST One question in, a text answer out, no session. For scripts, dashboards, and the Slack bot somebody will inevitably build. shell2http in front of the one-shot mode.
  3. Edge nginx terminates TLS, enforces basic auth, and routes both paths. The harness itself has no idea authentication exists, which is the correct division.

The one-shot mode built for CI in rung 5 is what makes the REST adapter a configuration file rather than a service rewrite. That was not planned in advance; it is what happens when the launcher is written to work when stdout is not a terminal.

The six surfaces, end to end

every piece, and the pi surface it uses
Piecepi strategyFile
Replacement system promptA · repo context.pi/SYSTEM.md, AGENTS.md
Kickoff commandB · prompt template.pi/prompts/ask.md
The answer loopC · skill.pi/skills/analytics-playbook/SKILL.md
SELECT guard and audit logD · tool gate.pi/extensions/sql-analytics.ts
Domain tools and /askE · custom tools/commands.pi/extensions/sql-analytics.ts
Single-command launchF · launch profilebin/pi-sql
Isolationoutside pidocker/
Service front dooroutside pideploy/

Where this sits

  1. Guardrails Five guardrails constrains a stock coding agent doing coding work. Same substrate, unchanged job.
  2. This page Same class of substrate, different job. The extension surfaces are used to change what the agent is for, not to fence in what it already does.
  3. Neighbours Promotion to the harness builds an agent loop from parts instead of inheriting one. Building a harness with iii is the other direction again: many small durable nodes rather than one reshaped agent.
The one-sentence version

If a coding agent exposes a system prompt, a tool registry, a way to drop the builtins, and a launch profile, it is a general agent harness wearing a coding costume. Take the costume off before you write a new one.