Proof or It Didn't Ship: Running AI Agents Like a Team

AI coding agents have a trust problem, and it's earned: they are brilliant at producing code that looks finished. The fix isn't better prompts — it's the same fix engineering teams invented for humans: evidence. Here is the discipline we run across our products, where most of the code is written by agents and all of it has to be true.

Every PR ships with recorded proof

In the Gema repo, pull requests carry a docs/pr-<n>-proof/ folder of short recordings named for the exact behavior they demonstrate: offline-add-no-duplicate.gif, stale-delete-reconnect.gif, two-user-reconnect-delete-guard.webm — that last one captured from two real clients reconnecting concurrently. Even the fun stuff gets proven: add-salad-fires-celebration.gif sits next to add-bread-no-celebration.gif, because a delight feature that fires on the wrong item is a bug like any other.

The rule is blunt: a feature counts as done when there's a recording of it working. An agent claiming “the reconnect logic handles stale deletes now” is a hypothesis. A GIF of two sessions doing it is a fact. Proof media lives inside the PR so reviewers see it in the diff, and a workflow strips it from main after merge — evidence is for review, not for the repo's waistline.

Audits are agents too — in parallel, with numbers

When we needed to know how far the native app trailed the web app, we didn't guess: three audit agents ran in parallel — one over the git history delta, one deep in chat features, one in core features — and their reports consolidated into a master parity table: 70+ features audited, 45+ at parity, 12 missing, 10 partial, each gap sized and prioritized. The punchline for planning: the small independent gaps were labelled “ideal for parallel worktree agents” — the audit output feeds straight back into agent work assignments.

The same pattern scales down: our “rule of five” review runs five reviewers over one change, each with a different lens. Redundancy catches what a single reviewer — human or model — reliably misses.

Agents get standing orders, not just prompts

Repos that agents work in carry an agent-tips doc — “hard-won lessons, read before touching code.” Real entries: never edit the migrations directory (edit the schema; the pipeline generates migrations); resolve conflicts by merging main in, never rebasing, because branch history matters to the compounding-PR workflow; only worry about type errors in files you touched. Every lesson an agent learns the hard way gets written down so the next agent doesn't. It's onboarding documentation — for a workforce that forgets everything between sessions.

Parallelism has rules too: agents work in separate git worktrees — one agent per lane, ever — so five parallel implementations never trample each other, and each lane's PR gets its own preview deployment to prove itself against.

Coverage is a matrix, not a feeling

For spectralChat we defined QA as a spreadsheet the harness must eventually drive: 16 surfaces × 15 personas, synthesized into 66 flows, each in five variants — happy path, boundary, RBAC-negative, concurrent/race, and failure-injection. Every row starts not-built: a written 0% baseline. That's the point. You can't ask agents to close a gap you haven't measured, and “the tests pass” means nothing if nobody wrote down which realities the tests represent.

At the top: an orchestrator that never writes code

For the largest efforts we run Gas Town (built on Steve Yegge's framework): work is decomposed into git-backed issues (“beads”), implemented by disposable worker agents, gated by read-only reviewers, merged by a dedicated role — coordinated by a “Mayor” agent whose defining property is that it orchestrates and never implements. The separation is the lesson, and it's as old as software teams: the one who plans the work shouldn't grade it, and the one who grades it shouldn't have written it.

The pattern under all of it

• Claims are cheap; artifacts are cheap to check. Demand artifacts.
• Write standing orders. Agents don't remember, but repos do — move every lesson from the conversation into the repo.
• Measure before you delegate: parity tables and coverage matrices turn “help me” into assignable work.
• Separate powers: implementers, verifiers and orchestrators should be different agents.

None of this slows the agents down. It's what lets you go faster — because the day you stop reading every line an agent writes (and that day comes quickly), the evidence system is what's still telling you the truth.