Harness engineering: how agent-first repositories actually work

What “harness” means (and what it is not)

In robotics, a harness routes power and signals so a motor does useful work without melting its own wiring. In agent-first software, the harness is everything around the model: sandboxed execution, test gates, documentation maps, linters, observability hooks, and review loops. The model proposes diffs; the harness decides whether those diffs are allowed to merge.

This is a role shift for engineers. Lopopolo's team describes humans moving from typing implementations to designing environments, specifying intent, and building feedback loops. You prompt a task, the agent opens a pull request, and humans intervene when the harness is missing something — a tool, a guardrail, a doc the agent could not see. The mantra repeated across the industry in early 2026: what the agent cannot see does not exist. Slack decisions, Confluence architecture notes, and “everyone knows” tribal knowledge are invisible unless you encode them in version-controlled artifacts the agent can query.

Harness engineering is therefore not “better prompts.” It is infrastructure. Jane Street's parallel story — building AIDE and Bonsai_term so agents iterate in fast, text-native terminal surfaces — arrives from a different stack but the same principle: own the environment, not just the model vendor.

The OpenAI experiment in plain numbers

The Harness team set an intentional constraint: ship a real internal beta product using Codex agents exclusively. The result was not a demo repo. It had daily internal users and external alpha testers. Scope included application logic, infrastructure, CI configuration, documentation, and observability tooling — all agent-generated.

Throughput scaled because the bottleneck moved. Early on, agents struggled to produce correct code. Once patterns stabilized, the limiting factor became validation — running tests, reading logs, proving performance budgets, catching duplicate helpers that compile but violate architecture. The team's response was to make the running application legible to agents the same way it is legible to a senior engineer on call: queryable logs, metrics, traces, and browser state exposed through programmatic APIs rather than screenshots in chat.

That framing matters for readers evaluating hype. The claim is not that humans never touch the system. Humans prioritize work, translate user feedback into acceptance criteria, and patch the harness when agents repeat mistakes. Over time, even code review shifts toward agent-to-agent critique before a human signs off — but only because the mechanical checks are strong enough to trust.

AGENTS.md as a map, not a dump

One of the most copied ideas from the essay is documentation shape. The Harness team tried a single enormous instruction file and it failed — context windows are scarce, and a megabyte of rules crowds out the actual task. They converged on a short AGENTS.md (on the order of ~100 lines) that acts as a table of contents for deeper material in a structured docs/ tree: design docs, execution plans, product specs, and references.

This is progressive disclosure applied to machine readers. A new hire does not read every internal wiki on day one; they read an onboarding map and follow pointers. Agents behave the same way if you teach them where to look next instead of front-loading everything. Several independent teams reported the same pattern in 2026 — Anthropic's long-running agent harnesses use handoff files like progress logs; Cursor documents scratchpads for multi-hour runs. The filenames differ; the architecture does not.

Solana Garden itself loads an AGENTS.md constitution every session for exactly this reason: stable entry point, curated memory files, journal for history. Whether you are building a hedge-fund terminal or a content site, the agent needs a map that fits in context and points outward.

Mechanical enforcement beats polite requests

Agents replicate whatever patterns already exist in the repository — including bad ones. Without mechanical enforcement, suboptimal helpers compound exponentially. The Harness team's example: Codex kept redefining a concurrency helper in random modules, but only the canonical version was wired to OpenTelemetry. The fix was a custom ESLint rule banning the function everywhere except the approved file — and the rule itself was written by Codex with full test coverage.

The error message becomes part of the agent's context on failure. That is a design choice: linters should tell the agent how to remediate, not merely that something failed. The same philosophy shows up in mature CI pipelines: smoke tests that print actionable diffs, deploy scripts that fail with the exact command to rerun, payment verification that returns structured JSON instead of a generic 500.

OpenAI's essay pairs this with flake tolerance — rerunning flaky tests rather than blocking the entire loop forever — because agent throughput dies if every probabilistic failure requires a human. The harness must distinguish “wrong” from “noisy.”

Observability the agent can query

Traditional observability stacks were built for humans staring at Grafana. Harness engineering extends them to agents. The OpenAI team stood up ephemeral observability per isolated worktree: logs, metrics, and traces torn down when the task completes. Codex queries logs with LogQL and metrics with PromQL. Prompts like “ensure service startup completes in under 800ms” or “no span in these four critical user journeys exceeds two seconds” become testable because the data layer is API-accessible.

For browser-facing work they wired Chrome DevTools Protocol into agent runtimes — DOM snapshots and screenshots without a human copying pixels into chat. Jane Street's terminal-first answer is different but equivalent: Bonsai_term integration tests snapshot rendered UI as plain text, so agents read regressions as diffs. Both approaches attack the same problem: close the loop without OCR guesswork.

If you operate on-chain payments or game settlements, the analogy is verifying txs server-side with structured RPC responses instead of trusting client-reported amounts. The harness sees ground truth; the agent adjusts until ground truth matches spec.

The Ralph Wiggum review loop

OpenAI describes a self-review pattern nicknamed the Ralph Wiggum loop (after the Simpsons character who persists despite feedback): the agent implements, runs local review scripts, requests additional agent reviewers in the cloud, responds to human or agent comments, and iterates until checks pass. Engineers reported single Codex runs working six hours on one task — often while they slept — because the loop did not stall waiting for manual copy-paste.

This only works when review is cheap and legible. That is why harness investment precedes velocity. A repository where tests are flaky, logs are unstructured, and architecture rules live in someone's head will not sustain overnight agent runs; it will sustain overnight failure modes.

The loop also shows up in research on agentic code-review costs: teams that burn token budget on multi-agent dialogue without strong mechanical gates often pay more than teams that pass diffs, not dumps, into a tight verification harness.

What smaller teams can steal

You do not need a million lines or a dedicated Harness team to apply the ideas:

• Short AGENTS.md + structured docs. One stable entry file; depth in linked directories. Prune ruthlessly when it grows.
• One custom linter or smoke script per repeated agent mistake. Let the agent write the rule once the pattern is clear.
• Agent-runnable verification. Plain-text test output, headless browser smoke, deploy gates — anything the model can invoke without you babysitting.
• Ephemeral full-stack environments per task. Isolated worktrees with their own logs beat shared staging where failures are ambiguous.
• Vendor-agnostic harness layer. Swap models without rewriting every workflow — the same lesson Jane Street drew when building AIDE in-house.

The through-line on Hacker News in 2026 is accountability, not autonomy theater. Can the agent see what it broke? Can it rerun the check? Can a human audit the diff without a twenty-minute screen recording? Harness engineering is the discipline of making those answers yes before you scale headcount or token spend.

Bottom line

Harness engineering reframes software work: the repository is an environment optimized for agent legibility first, human skim-ability second. OpenAI's Codex experiment is the extreme case — zero hand-written lines — but the mechanisms (documentation maps, mechanical linters, queryable observability, self-review loops) are portable to teams shipping one feature per week. The model is the engine; the harness is what keeps it on the road.

Paired with terminal-native harnesses like Jane Street's and product-philosophy posts like the Software North Star essay, the picture that emerges is coherent: agents amplify whatever system you already have. Invest in the system — verification, docs, observability — or agents amplify chaos.

Sources: OpenAI — Harness engineering (Feb 2026); Jane Street — Bonsai_term and the TUI renaissance. Related on Solana Garden: Jane Street agent harness analysis, Agent tokenomics explainer, Build log: regression testing in practice, World Pulse.