Harness Engineering: A Six-Month-Old Term for a Forty-Year-Old Idea

August 6, 2026 · 21 min read

Software Engineer

A small robot typing at a desk, ringed by a rig of curved rails, dial gauges, gates and feedback arrows looping back toward it

In February 2026 Mitchell Hashimoto gave a name to a habit, and within six months OpenAI, Anthropic, Thoughtworks and Red Hat had all published on it. The practices underneath the name are decades older than the name — which is the most interesting thing about it.

note

The harness landscape moves fast, and half the sources below are less than six months old. This reflects the state as of August 2026; benchmark numbers and tooling claims in particular are worth re-checking before you build on them.

The mental model: the agent is the model plus everything around it

Start with the equation that does most of the work in this discussion: agent = model + harness. The model is the weights some lab trained. The harness is everything else — and everything else is yours.

That framing comes from Birgitta Böckeler, a Distinguished Engineer at Thoughtworks, in her article on Martin Fowler's site, and it's deliberately blunt. When a coding agent does something stupid, there are only two places the fault can live. One of them you can't touch. The other is a pile of configuration, tooling and feedback loops you wrote, and can rewrite this afternoon.

Böckeler splits the harness into two kinds of control. Guides are feedforward: they "anticipate the agent's behaviour and aim to steer it before it acts" — system prompts, an AGENTS.md file, tool definitions, a plan document. Sensors are feedback: they "observe after the agent acts and help it self-correct" — a type checker, a test suite, a linter, a second model reading the diff.

The practical consequence is the whole discipline in one sentence. Telling an agent "follow our coding standards" in a prompt buys you probabilistic compliance; wiring a linter that blocks the merge buys you a guarantee. Harness engineering is the systematic preference for the second thing.

Where the term came from

The naming is unusually well documented, because it happened in public and fast.

On 5 February 2026, Hashimoto — co-founder of HashiCorp, and these days better known for the Ghostty terminal — published a six-phase account of getting from scepticism to daily use of artificial intelligence (AI) tooling, My AI Adoption Journey. Step five described a habit: every time an agent made the same mistake twice, he stopped correcting it by hand and built something into the environment that made the mistake impossible. Then came the sentence that stuck:

I don't know if there is a broad industry-accepted term for this yet, but I've grown to calling this "harness engineering."

Six days later OpenAI published Harness engineering: leveraging Codex in an agent-first world by Ryan Lopopolo, reporting an internal experiment: a three-person team, an empty repository in late August 2025, and five months later roughly one million lines of production code across about 1,500 merged pull requests (PRs) — with zero lines written by hand. Application logic, tests, continuous integration (CI) configuration, docs, observability, internal tooling: all of it generated by Codex, all of it shaped by the environment the team built around Codex.

That was the ignition. A term coined by a respected independent engineer, validated within a week by a frontier lab with a spectacular number attached, is about as good a launch as a piece of vocabulary can get.

It also arrived into a slot that was already shaped for it. The field had spent three years walking outward from the model: prompt engineering (2022–2024) worried about the wording of the instruction; context engineering (2025) worried about everything the model sees at the moment it answers — history, retrieved documents, tool output; harness engineering (2026) worries about the entire apparatus the model runs inside. Each layer subsumes the previous one rather than replacing it. You still write prompts. You just stopped believing the prompt was the lever.

What a harness is actually made of

A harness is not one artefact, and this is where new practitioners usually go wrong — they write a 400-line AGENTS.md, watch it fail, and conclude the idea doesn't work.

The most useful way to sort the parts uses two independent axes. The first is Böckeler's guides-versus-sensors split above. The second is how the check runs: computational checks are deterministic and fast (a compiler, a test, a permission rule); inferential checks use another large language model (LLM) call and are slow, expensive and non-deterministic (an evaluator agent grading a diff). Crossing the two gives four quadrants, and a healthy harness has something in all four — the teams that struggle usually have everything in one.

Anthropic's harness design write-up (24 March 2026) is mostly about the top-right and bottom-right of that picture — the sensors. Its central claim is that self-assessment doesn't work: "separating the agent doing the work from the agent judging it proves to be a strong lever", because models reliably overpraise their own output while an external evaluator can be calibrated to be honest. It recommends a planner/generator/evaluator split, sprint contracts negotiated before implementation, and deliberate context resets — clearing the window entirely and starting fresh — to avoid what it calls context anxiety, where a model rushes to declare victory as its context fills up.

The field guides converge on a surprisingly consistent set of components. HumanLayer's Skill Issue (Kyle, 12 March 2026) is the most opinionated and the most practical:

Keep the instruction file short. Under 60 lines, human-written, only rules that apply universally. "Less (instructions) is more."
Use Model Context Protocol (MCP) servers for tools, not for everything. Every tool description costs context: pile them up and "the context window fills up with tool descriptions, pushing you into the dumb zone much faster."
Prefer progressive disclosure. Skills that load when relevant beat everything loaded up front.
Treat sub-agents as context firewalls. Discrete work happens elsewhere and returns a conclusion, so intermediate noise never reaches the parent thread.
Build back-pressure. Verification that reports only failures, not successes — a type check that prints nothing when it passes is a better sensor than one that prints a wall of green.
Add configuration only after a real failure. Ship first, patch the environment when it actually breaks.

Red Hat's Marco Rizzi (7 April 2026) adds the institutional half: a repository impact map built with the Language Server Protocol (LSP) and MCP servers, reviewed by a human before any code is written, then a structured task template naming exact file paths and acceptance criteria. His two lines are worth keeping: "the more you constrain the solution space, the more predictable the output becomes", and — the one most teams skip — "skills, prompts, and MCP configurations are code. Version them, review them in PRs."

The forty-year-old idea underneath

Now read that component list again and ask which part is new.

A tool that reads your source and complains about things the compiler tolerates? That's lint, written by Stephen C. Johnson at Bell Labs in 1978. An automated gate that runs checks on every change and refuses the ones that fail? Continuous integration, standard practice since the late 1990s. A rig that sets up fixtures, exercises a component and reports pass or fail? That has been called a test harness for about thirty years — which is exactly where the new term borrowed its noun. Sandboxes, permission boundaries, structured handoffs, retry policies, observability: all present, all named, all decades old.

Stuart Miller makes this the whole of his case in Why the AI Industry's Newest Buzzword is an Old Idea (8 May 2026), and he does not hedge: "The 'thing' being described is not new. It is not even slightly new." He traces the lineage through platform engineering, middleware design, site reliability engineering (SRE), control plane design and the Toyota Production System, and dismisses the usual defence — that a stochastic component is a different problem — in one line: "wrapping something stochastic in a disciplined environment is a slightly different design problem than wrapping something deterministic. Slightly. Not fundamentally."

Werner Heijstek at the Software Improvement Group (SIG) (24 April 2026) reaches the same observation from the opposite mood — not as a debunking but as reassurance to organisations wondering what to buy: "quality gates, architecture standards, and governance infrastructure that good software engineering has always required are exactly what a production-grade harness is made of."

Both are right about the facts. The disagreement is about whether that makes the term worthless — and there's a decent counter-argument sitting in the etymology. Before "test harness" stabilised in the late 1990s, every shop had its own scattered vocabulary for runners, fixtures, mocks and assertion libraries. The umbrella term didn't invent any of those things. It made them designable as a unit — something a team could review, budget for and be bad at on purpose. A name that turns a pile of tactics into one object you can point at is doing real work, even when it invents nothing.

By every visible measure, yes — and the shape of the adoption says something.

Inside six months the term went from one blog post to publications by two frontier labs (OpenAI, Anthropic), two consultancies with strong methodology brands (Thoughtworks, Software Improvement Group), a major enterprise vendor (Red Hat), and Google's most-read web-performance writer (Addy Osmani, whose one-line version — "a decent model with a great harness beats a great model with a bad harness" — is the most quoted sentence in the whole literature). It has a 3.4k-star awesome list. It has dedicated academic benchmarks: Harness-Bench (May 2026, 106 tasks and 5,194 execution trajectories) exists purely to measure harness effects, and Agents' Last Exam (June 2026), built with 250+ industry experts, reports results at the model-harness pair level rather than by model.

That last detail is the strongest signal that this is more than a content-marketing cycle. When benchmark authors change what counts as a unit of measurement — from "the model" to "the model and its harness" — the concept has stopped being a blog post and become part of the measuring apparatus.

Two caveats on the trend. First, a lot of the mid-2026 material is search-engine filler: "complete guide" pages from vendors with no operating experience, all paraphrasing the same four primary sources. Second, there's a name collision — Harness is an established CI/CD (continuous integration and continuous delivery) company that also publishes on AI agents, which makes the search results genuinely confusing.

So does the harness actually matter?

Here the field splits, and the split is empirical rather than rhetorical. The question is simple: for a fixed budget, do you get more by switching models or by improving the environment?

The measurements disagree, and they disagree in an informative way.

Study	What it varied	Harness effect	Evidence strength
Agents' Last Exam (Jun 2026)	Models × harnesses, long-horizon real-world tasks	Model spread ≈ 3× harness spread	Benchmark, 250+ experts
METR time-horizon note (Feb 2026)	Claude Code and Codex vs. plain ReAct and Triframe scaffolds	None detected	Controlled, bootstrapped
Harness-Bench (May 2026)	106 tasks across model-harness pairings	Substantial across all metrics	Benchmark, 5,194 trajectories
Don't Blame the LLM (Jul 2026)	35 sequential harness releases, model held constant	Drives quality regressions	Controlled longitudinal study
OpenAI's Codex report (Feb 2026)	One team, one harness, five months	Decisive	Single uncontrolled case

Strong Good Mixed Weak

The result from METR — Model Evaluation and Threat Research, the nonprofit that measures how long a task a model can carry unaided — is the one harness enthusiasts should sit with. Measuring the time horizon of Opus 4.5 and GPT-5, they found that "neither Claude Code nor Codex outperform the default scaffolds METR uses" — despite both being elaborately prompted and explicitly tuned for their own model families, and despite the comparison scaffolds being generic. Claude Code beat plain Reasoning-and-Acting (ReAct) in 50.7% of bootstrap samples, which is a coin flip.

The reconciliation is that these studies are measuring different things. Agents' Last Exam and METR compare well-engineered harnesses to each other and find the differences small — which is unsurprising, since the top coding agents have converged on near-identical designs. Harness-Bench and Don't Blame the LLM compare harnesses that differ substantially and find large effects. Both can be true: the harness matters enormously up to a point of competence, and very little past it. What this means in practice is that the harness is where you lose, not where you win — a bad one costs you a great deal, a state-of-the-art one buys you little over a merely good one.

With that in hand, the positions line up coherently. Latent Space's Is Harness Engineering real? (5 March 2026) is the best snapshot of the argument at its peak, and several of the quotes below are drawn from it — it framed the split as "Big Model" versus "Big Harness", which is still the cleanest way to name the two camps.

Voice	Position	The argument
Mitchell Hashimoto (HashiCorp, Ghostty)	Harness-first	Every repeated agent failure should become a permanent environmental fix
Ryan Lopopolo (OpenAI)	Harness-first	Three people shipped a million lines with zero hand-written code
Birgitta Böckeler (Thoughtworks)	Harness-first, with caveats	Agent = model + harness — but the behavioural harness is still immature
Addy Osmani (Google)	Harness-first	Top coding agents resemble each other more than their underlying models do
Jerry Liu (LlamaIndex)	Harness-first	"The Model Harness is Everything"
Noam Brown (OpenAI)	Model-first	Complex scaffolding gets replaced as reasoning models improve
Boris Cherny & Cat Wu (Claude Code)	Model-first	Their own product aims to be "the thinnest possible wrapper over the model"
METR	Model-first (empirically)	Specialised scaffolds don't beat basic ones on time-horizon measurement
Stuart Miller	Sceptic, on novelty	The practices are real; the term is a rebrand of platform engineering
Werner Heijstek (SIG)	Sympathetic, on novelty	Same observation, framed as continuity with existing quality governance

The most quietly devastating entry is the Claude Code one. The team building the product most often cited as a great harness describes their goal as removing harness — Anthropic's own write-up makes the same point, that "every component in a harness encodes an assumption about what the model can't do on its own", and should be stress-tested and deleted when the assumption expires.

Where this goes next

Nobody serious thinks today's harnesses are permanent. The disagreement is only about the rate of decay.

The mechanism is well attested: each model generation absorbs the previous generation's patches. Context resets were a workaround for context anxiety until the models stopped panicking. Elaborate execution constraints got deleted when the models stopped needing them. Bruce's Window of Opportunity, Not a Forever Moat (8 May 2026) puts a date on it and argues the acute advantage closes around mid-2027: "the gap between 'well-harnessed agent' and 'poorly-harnessed agent' is currently massive, will narrow as models get strengthened, and will eventually be eaten by capability gains for routine tasks."

What survives absorption is not the same as what gets absorbed, and the distinction is the whole planning question:

Three directions are already visible past the current shape. Loop engineering — the subject of LoopsBench — narrows the focus from the whole environment to the quality of the observe-act-verify cycle itself, on the theory that the loop is where long tasks actually break. Agent-native systems invert the design: instead of documenting a codebase for agents, expose it through schema-driven interfaces an agent can discover at runtime, so no instructions are needed. And self-improving harnesses — Lilian Weng's Harness Engineering for Self-Improvement (4 July 2026) — treat the harness as the near-term substrate for recursive improvement, since editing code that shapes a model is far more tractable than editing weights. Weng's own forecast is the balanced one: "many harness improvements will be internalized into core model behavior, but the interface with external context and tools should remain."

Her constraint is the one to keep: "humans should move up the stack, not be removed from the loop." Böckeler lands in the same place — "a good harness should not necessarily aim to fully eliminate human input, but to direct it to where our input is most important."

The verdict

Three claims, held together.

The critics are right about the history. Nothing inside a harness was invented in 2026. It is lint and CI and sandboxes and code review, aimed at a new kind of unreliable worker. Anyone selling harness engineering as a novel discipline is selling you something you already own.

The advocates are right that the name earns its keep. Before it existed, the tactics were scattered across six vocabularies and nobody owned them as a unit. Now a team can review its harness, budget for it, and notice that it doesn't have one. That is what a good umbrella term does, and "test harness" did exactly the same job thirty years ago.

Both are wrong if they think the specifics are durable. Most of what a harness contains today is a patch for a model limitation with a shelf life measured in months. Write the disposable parts cheaply, invest only in the parts that survive the next model — the deterministic gates, the domain knowledge, the permission boundaries, and the evals that tell you whether any of it helped.

The habit Hashimoto described is the durable core, and it's the smallest part of all this: when an agent fails the same way twice, don't correct it again — change the world so it can't. That is a ratchet, and ratchets outlive the vocabulary used to describe them.

The mental model: the agent is the model plus everything around it​

Where the term came from​

What a harness is actually made of​

The forty-year-old idea underneath​

Is it trending?​

So does the harness actually matter?​

Where this goes next​

The verdict​

References​