The Product Needs a Harness

AI has made producing a software change radically cheaper. A coding agent can inspect a repository, propose a plan, modify several files, run tests, and prepare a pull request while a person is still forming a mental model of the task.

That speed changes the bottleneck. The difficult question is no longer whether software can be changed. It is whether the change can be trusted.

What guided the agent before it acted? What boundaries shaped its choices? Which checks observed the result? Which product promises were tested? What happens when the checks disagree, the evidence is incomplete, or the change looks correct while quietly making the experience worse?

The emerging answer is harness engineering: the deliberate design of the instructions, tools, constraints, tests, feedback loops, and operating environment around an AI agent.

That idea has consequences beyond coding. A coding harness can help an agent produce maintainable software. A product harness must go further. It must help the organization determine whether the changed product still keeps its promises to users, operators, buyers, and regulators.

The agent changes the software. The harness protects the conditions under which that change deserves trust.

The word harness names the missing layer

AI products are often described through the model they use. The model is treated as the intelligence, while the surrounding product appears to be packaging. In practice, the surrounding system determines what the model knows, what it can do, how far it can act, how its work is checked, and whether anyone can recover when it fails.

LangChain describes an agent as a model plus a harness. Its anatomy includes prompts, tools, infrastructure, orchestration, state, middleware, and feedback loops. OpenAI has used the same language to describe an agent-first engineering environment where repository structure, documentation, custom linters, browser access, logs, metrics, and recurring cleanup make a large codebase legible to agents.

This connects directly to The Prompt Is Not the Product. A durable AI product needs context, orchestration, evaluation, permissions, interface, approval, and observability. The prompt is one instruction inside a larger operating system.

Harness engineering gives that operating system a useful engineering vocabulary. It also reveals why improving the model alone rarely fixes a weak product. A stronger model operating inside unclear permissions, stale documentation, missing tests, and invisible production signals can create larger mistakes more efficiently.

Guides and sensors form the basic control loop

Birgitta Böckeler's article on martinfowler.com divides the coding-agent harness into two broad parts: guides and sensors.

Guides shape the work before and during execution. They include repository instructions, architecture rules, examples, templates, naming conventions, tool descriptions, permissions, and plans. They are feedforward controls. Their purpose is to make good action easier and undesirable action harder.

Sensors inspect what happened. They include tests, linters, type checks, architecture checks, screenshots, logs, runtime metrics, security scans, accessibility checks, and review agents. They are feedback controls. Their purpose is to compare the result with an expected condition and expose drift.

A second distinction matters just as much:

• Computational controls are deterministic. They can verify a type, compare a screenshot, enforce a dependency rule, measure a performance budget, or confirm that a contract test passes.
• Inferential controls use judgment. They can review whether a change matches intent, investigate an unexpected visual difference, assess whether an explanation is misleading, or decide which evidence deserves escalation.

The strongest harnesses combine both. Deterministic checks provide speed, consistency, and clear failure conditions. Inferential checks help when the meaning of a change cannot be reduced to one assertion.

Replacing dependable tests with an AI opinion would weaken the system. Using an agent to choose, run, connect, and interpret dependable tests can make the system much stronger.

Raw Drafts already contained the product-level version

Several ideas across Raw Drafts become clearer when placed inside the harness frame.

The Prompt Is Not the Product describes the system around the model.

Every Coding Agent Needs a Regression Agent describes product memory with tools: an agent that remembers accepted behaviour, predicts affected surfaces, runs relevant checks, investigates ambiguous differences, and prepares evidence for a human decision.

Digital Product Assurance Is the Evidence Behind Trust describes the broader discipline: connect product claims to traceable evidence, make release decisions from that evidence, and keep monitoring after launch.

Together, they form a nested control system:

1. The agent harness controls how an AI agent understands and performs a task.
2. The engineering harness controls whether the resulting software satisfies technical and architectural expectations.
3. The product-assurance harness controls whether the product still satisfies the promises that matter in real use.

The first layer helps the agent work. The second helps the codebase remain coherent. The third helps the organization justify confidence in the product.

A regression agent is an active sensor

Most software sensors are passive. A test runs and returns pass or fail. A monitor emits an alert. A screenshot tool exposes a pixel difference. Each signal is useful, but someone still has to connect it to the release, the product intent, and the likely consequence.

A regression agent can become an active sensor. It can inspect the change, identify the affected journeys, select relevant checks, open the product, reproduce a suspicious state, query telemetry, compare the result with prior evidence, and explain what may have moved backward.

Its job is not to invent quality through language. Its job is to coordinate evidence.

A useful regression agent should be able to:

• remember what the product previously promised
• identify which promises a change could affect
• run deterministic checks before using expensive judgment
• distinguish an intentional change from unexplained drift
• reproduce failures across realistic states and environments
• connect code, interface, telemetry, and user outcomes
• prepare a decision packet with evidence, severity, uncertainty, and recommended action
• turn confirmed recurring failures into new tests, rules, baselines, or monitoring

That last step closes the loop. The harness should become stronger after a failure. A human correction that remains only in a comment will be forgotten. A correction converted into an executable control becomes part of the product's memory.

The product promise is the unit of control

Code quality is one part of product quality. A release can compile, pass its unit tests, and still damage the product.

The interface may become harder to understand. Keyboard focus may disappear behind a modal. A page may become slower on an ordinary mobile connection. A data field may preserve its type while changing its meaning. A permission may expand without a clear user explanation. A recommendation may improve clicks while reducing trust. A workflow may still complete while creating more support requests, abandonment, or risk.

The useful unit for product-level harness engineering is the promise.

A promise can be explicit, such as an accessibility requirement, latency budget, API contract, retention rule, or regulatory obligation. It can also be an earned expectation, such as a stable checkout flow, a readable hierarchy, a predictable recovery path, or a clear distinction between generated and verified information.

A product-assurance harness should map promises across several surfaces:

• behaviour and critical journeys
• visual design and interaction states
• accessibility and assistive technology
• performance, reliability, and resource use
• data meaning, analytics, and model behaviour
• security, privacy, and permissions
• commercial, social, and human outcomes

This expands regression testing from protecting code to protecting continuity of meaning.

Design systems become executable policy

Harness engineering also changes how design systems should be understood.

A design system is often presented as a library of reusable components. For an AI-assisted product team, it can become part of the control environment. Tokens, components, states, interaction patterns, content guidance, accessibility requirements, and deprecation rules provide agents with a legible space in which to work.

The system becomes more valuable when its rules are executable:

• lint when values bypass approved tokens
• validate component properties and states
• test keyboard behaviour and accessible names
• compare important visual states across viewports
• enforce content and interaction conventions near the code
• identify deprecated patterns and unsafe exceptions
• expose examples that demonstrate the preferred implementation

Böckeler calls attention to harnessability: the qualities that make a codebase easier for agents to understand and change safely. Clear boundaries, local instructions, observable behaviour, reusable structures, and fast feedback improve both machine and human work.

A design system contributes to harnessability when it acts as a shared language, a set of constraints, and an evidence source. It helps a coding agent produce something plausible while helping a regression agent determine whether plausibility matches the product's actual standards.

The harness itself needs assurance

A harness can create false confidence. Tests may cover yesterday's architecture. Instructions may conflict. Screenshots may be accepted without understanding. A review agent may repeat the bias of the implementation agent. A green dashboard may mean that the wrong things were measured consistently.

The harness therefore needs its own assurance questions:

• Which important promises have no sensor?
• Which guides are stale, duplicated, or contradictory?
• Can a failing check be traced to an owner and decision?
• Are agents allowed to modify the tests that judge their own work?
• Do independent signals confirm high-risk conclusions?
• Are false positives teaching people to ignore alerts?
• Can the system detect sensor silence, missing telemetry, and skipped journeys?
• Does the evidence represent real users, devices, environments, and edge cases?

The goal is not maximum automation. The goal is justified confidence.

High-risk decisions should use separation of roles. The agent that implements a change should not be the only system evaluating it. Deterministic controls should remain difficult to bypass. Important baselines should have ownership. Human approval should concentrate on ambiguity, tradeoffs, and consequences rather than ceremonial clicking.

Human attention moves toward judgment

A strong harness does not remove humans from software development. It changes where human attention is most valuable.

People should spend less time rediscovering whether a button lost its accessible name or whether an analytics event disappeared. Machines are well suited to repetition, comparison, exhaustive checking, and signal collection.

People remain responsible for questions such as:

• Is the product promise still the right promise?
• Is an intentional change worth the tradeoff it creates?
• Does the interface communicate risk honestly?
• Are the chosen metrics hiding harm or exclusion?
• Is the available evidence sufficient for this release?
• Who carries the consequence when the system is wrong?

Harness engineering can make those decisions more informed. It cannot make them morally neutral.

A practical starting point

A team does not need a giant autonomous quality platform to begin. The first useful harness can be small and explicit.

1. Write down the promises. Identify the critical journeys, boundaries, thresholds, and human outcomes the product must preserve.
2. Add guides near the work. Put architecture rules, component guidance, permissions, examples, and decision records where agents and people can retrieve them.
3. Map each promise to sensors. Use tests, screenshots, accessibility checks, performance budgets, telemetry, data contracts, security controls, and structured review.
4. Run deterministic checks first. Reserve inferential review for ambiguity, investigation, prioritization, and explanation.
5. Create an independent regression role. Give a separate agent or workflow access to the change, the accepted baselines, the product context, and the evidence tools.
6. Promote learning into controls. When a human confirms a failure, convert the lesson into a test, rule, template, baseline, runbook, or monitored claim.

Over time, the harness becomes a living assurance case. It records what the product claims, how those claims are evaluated, what evidence exists, where uncertainty remains, and what should happen when reality changes.

From agent harness to product assurance

Harness engineering began as a practical answer to a coding problem: agents need a structured environment if they are going to change large software systems reliably.

The product-level implication is larger. Faster code generation increases the number of changes an organization can attempt. Each change increases the need for memory, constraints, evidence, and accountable decisions.

The agent harness regulates action. The engineering harness regulates technical coherence. The product-assurance harness regulates trust across the full experience and lifecycle.

Speed without memory becomes drift. Autonomy without evidence becomes theatre. A product needs a harness because the work around the model is where promises become controls, changes become evidence, and confidence becomes something an organization can defend.

AI authorship note

This essay was developed from Kris Haamer's editorial direction and HAAM's work across AI systems, interaction design, regression testing, software delivery, and digital product assurance. AI supported research, synthesis, drafting, and implementation. Kris is responsible for the framing, argument, source selection, and decision to publish it.

Connected reading

• The Prompt Is Not the Product
• Every Coding Agent Needs a Regression Agent
• Digital Product Assurance Is the Evidence Behind Trust
• The Product Is Becoming an Organization