Harness Engineering for Reliable Agents

Teams argue about models. Incidents usually blame the harness. The model proposed a tool call; your process ran it with the wrong credentials, no timeout, and no record of why. That is not a reasoning failure. That is scaffolding failure.

Harness engineering is building the runtime around the model: tools, permissions, state, stop conditions, retries, and telemetry. The model is a component. The harness is the product surface that can hurt you.

Thin model, thick harness#

A reliable agent looks boring from the outside:

• The model proposes the next step.
• The harness checks allowlists, schemas, and budgets.
• Typed tools execute side effects.
• Results append to state the next call can see.
• Something deterministic decides when to stop.

If your “agent” is a while-loop that trusts free-form JSON and shell access, you skipped the harness and shipped a demo.

The practical test is simple: can you swap the model behind the same allowlist, timeouts, and trajectory schema without rewriting product code? If the answer is no, the “agent” is really a pile of prompt-coupled scripts. Harness engineering is how you get to yes.

What the harness must own#

Notice what is missing from that table: “clever system prompt.” Prompts matter. They do not replace tool contracts.

type HarnessConfig = {
  allowTools: Set<string>;
  maxSteps: number;
  maxTools: number;
};

async function step(cfg: HarnessConfig, state: AgentState, proposal: ToolProposal) {
  if (state.steps >= cfg.maxSteps) return { type: "stop" as const, reason: "max_steps" };
  if (!cfg.allowTools.has(proposal.name)) {
    return { type: "reject" as const, reason: "tool_not_allowed" };
  }
  const result = await runTool(proposal.name, proposal.args, { timeoutMs: cfg.maxTools });
  return { type: "ok" as const, result };
}

Reject paths are first-class. Silent “try something else” without logging is how you debug for a week.

When a tool is rejected, return a structured reason the next model call can see — tool_not_allowed, schema_invalid, timeout — not a vague “something went wrong.” Models recover better from named failures. On-call recovers better from named failures too.

Failure modes the harness prevents#

• Unbounded loops — no max steps, no wall clock
• Tool sprawl — every internal API exposed “just in case”
• Credential bleed — secrets in prompts or tool outputs the model can echo
• Blind retries — same failing call five times, five bills

Fix those in code. Do not ask the model to “be careful.”

Also watch partial success. A tool that writes a draft record and then times out leaves the world half-updated. Idempotency keys and compensating actions belong in the harness, not in a hope that the next loop iteration “notices.” If you cannot replay a failed step safely, you do not have a production tool — you have a demo button.

Where to start#

1. One workflow, five tools, hard allowlist.
2. Schema validation on every tool argument.
3. Trajectory logging before you add a second agent.
4. A degraded mode: smaller tool set or human handoff when the harness trips a budget.

Upgrade the model when the harness already refuses bad plans. Doing it the other way around is how you buy a faster way to call the wrong API.

Harness vs prompt vs workflow#

People mix these up:

• Prompt / context — what the model sees this turn.
• Harness — what code is allowed to run because of what the model said.
• Workflow — durable steps across time (approvals, waits, human gates).

You need all three eventually. Day-one reliability almost always comes from hardening the harness, not from a longer system message. A workflow engine on top of an unsafe tool runner just schedules incidents.

A useful sequencing for a new team: ship one workflow with a strict harness, prove you can stop and audit it, then widen tools. Context budgets and durable workflows come next. Jumping straight to multi-agent orchestration before allowlists and trajectory logs exist is how platforms earn a reputation for “AI that nobody trusts.”

Summary#

Harness engineering makes agents operable. Own tools, permissions, state, stops, and telemetry in deterministic code; keep the model thin and replaceable. If the scaffolding is vague, no frontier checkpoint will save the on-call rotation.