Harnesses That Rewrite Themselves

For two years the industry optimized the model and hand-wrote the harness. That balance has inverted. Base capability moves in six-month steps; the harness — the retrieval policy, the tool schema, the retry logic, the verifier — moves every week, and it is where most of the remaining headroom lives.

A self-evolving harness treats its own configuration as a search space. Prompts, tool descriptions, decomposition depth, and model assignment per subtask become parameters, and a scoring loop over a held-out task suite decides which mutations survive. The interesting engineering is not the mutation step; it is the evaluation substrate that makes a mutation trustworthy.

We keep three invariants. Every proposed change must be attributable to a measurable delta on a frozen benchmark. Every change must be reversible in a single commit. And every change must be cheaper to evaluate than to deploy, otherwise the loop degenerates into production A/B testing with extra steps.

The failure mode we watch for is benchmark capture: a harness that learns the shape of the eval rather than the shape of the task. Rotating hidden task pools and adversarially generated variants keep the loop honest, and a human review gate stays on any mutation that touches tool permissions.