self-improving-harness

Self-Improving Harness — systems where an LLM agent improves its own harness (prompts, tools, workflow, memory, control flow) through a propose-evaluate-accept loop, without changing the base model’s weights. The harness becomes both the tool and the optimization target (2026-07-07-harness-engineering-self-improvement).

STOP (Self-Taught Optimizer) — Zelikman et al. 2023

One of the earliest examples of recursive scaffolding improvement. An improver function $I$ takes a solution $s$, a utility function $u$, and a model $M$, returns an improved solution. STOP’s goal is to improve I itself:

$I_t=I_{t-1}(\hat{u},I_{t-1};M)$

The meta-utility measures the average performance of improver $I$ over downstream tasks $𝒟$.

Discovered strategies

Through recursive self-improvement, STOP discovered: genetic algorithms, decompose-and-improve, multi-armed prompt bandits, simulated annealing, varying temperature, beam/tree search.

Cautionary result

STOP improved mean performance with GPT-4 but degraded with weaker models (GPT-3.5, Mixtral). Recursive structure alone is insufficient — the base model must be capable enough to improve the mechanism. This implies: harness improvement enables better deployment, but intelligence remains core.

Self-Harness — Zhang et al. 2026

A propose-evaluate-accept loop with three stages:

1. Weakness mining

• Run current harness $h_t$ on tasks; collect execution traces.
• Cluster failures into verifier-grounded failure patterns.
• Distinguish surface verifier outcome (e.g. timeout) from causal mechanism (rich failure record with terminal cause, causal agent behavior status, abstract mechanism exposed by trace).

2. Harness proposal

• Same model under $h_t$ acts as proposer.
• Bounded proposal context: editable harness surfaces, failure patterns, passing behaviors to preserve, previously attempted edits.
• Edits should target recurrent addressable patterns (not task-specific difficulty), be narrow, and be diverse.

3. Proposal validation

• Regression tests on held-in $D_{\text{in}}$ (is weakness resolved?) and held-out $D_{\text{out}}$ (no new regressions?).
• Accept only if no regression on both splits.
• Rejected candidates logged; accepted ones merged to $h_{t+1}$.

Results

Tested with MiniMax M2.5, Qwen3.5-35B-A3B, GLM-5 on TerminalBench-2. Learns model-specific harness instructions targeting each model’s unique weaknesses.

DGM (Darwin Gödel Machine) — Zhang et al. 2025

Evolutionary harness code evolution (see also evolutionary-search-llm). Fixed model; evolving harness codebase. Parent agent examines its own benchmark logs, proposes harness improvements, offspring evaluated and kept if strong.

Claude 3.5 Sonnet results: SWE-bench Verified 20% → 50%.

Safety and design concerns

Weng flags that self-improving harnesses raise abstraction-boundary concerns:

• If a program can edit the OS/evaluator, abstraction boundaries break.
• The editable surface must be properly designed.
• Permission control and security layers must live outside the self-improvement loop.
• All reward-hacking challenges remain — the system optimizes whatever signal it’s given.

The evaluator should sit outside the loop, with held-out tests, trace audits, and human review at decision points that matter. How much oversight can be automated is an open research area.

Relation to other concepts

• harness — the runtime frame; self-improving harness applies optimization to the harness.
• meta-harness — the outer-loop optimizer; closely related but Meta-Harness is researcher-initiated search, while Self-Harness runs autonomously from the agent’s own failures.
• recursive-self-improvement — self-improving harness is one realization of RSI at the deployment-system level.
• evolutionary-search-llm — DGM and related methods use evolutionary search as the improvement mechanism.