RRSI Regularizes Agent Harness Self-Improvement to Prevent Overfitting Across Benchmarks

On September 21, 2026, Peng Xia, Rujun Han, Zifeng Wang, and colleagues at Google Cloud AI Research, Stanford University, and Washington University in St. Louis published RRSI (arXiv:2609.24972), a framework for regularizing recursive agent harness self-improvement. The core finding: standard harness evolution methods overfit to the tasks used during optimization, and gains that look strong on the evolve split largely disappear on held-out benchmarks. RRSI prevents this through constraints on both how edits are proposed and which candidates are accepted, gaining up to 14.1 points on the evolve split and up to 4.7 points on five out-of-distribution benchmarks, while running on 30% fewer policy tokens than unregularized evolution. An Apache-2.0 implementation is available at github.com/google-research/rrsi.

The Overfitting Problem in Harness Evolution

An LLM agent’s capability depends on its backbone model and its harness — the prompts, control flow, tools, memory, and context management wrapped around the frozen weights. Much recent progress in agent capability has come from harness engineering rather than from new model weights. Automated harness evolution methods use LLMs to propose and evaluate changes to the harness, turning the agent’s own performance into a feedback signal for improving the system that shapes its behavior.

The problem is that this feedback loop is adaptive: proposals at round t depend on measurements taken from the same tasks in earlier rounds. The paper frames this as “adaptive empirical optimization over an unusually expressive search space” applied to a finite evolve set. Each evaluation is another adaptive look at the same tasks, so the search gradually encodes those tasks into the harness rather than discovering reusable mechanisms. Multiple recent studies find the same pattern — the paper notes that “apparent improvements can arise from task-specific fitting or increased test-time computation rather than reusable mechanisms.” The paper’s Figure 1 shows this directly: prior methods retain little of their evolve-set gain on out-of-distribution tasks, and several drop below H0, the initial harness, on held-out splits.

The paper identifies three specific failure modes: benchmark-specific encoding (harness components that encode task names, entities, or specific answers), noise chasing (candidates that won by stochastic variation rather than genuine improvement), and complexity accumulation (harness growth that improves evolve-set scores without improving the underlying agent mechanism). Each widens the gap between evolve-set and transfer performance.

Three Regularization Mechanisms

RRSI regularizes both sides of the evolution loop — how candidates are proposed and how they are selected — using principles borrowed from classical machine learning regularization. The harness edit space remains fully open: prompts, control flow, configuration, tools, memory, context management, and subagents may all be modified, added, or removed. Regularization constrains the search trajectory through that space, not which components may be touched.

L0-Style Annealed Edit Budget

An unconstrained proposer can bundle many unrelated modifications into one candidate, making any measured change difficult to attribute to a specific mechanism and increasing the capacity to fit evolve-set idiosyncrasies. RRSI caps the number of independently attributable edits per candidate using a cosine annealing schedule:

b_t = ⌈b_min + (b_max – b_min) × 0.5 × (1 + cos(π × t / T))⌉

The schedule starts at b_max (permissive, allowing coordinated changes to discover new mechanisms) and finishes at b_min (sparse, where changes are individually attributable). This is the paper’s closest analogy to an L0 cardinality constraint: it directly limits the number of independently active edits per update. A small portion of each round’s proposal budget is also reserved for harness components not yet exercised in the run, preventing the search from collapsing onto a narrow edit family — for example, repeatedly rewriting prompts while leaving control flow and tool design untouched.

Leakage Screening

Before full evaluation, a critic reads each candidate diff and rejects edits that explicitly encode task names, entity names, task-specific values, answers, or logic specific to the evolve benchmark. This screening happens before evaluation — a key timing choice. The paper notes: “a leaking candidate never receives the inflated evolve-set score that could make it attractive to subsequent rounds.” Generic prompt or tool-description improvements remain valid candidates; the screen targets content specificity, not component type. Screening before evaluation is also cheaper than running a full evaluation on every candidate before discovering it was benchmark-specific.

Complexity-Aware Acceptance and Structural Pruning

For a candidate whose measured score gain exceeds the empirical noise band δ, an extra cost condition must hold: the relative increase in policy-token cost must satisfy ΔC ≤ β_0 + β_1 × ΔS, where β_0 sets the cost increase tolerated for a negligible score gain and β_1 controls how much additional cost is allowed as measured improvement increases. These parameters are selected on the evolve set and kept fixed for all transfer evaluations. This prevents unconstrained token-cost growth from being passed through score improvements — the paper’s analogy to Ridge/L2-style shrinkage, penalizing growth in the aggregate resource footprint without requiring any particular component to be removed.

Complementing this, Lasso-style structural pruning tracks whether recently exercised harness components have produced measurable gains over a fixed pruning window. Components that remain unproductive are reported to the proposer as deletion targets in subsequent rounds. The retained harness becomes structurally sparser — a mechanism must continue to earn its place rather than persist because score-only evolution has no incentive to remove it.

Experimental Results

RRSI is evaluated on eight benchmarks across three domains. For coding: Terminal-Bench 2.1 (89 containerized terminal tasks verified by unit tests) and SWE-bench Verified (repository-level bug fixing). For agentic workspace: Harvey LAB (legal-work benchmark spanning 25 practice areas, with 120 evolve tasks and 40 in-distribution held-out tasks), JobBench, GDPval, and APEX-Agents. For engineering design: EngDesign (61 design tasks graded by frozen simulators) and Frontier-Eng. In each domain, the harness is evolved on one suite and then run unchanged on held-out benchmarks that differ in task type, tooling, and verifier.

RRSI gains up to 14.1 points on the evolve split. On five out-of-distribution held-out benchmarks, it gains up to 4.7 points (JobBench) and 4.3 points (Frontier-Eng), and improves all six held-out evaluations across the three domains. Compared to the average of four prior harness evolution methods, RRSI outperforms by up to 22.9% on held-out environments. It achieves these gains while the evolved harness runs on 30% fewer policy tokens than unregularized evolution — the pruning mechanism removes components that accumulated without contributing, reducing both the token footprint and the cognitive complexity of the deployed harness.

The contrast with prior methods is stark. The paper’s Figure 1 shows the average of four baselines retaining little of their evolve-set gain and several methods ending below H0 on OOD splits. RRSI’s held-out scores move up across all six held-out evaluations. The improvement on OOD tasks, not just on the evolve split, is the central claim: RRSI “learns broadly useful harness changes” that “generalize across substantially different tasks and evaluation settings.”

Limitations and Open Questions

Harness evolution is expensive regardless of regularization. Candidate harnesses must be run against evaluation tasks to collect performance measurements, and the quality of the evolved system depends heavily on whether those evaluations represent future usage. RRSI reduces wasted search capacity through its regularization constraints but does not eliminate the fundamental cost of evaluating many candidates across many tasks.

The regularization hyperparameters — b_min, b_max, β_0, β_1, noise band δ, pruning window — are selected on the evolve set and kept fixed for transfer evaluations. The paper does not evaluate sensitivity to these choices or provide a principled method for setting them on a novel domain. Teams applying RRSI to a new task type would need domain-specific experimentation to calibrate these values.

The open-source implementation covers coding, agentic workspace, and engineering design. It does not currently cover retrieval-heavy tasks, multimodal agents, or long-horizon planning environments with different verifier structures. Whether the regularization mechanisms transfer cleanly to those settings is not established.

What This Means for Engineering Teams

The emerging picture of agent improvement has two distinct axes: training better model weights and engineering better harnesses. RRSI makes explicit that harness improvement — like weight optimization — can overfit, and that the remedies are analogous: budget how much changes per iteration, penalize complexity growth, prune what doesn’t contribute, and hold evaluation tasks out from the optimization loop entirely.

For teams running agent systems in production, this has a concrete implication: measuring harness improvement only on the tasks used to generate that improvement is not sufficient evidence of generalization. The same discipline applied to model evaluation — held-out test sets, OOD benchmarks — applies to harness evaluation. A harness that improves on your development benchmark without improving on production tasks has overfit, and RRSI’s results suggest this happens more often than expected with unregularized evolution.

There is also a structural implication. RRSI evolves prompts, control flow, tools, memory, context management, and subagents — all components — while keeping the backbone weights frozen. Harness evolution and model fine-tuning become separate, independently applicable improvement axes. Teams with frozen deployed models (for cost or stability reasons) can still improve agent capability through regularized harness evolution. The memory and learned policy systems that govern agent behavior are first-class optimization targets, not fixed infrastructure. RRSI’s constraint framework gives those optimization runs a principled way to avoid the overfitting that makes unregularized evolution unreliable for production deployment.

Key Takeaways

• Standard harness evolution overfits to the evolve set: prior methods retain little evolve-set gain on OOD benchmarks, and several end below the initial harness on held-out splits.
• RRSI gains up to 14.1 points on the evolve split and improves all six held-out OOD benchmarks by up to 4.7 points, while using 30% fewer policy tokens than unregularized evolution.
• Three regularization mechanisms: annealed edit budget (L0-style), leakage screening by a critic before evaluation, and complexity-aware acceptance with structural pruning (L1/L2-style).
• The harness edit space remains fully open — any component may be modified; regularization constrains the search trajectory through that space, not the hypothesis space itself.
• An Apache-2.0 open-source implementation covering coding, agentic workspace, and engineering design domains is available at github.com/google-research/rrsi.

Work With Origins AI

Origins AI builds production AI systems for engineering teams. If your team is investing in agent harness engineering and needs measurement discipline to distinguish genuine generalization from evolve-set overfitting, talk to our team.