Harness-Zero: Distill Optimized Agent Harness Behavior Into Model Weights, Then Remove the Harness

On September 21, 2026, Haoran Ye, Yuxing Lu, Haonan Dong, Zhaochen Su, and Guojie Song at Peking University and Google published Harness-Zero (arXiv:2609.24974), a method for transferring the gains of a specialized agent harness directly into model weights via supervised fine-tuning — and then removing the harness entirely at deployment. The result on Qwen3.5-9B: macro-average task success of 44.3%, compared to 23.3% without any harness and 41.7% when the specialized harness remains attached. The distilled model, running under a minimal one-tool harness, outperforms the original model running under the richer harness it was trained from.

The Problem With Accumulating Harnesses

An agent harness is everything around the model weights: tools, middleware, context management, memory, subagents, and control flow. Harness engineering has become one of the main levers for improving agent performance. Modern coding agents like Claude Code, Codex, and Kimi Code each embody different design philosophies within this space. Automated harness optimization — meta-harness or harness evolution — can automatically improve these components by analyzing failure trajectories and proposing updates.

The problem is that harness gains stay coupled to the harness. A specialized harness for scientific tasks behaves differently from one for spreadsheet manipulation. A general-purpose agent must either use a shared harness that sacrifices some specialized gains, or maintain a growing collection of specialized harnesses with routing logic, context overhead, and orchestration costs at every inference. The authors write that “neither choice moves the discovered harness improvements into the model.” The best harness varies across domains, instances, and base models, so maintaining many harnesses incurs “recurring costs in context, model calls, tool calls, and orchestration.”

Agent-as-Harness: Translating Guidance Across Action Spaces

The central challenge in harness distillation is that the specialized harness and the target deployment harness differ in both action space and available information. Trajectories collected under the specialized harness cannot serve directly as supervision for the target harness — if the specialized harness has tools the target harness lacks, those constructs don’t translate into valid instructions under the minimal harness.

Harness-Zero solves this with an “agent-as-harness” mechanism. A separate harnessing agent sits at the student model’s response boundary during training data collection. At each turn, the student proposes a response under the fixed minimal target harness. The harnessing agent reviews that proposal using a private reference harness adapted from the specialized harness. It either passes the proposal unchanged or replaces it with the smallest coherent correction that follows the specialized harness’s guidance — expressed as a complete valid response in the target harness’s action space. The student model never sees the review discussion; only the accepted response enters the student’s visible trajectory.

The adaptation step converts specialized-harness components into harnessing-agent guidance. A middleware rule that blocks risky actions in the specialized harness becomes a review-time warning in the reference harness: when the student proposes such an action, the harnessing agent sees the warning and replaces the proposal. A memory file that gives the student domain knowledge in the specialized harness becomes diagnostic criteria and intervention guidance. Tools that extend the student’s action space in the specialized harness become specifications for constructing student-native equivalents in the target action space. The harnessing agent cannot inspect the student’s sandbox or access hidden task solutions — any additional evidence it needs must come from proposing an action the student can execute through the target harness.

Learning From Reviewed Trajectories

The reviewed trajectories — containing both unchanged student proposals and harness-guided replacements — serve as supervised fine-tuning data. Replacements are written from the student’s perspective, as complete responses valid under the target harness. Because they are generated within the private review context, they may inadvertently include reviewer-perspective reasoning; such reasoning is masked from the training loss. The resulting fine-tuned model deploys under the minimal target harness alone, with no harnessing agent, no specialized harness, and no additional infrastructure.

The paper compares harness-guided review against four alternatives: direct trajectories from a stronger model, trajectories generated under the specialized harness directly, review without the private reference harness, and review given only the task answer. Harness-guided review achieves 30% task success on the held-out evaluation. The alternatives achieve 3–15%. The reference harness — the adapted version of the specialized harness used only during review — is what makes the difference: without it, the harnessing agent lacks the domain-specific guidance to know when and how to intervene.

Experimental Results

Experiments cover three task domains. SpreadsheetBench Verified contains 400 real-world spreadsheet-manipulation tasks; 300 are used for harness evolution and training data collection, 100 are held out for evaluation. AppWorld evaluates interactive tool use across simulated applications. USPTO Retrosynthesis tests scientific reasoning and multi-step chemical planning. The target harness is a fixed, minimal mini-SWE-agent with a single Bash execute tool — deliberately constrained to make the deployment requirement tight.

For the agent-as-harness evaluation without any training — frontier models using agent-as-harness at inference time — agent-as-harness achieves 81.1% versus 78.1% for code-as-harness, averaged across six benchmark-model settings. This shows the mechanism works even before fine-tuning: a harnessing agent translating one harness’s guidance into another harness’s action space outperforms simply running the student directly under the evolved harness.

For the full Harness-Zero distillation on Qwen3.5-9B: the base model under the minimal target harness achieves 23.3% macro-average task success. Adding the specialized harness raises it to 41.7%. Harness-Zero — fine-tuning on reviewed trajectories, then removing the specialized harness — achieves 44.3%. The distilled model under the minimal harness beats the original model under the richer harness. Behavioral analysis finds 82.3% average recovery of 28 behavioral patterns induced by the specialized harness but absent from the base model, measured across the three domains.

Limitations and Open Questions

Harness-Zero does not claim that arbitrary complex harnesses can vanish into weights. Some capabilities depend on runtime information or external effects that fundamentally cannot be stored in model parameters — a harness that fetches real-time data, calls external APIs, or maintains persistent state across sessions provides things that fine-tuning cannot internalize, because those things change at runtime.

Distillation introduces a training cycle, which takes time and compute. The harnessing agent requires a capable frontier model for the review step — a cost at data-collection time even if the deployment model is smaller. Workflows that change rapidly may be better handled externally rather than through periodic re-distillation cycles.

The study covers three specific domains. Whether agent-as-harness generalizes to other task types — very different action spaces, longer time horizons, or domains requiring persistent external state — is not yet established. The paper notes that one controlled coding-agent study found that changing the evaluation harness affected performance more than the training method, suggesting that harness-weight coupling is a real concern the field is still working through.

What This Means for Engineering Teams

The standard approach to agent infrastructure treats the harness as permanent serving infrastructure — code that runs alongside the model in production at every inference call. Harness-Zero suggests a different lifecycle: build the specialized harness, use it to collect training data, distill those behaviors into the model, then simplify or remove the harness in production. Harness complexity becomes an investment in training data quality rather than a permanent operational cost.

For teams running AI agent systems in production, this creates a useful engineering pattern. The harnessing agent that translates guidance is expensive to run, but it only runs during data collection, not during inference. The deployed system has a minimal harness, lower orchestration overhead, and less routing logic. When an improved harness is discovered — through automated evolution or manual engineering — the upgrade path becomes: collect reviewed trajectories under the new harness, fine-tune, validate, redeploy the simpler system.

This also changes how teams should think about agent engineering investment. Time spent building a better harness is not wasted when the harness is eventually removed — it becomes the source of training signal. The harness is a scaffolding tool for building a more capable model, not just a runtime dependency. That framing is useful when justifying the cost of elaborate harness engineering on the grounds that production deployment will eventually simplify.

Key Takeaways

• Harness-Zero distills specialized agent harness behavior into Qwen3.5-9B via supervised fine-tuning, then removes the harness at deployment.
• The distilled model achieves 44.3% macro-average task success under a minimal one-tool harness, versus 23.3% base and 41.7% with the specialized harness still attached.
• Agent-as-harness at inference time outperforms code-as-harness: 81.1% versus 78.1% averaged across six frontier model-benchmark settings, without any fine-tuning.
• 82.3% of 28 harness-induced behavioral patterns absent from the base model are recovered in the distilled model, measured across three task domains.
• Harness-guided review (30% task success) substantially outperforms alternatives including trajectories from a stronger model (3–15% task success range).

Work With Origins AI

Origins AI builds production AI systems for engineering teams. If your team is managing growing agent infrastructure that needs to be simplified without sacrificing task performance, talk to our team.