Context Language Models Let Agents Rewrite Their Own Context

A joint team from the University of Washington and Meta Superintelligence Labs, led by Rulin Shao, Shannon Zejiang Shen, and Junjie Oscar Yin, published the CLM paper on September 29, 2026, proposing that language models should not just generate into their context but be allowed to edit it, arbitrarily and on demand. The system they describe, Context Language Models (CLMs), treats the model’s live context as a directly writable file synchronized with its generation state. The motivation is simple to state but hard to ignore: every existing context management strategy forces the model to operate within a human-defined action space, and that constraint is now the bottleneck for long-horizon agent tasks.

Why Context Management Is Currently Infrastructure, Not Model Behavior

Standard language models operate under an append-only constraint. At each step, the context grows by exactly the tokens the model generates. Formally: c_{t+1} = c_t ⊕ f_θLM(c_t). Nothing in the context is ever removed or changed mid-trajectory. What gets dropped, compressed, or reorganized happens above the model, in a harness written by a human engineer, executing a policy the engineer chose.

The industry has built a catalog of such policies. AutoCompact and Self-Compact let the model decide when to compact, but not how. Context-as-a-Tool exposes model-triggered compaction over a predefined portion of context with a fixed method. ACM adds model-triggered offloading and retrieval. Sculptor lets the model select context fragments to operate on. Each approach extends the model’s autonomy over timing or targeting, but the action space stays fixed by whoever designed the harness.

The CLM paper frames this as the same mistake that The Bitter Lesson identifies in game-playing AI: “we should let LMs search for and learn better strategies that go far beyond existing human priors.” Context management has been an engineering concern configured around the model. CLMs invert that: make context management part of what the model does.

Any team running production AI agent systems on multi-hour tasks has encountered this directly. Context runaway is not a scaling problem you solve by throwing more memory at it. It is a policy problem, and fixed human-designed policies are the bottleneck.

The Core Mechanism: Context as an Editable File

The CLM design replaces the append-only constraint with a general context function: c_{t+1} = f_θCLM(c_t). The model’s context at each step is the result of applying an arbitrary transformation to the previous context. That transformation can add, remove, reorder, or rewrite anything.

The implementation is the interesting part. The authors mirror the model’s live context as a file that the model can edit using ordinary Bash commands. When the model edits this file, those edits are automatically synchronized with the model’s actual context for continued generation. When the model does not edit the file, tokens are appended by default, preserving standard behavior. The mechanism requires no changes to the model’s attention mechanism or weight structure. It is an interface change: the model gains write access to what was previously read-only.

The multi-agent extension follows naturally. Multiple context files can coexist, each synchronized with a separate LLM server. An agent swarm is initialized by creating multiple context files in a shared workspace; agents read each other’s context and update their own based on what others have written. On a 24-hour, six-repository swarm task, CLMs achieve 65% greater end-to-end speedup at the same compute budget, coming directly from agents pruning shared context rather than each maintaining a full accumulating history.

What’s easy to miss: the editing mechanism is not a new tool the model calls. It is the context itself becoming mutable. The action space for context management becomes as broad as the model’s ability to write Bash commands, which is essentially unbounded.

ContextBench: Where Fixed Strategies Fail

The authors built ContextBench, a diagnostic benchmark with four synthetic tasks designed to isolate context management from reasoning and world knowledge. The tasks are deliberately simple: the question is not whether the model can reason, but whether it can manage context correctly.

Needle Retention tests selective verbatim retention: preserve specific important information as the context fills up. Sudoku Sketchpad tests surgical in-place updates: maintain an accurate Sudoku board as a user streams moves. KV Store tests offloading and retrieval of massive values that cannot fit in active context. Log Triage tests offloading and retrieval of working logs across a long task session. All tasks were evaluated at a 32K context limit with context pressure up to 24 times the nominal window.

The results are worth reading carefully. Summary-based compaction methods lose or hallucinate information on Needle Retention and Sudoku Sketchpad, because summarization cannot preserve verbatim content or exact board state. Methods without flexible in-place editing must regenerate the entire Sudoku board for every single move, which is expensive and error-prone at scale. Standard coding tools can offload data on KV Store and Log Triage but cannot evict content from the live context on demand. None of the existing methods performs correctly on all four tasks. These are not hard reasoning problems; they are context hygiene problems, and fixed strategies fail them at 32K context pressure that any serious agent task will encounter.

Benchmark Results

The headline number is BrowseComp-Plus, a deep-research benchmark requiring extended web browsing and multi-source synthesis. CLMs achieve 11.4% higher accuracy than the strongest baseline while using 21.5% fewer prefix-reuse FLOPs. Prefix-reuse FLOPs is a metric the authors introduce to capture trajectory-wide inference costs across decoding, prefilling, and re-prefilling. It is a more honest cost measure than per-call FLOPs for agentic workloads, because re-prefilling costs compound over a long trajectory.

On TerminalBench 2.1, CLMs match the strongest baseline’s accuracy with 29.5% fewer prefix-reuse FLOPs. On mathematical optimization, CLMs outperform OpenEvolve by 16.8% on Heilbronn and 3.0% on circle packing, tasks where the agent must maintain, update, and prune a working set of candidate solutions across many iterations.

EdgeBench, a 12-hour, 10-task subset of a long-horizon evaluation, shows CLMs scoring 5% higher while using 59% fewer prefix-reuse FLOPs. For a 12-hour task, that compute difference is not marginal.

Learning Context Management: In-Context and RL

CLMs support two distinct paths for improving context management without retraining the model from scratch.

The first is in-context learning via a skill-evolution loop. The authors evolve an in-context skill document, a natural-language description of useful context-management procedures, through an iterative process that tests candidate skills on held-out ContextBench tasks and retains the ones that improve accuracy. The resulting skill document improves held-out ContextBench accuracy by up to 35.9 points at lower compute than retraining. This means a team can steer CLM context behavior through natural-language instructions without touching model weights, a practically important property for deployment.

The second path is online reinforcement learning. The authors introduce a success-gated efficiency advantage in stepwise GRPO that rewards compute-efficient CLM trajectories among the successful ones, without trading accuracy for compute. Training Qwen3.5-9B this way lifts CLM performance from 28.8% to 42.5% on BrowseComp-Plus. The trained CLM outperforms a Codex-style summary harness trained with the identical recipe by 0.4 points while using 38.8% fewer FLOPs. The same training recipe is the important control: the gain is from the CLM structure and efficiency-gated reward, not a bigger budget.

The emergent behaviors the authors document are worth noting. Models trained with this recipe develop novel strategies not present in training data: tracker data structures for multi-agent orchestration, a new chat role for internal notes invisible to downstream stages, and reusable context-management functions stored as code the model calls across turns. None were prescribed. They emerged because the model had write access to its own context.

Suffix Cache Reuse: The Serving Co-Design

CLMs create a serving problem. Standard engines reuse cached KV states only for matching prefixes. An edit in the middle of the context invalidates all cached states for everything that follows, forcing a full re-prefill of the suffix. For CLMs that make frequent surgical edits, that overhead would negate the compute savings from context management itself.

The authors address this with Suffix Cache Reuse (SCR). SCR reuses cached states beyond the matching prefix for tokens that were not edited, even when tokens before them changed. Many edits are localized: a model removes a stale section from the middle of its context, but the suffix is unchanged and only weakly affected by the edit. SCR exploits this to reduce re-prefilling while empirically preserving task performance, cutting server-side compute by 35% relative to standard SGLang at matched performance. SCR is not a standalone optimization; it is the serving-layer complement that makes CLM context editing economical in production.

Limitations and Open Questions

ContextBench’s synthetic design means the gap between controlled evaluation and real deployed workloads is uncharacterized. A Needle Retention task with fixed content structure is easier to measure than the same task inside a live research agent where “important information” is task-dependent and ambiguous.

The 42.5% absolute score on BrowseComp-Plus also tells you the benchmark is genuinely difficult and that the accuracy advantage over the summary harness, 0.4 points, is narrow. Whether that gap grows with larger models or better training is undemonstrated.

Suffix Cache Reuse preserves task performance “empirically.” The theoretical boundary conditions for when reusing suffix KV states beyond an edit point introduces meaningful attention errors are not established. Deploying CLMs in production requires validating SCR behavior on specific workloads before assuming the compute savings are universal.

The 65% multi-agent speedup comes from a single reported workload. The scaling properties of synchronized context files when edits from multiple agents conflict, or when the swarm grows to tens of agents, are not analyzed.

What This Means for Engineering Teams

Context management is becoming a first-class capability to train and evaluate, not a harness to configure. Every team running long-horizon agents today has some version of a fixed policy: a compaction threshold, a summarization prompt, an eviction heuristic. CLMs suggest the right question is not which fixed policy to use but how to give the model the ability to develop its own.

For teams building production AI agent systems that run for hours or across many tool calls, the compute numbers are directly relevant. A 59% reduction in prefix-reuse FLOPs on a 12-hour task, or a 29.5% reduction on a terminal coding benchmark at matched accuracy, is real infrastructure cost savings at scale. The mechanism, surgical context editing rather than append-then-compact, requires no changes to model weights.

The serving co-design is the practical constraint. CLMs without SCR pay heavy re-prefill costs on every context edit. The agent serving stack design choices made early determine whether context management gains are realizable in production or consumed by serving overhead. Teams considering CLM adoption need to evaluate serving infrastructure alongside model behavior, not separately.

The skill-evolution loop is the lowest adoption barrier. A team does not need to retrain to improve context management: it evolves a natural-language skill document and provides it in context. The 35.9-point improvement on held-out ContextBench tasks is large enough to be meaningful at deployment without a training budget. That matters for teams working to extend large language model capabilities into long-horizon settings but cannot afford a full training run.

Key Takeaways

• CLMs treat the model’s live context as a directly editable file, replacing the append-only constraint (c_{t+1} = c_t ⊕ f_θLM(c_t)) with an arbitrary context function (c_{t+1} = f_θCLM(c_t)), implemented via Bash-editable files synchronized with the model’s actual context.
• On BrowseComp-Plus (deep research), CLMs achieve 11.4% higher accuracy with 21.5% fewer prefix-reuse FLOPs than the strongest baseline; on TerminalBench 2.1 they match accuracy with 29.5% fewer FLOPs; on EdgeBench’s 12-hour subset, 5% higher score at 59% fewer FLOPs.
• Online RL with a success-gated efficiency advantage in stepwise GRPO trains Qwen3.5-9B from 28.8% to 42.5% on BrowseComp-Plus, outperforming a Codex-style summary harness trained identically by 0.4 points while using 38.8% fewer FLOPs.
• Suffix Cache Reuse, the serving co-design, reuses cached KV states beyond the matching prefix after context edits, reducing server-side compute by 35% relative to standard SGLang at matched task performance, making arbitrary context editing viable in production.
• None of the existing context management approaches (AutoCompact, Self-Compact, Context-as-a-Tool, ACM, Sculptor) performs correctly on all four ContextBench tasks at 24× context pressure, confirming that fixed human-designed policies are the bottleneck, not the model’s reasoning ability.
• The skill-evolution loop improves held-out ContextBench accuracy by up to 35.9 points via an evolved natural-language skill document, providing a path to better context management behavior without model retraining.

Work With Origins AI

Origins AI builds production AI systems for engineering teams. If your long-horizon agents are hitting context runaway, compounding re-prefill costs, or degrading task accuracy past the two-hour mark, talk to our team.