ContextRender: Agent Context Built From Execution Dependencies

Savini Kashmira, Jayanaka L. Dantanarayana, Lingjia Tang, and Jason Mars at the University of Michigan identified a gap that recency-based and relevance-based context management both miss: neither tracks how earlier tool results are actually used in subsequent execution. The ContextRender paper (arXiv:2609.37743, September 2026) addresses this by building a persistent execution graph that records observed reuse as the agent runs, then uses that evidence alongside recency and semantic relevance to select tool results before each model invocation, matching full-history performance at a 6K token budget and cutting inference cost by 10.2%-32.2%.

Why Recency and Relevance Fail Long-Running Agents

The problem shows up clearly in a concrete example from the paper. The benchmark task is settling a shared dinner bill among roommates using Venmo. The agent works through it across 18 turns. At Turn 8, the agent calls search_contacts(relationship="roommate") and gets back contact information for three roommates: Kathryn, Troy, and Nancy, including names, email addresses, and other identifiers. At Turn 9, the agent uses the three names to calculate the bill split across four people. Then at Turns 15 and 16, the agent uses Kathryn’s and Troy’s email addresses to create payment requests. At Turn 18, the agent needs Nancy’s email address to complete the final payment request.

Nancy’s email address is only in the Turn 8 result. It was not copied forward, not repeated in any subsequent turn, not referenced in any intermediate output. At Turn 18, both recency-based and relevance-based selection independently fail to include it.

Recency fails because Turn 8 happened long ago. By Turn 18, there are many more recent tool results, and the budget constraint pushes Turn 8 out of the retained window. The fact that the information is still needed does not change how old it is.

Relevance fails for a different reason. At Turn 18, the agent is in the middle of processing a payment, so the most semantically relevant tool results are the ones about Venmo payment APIs, payment request structures, and account identifiers, not a contact search from the early part of the session. The semantic distance between a contact lookup from Turn 8 and the payment operation at Turn 18 is large enough that relevance scoring deprioritizes it.

What’s easy to miss: the payment calls at Turns 15 and 16 both drew from the Turn 8 result. Those uses were observable. They are direct evidence that Turn 8 continues to contribute. Recency and semantic similarity are inferred proxies for usefulness; observed reuse is evidence from execution. Neither conventional approach captures this because neither tracks the provenance of values flowing through the agent’s steps.

ContextRender’s Architecture

ContextRender has three components: a persistent context graph storing the full execution history, a Tool-Flow Analysis (TFA) module that identifies and records reuse relationships, and a renderer that combines multiple signals to select what goes into the context window before each model invocation. The design commitment is a clean separation between durable agent state and the temporary projection the model sees at each step. The persistent graph is the ground truth; the context window is a view over it, analogous to a materialized view in a database: the query processor gets a sized representation of the underlying data, but the underlying data stays complete. Unselected tool results remain in the graph and can be selected at any later invocation. Nothing is permanently discarded.

Tool-Flow Analysis: Tracking Observed Reuse

Tool-Flow Analysis (TFA) is the module that produces what the paper calls “observed reuse.” It inspects each new model output and each new tool call against the set of earlier tool results, looking for values that originated in those earlier results and now appear in the current output or call.

The matching is exact and case-sensitive. TFA applies extraction rules to identify qualifying values, the kinds of values that propagate through agent execution: entity identifiers, email addresses, numeric IDs, account references, and other specific strings that are unlikely to appear by coincidence. When TFA finds a qualifying value in a model output or tool call, it links that use back to the earliest tool result that introduced the value. Intermediate copies in model outputs do not get credited as original sources. The attribution goes to the first result that introduced the value into the execution.

Each detected use increments the reuse score c_t(r) for source result r at time t. When TFA detects Kathryn’s and Troy’s email addresses appearing in the Turns 15 and 16 payment calls, it links those uses back to the Turn 8 result and records them as reuse edges in the persistent graph. Nancy’s email has not yet been used, but Turn 8’s reuse score has increased and the reuse edges provide explicit evidence that this result is still in active use, evidence that neither recency nor semantic relevance can supply.

The interesting part is that TFA requires no model capability. It is a deterministic inspection pass over token sequences. The agent does not need to be aware of it, and it does not affect model outputs. It runs as an interception layer that observes what the agent does and records the dependency structure that emerges from actual execution.

The Persistent Context Graph

The persistent context graph G stores every message, tool call, and complete tool result in chronological order. For each tool result, the graph records the complete content of the result, its position in the execution sequence, its timestamp, and a content embedding computed at storage time. The reuse edges that TFA identifies are also stored in the graph, linking later uses back to the source results that introduced the reused values.

The embedding supports semantic relevance; position and timestamp support recency; reuse edges support the observed reuse signal. Storing all three at graph construction time means the renderer can apply any combination at selection time without re-examining raw content. A runtime adapter records entries into the graph as execution happens, as a lightweight interception layer that does not modify the agent’s execution logic.

Nothing is removed from the graph. A tool result not selected at Turn 12 is still in the graph at Turn 18 and can be selected then. This is a different design commitment from pruning-based systems, which delete messages permanently, and from compaction-based systems, which replace earlier interactions with shorter summaries. Once pruned or summarized, specific details are gone. The ContextRender graph preserves complete tool result content across the full execution history, keeping it available at any future invocation.

The Renderer: Combining Three Signals

Before each model invocation, the renderer selects which historical tool results to include in the context window, up to history budget B. In the experimental setting, B is 6K tokens, well below the models’ maximum context windows. If accumulated history fits within B, the renderer includes everything. When it does not, the renderer combines three signals to rank tool results: recency (position and age), semantic relevance (embedding similarity to current state), and observed reuse (the reuse score c_t(r) accumulated by TFA). The ablation results make clear that each signal contributes independently, and that observed reuse produces the largest gains in exactly the cases where recency fails: results that were used early and continue to be used late.

Tool results not selected at a given invocation remain in the graph, their reuse scores can still increase, and they remain candidates at any future invocation. This is a key architectural difference from methods that treat selection as a permanent decision.

Results

The evaluation covers AppWorld (Trivedi et al., 2024) and an 8-objective QA benchmark across three execution models. At the 6K history budget, ContextRender consistently achieves the highest task performance among managed-context baselines on both benchmarks. The comparison is against other methods also operating under a constrained history budget, not against unlimited context. Within that class, tracking execution dependencies outperforms selection by recency alone, by semantic relevance alone, and by combinations that exclude observed reuse.

Mean standardized inference cost reduction relative to passing full accumulated history is 10.2% to 32.2%, depending on benchmark and execution model. The upper end corresponds to tasks where many accumulated tool results are never reused after introduction, allowing the renderer to exclude them safely. Performance is close to or above that of passing full accumulated history, validating the design assumption: for tasks with localized dependency structure, a well-selected 6K context can match or exceed the performance of passing everything.

The ablation findings are the most informative part of the results. Adding observed reuse to recency and semantic relevance improves task performance, improves retention of results reused later, and produces the largest recall gains for results reused after long intervals. That last finding confirms the system solves exactly the failure mode the AppWorld example illustrates. The reuse edges recorded at Turns 15 and 16 provide the signal that keeps the Turn 8 result in scope at Turn 18.

Limitations and Open Questions

The authors are specific about where TFA breaks down. Three cases undermine the tracking. Free-form browser interaction does not always expose the specific values moving between steps; if a value appears visually on a page but not in structured tool output, TFA cannot observe it. Implicit side effects are similarly invisible: if a tool call changes state that later calls depend on, but the changed state does not surface as a value in any tool result, the dependency is missing from the graph. And synthesized information, where the model combines or transforms values from multiple sources into something new, may not match any exact string from any individual tool result.

These are not edge cases. Web browsing, form interaction, and multi-step data transformation all produce implicit and synthesized dependencies. The authors describe ContextRender as not yet a universal replacement for semantic context management. The correct framing is that it works well for tasks where execution has structured, observable data flow, and should complement rather than replace semantic retrieval where dependencies are implicit. Scale is also an open question: the persistent graph grows without bound, and the per-invocation TFA scoring overhead is not yet characterized for multi-session or open-ended agents with very long task horizons.

What This Means for Engineering Teams

The core architectural commitment for teams building production AI agent architectures: the agent’s durable state is the execution graph, not the context window. Most current agent frameworks treat the context window as working memory, and context management as deciding what to include or exclude from a flat message list. That framing makes the third signal impossible to obtain, because the message list contains no record of which tool results contributed to which later operations.

Context management cannot be done well as a post-hoc filter. Recency and relevance are inferred proxies; observed reuse is evidence from execution itself. Teams thinking about agent serving stack design should consider where provenance tracking fits, and whether the execution layer can expose dependency information to the context manager.

For tasks with structured tool I/O, the 10.2%-32.2% cost reductions are achievable at a 6K history budget without accepting performance degradation. The more important implication is reliability. A context manager that loses a critical identifier from 10 turns ago because it looked semantically irrelevant at selection time produces failures that are hard to debug: the model’s output is coherent but based on an incomplete task state. ContextRender gives teams an observable, inspectable dependency structure to reason about when those failures occur. The reuse edges in the persistent graph are auditable artifacts. For teams working on AI engineering services that require reliability guarantees, that matters more than the efficiency numbers alone.

Key Takeaways

• Recency and semantic relevance both fail on long-horizon tasks because they use inferred proxies for usefulness, and the AppWorld dinner bill example shows a realistic case where both fail simultaneously: a contact lookup from Turn 8 that remains critical at Turn 18 but looks old and irrelevant from both perspectives.
• ContextRender’s Tool-Flow Analysis adds a third signal by tracking which values from earlier tool results appear in later model outputs and tool calls, attributing each use to the first result that introduced the value and accumulating a reuse score that increases with each observed use.
• The persistent context graph is the architectural commitment that makes this work: complete tool results, timestamps, embeddings, and reuse edges all stored durably, with the context window treated as a temporary projection rather than the ground truth, so no information is permanently discarded.
• At a 6K history budget, ContextRender matches or exceeds full-history performance across AppWorld and 8-objective QA, with inference cost reductions of 10.2% to 32.2%, but the approach works best when tool I/O exposes clean structure and breaks down for free-form browser interaction, implicit side effects, and synthesized values the model produces internally.

Work With Origins AI

Origins AI builds production AI systems for engineering teams. If your agents are dropping critical tool results mid-task because conventional context management cannot distinguish what execution actually depended on from what merely looks recent or relevant, talk to our team.