Nym Splits Its Agent Into a Decision Plane and a Generation Plane

On September 20, 2026, Ian T. Butler at Nym published a detailed technical account of rebuilding the company’s consumer agent around TypeSafe’s Jev classifier as a core execution primitive. What makes the post worth careful reading is not the speed improvement numbers, which are real but modest at the system level. It is the architectural diagram that emerges: a working agent that separates bounded decision-making from open-ended language generation into two distinct computational planes, with deterministic code retaining authority over anything that involves money, credentials, or security.

Why the Autoregressive LLM Is an Expensive Branch Statement

Most AI agents today route every decision through a frontier language model. The model selects which tool to call, decides whether to ask for user confirmation, checks whether credentials should be released, evaluates whether a browser action is safe, and generates the text of the actual tool argument. This works, and frontier models are genuinely good at all of these things. The cost is that autoregressive generation is slow for choices that do not require generation.

Autoregressive models produce output one token at a time. They depend on prefix caching to avoid reprocessing the entire conversation on each turn, but each new token still requires a forward pass proportional to the generation length. For a decision like “should this agent ask for approval before proceeding?” the correct answer is binary. Generating twenty tokens of reasoning before arriving at “yes” is expensive relative to what is actually being computed.

Ian T. Butler frames this directly: “Gemini Flash is not a bad model, but it’s an LLM. LLMs autoregressively generate tokens, often one at a time. They rely on prefix or KV caching to reduce repeated processing of previous tokens, but still do calculations for each new token. This makes them slower for these classification tasks than a model that produces its decisions in parallel.”

Nym previously ran seven Gemini 3.1 Flash Lite classifiers on every user request: action approvals, credential release, checkout state, commerce abuse detection, approval replies, capability selection, and memory relevance. These reviewers fed checks that were themselves still enforced in deterministic code. Spending limits, purchase authorization, and credential protections remained in application logic. The classifiers were the decision signal, not the authority. This distinction matters when thinking about how to replace them.

What Jev Is and How It Differs

Jev is a non-generative model from TypeSafe. You give it state (context, conversation, page content), questions, and the possible answers for each question, and it returns typed decisions with probabilities and confidence scores. TypeSafe describes its training methodology as Reinforcement Learning for Calibrated Decisions (RLCD), with outputs produced in parallel rather than sequentially generated token by token.

The confidence score is the property that makes Jev composable with a generative model rather than simply a replacement for one. An application can define thresholds: above 0.9 confidence, execute the decision automatically; below that, gather more context, try again with a richer observation, or fall back to a frontier LLM. This turns confidence into a programmatic routing signal rather than an internal model property that is invisible to application code.

For a practical sense of reported performance characteristics: TypeSafe states 70 to 500 ms end-to-end latency, a 32,000-token context window, and input pricing around $0.042 per million tokens with output reported as free. As the Langfuse integration post notes, Jev supports Choice, Score, and Noul decision primitives, and multiple questions can be evaluated against the same state in a single call, making it efficient for the multi-check pattern Nym was running.

Replacing the Seven Classifiers

Nym ran both implementations, Gemini 3.1 Flash Lite and Jev, against the same evaluation set: 167 calls across 88 distinct inputs covering all seven classifier types. The results are specific enough to quote directly from the source:

The headline number is a 4.1 to 5.7× speedup in median reviewer latency. Less reported but equally notable: Gemini had four errors and three wrong answers across the evaluation set. Jev had zero. The checkout state classifier went from 16/20 correct to 20/20. For a consumer agent handling purchases and credentials, that delta matters more than the speed gain.

Extending Jev Into the Browser Loop

Nym went further than swapping classifiers. Butler describes replacing the core browser execution loop, previously driven entirely by DeepSeek making sequential decisions about page actions, with a combined architecture where Jev handles bounded choices and DeepSeek handles generation and uncertain decisions.

The old loop:

  DeepSeek → inspect page → decide action → execute → inspect → decide → ...

The new loop:

  Jev → score possible actions + targets from page view
       │
       ├── high confidence → execute
       │
       └── uncertain → fresh observation → Jev → still uncertain → DeepSeek

Text fill fields are delegated separately to a generative model. Everything else, action type, target element, sequence ordering, is scored by Jev first. DeepSeek becomes the exception path, invoked only when the classifier lacks sufficient confidence.

The same pattern was then applied to main-loop tool selection. Jev receives the current context and the list of available tools and returns probabilities per tool. If confidence is high, the selection is used and the LLM generates only the tool arguments, with thinking disabled and the cached prompt prefix reused. If confidence is low, DeepSeek handles the full tool selection decision. The result was 41 generative model calls versus 72 on a 17-task benchmark, with both implementations completing 17/17 tasks.

What the Benchmark Actually Shows

This is where the post is most useful for skeptics. The full 17-task benchmark is shown task by task, with wall-clock times in seconds:

• Greeting: 5.083s vs 4.220s (Jev slower)
• Calendar question: 4.115s vs 6.576s (Jev faster)
• Wikipedia fact lookup: 15.870s vs 13.151s (Jev slower)
• Compare Wikipedia facts: 19.405s vs 27.547s (Jev faster)
• Shopping without a payment card: 21.939s vs 38.873s (Jev faster, large delta)
• Navigate Quince to men’s section: 21.736s vs 27.974s (Jev faster)
• Find products on LTT Store: 39.479s vs 25.381s (Jev slower, large delta)
• Total: 224.725s vs 234.123s

The task-level variance is substantial. Jev wins on some tasks by a wide margin and loses on others. The aggregate improvement is 4.0% faster wall-clock time across the 17 tasks. That is a real improvement, but it does not scale linearly from the 4.1–5.7× classifier speedup. End-to-end agent time is dominated by tool execution, network latency, and browser rendering. Replacing decision latency removes one bottleneck, not the primary one.

The cost story is cleaner at the unit level. Generative model spend dropped 36% from $0.18648 to $0.11930 for one benchmark pass. Adding estimated TypeSafe charges of $0.05916 (estimated from hourly usage exports, not direct per-request measurement), the combined model saving is approximately 15%. A later run with provider-reported TypeSafe token usage measured 14.88% combined savings on the same 17-task suite with 17/17 passing. Butler is transparent about the estimation methodology and its assumptions.

For the five software-building tasks (running tracker, calorie tracker, budget helper, habit tracker, reading list), the improvement was larger: 17.1% less time from request through delivery, measuring end-to-end through to the finished artifact. Total: 1,024.531s versus 1,235.307s with both completing 5/5.

Limitations and Open Questions

The evaluation set is small: 167 classifier calls and a 17-task general benchmark run on a system in private beta. Butler’s footnotes are unusually honest about this. Passing the evaluation set “does not establish correctness on all future inputs.” Task comparisons used separate runs and software versions. The six-case app run used a different browser implementation. Results vary with software version and service conditions.

TypeSafe costs are estimated from hourly usage exports, allocating tokens across runs in proportion to logged request counts and assuming equal average request sizes. This is a reasonable approximation but not a direct measurement. The export does not separate individual requests.

Jev’s RLCD training methodology is not publicly documented in sufficient detail to independently evaluate. TypeSafe has not published model weights, architecture details, or a reproducible benchmark harness. The vendor-reported figures of 200× faster and 400× cheaper than frontier models on classification tasks refer to TypeSafe’s own testing and should be treated as such.

There is also an open question about out-of-distribution inputs. A trained classifier has a defined decision space. When user behavior shifts, a new product decision is added, or an edge case falls outside the training distribution, the confidence score should flag the uncertainty and route to the generative model. Whether that routing holds reliably in practice is something that production data will reveal, not a 17-task benchmark.

What This Means for Engineering Teams

The practical takeaway is less about Jev specifically and more about what the architecture makes possible. Teams building agentic automation pipelines should distinguish between decisions that require generation and decisions that require classification. They are not the same computation, and treating them identically is expensive.

A decision like “which tool should I call next?” is bounded: the tool list is fixed, the context is structured, the answer is categorical. Routing this through an autoregressive model adds generation overhead to a problem that does not require generation. A confidence-producing classifier handles the common case faster and leaves the generative model available for argument construction, reasoning, and synthesis, where it is actually necessary.

The confidence threshold pattern is also worth noting for safety-critical paths. Nym’s credential release and checkout classifiers now route to DeepSeek when confidence is below threshold, rather than executing on a lower-confidence classification. This is different from simply running a faster model; it is a programmatic escalation policy written in application code, not hidden inside the model’s generation.

For teams thinking about how AI agents interact with language model inference at scale, the implications connect directly to how LLM inference actually works: KV cache reuse, prefix consistency, and generation length all affect cost and latency. Nym’s approach of turning off thinking for routine argument generation and keeping conversation routes consistent to reuse cached prefixes is applied inference optimization layered on top of the architectural split, not a separate concern.

Butler closes with the next experiment: using Jev in a sliding window over conversation history excerpts to select which context to include, with subsequent passes to decide whether to expand excerpts to full fact lists, and then to full stored messages. The same classification primitive that replaced seven reviewers could, if the pattern holds, control context selection for the entire agent memory system. That would be a larger claim; it is not yet tested.

Key Takeaways

• Nym replaced seven Gemini 3.1 Flash Lite classifiers with TypeSafe’s Jev, achieving 4.1 to 5.7× faster median latency per reviewer and improving correctness from 160/167 to 167/167 on the same evaluation set.
• Tool selection in the main agent loop was also moved to Jev, reducing generative model calls from 72 to 41 (43% fewer) while completing 100% of a 17-task benchmark.
• End-to-end wall-clock improvement was 4.0% across the 17 tasks, substantially smaller than the per-classifier speedup, confirming that decision latency is not the dominant factor in overall agent time.
• Combined model cost savings were approximately 15%, with generative model spend down 36% and TypeSafe charges adding back part of the saving.
• Authority stays in deterministic code: spending limits, purchase authorization, and credential controls were not moved to Jev. The classifier feeds those checks; it does not replace them.
• The confidence score is the architectural key: it allows programmatic escalation to a frontier LLM when classification certainty is insufficient, making the split composable rather than a hard cutover.

Work With Origins AI

Origins AI builds production AI systems for engineering teams. If your agent architecture routes every decision through a frontier LLM and you need to separate decision cost from generation cost at scale, talk to our team.