Your agent's bill
is set by the orchestration
layer, not the model
When it comes to optimizing agent cost, most people fixate on model choice and wait for per-token prices to fall. A controlled experiment from Writer closes off that path: lock the tasks and the models, swap only the harness (orchestration layer), and cost drops 33% to 61%. Under the baseline, moving from the most to the least expensive model saves only 36%.
The spend is in the orchestration layer, not that one model call
An agentic task is never a single model call. The paper opens with "reconcile these two contracts and draft a revision memo" — a request that unfolds into a dozen turns: system prompt, tool definitions, retrieval results, intermediate reasoning, tool output. In a naive implementation, every one of those is replayed in full on each subsequent turn.
The task's token bill is the sum of that loop, and the loop is set by the software around the model, not by the model. The paper calls that software the harness: it decides what enters the context window, which tools are visible, when to retrieve, when to retry, when to delegate, when to stop.
The paper observes that the industry's default response to rising agent capability is to spend more tokens: reasoning models emit thousands of deliberation tokens per answer, agent frameworks replay conversation history quadratically in the number of turns, tool ecosystems inject every definition into every call. The paper names this trajectory token maxing — buying capability with tokens.
"buying capability with tokens — longer reasoning traces, more agent turns, wider tool payloads, larger replayed contexts — so that tokens per task grow faster than task value."
Falling per-token prices only mask the problem, the paper argues; they do not fix it, and total spend rises anyway. It calls this a textbook Jevons dynamic (efficiency gains in a resource raise its total consumption): as the unit price drops, teams treat tokens as nearly free at the margin and scale usage to match. The paper's judgment is that the lever against token maxing is the harness, because four of the five input terms per task, plus retries, are code, not model.
Lock the tasks and the models, swap only the harness
The layered architecture gives the paper a natural experiment: pin the tasks and the models, vary only the orchestration layer. The team ran the same 22 locked evaluation tasks across six foundation models, comparing two harnesses: a conventional production agent loop (the baseline) and the Writer Agent Harness. Every task, prompt, model, judge, and price list is identical; the only variable is the orchestration code.
The six models span five vendors and three weight classes: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, and Palmyra X6. The paper stresses the spread is deliberate: the claim is about the layer above the model, so it must be tested across weight classes and vendors, not staked on one flagship.
| Dimension | Baseline | Harness | Change | Paper's reading |
|---|---|---|---|---|
| Quality (task completion) | 0.78 | 0.81 | +0.03 | a wash at n=22 |
| Cost per task | $0.21 | $0.12 | −41% | decisive |
| Latency per task (median) | 48 s | 27 s | −44% | decisive (1.8× faster) |
| Tokens per task | 14.2k | 8.8k | −38% | decisive |
| Quality per dollar η$ | 3.71 | 6.75 | +82% | derived |
| Completions per million tokens | 54.9 | 92.0 | +68% | derived |
The paper treats the two kinds of result differently: at n=22 the quality difference is only directional, not statistically significant, so +0.03 is reported as a "wash" rather than a gain; while the cost, token, and latency differences are large and sign-consistent across all 22 prompts and all six models, and so decisive at this sample size. The single most robust result is the distribution of the cost cut across models: every model got cheaper, from −33% (Gemini 3.1) to −61% (Flash 3.5), with no exception.
"moving from the most to the least expensive model under the baseline saves 36%, while keeping any model and adopting the harness saves 33% to 61%."
The largest relative gain lands on the fast tier: Flash 3.5 drops 61% in cost and 55% in latency. The paper explains this via the cost decomposition: on a small, cheap model the harness overhead (replayed history, broadcast tool definitions) is a larger share of the bill, so removing it removes a larger fraction.
Why the harness sets the bill: write the cost as a formula
The paper does not stop at "it saves money" — it writes the bill as a formula. Take a task as a k-turn agent loop, each turn submitting input tokens and receiving output tokens. The root of runaway cost is one thing: a naive harness replays the entire history on every turn.
The input side decomposes into five terms the harness assembles: system prompt, history, tool definitions, retrieval, and this turn's user input. Across these, whether the system prompt is replayed or cached, history replayed or compacted, tool definitions broadcast or scoped, how large the retrieval payload is, whether a retry fires — all set by the harness.
If each turn replays the full history, total input tokens grow quadratically in turn count: turn 10 replays the first 9 turns, turn 50 replays the first 49, and the history snowballs. A harness that compacts history, caches the invariant prefix, offloads bulky tool output, and truncates retrieval to the evidential minimum turns that quadratic term into roughly linear. The paper's line: nothing about the model changed; the bill did.
The paper adds one key fact that sharpens the bill: the price of an input token is not one number. Providers bill tokens that repeat a previously seen prompt prefix at the cache rate, about 0.1× list (a tenth). If most input is cache reads, the dominant term is billed at a tenth. The line the paper drives home:
"Crucially, h is neither a model property nor a provider favor: it is a function of prompt byte-stability across turns, which is set entirely by how the orchestration layer assembles context."
So the harness controls both factors of the bill: how many tokens are submitted, and the price at which the dominant ones are billed. The escape from token maxing is not cheaper tokens but fewer tokens for the same work — and that route sits squarely in the harness.
Where the savings actually come from: six mechanisms
The paper maps the observed cuts back onto the cost formula; the savings must come from the terms the harness rewrote. The design goal, in one sentence: maximize the fraction of tokens that are cached, decision-relevant, and spent inside recoverable work — and enforce all three with structure, not model behavior. Each mechanism below gets a concrete picture.
Two-zone prompt: weld the invariant to the front
Every prompt is split in two. The top zone never changes byte-for-byte: tool definitions, system prompt, the append-only transcript. The bottom zone is rebuilt each turn: clock, file listings, one-shot reminders.
The rule is strict: anything that changes per turn is structurally banned from the top zone, and the cache-marker logic refuses to place a breakpoint at or after the first volatile message. Up to four provider breakpoints are pinned (after the schema catalog, after the system prompt, sliding along the two newest durable turns), with one-hour retention latched per session so the policy never flips mid-run — because providers only give the deep discount to a prefix identical to last time.
Don't delete history — fold it into a checkpoint
History snowballs. At 80% of the model's input budget, the harness doesn't crudely truncate; it folds older history into a typed checkpoint: durable memory of decisions and constraints, an eight-section resumable summary, verbatim user requirements, and skill references. The 4 to 12 most recent messages stay verbatim.
Two subtleties: the summary runs on a cheaper helper model off the paying loop; and the checkpoint is a durable row, so the rebuilt prompt becomes a new cacheable prefix. Compaction and caching don't fight — this is the step that turns the quadratic snowball into linear. Two safety rails on top: an empty or degraded summary aborts the compaction rather than persisting it, and if the prompt still doesn't fit, a deterministic ladder (shrink the tail, then middle-truncate with a restate-what-matters nudge) guarantees a sendable request.
Information stays reachable, without taking up space
A family of tricks keeps information available without keeping it in context. A sub-agent acts as a firewall: it reads a large body in its own context and returns a summary capped at 8KB, with citations on a sidecar the parent never reads; delegation is depth-capped and idempotent under retries, so delegated exploration cannot inflate the parent loop. Skills use progressive disclosure: the prompt carries only a name-and-description table; the full document is read only when invoked.
Bulky tool output spills to files: shell output beyond 20K characters is head-and-tail previewed, the full output written to a workspace file under a banner forbidding the model to infer success from the preview. Plan state is projected once per turn as a compact rendering, which doubles as objective recitation against long-horizon drift. In one line: the filesystem is unbounded memory, the context holds pointers.
Waiting costs nothing; a crash isn't re-bought
An agent mid-run needs a human approval, or a long background job. The naive way loops and polls ("done yet?"), burning a turn's tokens each time. Here it suspends durably at zero cost and resumes on an ingress event.
The same durability layer bounds catastrophe: every event is journaled to a write-ahead log before it is streamed, a crashed or preempted run resumes under generation fencing at the exact next sequence number, and tool results are persisted before they are shown. The paper's line is the clearest: "A crash that loses a 40-turn run means re-buying 40 turns of tokens; here it means resuming from durable state."
Bound the retry-and-doom-loop multiplier
Agents fail, retry, get stuck in doom loops — these are the multiplier on the bill, and no per-token discount touches them. This mechanism disciplines failure: every failure is typed first (rate limit / stall / timeout / malformed stream / provider outage / permanent), and only whitelisted classes fall through to the next provider.
A circuit breaker halts a model that re-issues a byte-identical failing tool call a third time, with cause-aware steering. Mid-stream failures are discarded, and a discarded attempt leaves no side effect — precisely the piece generic library fallbacks omit: streaming failover in orchestration libraries is documented to work only before the first chunk.
Swap any model, ride the same floor
Swapping models shouldn't change code. Which model runs, over which providers, in what fallback order, is a typed route plan supplied as data; the loop never branches on a model name, and each provider stream is normalized into one contract before the loop sees it.
It also cleans up after weaker models: refs are inlined, double-encoded JSON arguments recovered, framework internals scrubbed from errors, and overloaded tool schemas split when weaker models misuse them. The paper's closing line: "the harness fixes the floor; the model sets the ceiling." That's what makes "swap only the harness, save 33–61%" hold for every model.
The paper draws the through-line of all six: token economy and output quality are one lever, pulled once. Long, distractor-dense contexts measurably degrade every frontier model tested; a mechanism that removes stale or bulky tokens is simultaneously cutting the bill and cleaning the model's working set.
"Aren't these all known techniques?" The paper answers it
Cached prefixes, compacted history, sub-agent isolation — taken one by one, each is familiar, and the paper does not pretend otherwise. It responds with a table: six widely used agent systems, checked cell by cell against these mechanism families (assessed from public documentation and a design-time source study, not head-to-head measurement). The claim is narrow and economic: among these systems, token economics is nowhere a first-class, published contract. Knowing a technique and enforcing it as structure, with a per-task accounting surface behind it, are two different things.
| System | Deployment class | Model-portable | Structural cache policy | Compaction contract | Firewalled delegation | Zero-token waits | Per-task accounting |
|---|---|---|---|---|---|---|---|
| Claude Code / Cowork | single-user client | no | yes | yes | partial | no | no |
| LangGraph | library | yes | app | app | app | partial | no |
| CrewAI | framework | yes | app | partial | no | no | no |
| AutoGen / AG2 | framework | yes | app | app | no | partial | no |
| Hermes Agent | personal agent | yes | partial | yes | yes | partial | no |
| Writer Harness | multi-tenant runtime | yes | yes | yes | yes | yes | yes |
The paper's verdict, class by class. Claude Code and Claude Cowork are called among the most sophisticated harnesses in wide deployment; several of this harness's patterns (cache-breakpoint latching, byte-stable prefixes, sub-agent task splitting) were adopted from or validated against them. The differences are deployment class and contract: single-user, client-side tools bound to one model vendor, with token management not exposed as a per-task accounting surface. LangGraph supplies excellent low-level primitives (graph orchestration, checkpointing, interrupts) and deliberately leaves cache policy, compaction, offload, and failure governance to the application — token economics becomes the application team's unbudgeted responsibility. CrewAI and the AutoGen lineage organize work as shared-transcript conversations, which is a token multiplier by construction: each participating agent re-reads the growing conversation and carries its own role preamble; the paper cites Anthropic's own measurement of agents at roughly 4× chat-level tokens and multi-agent systems at roughly 15×. Hermes Agent is a genuinely model-agnostic open-source harness with isolated sub-agents, but the design-time source study found it does not place its tool schemas inside a cached prefix — forfeiting the largest single discount on an input-dominated workload.
The last column is the whole paper in miniature: without per-task accounting built into the orchestration layer, token maxing is unobservable — and what cannot be observed cannot be managed.
A counterintuitive boundary: the harness picks the model
The efficiency gain is unconditional; the quality gain is not — it tracks model capability. The paper collapses each model's average quality gain across eight capabilities to a single number and plots it against baseline strength; the relation is near-linear (r = 0.99, six points, flagged by the paper as suggestive not conclusive). It names the slope harness leverage: the rate at which a model converts orchestration structure into quality.
| Model | Quality gain | Sub-agent delegation completion |
|---|---|---|
| Palmyra X6 | +0.079 | 0.86 · above bar |
| Sonnet 4.6 | +0.073 | 0.85 · above bar |
| Gemini 3.1 | +0.050 | 0.70 · degraded |
| GLM 5.1 | +0.028 | 0.58 · degraded |
| Flash 3.5 | +0.010 | 0.45 · unreliable |
| Qwen 3.6 | −0.031 | 0.42 · unreliable |
Across the 48 capability×model cells, 30 improve, 11 hold, 7 regress — and all 7 regressions land on the three smaller models (Flash 3.5, Qwen 3.6, GLM 5.1), concentrated in the most orchestration-heavy capabilities (tool use, multi-step Playbooks, presentation generation).
"The same richer harness that a strong model converts into quality, a weaker model experiences as load."
The asymmetry has a corollary: a weak model on the harness still gets its 44% to 61% cost cut — it just doesn't get better at the same time. That decouples the two reasons to adopt a harness and prices each on its own. The one genuinely new capability the harness adds — delegating work to spawned sub-agents — puts the boundary at its sharpest: delegation completion clears a usable bar only on the two strongest models, and is unreliable on the fast tier. The paper calls this a capability floor: an orchestration feature carries a capability floor, and below it, exposing the feature produces failure, not function.
From this the paper offers a direction for routing research: requests should be routed by capability demand, not just prompt difficulty. A request that will use a sub-agent belongs on the strongest model no matter how simple its text; a grounded Q&A request can take the 61%-cheaper fast tier with no quality penalty.
"Token maxing is invisible in benchmark tables, which report quality, and painfully visible in cloud invoices, which report tokens."
Writer · The Harness Effect · arXiv 2607.06906
Two corroborations: the same thing from the other end
What follows is corroboration, not the paper's main body. Two datasets from practitioners, from the concrete settings of "AI coding" and "multi-agent," back the paper's claim that the load is input-dominated and the orchestration sets the bill.
Cursor data Third party · vendor self-reported
Cursor's report on two years of aggregated usage has one direct line: 90% of token usage is input tokens, most spent reading the existing codebase and docs, with output code a minority. It echoes the classic "read vs write" ratio well over 10:1 from Clean Code, and matches the paper's input-dominance from the data side.
Cursor quantifies caching to the extreme: without caching, token cost would be 10× higher; with caching counted, output tokens are only 0.6%, the rest split cache read 90% / cache write 2.5% / input 7%. Gergely Orosz's line ties it straight to building your own harness: "if you roll your own agent harness, you also need to put an efficient caching layer in place to match the efficiency of tools like Cursor." Which is exactly what Mechanism 1 is about.
This data is Cursor's own, and Cursor has an incentive around its Composer model, so read it with that caveat.
Factory's multi-agent retro Third party · conference talk, self-reported
Factory showed a multi-agent system running production code, backing the paper from the angle that orchestration, constraints, and boundaries set reliability and cost. Its core judgment matches the paper's: the agents aren't the problem, the coordination is. The production architecture splits three agents by cognitive mode (generating / criticizing / coordinating), not task type: the Drafter sees only task and code, never its own failures; the Reviewer is pure criticism, never sees the original spec, structurally eliminating self-review bias with a different context; the Integrator has the broadest context but the narrowest action space — it can merge or reject but cannot write code itself.
"Constraints eliminate failure classes" is its refrain: every constraint added removes a class of failure. On economics, it rebuts the "multi-agent is more expensive" misread: multi-agent costs about 2.3× per task versus single-agent, but with 3.1× the success rate and 74% lower rework; counting the fully-loaded cost of an engineer redoing failed work, multi-agent is about 40% cheaper per successfully completed task. To be clear, these figures — 91%, 2%, 2.3×, 3.1×, and the ~15% breakeven — come from Factory's conference talk: self-reported and third-party unverified.
| Cost basis | Single agent | Multi-agent |
|---|---|---|
| Cost per task | $0.12 | $0.28 (2.3×) |
| First-pass success rate | 31% | 96% |
| Cost per successful task | $0.39 | $0.29 |
Breakeven is roughly a 15% improvement in success rate; a 3-to-5 specialized-agent setup consistently beats single-agent on cost-per-successful-completion. Numbers self-reported from Factory's production system, early 2026.