From Five Agents
Down to One Loop.

Respan is an agent observability and evals platform — providing tracing, evaluator pipelines, experiment comparison, and more. Their product has an embedded AI agent that helps users operate platform features: browse logs, build evaluators, run experiments, configure automations.

The rewrite target was this embedded agent. Not an agent users deploy for their own tasks, but Respan's own dogfooding experiment subject. This allowed strict controlled comparison — same model, same parameters, same evaluation slate, with architecture as the only variable.

This "dogfooding your own platform" setup maximizes data comparability, but comes with limitations: sample size = 1, and Respan is the vendor selling the evals platform, making the article both a case study and product marketing. The specific frontier model used was not disclosed — only "same model and settings."

V1 was the classic 2024 multi-agent pattern: a top-level router agent (~10K char prompt) reads user messages and dispatches to four domain specialists — logs / evaluators / automations, etc. Each specialist has its own ~50K char focused prompt and narrow tool set. The router uses each specialist's return to decide next steps.

V2 eliminated the router and all specialists. A single agent decides each action step in one continuous reasoning chain until the task is complete — same model, same parameters, architecture as the only variable.

But giving the agent full context comes at the cost of a ballooning resting context. The key to making single loop practical is two-tier tool loading — core tools always loaded, long-tail tools behind search, schemas pulled on demand.

Respan dogfooded their own platform — datasets pulled from production traces, evaluator pipelines self-built, experiments compared V1 vs V2 on the same slate. All three evaluators use 1-100 strictness-anchored scoring, judged by claude-haiku-4-5.

The same pattern held on OOD real customer questions, with Helpfulness & Completeness showing the most significant gains — in real scenarios, "stopping at confirmation" means "the customer didn't get what they wanted."

The article's verdict: regression net ensures known failure modes don't regress, OOD set ensures improvements generalize to unseen questions. Both layers are indispensable.

Back when V2's prompt was still a rough draft, the Respan team noticed the agent doing something they hadn't told it to. When it hit an unfamiliar request or an unexpected tool error, it would call docs_search on its own, multiple times, verifying the current schema, double-checking which ID type a field expected, working out why a tool had failed.

Nothing in the prompt said "look up the docs when unsure." The loop just had the room.

V1 couldn't do this. A specialist that hit an unfamiliar error returned a string to the router, which had to decide on that summary without re-entering the loop. V2's loop takes the error, reads the docs, retries with the right shape, and only then turns back to the user — all in one pass, no state lost.

This is not prompt engineering — it's a feature of the architecture, not the prompt.

Respan's argument has boundaries — the article explicitly states "both patterns are valid; the choice has trade-offs around context fidelity, prompt blast radius, latency, and how much of the model's reasoning is visible to the user." The core test: if your agents are passing context between each other rather than doing truly independent work, you probably just need one loop with full context.

Tool Loading is the key to making Single Loop work — not "give the agent all tools and pray," but core set always present + long tail behind search. This is essential prerequisite engineering.

Regression Net is the minimum viable eval for Agent Rewrites — fixed internal probes (covering known failure modes) + OOD real customer questions (verifying generalization). Both layers are indispensable.