Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures — Benjamin Rombaut, arXiv:2604.03515v2 (cs.SE), April 2026.
The gap the paper fills
The scaffolding around an LLM — the control loop, tool definitions, state management, context strategy — increasingly is the agent. The model is a commodity; the scaffold decides what it sees, what it may touch, and what happens when it fails. Yet the research literature has been studying agents through two lenses that both miss the scaffold. Capability-based surveys classify agents as "tool-using, planning, reflective" — a description that applies equally to a system running Monte Carlo Tree Search over candidate patches and to one running a bare while loop with test-driven retries. Trajectory studies watch what agents do at runtime, but treat the agent as a black box: they observe behavior without examining the scaffold code that produced it. The result is a gap this blog has been circling for months: the deployable unit is the model+harness pair, and nobody had actually mapped the harness side.
Rombaut's paper is the first attempt to map it at the implementation level. It is a source-code analysis of 13 open-source coding agent scaffolds, at pinned commit hashes, organized into 12 dimensions across three layers, with every taxonomic claim grounded in a file path and line number. This review covers what the paper found, what it missed, and why it matters for anyone designing or studying agent harnesses.
What the paper did
The corpus spans the open-source ecosystem's range of adoption: OpenCode (135k stars), Gemini CLI (100k), Codex CLI (72k), OpenHands (70k), Cline (60k), Aider (43k), SWE-agent (19k), mini-swe-agent (4k), AutoCodeRover (3k), Agentless (2k), Prometheus (1k), Moatless Tools (600), and DARS-Agent (70). Selection was disciplined, and the exclusions are as informative as the inclusions: Claude Code was excluded because it ships as a compiled binary with no readable source, MASAI because it never released code, Copilot Workspace/Cursor/Windsurf because they are proprietary, and MetaGPT/CrewAI because their unit of analysis is agent coordination rather than scaffold architecture.
The methodology is a qualitative case study with open coding: nine analysis dimensions were derived iteratively from a pilot on two architecturally contrasting agents (Aider and OpenHands), then applied to all 13 through a three-level template that separates observation (what the code does, with file references) from classification (how it maps to a dimension) from evidence (the pinned commit path and line). The honesty of the evidence trail is the paper's best methodological feature: a post-hoc verification pass checked 296 claims against the cloned repos, confirming 267, correcting 19 (mostly line-number drift), and accepting 10 as minor simplifications. Every claim in the results is independently checkable — which is rare in this literature and should be the norm.
The taxonomy: three layers, twelve dimensions
The framework is the paper's most reusable artifact, and it maps cleanly onto how this blog has been thinking about harnesses:
- Layer 1 — Control architecture (how the agent decides what to do next): control loop topology, loop driver, control flow implementation.
- Layer 2 — Tool and environment interface (how the agent touches code and execution): tool set design, edit and patch format, tool discovery, context retrieval, execution isolation.
- Layer 3 — Resource management (how the agent handles context, state, and models): state management, context compaction, multi-model routing, persistent memory.
Nine of these were the pilot dimensions; loop driver and edit/patch format were split out as independently discriminating sub-properties, and control flow implementation emerged from the code as an orthogonal axis. That last one is a nice catch: an agent's semantic loop strategy (ReAct, pipeline) is implemented via while loop, recursion, graph-as-control-flow, or exception-based signaling, and those implementations carry real architectural consequences — Cline's recursive main loop grows the JS call stack with every tool-use turn, while Prometheus's compiled LangGraph state machine makes control flow inspectable, serializable, and checkpointable.
Finding 1 — Spectra, not categories
The persistent result across all 12 dimensions: scaffold architectures resist discrete classification. Control strategies run from Agentless's fixed 10-stage pipeline (no feedback loop) to Moatless Tools' full MCTS with reward backpropagation; tool counts run from 0 (Aider — the user drives all navigation, the LLM has no callable tools) to 37 action classes (Moatless); context compaction spans seven distinct strategies; state management runs from Aider's destructive two-list overwrite to OpenHands' event-sourced immutable EventStream. Agents occupy positions on these spectra, and the positions reflect genuine tradeoffs, not arbitrary implementation choices.
The most consequential dimension is loop driver — who decides what happens next. Aider is user-driven: the LLM never runs grep, never opens a file it wasn't given, and the user absorbs the bug-localization burden entirely. Agentless and AutoCodeRover are scaffold-driven: the pipeline sequences phases and calls the LLM at fixed points. Nine agents are LLM-driven, with Prometheus a hybrid (LLM drives tool selection within each graph node, scaffold controls the edges). This dimension cascades: user-driven agents sidestep the localization bottleneck that trajectory studies keep finding is the primary failure point, while LLM-driven agents must solve localization as part of the task — which is why retrieval strategy correlates with loop driver, and why scaffold dimensions cannot be evaluated in isolation.
The tree-search gradient deserves special attention because it shows the same "search" label covering radically different machinery. Agentless samples ~20–40 patches independently and picks by majority vote — no tree, no interaction between candidates. DARS-Agent builds a tree with an LLM critic choosing among branch alternatives, but has no numeric rewards and no backpropagation, and recovers branch state by resetting the Docker environment and replaying every action from the root — correct, but expensive at depth. Moatless Tools runs real MCTS with rewards from −100 to +100, visit counts, and backpropagation, and solves the branching cost with shadow-mode execution: file modifications tracked in memory rather than written to disk. And the deepest structural insight is Moatless's: its ActionAgent (a single-step executor) is decoupled from its orchestrators (AgenticLoop or SearchTree), so whether the agent explores sequentially or with tree search is a configuration decision, not an architectural one.
Finding 2 — Five composable loop primitives
The reason the space is spectral rather than categorical is that the control structures compose. The paper names five loop primitives — ReAct, generate-test-repair, plan-execute, multi-attempt retry, tree search — and finds that 11 of 13 agents layer multiple primitives rather than relying on a single control structure. AutoCodeRover runs a full multi-turn ReAct interaction inside each stage of its phased pipeline; Aider's user-driven outer loop wraps an autonomous generate-test-repair inner loop; SWE-agent's RetryAgent combines iteration within each attempt with sampling across attempts. Only the two deliberately minimalist agents — Agentless and mini-swe-agent — come close to single-primitive purity.
This is the finding most relevant to this blog's harness patterns series: the pattern catalog is not a menu of mutually exclusive architectures but a set of composable primitives, and real agents stack them. Assigning a single label ("a ReAct agent") obscures the decisions that actually differentiate a system. The corollary for evaluation is pointed: a study comparing "ReAct agents" against "pipeline agents" conflates loop topology, loop driver, tool design, and context management into one binary.
Finding 3 — Convergence where constrained, divergence where open
The distribution of answers across dimensions is the paper's third result, and it is the most useful for design strategy. Dimensions converge where the constraints are external to the scaffold designer. Every LLM-driven agent converged on the same four capability categories — read, search, edit, execute — because those are the operations software engineering tasks require. Five independently-developed agents converged on string-replacement editing (str replace: old string, new string) over line-number or unified-diff formats, because exact string matching is more reliable for LLM-generated patches. Benchmark agents converged on Docker for execution isolation, because autonomous code execution without sandboxing is unacceptable in unattended evaluation.
Dimensions diverge where open design questions remain. Context compaction shows seven strategies across 13 agents, from mini-swe-agent's none-at-all (it crashes on ContextWindowExceededError) to Cline's LLM-initiated condense tool and Gemini CLI's summarization-with-verification "Probe" turn. State management runs from destructive overwrite to event sourcing, with tree-structured, graph-scoped, and SQLite-backed variants between. Multi-model routing runs from single-model simplicity to Gemini CLI's seven-layer classifier chain with an optional local Gemma model making client-side routing decisions — the only client-side model selection in the corpus. The paper reads this pattern as a frontier map: converging dimensions are candidates for standardization (MCP is an early transport-layer attempt, but no semantic contract for the four tool categories exists), while diverging dimensions are where research investment is most needed — no existing compaction strategy fully solves the "token snowball" problem that resource studies keep measuring.
The paper also delivers five cross-cutting themes that refuse to reduce to a single axis. Sampling vs. iteration (generate independent attempts, or refine one with feedback): Agentless is pure sampling, six of nine LLM-driven agents are pure iteration, and the distinction poses an acute evaluation problem — an agent that samples 40 mediocre patches and votes beats an agent that iterates one good patch, but benchmark scores conflate the two. Sub-agent delegation: five agents implement it through five different mechanisms, and who controls the delegation decision mirrors the loop-driver spectrum. Online vs. offline selection in tree search: DARS-Agent's leftmost-path extraction trusts its online critic entirely, while Moatless's discriminator re-evaluates all completed trajectories. Ecosystem maturity: DARS-Agent forked SWE-agent — a 700-line copy of the Agent class — while mini-swe-agent reuses SWE-agent's environments through structural typing, and the coexistence of fork-based and dependency-based reuse for the same upstream project means clean extension points have not yet emerged. And IDE as architecture: Cline's VS Code coupling buys it diagnostics, terminal, and file-change context no CLI agent can get, at the cost of platform lock-in.
Why this matters here
Three of this paper's themes are this blog's recurring arguments, now with a source-code evidence base attached.
First, the scaffold-model confound is real and it invalidates naive comparisons. The paper documents that trajectory studies compared agents running different models — OpenHands on Claude 3.5 Sonnet against Prometheus on DeepSeek-V3 — making it impossible to attribute behavioral differences to the scaffold or the model. That is exactly the better-harnesses-smaller-models finding generalized: model-swap failure is a harness artifact, and this taxonomy gives the vocabulary to say which harness dimension is the artifact. The paper even cites Bui's conclusion — "tool reliability matters more than model capability" — which is this blog's harness-is-the-product thesis in five words.
Second, evaluation must decompose. Benchmark scores conflate scaffold, model, and configuration in a single number; the taxonomy names the variables a controlled comparison would need to hold constant (same tool set, different loops; same loop, different compaction; model fixed). This is the empirical-SE discipline applied to scaffolds: attribute before you compare. And the sampling-vs-iteration finding says we need both single-attempt and multi-attempt metrics — a leaderboard number cannot tell you whether an agent's score comes from one good patch or forty sampled ones.
Third, token economics is scaffold economics. The compaction dimension is where the token snowball effect lives: naive history accumulation grows input tokens linearly with API calls, and the paper's seven strategies are seven different answers to where that budget goes. SWE-agent's polling parameter — keeping the prompt prefix stable to preserve provider prompt-caching across steps — is a single-line reminder that context strategy and API cost are the same dimension.
And there is a quiet connection to this blog's systems-of-systems post: the loop-driver dimension is the scaffold-level version of the directed-vs-LLM-driven question, and the convergence/divergence pattern is Maier's principle in miniature — dimensions converge where external constraints dominate, diverge where the design space is genuinely open. The paper is, among other things, an empirical map of where agent-architecture standards are ready to be written and where they would be premature.
What's missing
A review should say what the paper does not do, and the paper is unusually candid about its own limits.
Single-author analysis. All 13 analyses were conducted by one researcher, with LLM-assisted code navigation. The file:line evidence trail and the 296-claim verification pass mitigate this, but the verification was self-verification. The paper explicitly calls for independent replication, and the dimension classifications most needing it are the judgment calls — like whether Prometheus's graph-scoped state is a compaction strategy or a separate pattern.
The corpus is open-source-only, Python-heavy, and a snapshot. The most widely used coding agent in the world, Claude Code, is absent because its source is compiled — a survivorship bias the paper acknowledges: open-source agents may differ systematically from proprietary ones optimized for UX rather than benchmarks. Ten of 13 agents are Python, a consequence of SWE-bench's Python-only tasks, so multi-language architectural variation is unmeasured. And the analysis is pinned to specific commits while several agents were under active development — the taxonomy decays in real time, and the paper's proposed longitudinal re-analysis is the right answer but has not happened.
Static analysis and no performance claims. The taxonomy describes architectural capability, not runtime behavior — whether configurable features (Moatless's pluggable selector, MCP tool discovery) are actually used in deployment is invisible to source reading. And the paper deliberately runs no benchmarks, because SWE-bench scores confound scaffold with model and configuration, and documented solution leakage makes raw pass rates unreliable. This is methodologically right — it is the reason the taxonomy is trustworthy — but it means the paper is a map without a compass: it names the dimensions but cannot tell you which positions are better. That is left, correctly, to the controlled experiments the taxonomy enables.
Bottom line
This is the periodic-table moment for harness engineering: the field had elements scattered across codebases and blog posts, and now it has a named design space with verified positions. The paper's three findings — spectra, not categories; composable primitives; convergence where constrained — will age better than any specific agent classification, because they describe how the design space is structured rather than what it currently contains. Its value for this blog's readers is the vocabulary: when you can say "this agent differs on loop driver, compaction strategy, and routing," you can start attributing behavior, designing experiments, and comparing harnesses instead of models. What the taxonomy does not do — tell you which position on each spectrum wins — is honest work for the next round of controlled studies. The map is drawn; the surveying was overdue.