Anthropic Publishes an AI Agent Architecture Guide: Ask Three Questions Before Deciding Whether to Go Multi-Agent
Multi-agent systems can boost performance by 90.2%, but they also cost 10-15x more in tokens — this three-question framework helps you decide whether it's worth it before you build.
- Anthropic published an enterprise-grade AI agent architecture guide, drawing on real deployments from customers like Coinbase, Intercom, and Thomson Reuters to lay out architecture patterns and a selection method.
- The guide covers six core architectures: single agent, hierarchical (supervisor), collaborative, sequential workflow, parallel workflow, and evaluator-optimizer workflow.
- Anthropic's internal research shows that for complex tasks requiring simultaneous exploration of multiple independent paths, multi-agent systems outperform single agents by 90.2% — but consume 10-15x more tokens.
- The guide offers a three-question decision framework (how much control you need, how many domains the problem spans, what your resource budget is) to help you pick the right architecture instead of defaulting to the most complex one.
- The core method is "start simple, evolve gradually": validate value with a single agent first, then upgrade based on business needs and data feedback — rather than building a complex system from day one.
An enterprise agent architecture handbook from Anthropic
Anthropic recently published an enterprise-facing AI agent architecture guide, compiling real deployment cases from customers like Coinbase, Intercom, and Thomson Reuters into six architecture patterns and a selection decision framework.
The white paper opens with a line that states its position outright: generative AI answers questions, AI agents solve problems. And what this handbook sets out to answer is the next, more practical question — for a given business, should you run one AI agent solo, or build out a whole multi-agent system? It breaks that choice down into an actionable decision framework, backing each architecture with quantified data from real customers.
Why it's worth reading: the guide surfaces one key trade-off — for complex tasks that require exploring multiple independent directions at once, multi-agent systems outperform single agents by 90.2%; but they also consume 10-15x more tokens than a single agent. This one trade-off is the foundation for every architecture decision in the entire guide.
Let's get one thing straight first: AI agents and ordinary automation scripts work very differently. Traditional automation requires every step to be hard-coded in advance — the process follows the script step by step, and it gets stuck the moment it hits a situation nobody anticipated. An agent's approach is to evaluate the task, pick the right tool, try an approach, check whether the result is good, and adjust its strategy — running that loop until the job gets done.
Every step has to be pre-written. The process is fixed and traceable, but it only ever walks the path laid out in advance — and breaks the moment it hits an edge case.
Given a task, it plans on its own: reads the problem, checks account history, searches the knowledge base, drafts a reply, and loops in a human when needed — adjusting its strategy the whole way based on intermediate results.
The guide opens with a batch of customer results, meant to show what these systems have delivered in real production environments.
These four figures aren't about scale — they're about coverage: customer support, security operations, fraud review, and cross-industry resolution rates — four completely different jobs, all held up by agents in production. What they prove is "this is worth investing in," not "which architecture to pick." The guide also gives a more relatable, day-to-day example: a retail bank used an agent to process credit risk memos. What used to take relationship managers weeks of manually cross-checking ten data sources now delivers a 20% to 60% productivity gain, cutting credit turnaround time by 30%. These numbers establish a premise — agents are worth investing in — but which architecture you use to build one is what this guide is really here to teach. It covers six architectures in total, which really fall into three families: ones where the AI decides for itself (single agent, hierarchical/supervisor, collaborative), ones where the process is pre-scripted by humans (sequential and parallel workflows), plus one dedicated to quality control (evaluator-optimizer). Let's go through them from simplest to most complex.
Step one: check whether a single agent is enough
The first principle the guide keeps hammering is "start simple." Don't jump straight to a complex system — first check whether a single agent can solve the problem. It's cheaper, easier to debug, and its metrics map more cleanly to business outcomes.
A single-agent system is just one AI agent running a continuous loop: sense the environment, decide the next step, take action, and repeat until the task is done or it hits a stop condition like "pause for human review." A few components hold this together at its core.
Two of the three components in the diagram are specific terms worth explaining up front.
A standard interface that lets an agent connect to external systems — databases, web search, internal tools. It's what lets an agent actually go query data and operate tools, rather than just chat. Think of it as a universal adapter for AI: whether it's plugging into a database or a search engine, the interface is standardized, so you don't need custom integration code for every connection.
Domain expertise, standard procedures, and tool usage packaged into a module that an agent can call on demand, instead of cramming all domain knowledge into the prompt. Think of it as giving the agent a set of specialized toolboxes — pull out the legal toolbox for legal questions, swap in the finance one for finance questions — one agent doesn't need to carry every domain in its head at once.
Good fit: open-ended problems where the path forward isn't clear at the start — you don't know how many steps it'll take or what obstacles you'll hit — and you want the agent to figure it out and adjust as it goes.
Not a fit: scenarios that demand a perfect answer on the first try. A single agent struggles here — either add specialized Skills to boost accuracy, or only then consider going multi-agent. Before upgrading, ask yourself: would adding Skills to the single agent already be enough?
Expand: how a research agent walks through a complex query in 8 steps
An employee asks: "Research the remote-work productivity tools engineering teams are adopting, and see if any correlate with our internal productivity metrics." This agent, wired up via MCP to a content library, business tools, and a dev environment, handles it like this:
- ① Receive the question → ② Initial analysis: determines this needs two data sources — external tool research and internal company metrics — and that the external research doesn't initially depend on internal data, so it can be split into parallel searches and correlated at the end.
- ③ Activate Skills: calls on research methodology, data correlation, and business insight skills, applying proven frameworks instead of reasoning from scratch.
- ④ Task decomposition and planning: settles on a plan of external web search, internal database query, parallel execution, and correlation-based synthesis.
- ⑤ Parallel tool calls: runs a web search (for efficiency-tool trends) and a SQL database query (for internal productivity metrics) concurrently, sharply cutting total time.
- ⑥ Iterative refinement: reviewing the initial results, it realizes it needs more specific engineering-team data, and issues a more targeted follow-up query.
- ⑦ Correlate and synthesize → ⑧ Generate the result: cross-references external trends against internal metrics to produce an integrated conclusion.
Signs you've outgrown a single agent — and need to go multi-agent
Multi-agent only enters the picture once a single agent hits a fundamental ceiling. The guide gives three clear trigger signals — but it also puts the cost right on the table.
Multi-agent means splitting a task apart and handing it to multiple agents, each with their own specialty, who then combine their work into one coherent answer. Anthropic's internal research gives the core number: for complex tasks requiring simultaneous exploration of multiple independent directions, multi-agent systems outperform single agents by 90.2%. Its take is that once intelligence crosses a certain threshold, "multi-agent systems become the critical path to scaling performance," because what a group of agents can accomplish far exceeds what a single one can — just like human organizations.
Put these two bars side by side and you get the anchor for the whole piece: performance really can jump substantially, but cost climbs just as much. Also, watch the scope of that 90.2% closely — it's the most easily misquoted number in the whole guide. It was only measured on tasks that "require exploring multiple independent directions at once." Don't cite it to justify every scenario. Hence the direct warning: simple queries shouldn't trigger an expensive multi-agent pipeline — the system needs to allocate effort dynamically based on task value.
① Open-ended problems: it's genuinely hard to predict how many steps it'll take, and the investigation needs to be able to pivot and explore side paths as it goes.
② The task spans multiple unrelated domains: research shows that once a single task mixes in two or more unrelated specialties (say, reviewing legal terms and running financial models at once), a single agent's performance drops sharply as it tries to juggle both.
③ Broad parallelism: the problem requires chasing multiple independent directions at once, and parallel processing delivers a real performance gain.
And the cost isn't just tokens: multi-agent systems make debugging much harder. Traditional debugging methods stop working here, because agents make dynamic decisions and every run is non-deterministic. As the number of decisions multiplies, you need tracing that captures how the agents communicate, delegate, and synthesize results with each other — otherwise, the moment coordination goes wrong, it's nearly impossible to trace.
Hierarchical (Supervisor): one lead managing a team of specialists
Once you've decided to go multi-agent, the first question is how to organize the group: with someone in charge, or without. The managed version is called hierarchical (supervisor): a central supervisor analyzes the request, delegates tasks to the right specialist agents, and synthesizes the results — forming a clear chain of accountability. It's also the most common multi-agent pattern.
The key mechanism: in a hierarchical system, each specialist sub-agent is treated as a "tool," and the supervisor decides which specialist to invoke via tool calls. This mirrors how a high-performing human team works — specialists focus on their own domain, while the coordinator handles delegation and integration. Sub-agents can themselves have sub-agents beneath them, and those lower layers are hidden from the layer above — the supervisor only deals with the lead of each sub-team and has no idea how many layers exist below.
Let's walk through the flow using a marketing team example.
The single biggest pitfall in this pattern is context management — worth calling out on its own.
As a system runs longer, conversation history and intermediate results fill up the AI's "brain capacity" (context window), showing up as window overflow, degraded reasoning, and broken coordination between agents. There are three fixes:
- Context editing: as the token limit approaches, automatically prune stale tool-call records and results while keeping the conversation flow intact.
- Memory tools: store information in a file system outside the context window, and retrieve it when needed — it can even persist across sessions.
- Built-in tool throttling: add pagination, range selection, filtering, and truncation to tools, and cap single-response size at a sane default (roughly 25,000 tokens, for example) to keep the context from overflowing.
The guide also admits this pattern uses more tokens overall, but its judgment is that the math works out: for high-value, complex tasks that need specialized expertise or exceed a single agent's context limit, the performance gain is worth the cost.
Collaborative: agents talking directly to each other to solve problems
This is the "no one's in charge" version — and it's also the one most easily confused with hierarchical/supervisor. The essential difference between the two comes down to one thing: nobody manages anybody. Coordination emerges organically from how the agents interact, rather than being imposed by a central authority.
In a collaborative system, agents are autonomous peers who communicate directly with each other, negotiate their roles dynamically, and work together to solve a complex problem. It reaches consensus through three mechanisms.
Group-chat style discussion: multiple agents share a common conversation thread, working through problems, making decisions, and cross-checking each other as they talk — like a project group chat.
Event-driven coordination: agents use "events" as a shared language — every action gets broadcast as a structured update, and other agents pick up work and sync context based on it.
Blackboard-style shared knowledge base: a central "blackboard" that every agent can read and write, where anyone posts a finding, acting as collective memory.
The risk in collaborative systems grows directly out of its freedom. Frequent inter-agent communication drives up compute cost and complexity; worse, "emergent behavior" can appear — behavior nobody explicitly programmed — where a small tweak can unpredictably ripple through how the whole system behaves. The way to handle this isn't hard rules, but a solid collaboration framework: define roles, how problems get solved, and the budget — while also preventing agents from endlessly punting tasks to each other, and setting up a conflict-resolution mechanism.
Put these two multi-agent patterns side by side and the difference becomes obvious at a glance. Click the tabs below to switch between the two structures.
A tree-shaped chain of command. One supervisor at the top, specialists below — tasks flow down, results flow back up.
- Clear chain of accountability — auditable, traceable
- The supervisor can enforce business rules and retain oversight
- Good fit for medium-control scenarios that need both flexibility and oversight, like customer support, content creation, or data analysis
A mesh of peer connections. All agents are equals who can talk directly to each other and reach consensus through discussion.
- No central decision-maker — coordination emerges from interaction
- Behavior is harder to predict — a feature, not a bug, in exploratory scenarios
- Good fit for low-control scenarios like research, brainstorming, or complex analysis
Sequential vs. parallel: should tasks queue up or run at the same time
The previous patterns all involve dynamic agent decision-making. These next two are pre-scripted, static agentic workflows that define what order agents execute tasks in and under what conditions. There's really just one core difference: do the tasks queue up, or run at once.
| Sequential Workflow | Parallel Workflow | |
|---|---|---|
| How it runs | Runs in a single line, one step at a time; each step's output feeds the next step's input | Multiple lines run at once, each producing its own result independently, then aggregated at the end |
| Core value | Trades time for accuracy: spending a bit more time turns every AI call into a more focused, less error-prone simple task; the process is predictable, cost is estimable, and you can debug stage by stage | Built for speed and multiple perspectives: several lines run at once, covering the problem from different angles for a more well-supported conclusion |
| When to use it | The task splits cleanly into fixed subtasks with a clear linear dependency, and needs an audit trail (compliance checks, approval chains, draft → review → publish) | Subtasks are independent and can run at the same time; multiple perspectives improve quality; speed matters more than coordination overhead (multi-perspective risk assessment, guardrail review where one model does the work and another screens for inappropriate content, multi-track parallel evaluation) |
| When to skip it | A single agent can handle it in just a few steps; agents need to collaborate rather than hand off; the process needs to loop back and iterate | Agents need to build on each other's output; a specific execution order is required; shared state needs to be modified without a conflict-resolution strategy; result-aggregation logic is so complex it hurts quality instead of helping |
Each workflow comes with a real case study. The sequential one is a data-science insight pipeline: an analysis request first goes to a scoping agent (determines the analysis type, routes it), then to a data engineering agent (extracts and cleans data), then to an analysis agent (runs statistical modeling, or flags complex requests for human handoff), and finally into human review. The parallel one is financial risk assessment.
Evaluator-optimizer: getting AI to grade and revise its own work
This is the last workflow — two AIs split the labor, running a loop that polishes quality until it hits the bar.
One AI generates content; another is dedicated to spotting flaws and giving actionable feedback, and the two go back and forth until it hits the standard — instead of generating once and calling it done. Like a writer paired with an editor who keeps sending drafts back for revision until it's good enough to publish.
The API documentation generation case study runs like this: a generation agent analyzes the codebase and writes a first draft (endpoint descriptions, parameters, examples, auth requirements); a technical evaluator agent checks it line by line against the actual code implementation (are parameter types right, is endpoint coverage complete, do the examples actually run), sends issues back, and the generation agent absorbs the feedback and revises.
Good fit: scenarios with clear evaluation criteria where iterative polishing delivers visible value — literary translation, code generation with safety requirements, professional documents that demand precise tone, or research tasks that need multi-step reasoning plus verification.
Skip it for: scenarios where the first draft already meets the bar, evaluation criteria are subjective or vague, or the time cost outweighs the quality gain; also skip it for real-time applications that need instant responses, simple tasks like basic classification, token-constrained environments, and cases where the evaluator itself lacks the domain expertise to give meaningful feedback.
The three-question framework + a real evolution roadmap
Let's straighten out the chain we've walked through: if a single agent is enough, don't add more; only go multi-agent once you hit one of the three signals (open-ended, cross-domain, broad parallelism); once you've gone multi-agent, first choose managed or unmanaged; whatever process can be pre-scripted goes to a workflow, and only what can't gets left to agent decision-making. Converge that chain and you get the single most valuable part of the whole guide: three must-answer questions. Get clear on these three first, then match them to an architecture — don't just stack on technical complexity. Below is a selector you can click through with your own situation to see which architecture it points to.
The guide adds a fourth question: do you need deep domain expertise? A single domain with established processes just needs a single agent equipped with specialized Skills; multiple distinct domains that must coordinate (say, legal review working alongside financial analysis) call for a multi-agent system, each with its own specialized Skills. Below is the full checklist version of this three-question framework, item for item matching the original — take it and apply it directly to your own project.
1. What level of control do you need? High control requirements (regulatory compliance, financial transactions, safety-critical operations) → Start with single agents or sequential workflows Moderate control requirements (customer support, content creation, data analysis) → Consider hierarchical multi-agent systems Low control requirements (research, brainstorming, complex analysis) → Collaborative multi-agent systems become viable 2. How complex is your problem domain? Single domain problems (answering product questions, processing returns, generating reports) → Single agents handle these efficiently Multi-domain but predictable problems (employee onboarding, compliance workflows, standard analysis tasks) → Sequential or parallel workflows Complex, open-ended problems (strategic analysis, research projects, system troubleshooting) → Multi-agent architectures 3. What are your resource constraints? Limited budget/tokens → Single agents or carefully designed parallel workflows Time-to-market pressure → Start with single agents, plan an evolution path Long-term strategic initiative → Design for modular evolution 4. Do you need deep domain expertise? Single domain with established workflows → Single agent with specialized Skills Multiple distinct domains requiring coordination → Multi-agent systems with specialized Skills
Quick-reference table: which pattern fits which scenario
The constraints above translate into a clear lookup table — check it directly when deciding which category a given business case falls into.
Single agent works best for: - Customer support flows with clear category boundaries - Document processing with well-defined business rules - Code review and basic development tasks - Routine analysis and reporting Sequential workflow works best for: - Multi-step approval processes - Content production pipelines (draft → review → publish) - Data transformation and validation - Compliance checks against multiple criteria Parallel workflow works best for: - Scenarios where multiple perspectives improve quality - Independent analyses that can run at the same time - Scenarios where speed matters more than coordination overhead - Risk assessments that need diverse viewpoints Multi-agent systems work best for: - Complex problems that need expertise across domains - Research and analysis projects - Dynamic customer interactions spanning multiple systems - Strategic planning and decision support
Original English text (three-question checklist / quick-reference table / evolution template)
1. What level of control do you need? High control requirements (regulatory compliance, financial transactions, safety-critical operations) → Start with single agents or sequential workflows Moderate control requirements (customer support, content creation, data analysis) → Consider hierarchical multi-agent systems Low control requirements (research, brainstorming, complex analysis) → Collaborative multi-agent systems become viable 2. How complex is your problem domain? Single domain problems (answering product questions, processing returns, generating reports) → Single agents handle these efficiently Multi-domain but predictable problems (employee onboarding, compliance workflows, standard analysis tasks) → Sequential or parallel workflows Complex, open-ended problems (strategic analysis, research projects, system troubleshooting) → Multi-agent architectures 3. What are your resource constraints? Limited budget/tokens → Single agents or carefully designed parallel workflows Time-to-market pressure → Start with single agents, plan an evolution path Long-term strategic initiative → Design for modular evolution 4. Do you need deep domain expertise? Single domain with established workflows → Single agent with specialized Skills Multiple distinct domains requiring coordination → Multi-agent systems with specialized Skills Single agents work best for: customer service / document processing / code review / routine analysis and reporting Sequential workflows work best for: multi-step approvals / content pipelines (draft → review → publish) / data transformation / compliance checking Parallel workflows work best for: multiple perspectives / independent analyses / speed over coordination / diverse-viewpoint risk assessment Multi-Agent systems work best for: complex problem-solving / research and analysis / dynamic multi-system interactions / strategic planning Phase 1: Single agent for customer inquiries (proving value) Phase 2: Routing pattern separating order status, product questions, complaints Phase 3: Specialized agents for each category with shared context Phase 4: Multi-agent system with inventory, payment, and shipping coordination Phase 5: Evaluator agents for quality assurance and continuous improvement
A real evolution roadmap: one e-commerce platform's five phases
The guide closes with a real path: an e-commerce platform's five phases from single agent to multi-agent. It wasn't a one-shot build of a complex system — each phase only added complexity once the prior one had proven measurable value.
Production systems often evolve into hybrid architectures too: parallel processing nested inside a hierarchical supervisor, dynamic routing woven into a sequential workflow, a single agent that automatically triggers a multi-agent system when it hits an edge case. When the business value justifies the complexity, these combinations unlock capabilities no single pattern can achieve alone.
It always comes back to this: architecture evolves with need, start simple, measure everything, and only add complexity when it delivers measurable value. The best architecture today is the simplest one that meets today's requirements while leaving a clear path for tomorrow. Building Effective AI Agents · Anthropic
An AI agent isn't better just because it's more complex — ask three questions before deciding whether to build one or a whole crew
Anthropic put out an enterprise-grade agent architecture guide, sorting six approaches into three families and pairing them with a three-question selection method — here it is in one page, with a diagram.
↓ Read the whole thing in one scroll · one diagram animates
AI agents (agent: AI that can decide its own next step, pick tools, and adjust after trial and error — not a script that just follows a fixed sequence) are increasingly being put to work on more and more complex jobs at companies. The catch: there are several ways to build one, and jumping straight to the most complex setup often just burns money for nothing.
A script only ever walks the path laid out for it — it can't evaluate on its own, switch approaches, or try and fail. That's exactly the gap agents fill: they evaluate the task, pick a tool, check the result, and adjust
But "can adapt on its own" doesn't mean "build whatever." Getting the architecture wrong has a real cost: forcing a full multi-agent system onto a job a single agent could've handled can burn through 10-15x more tokens (the billing unit for how much text an AI processes — the more it uses, the bigger the bill). This guide exists to teach you how to choose.
The guide sorts agent architectures into six types, which really fall into three families: ones where the AI decides for itself (a single agent working solo, one lead managing specialists, several peers talking directly to each other), ones with a pre-scripted process (sequential — queue up — and parallel — run at once), and one dedicated to quality control (two AIs, one writes and one critiques, going back and forth). The three in the first family are the easiest to mix up — the diagram below lays them out side by side using "XiaoHu needs to put together a competitive analysis report" as the example; the other three get full diagrams in the article body.
Picking an architecture shouldn't mean defaulting to the most technically complex option — answer three questions first, and the answers point straight to the right one.
Going on gut feel, or assuming "more complex = more advanced," and jumping straight to multi-agent.
Ask three questions first, match to the right fit, and only add complexity when it's actually worth it.
- How much control do you need? Need to explain every step to regulators or auditors → single agent or sequential workflow; can tolerate some unpredictability → multi-agent becomes worth considering.
- How many domains does the problem span? Single domain (answer product questions, generate reports) → one agent is enough; spans several unrelated domains → then go multi-agent.
- How big is the budget? Tokens are tight → single agent; long-term investment → build in upgrade-ready interfaces from the very first agent.
The article turns these three questions into a clickable selector, plus a quick-reference table of "which scenario uses which pattern" and a real five-phase upgrade path an e-commerce platform took from single agent to multi-agent — you can apply it directly.
Multi-agent really is more powerful — but that power comes at a cost. Get this math straight before deciding whether it's worth it.
running the same batch of queries
on the same batch of queries
All of these figures come from Anthropic's internal research or customer self-disclosure, with no independent third-party reproduction — the method is worth learning, but take the numbers with a grain of salt.
Just build the fanciest setup!
tokens (AI's billing unit) got burned
why's it cost so much more?!
→ just use one
→ one is enough
→ don't build a crew
prove it's worth the money.
don't mistake
"more complex" for "better."
Members can copy or download the full Markdown, and let their AI subscribe and ask.
Found this useful? Leave your email — every new explainer, the moment it's out. Unsubscribe anytime.