Why Multi-Agent Systems Fail in Production: Infinite Loops, Token Explosions, and The Evaluator-Optimizer Alternative

Consider an autonomous software engineering squad constructed inside a popular multi-agent framework. The squad consists of four specialized personas: an Architect, a Coder, a Test Engineer, and a Security Reviewer.

The task is routine: upgrade an internal service from Python 3.10 to Python 3.12 and resolve any deprecated library warnings.

The Coder modifies the dependencies and opens a pull request. The Test Engineer executes the test suite, observes a deprecation warning in an external database driver, and rejects the PR. The Coder rewrites the database wrapper. The Security Reviewer reviews the code and notes that the rewritten wrapper does not use parameterized queries. The Coder implements parameterized queries. The Architect reviews the change and objects that parameterization was implemented using a custom regex parser rather than the ORM's native dialect. The Coder rewrites the code using the native dialect. The Test Engineer flags that the native dialect fails an older integration test.

The conversation loops.

This is an illustrative scenario, not a reported customer incident. Measure rounds, input/output tokens, tool calls, and total spend from traces to establish the actual cost of a run.

Unbounded retries and repeated context can raise cost. Apply explicit limits and evaluate the workflow with traces.

1. Network Topologies: Token Growth in Group Chats

In many developer tutorials, multi-agent coordination is designed as a decentralized "group chat": every agent listens to every other agent.

A simple model for cumulative input tokens

Let:

• $M$ = Number of agents in the swarm
• $T$ = Number of conversation rounds
• $L_{\text{msg}}$ = Average message length (in tokens)
• $P_{\text{sys}}$ = System prompt overhead per agent

For a simplified upper-bound model, suppose each of M agents receives the full message history each round. Actual systems may route messages selectively or summarize history. On round $t$, the context window size of each agent is:

$$\text{Context}(t) = P_{\text{sys}} + (t-1) \cdot L_{\text{msg}}$$

When all $M$ agents ingest this context on round $t$, the total input tokens consumed across the swarm for that round is:

$$\text{Round Tokens}(t) = M \cdot \left( P_{\text{sys}} + (t-1) \cdot L_{\text{msg}} \right)$$

Integrating across all $T$ rounds:

$$\text{Total Input Tokens} = \sum_{t=1}^T M \cdot \left( P_{\text{sys}} + (t-1) \cdot L_{\text{msg}} \right) = M \cdot T \cdot P_{\text{sys}} + M \cdot L_{\text{msg}} \cdot \frac{T(T-1)}{2}$$

Notice the asymptotic complexity:

$$\mathcal{O}(M \cdot T^2 \cdot L_{\text{msg}})$$

Under this simplified model, cumulative input grows as $O(M T^2 L_{msg})$ as rounds increase. This is not cubic in $M$ and $T$ jointly. Actual frameworks differ in routing, summarization, and truncation.

Actual usage depends on system prompts, message routing, output size, summaries, and retries; instrument a run to calculate the total.

2. Four Inherent Failure Modes of Multi-Agent Swarms

2.1 The Polite Deadlock (Echo Chamber Cycles)

Without explicit termination conditions, conversational agents default to procedural politeness and pedantic nitpicking:

• "Thank you for providing that update. Could you also ensure that line 4 adheres to PEP 8 standards?"
• "Certainly! Here is the code formatted to PEP 8. Does this meet your expectations?"
• "Almost! However, could we adjust the variable name to be more descriptive?"
• Each exchange incurs a full forward pass over the entire conversation history, costing dollars for minor aesthetic revisions.

2.2 State Desynchronization and Divergent Realities

When agents operate with private memories or selectively summarize previous steps, their internal world states drift apart:

• Agent 1 migrates a database table from users to accounts.
• Agent 2 (handling API routes) receives a truncated summary that omits the migration details.
• Agent 2 generates endpoint code querying the non-existent users table.
• Agent 3 (QA) flags an error, causing Agent 1 to attempt a second, conflicting migration.
• Result: The system fragments into contradictory code paths, generating merge conflicts and corrupting database states.

2.3 Cascading Tool Invocations and Race Conditions

When multiple autonomous agents share write permissions to common environments (e.g., bash shells, AWS CLI, file systems), they execute competing tool calls simultaneously. One agent runs git rebase while another executes rm -rf, creating environmental inconsistencies that trigger cascading error-handling loops.

3. The Production Alternative: The Evaluator-Optimizer Pattern

Anthropic engineering guidance recommends starting with simple, composable workflows and adding agentic complexity when it improves results. This is practitioner guidance, not a universal controlled finding that one architecture always outperforms another. See Building Effective Agents.

A bounded workflow with deterministic checks is one option; select the architecture based on task results and operational constraints.

Why the Evaluator-Optimizer Pattern Succeeds:

1. Separation of Proposing and Evaluating: The Proposer and Evaluator do not converse. The Proposer generates a candidate artifact; the Evaluator runs deterministic linters, test suites, and schema validators.
2. Pruned Feedback Diffs: When a candidate is rejected, the system does not append the entire conversation history. It passes only the precise failing assertion or error diff, keeping context size constant ($O(1)$) rather than quadratic ($O(T^2)$).
3. Hard Infrastructure Bounds: The cycle enforces a hard ceiling ($N \le 3$ retries). If the Proposer cannot pass deterministic gates within 3 attempts, the system halts and escalates to a human engineer.

4. Prove the Boundary: The Loop Step Cap Diagnostic

To verify whether your autonomous agent architecture is protected against the infinite handshake, run this verification test:

# The Reference-Monitor Step Breaker
class ExecutionBoundaryMonitor:
    def __init__(self, max_steps: int = 4, max_budget_usd: float = 0.50):
        self.max_steps = max_steps
        self.max_budget_usd = max_budget_usd
        self.current_step = 0
        self.spent_usd = 0.0

    def record_step(self, step_input_tokens: int, step_output_tokens: int, pricing: dict):
        self.current_step += 1
        cost = (step_input_tokens / 1e6) * pricing["in"] + (step_output_tokens / 1e6) * pricing["out"]
        self.spent_usd += cost

        # HARD BOUND 1: Step Ceiling
        if self.current_step > self.max_steps:
            raise SystemExit(f"[BREAKER TRIPPED] Maximum execution steps ({self.max_steps}) exceeded!")

        # HARD BOUND 2: Budget Ceiling
        if self.spent_usd > self.max_budget_usd:
            raise SystemExit(f"[BREAKER TRIPPED] Financial step limit (${self.spent_usd:.2f}) exceeded!")

        print(f"[STEP {self.current_step}] State Valid. Accumulated Cost: ${self.spent_usd:.4f}")

Inject this monitor at the network proxy layer, outside the agent's application code. If your agent framework bypasses or crashes without this monitor, your system is vulnerable to runaway token exhaustion.

Summary: Designing for Deterministic Convergence

The allure of multi-agent swarms is the dream of emergent intelligence: assigning human personas to models and expecting self-organizing problem solving.

In production engineering, emergent behavior is another name for unbounded, unrepeatable systems variance.

Production-grade AI systems do not debate. They propose, validate against deterministic test runners, prune context diffs, and terminate under immutable mathematical bounds.

Sources and methodology

Performance and cost figures not linked to a named external benchmark are illustrative examples, not audited customer results. Verify provider model, tier, region, cache rules, and price before publication; measure latency and quality on a representative evaluation set. References:

• Anthropic, Building Effective Agents
• Yao et al., ReAct paper