A2A Circuit Breakers: Design for Reliable Agent Calls

When one AI agent calls another, put a circuit breaker around the remote call so repeated timeouts or failures do not consume resources or cascade through the system. Track outcomes per destination, stop sending ordinary requests after a failure threshold, probe again after a cool-down, and keep retries, deadlines, and fallback behavior bounded.

What an A2A circuit breaker protects

An agent-to-agent (A2A) call is a dependency just like a call to a service: the receiving agent may be slow, unavailable, overloaded, or unable to complete a particular task. A circuit breaker is a local control in the caller that changes how it handles calls when evidence suggests the dependency is unhealthy. It does not repair the other agent, guarantee task completion, or replace monitoring.

Without a breaker, a failing recipient can trigger a feedback loop. The caller waits for responses, its concurrent work accumulates, and retries add more traffic to the already struggling recipient. If several agents depend on one another, the delays can spread. A breaker limits this amplification and gives the caller a defined way to fail quickly or choose another path.

How the states work

A common design has three states:

• Closed: Calls proceed normally and the breaker records outcomes. “Closed” means the path is open for traffic.
• Open: Calls to the affected destination are rejected locally, usually with a fast, explicit failure. The caller waits for a cool-down instead of repeatedly testing a broken dependency.
• Half-open: After the cool-down, a small number of probe calls are allowed. Success returns the breaker to closed; failure opens it again.

Define what counts as a failure. A connection refusal, timeout, or selected server-side failure may indicate dependency trouble. A valid response saying “I cannot perform this task” may be an ordinary business outcome, not an infrastructure failure. Likewise, caller errors such as invalid input should not normally trip the recipient’s breaker. Classify outcomes deliberately rather than treating every non-success result as identical.

Choose the breaker’s scope and signals

Use a breaker key that isolates the failing dependency without creating unnecessary state. In practice, that might be the recipient service or agent endpoint, optionally separated by operation or protocol when those paths fail independently. Avoid one global breaker for every agent: a single unhealthy recipient should not block unrelated destinations. Avoid keys based on unbounded values such as arbitrary user prompts, which can create excessive state.

Choose a measurable failure signal and a window. Options include a consecutive-failure threshold or a failure proportion over a rolling window, provided the window has enough calls to be meaningful. A percentage calculated from only one or two requests is noisy. A consecutive-failure rule is simple but can be sensitive to intermittent faults. Record latency as well as outcome; slow calls can be harmful even when they eventually succeed.

Set the timeout from the caller’s task budget, not from an assumption that every agent call deserves a long wait. The deadline must leave time for any remaining reasoning, tool calls, or response delivery. If the protocol supports request deadlines, propagate a deadline or remaining time rather than restarting a full timeout at every hop.

Design retries, probes, and fallbacks

A breaker and a retry policy solve different problems: retries address some transient failures, while a breaker limits calls during sustained trouble. Combining them carelessly multiplies traffic. Keep attempts bounded, apply backoff with jitter where appropriate, and do not retry non-idempotent actions unless the operation has a safe deduplication mechanism. A retry should fit within the original overall deadline.

For half-open probes, limit concurrency so many callers do not all rush the recipient as soon as the cool-down ends. A single probe or a small configured allowance is often easier to reason about than releasing all queued requests. Do not queue calls indefinitely while the breaker is open; return a clear unavailable result or use an explicit fallback.

Fallbacks must preserve task meaning. An alternate agent may have different capabilities, permissions, data freshness, or output quality. A cached answer may be stale; a local response may be less complete. Make the fallback visible in the result or trace so downstream agents do not mistake degraded output for a normal completion. If no safe fallback exists, fail promptly and let the caller decide whether to ask a person or report that the task could not be completed.

Thresholds, recovery, and operational visibility

There is no universal threshold or cool-down that fits every agent system. Start with the expected request volume, normal latency, task deadline, and cost of failed calls. A low-volume destination may need a consecutive-failure rule; a high-volume path may support a rolling failure ratio. Tune using observed behavior and controlled fault tests, not guesswork alone.

Expose the breaker’s state and transitions in logs, metrics, or traces. Useful signals include calls attempted, rejected while open, timeouts, classified failures, probe outcomes, and time spent in each state. Include the destination and operation in a bounded, privacy-conscious way. Do not log prompts or sensitive payloads merely to diagnose breaker behavior.

Make state changes observable but avoid a noisy alert for every rejected request. Alert on sustained inability to complete important work or repeated breaker opening, and distinguish a breaker reacting correctly from a breaker malfunctioning. If callers run in many processes, decide whether each process has a local breaker or whether shared state is truly needed. Local state is simpler and avoids a central dependency, but different instances may make different decisions; shared state can coordinate decisions while adding its own availability and consistency concerns.

Common design mistakes

• Counting task-level refusals as outages: Separate transport and service-health failures from normal capability or policy responses.
• Retrying inside every layer: Multiple nested retry loops can multiply attempts. Assign retry responsibility and enforce an overall deadline.
• Opening globally: Scope failures to the affected recipient or independent operation so healthy routes remain available.
• Using an unlimited half-open flood: Restrict probes and avoid releasing accumulated work all at once.
• Hiding degraded results: Mark fallback or partial completion clearly, and preserve provenance for downstream decisions.
• Treating the breaker as a security boundary: It is a reliability mechanism, not authorization, input validation, or protection against malicious agent behavior.

Frequently Asked Questions

Should each agent have its own circuit breaker?
Usually, the caller maintains breaker state for each relevant remote destination or independent operation. This isolates failures and lets each caller account for its own deadlines and traffic. Shared state is an architectural choice, not a requirement.

Does a circuit breaker replace retries?
No. A bounded retry can help with a brief transient fault; the breaker stops repeated calls when failures persist. Coordinate the two so retries do not exceed the task deadline or overwhelm the recipient.

What should count as a failed agent call?
Typically, count events that indicate the dependency could not serve the call, such as connection errors, timeouts, or selected server failures. Do not automatically count valid task refusals, caller mistakes, or every unusual response as an outage.

How long should the breaker stay open?
Choose a cool-down based on the service’s recovery behavior and the urgency of the caller’s task. Use observation and fault testing to tune it. Half-open probes should be limited so recovery checks do not create a traffic surge.

What should happen when the breaker is open?
Fail quickly with a clear dependency-unavailable result, or use a documented and safe fallback. Do not silently return stale or lower-quality output as if the requested agent completed the task normally.

Related

• DC Function Score Calculator: How It Works
• Cumulative Probability Z Score Calculator
• AP Test Score Calculator: How It Works