What Is the Circuit Breaker Pattern? A Practical Guide

What Is the Circuit Breaker Pattern? A Practical Guide for Developers Imagine your application depends on a payment service. Everything works normally until the payment service starts responding slowly. Your application keeps sending requests, each request waits longer than usual, and eventually more requests start piling up. Now imagine this happening across several services at the same time. One small failure can quickly turn into a much larger outage. This is one of the problems the Circuit Breaker pattern is designed to solve. The circuit breaker pattern is a resilience technique that prevents an application from repeatedly calling an unhealthy or failing service. Instead of allowing failed requests to continue piling up, the circuit breaker temporarily stops requests and gives the failing service time to recover. In this guide, we'll look at what the circuit breaker pattern is, how it works, its three states, why it's important in microservices, how it differs from retries and timeouts, common implementation strategies, and when you should use it in production. What Is the Circuit Breaker Pattern? The Circuit Breaker pattern is a software design pattern used to prevent repeated calls to a service that is currently failing. It works similarly to an electrical circuit breaker. When an electrical system detects a serious problem, a circuit breaker cuts the connection to prevent further damage. A software circuit breaker does something similar: Healthy service ↓ Requests allowed ↓ Service starts failing ↓ Circuit opens ↓ Requests stopped ↓ Service gets time to recover ↓ Circuit tests service ↓ Service healthy? │ ├── Yes → Close circuit │ └── No → Keep circuit open The important idea is simple: When a dependency is failing, stop repeatedly calling it until it has had a chance to recover. Why Do We Need Circuit Breakers? Modern applications rarely work in isolation. A typical SaaS application might depend on: Your API │ ├── PostgreSQL ├── Redis ├── Payment API ├── Email Provider ├── Authentication Service ├── AI API └── Other Microservices If one of these dependencies becomes unavailable, your application can start experiencing failures too. For example: Order Service │ ▼ Payment Service │ X DOWN If the Order Service continues calling the Payment Service thousands of times, those requests may: consume connection pools consume worker threads increase memory usage increase latency create request queues trigger more timeouts make the Order Service unhealthy Eventually, the failure can spread. This is known as a cascading failure. What Is a Cascading Failure? A cascading failure happens when a failure in one component causes problems in other components. Consider this example: Payment Service ↓ DOWN ↓ Order Service waits ↓ Requests accumulate ↓ Worker pool exhausted ↓ Order Service becomes slow ↓ API requests start timing out ↓ Entire application becomes unstable The original problem was the Payment Service. But now multiple services are affected. A circuit breaker helps stop this chain earlier. Payment Service ↓ DOWN ↓ Circuit Breaker ↓ Stop calling Payment Service ↓ Order Service remains responsive This is one of the biggest reasons circuit breakers are important in distributed systems. How Does a Circuit Breaker Work? A circuit breaker typically has three states: Closed Open Half-Open Understanding these three states is the key to understanding the circuit breaker pattern. 1. Closed State The Closed state is the normal state. Requests are allowed to reach the dependency. Client ↓ Application ↓ Circuit Breaker ↓ Payment Service ↓ Response The circuit breaker monitors the requests. For example, it may track: failed requests successful requests timeouts latency error percentage Suppose the configuration is: Failure threshold: 50% Minimum requests: 20 If enough requests start failing, the circuit breaker can decide that the dependency is unhealthy. The circuit then changes from: CLOSED ↓ OPEN 2. Open State When the circuit is Open, requests are no longer sent to the failing dependency. Instead, the circuit breaker fails fast. Client ↓ Application ↓ Circuit Breaker │ └── OPEN ↓ Don't call service ↓ Return fallback/error This is extremely important. Without a circuit breaker: Request ↓ Payment API ↓ Timeout ↓ Wait ↓ Retry ↓ Timeout ↓ Wait With an open circuit: Request ↓ Circuit Breaker ↓ OPEN ↓ Fail immediately The application doesn't waste resources waiting for a dependency that is already known to be unhealthy. 3. Half-Open State The circuit shouldn't remain open forever. Eventually, the dependency might recover. That's where the Half-Open state comes in. After a configured period, the circuit breaker allows a small number of test requests through. OPEN ↓ Wait ↓ HALF-OPEN ↓ Test request ↓ Service healthy? If the test succeeds: HALF-OPEN ↓ Success ↓ CLOSED If the test fails: HALF-OPEN ↓ Failure ↓ OPEN This gives the dependency an opportunity to recover without immediately sending a large amount of traffic back to it. Circuit Breaker State Diagram The complete lifecycle looks like this: ┌───────────────┐ │ CLOSED │ │ Normal traffic│ └───────┬───────┘ │ Failure threshold │ ▼ ┌───────────────┐ │ OPEN │ │ Fail fast │ │ No requests │ └───────┬───────┘ │ Recovery time │ ▼ ┌───────────────┐ │ HALF-OPEN │ │ Test requests │ └───────┬───────┘ │ ┌────────┴────────┐ │ │ Success Failure │ │ ▼ ▼ CLOSED OPEN A Simple Real-World Example Suppose your application uses a third-party payment API. Normally: Order API ↓ Circuit Breaker ↓ Payment API ↓ Success Now the payment provider starts failing. The first few requests fail: Request 1 → 500 Request 2 → 500 Request 3 → timeout Request 4 → 500 Request 5 → timeout The circuit breaker detects the failure rate. It opens the circuit: Order API ↓ Circuit Breaker ↓ OPEN ↓ Don't call Payment API New requests fail immediately or use a fallback. After a configured recovery period: OPEN ↓ HALF-OPEN ↓ Test Payment API If the payment service is healthy: Test succeeds ↓ CLOSED ↓ Normal traffic resumes Circuit Breaker vs Retry Circuit breakers and retries are often confused because both deal with failures. But they solve different problems. Retry A retry says: "This request failed. Let's try it again." For example: Request ↓ Failure ↓ Retry ↓ Success Retries are useful for temporary failures. For example, a network connection may fail once but succeed immediately afterward. Circuit Breaker A circuit breaker says: "This dependency appears unhealthy. Stop calling it for now." For example: Repeated failures ↓ Circuit opens ↓ Stop requests ↓ Wait for recovery So: Retry = Try again Circuit Breaker = Stop trying for a while They are often used together. Circuit Breaker + Retry A resilient application might use: Request ↓ Circuit Breaker ↓ Retry ↓ Dependency For a temporary failure: Request ↓ Dependency ↓ Failure ↓ Retry ↓ Success For a persistent failure: Request ↓ Dependency ↓ Failure ↓ Retry ↓ Failure ↓ Circuit opens ↓ Future requests fail fast The order and exact behavior depend on your architecture and libraries, but the important point is that retries should not blindly continue when a dependency is persistently failing. Circuit Breaker vs Timeout A timeout controls how long a request is allowed to wait. For example: Timeout = 3 seconds If the dependency doesn't respond within three seconds: Request ↓ Wait 3 seconds ↓ Timeout A circuit breaker controls whether requests should be sent at all based on observed failures. The two mechanisms work well together: Request ↓ Circuit Breaker ↓ Timeout ↓ Dependency A timeout prevents an individual request from waiting forever. A circuit breaker prevents the application from repeatedly sending requests to a dependency that is consistently failing. Circuit Breaker vs Rate Limiting These mechanisms solve completely different problems. Rate limiting Controls traffic volume. 100 requests/minute Circuit breaker Controls traffic based on dependency health. Dependency failing ↓ Stop sending requests You can use both: Client ↓ Rate Limiter ↓ Circuit Breaker ↓ Backend Rate limiting protects your system from excessive traffic. Circuit breaking protects your system from unhealthy dependencies. What Metrics Should a Circuit Breaker Monitor? A circuit breaker needs some way to determine whether a dependency is unhealthy. Common signals include: Error count For example: 10 failures within the last 20 requests Error percentage For example: Failure rate = 60% Timeouts Repeated timeouts are often a strong indicator of an unhealthy dependency. Latency A service may technically return HTTP 200 responses while becoming extremely slow. For example: Normal latency: 100ms Current latency: 500ms 1s 2s 5s Depending on your requirements, excessive latency can be treated as a failure condition. Failure Thresholds A circuit breaker usually needs a threshold that determines when the circuit should open. For example: Minimum requests: 20 Failure threshold: 50% The circuit doesn't immediately open after one failed request. Instead, it waits until enough data is available. Example: 20 requests 12 failures Failure rate = 60% If the configured threshold is 50%, the circuit can open. This prevents a single temporary failure from unnecessarily taking the circuit offline. Failure Count vs Failure Percentage There are two common approaches. Failure count Open the circuit after a certain number of failures. 5 consecutive failures → OPEN Simple, but it may not work well for services with highly variable traffic. Failure percentage Open the circuit when the percentage of failures exceeds a threshold. 20 requests 12 failures 60% failure rate → OPEN This can provide more context because it considers both successful and failed requests. Consecutive Failure Detection Another simple strategy is tracking consecutive failures. For example: Success Success Failure Failure Failure Failure Failure Configuration: 5 consecutive failures The circuit opens after the fifth consecutive failure. This approach is easy to understand and can work well for certain services, although it doesn't capture all traffic patterns. How Long Should a Circuit Stay Open? The open state usually has a cooldown period. For example: Open duration = 30 seconds After 30 seconds: OPEN ↓ HALF-OPEN The correct duration depends on the dependency. If you test too quickly: Service still recovering ↓ Test request fails ↓ Circuit opens again If you wait too long: Service recovered ↓ Traffic still blocked ↓ Unnecessary downtime A good value should be based on the recovery characteristics of the dependency. What Should Happen When the Circuit Is Open? This is one of the most important design decisions. The application can: Return an error For example: 503 Service Unavailable with: { "error": "dependency_unavailable", "message": "Payment service is temporarily unavailable" } Return cached data For read operations, stale data may sometimes be better than no data. Request ↓ Circuit OPEN ↓ Cached response ↓ Client Use a fallback For example: Recommendation service unavailable ↓ Return popular products Queue the request For operations that don't need an immediate response, you may be able to queue work for later processing. The right fallback depends heavily on the business operation. Circuit Breakers in Microservices Circuit breakers are particularly useful in microservice architectures. Imagine: API Gateway │ ┌───────┼───────┐ ▼ ▼ ▼ Users Orders Payments │ ▼ Inventory If Inventory becomes unavailable: Orders ↓ Inventory X DOWN Without a circuit breaker, Orders may repeatedly call Inventory. With a circuit breaker: Orders ↓ Circuit Breaker ↓ Inventory After repeated failures: Orders ↓ Circuit Breaker OPEN ↓ Don't call Inventory The Orders service can continue handling requests that don't depend on Inventory. This helps isolate failures. Circuit Breakers at the API Gateway Circuit breakers don't have to live inside individual applications. They can also be implemented at an API gateway or edge proxy. For example: Internet │ ▼ API Gateway │ Circuit Breaker │ ┌────────────┼────────────┐ ▼ ▼ ▼ Service A Service B Service C The gateway can monitor origin health and stop forwarding requests when an origin becomes unhealthy. This can be particularly useful when you have multiple applications or services that share the same infrastructure. For example, EdgeWrap provides an edge API gateway layer that can sit in front of your APIs, while the EdgeWrap documentation provides configuration and implementation details. Circuit Breaker and Caching Caching can complement circuit breakers. Suppose: GET /api/products normally returns product information. If the origin becomes unavailable: Client ↓ API Gateway ↓ Origin DOWN Instead of returning an error immediately, an edge gateway may be able to serve a previously cached response, depending on the cache policy. Client ↓ API Gateway ↓ Origin unavailable ↓ Cached response ↓ Client This is sometimes called stale-if-error behavior when implemented through appropriate HTTP caching semantics. It can make applications more resilient during short backend failures. Circuit Breaker and Failover Circuit breakers can also work with multiple origins. For example: API Gateway │ ┌─────────┴─────────┐ ▼ ▼ Primary API Secondary API │ │ DOWN Healthy The gateway can detect that the primary origin is unhealthy and route traffic to a secondary origin, depending on the platform's failover capabilities. A more resilient architecture might look like: API Gateway │ Health Monitoring │ ┌──────────┴──────────┐ ▼ ▼ Primary Origin Backup Origin │ │ DOWN Healthy │ │ └──────────┬──────────┘ ▼ Client Circuit breaking and failover aren't exactly the same thing, but they can complement each other. Circuit Breaker Implementation Example The following simplified pseudocode demonstrates the basic idea: class CircuitBreaker { constructor(action, threshold = 5, timeout = 30000) { this.action = action; this.threshold = threshold; this.timeout = timeout; this.failures = 0; this.state = "CLOSED"; } async execute() { if (this.state === "OPEN") { throw new Error("Circuit is open"); } try { const result = await this.action(); this.failures = 0; return result; } catch (error) { this.failures++; if (this.failures >= this.threshold) { this.state = "OPEN"; setTimeout(() => { this.state = "HALF_OPEN"; }, this.timeout); } throw error; } } } This is intentionally simplified. A production implementation needs to consider things such as: concurrent requests half-open request limits failure windows latency distributed state race conditions metrics fallback behavior recovery detection For production systems, using a well-tested resilience library or managed infrastructure is usually preferable to maintaining your own implementation. Distributed Circuit Breakers A distributed system introduces another challenge. Imagine: API Gateway │ ┌───────────┼───────────┐ ▼ ▼ ▼ Server A Server B Server C If each server has its own circuit state: Server A → OPEN Server B → CLOSED Server C → CLOSED some servers may continue sending traffic to an unhealthy dependency. Depending on your architecture, this may be acceptable or undesirable. A centralized or edge-level circuit breaker can provide a more consistent view of dependency health. However, distributed circuit state also introduces its own complexity. For many applications, a local circuit breaker is sufficient because each service instance can independently protect itself. Common Circuit Breaker Configuration A circuit breaker might have configuration such as: Failure threshold: 50% Minimum requests: 20 Open duration: 30 seconds Half-open requests: 3 Request timeout: 5 seconds These values are only examples. You should tune them based on your traffic and dependency behavior. Circuit Breaker Best Practices 1. Always configure timeouts A circuit breaker is not a replacement for timeouts. A request should not be allowed to hang indefinitely. Use: Timeout + Circuit Breaker rather than relying on either mechanism alone. 2. Don't open the circuit too aggressively A single temporary error doesn't necessarily mean the service is unhealthy. Use a minimum request count or appropriate failure window before opening the circuit. 3. Monitor latency as well as errors A service that returns successful responses after 20 seconds may be just as problematic as a service returning errors. 4. Keep the half-open state controlled Don't send hundreds of requests immediately when testing recovery. Allow a small number of test requests first. 5. Choose meaningful fallbacks Don't return fake success responses for operations such as payments or account creation. For critical operations, it may be safer to return a clear error or queue the operation. 6. Monitor circuit state changes Track events such as: CLOSED → OPEN OPEN → HALF-OPEN HALF-OPEN → CLOSED HALF-OPEN → OPEN These transitions can provide valuable operational information. 7. Combine circuit breakers with other resilience techniques A robust API architecture may use: Timeout + Retry + Circuit Breaker + Rate Limiting + Caching + Health Checks + Failover Each solves a different part of the reliability problem. Common Circuit Breaker Mistakes Mistake 1: Treating every error as a failure Some HTTP errors are caused by invalid client requests. For example: 400 Bad Request 401 Unauthorized 404 Not Found These don't necessarily indicate that the dependency is unhealthy. You need to carefully define which responses should contribute to the circuit failure threshold. Mistake 2: Retrying endlessly Retries without limits can make an outage worse. A failing dependency can receive even more traffic precisely when it is struggling. Mistake 3: Using an extremely short recovery period If the dependency needs 60 seconds to recover and your circuit tests it every 5 seconds, you'll repeatedly hit an unhealthy service. Mistake 4: No fallback strategy Opening the circuit is only part of the solution. You also need to decide what your application should return to the user. Mistake 5: Ignoring observability If you don't monitor circuit transitions, you may not know why requests are failing. When Should You Use the Circuit Breaker Pattern? Circuit breakers are particularly useful when your application depends on: third-party APIs payment providers authentication services microservices AI APIs external databases messaging services internal HTTP services They are especially valuable when a dependency failure could cause your own application to become unstable. When Should You Not Use a Circuit Breaker? Not every function needs a circuit breaker. For example, a simple in-process function: calculateTax() probably doesn't need one. Circuit breakers are most useful around remote or failure-prone dependencies where failures can consume significant resources. Adding a circuit breaker everywhere can also make systems unnecessarily complicated. Use it where dependency failures actually pose a resilience risk. A Practical Resilience Architecture For a production SaaS API, you might end up with something like: Users │ ▼ ┌─────────────────┐ │ Edge Gateway │ │ │ │ DDoS Protection │ │ WAF │ │ Rate Limiting │ │ Caching │ └────────┬────────┘ │ ▼ Application │ ┌──────┴──────┐ ▼ ▼ Circuit Breaker Redis │ ▼ External Service │ ┌────┴────┐ │ │ Healthy Down │ │ ▼ ▼ Success Fallback The goal isn't to eliminate every failure. That's impossible. The goal is to contain failures and prevent them from spreading through the system. Circuit Breaker Pattern: Key Takeaways The circuit breaker pattern can be summarized in a few ideas: Closed Requests flow normally. Request → Dependency Open The dependency is considered unhealthy. Request → Fail Fast Half-Open The system tests whether the dependency has recovered. Request → Test Dependency And the most important principle is: Don't keep hammering a dependency that is already failing. A circuit breaker gives your system a way to recognize failure, stop unnecessary traffic, and recover gracefully. Frequently Asked Questions What is the circuit breaker pattern? The circuit breaker pattern is a resilience design pattern that prevents an application from repeatedly calling an unhealthy dependency. It temporarily stops requests when failures exceed a configured threshold and later tests whether the dependency has recovered. What are the three states of a circuit breaker? The three common states are Closed, Open, and Half-Open. Closed allows normal traffic, Open blocks calls to the dependency, and Half-Open allows a limited number of test requests to determine whether the dependency has recovered. What is the difference between a circuit breaker and a retry? A retry attempts a failed operation again. A circuit breaker stops sending requests when a dependency is consistently failing. They are often used together. What is the difference between a circuit breaker and a timeout? A timeout limits how long an individual request can wait. A circuit breaker prevents new requests from being sent to a dependency that appears unhealthy. Is a circuit breaker useful in microservices? Yes. Circuit breakers are commonly used in microservice architectures to prevent failures in one service from cascading into other services. Can an API gateway implement a circuit breaker? Yes. API gateways and edge proxies can monitor backend health and stop forwarding requests to unhealthy origins. This can provide centralized resilience across multiple APIs. Should every API use a circuit breaker? No. Circuit breakers are most useful for remote or failure-prone dependencies. Adding them to every function or internal operation can create unnecessary complexity. Can a circuit breaker improve API reliability? Yes. A circuit breaker can prevent repeated calls to failing dependencies, reduce resource exhaustion, and help isolate failures. It does not make the dependency itself more reliable, but it can make your overall system more resilient to its failures. Final Thoughts Failures are inevitable in distributed systems. Servers go down. Networks become unreliable. Third-party APIs experience outages. Databases become overloaded. External services become slow. The goal of resilient architecture isn't to pretend these failures won't happen. It's to make sure one failure doesn't bring down everything else. The circuit breaker pattern is one of the simplest and most useful patterns for achieving that. By combining: Timeouts + Retries + Circuit Breakers + Rate Limiting + Caching + Health Checks + Failover you can build APIs that continue operating gracefully even when individual dependencies aren't healthy. For teams that want to enforce resilience at the infrastructure layer, an API gateway can provide another useful control point. EdgeWrap is designed to sit in front of APIs and provide edge-level traffic management, security, caching, routing, and reliability features. You can learn more about its architecture and configuration in the EdgeWrap documentation. The most important lesson is simple: When a dependency fails, protect your application first. Stop unnecessary calls, fail fast when appropriate, and give the dependency time to recover.