Circuit Breaker

How to protect your services from cascading failures using the Circuit Breaker pattern with states, fallbacks, and automatic recovery.

What problem it solves

In a microservices architecture, services depend on one another. When a downstream service becomes slow or stops responding, the services calling it accumulate open connections, exhaust their thread pools, and eventually fail as well. This domino effect is known as a cascading failure.

Example of the problem

Usuario → API Gateway → Servicio A → Servicio B (lento/caído)
                                         ⏳ timeout 30s
                                         ⏳ timeout 30s
                                         ⏳ timeout 30s
                         ↓
              Servicio A agota sus threads
              esperando respuesta de B
                         ↓
              API Gateway agota sus threads
              esperando respuesta de A
                         ↓
              Todo el sistema deja de responder

Without protection, a single slow service can take down the entire platform. The Circuit Breaker acts as an automatic switch that cuts off communication with a failing service, preventing the problem from spreading.

The pattern was popularized by Michael Nygard in his book Release It! and takes its name from the electrical circuit breakers that cut the circuit when they detect an overload.

Real impact of cascading failures

How it works

The Circuit Breaker is a proxy placed between a service and its dependencies. It monitors the calls and, based on the failure rate, decides whether to allow or block requests.

The three states

                    Fallos > umbral
    ┌──────────┐ ──────────────────► ┌──────────┐
    │ CERRADO  │                     │ ABIERTO  │
    │ (normal) │ ◄──────────────── │ (cortado)│
    └──────────┘   Prueba exitosa    └────┬─────┘
         ▲                                │
         │         Timeout expira         │
         │                                ▼
         │                          ┌───────────┐
         └────── Prueba exitosa ────│SEMI-ABIERTO│
                                    └───────────┘
                  Prueba falla ────► ABIERTO

Closed state

This is the normal state. All requests pass through to the downstream service. The circuit breaker monitors the results:

• It counts recent failures (errors, timeouts)
• If the failure rate exceeds a threshold (e.g., 50% failures in the last 10 calls), it switches to Open

Open state

The circuit is cut. Requests do not reach the downstream service. Instead, the circuit breaker responds immediately with an error or executes a fallback.

• It avoids sending traffic to a service we know is failing
• It releases resources (threads, connections) that would be wasted waiting for timeouts
• After a wait period (e.g., 30 seconds), it switches to Half-Open

Half-Open state

A test state. The circuit breaker lets a limited number of requests through to check whether the service has recovered:

• If the test requests succeed → it returns to Closed
• If the test requests fail → it returns to Open

Configuration parameters

Fallback strategies

When the circuit is open, instead of returning a raw error, you can implement fallbacks:

Full flow example

Estado: CERRADO
  Llamada 1 → Servicio B → ✅ OK
  Llamada 2 → Servicio B → ✅ OK
  Llamada 3 → Servicio B → ❌ Timeout
  Llamada 4 → Servicio B → ❌ Error 500
  Llamada 5 → Servicio B → ❌ Timeout
  → Tasa de fallos: 60% > umbral 50%
  → Cambio a ABIERTO ⚡

Estado: ABIERTO
  Llamada 6 → ⚡ Fallback inmediato (no llega a B)
  Llamada 7 → ⚡ Fallback inmediato
  ... (30 segundos de espera)
  → Cambio a SEMI-ABIERTO

Estado: SEMI-ABIERTO
  Llamada 8 → Servicio B → ✅ OK (prueba)
  Llamada 9 → Servicio B → ✅ OK (prueba)
  Llamada 10 → Servicio B → ✅ OK (prueba)
  → 3 éxitos consecutivos
  → Cambio a CERRADO ✅

Practical example: Circuit Breaker in a payment service

Imagine an order service that calls an external payment provider:

// Pseudocódigo de configuración del Circuit Breaker
circuitBreaker = new CircuitBreaker({
  name: "payment-provider",
  failureThreshold: 50,        // 50% de fallos
  requestVolume: 10,           // mínimo 10 requests para evaluar
  sleepWindow: 30000,          // 30 segundos en estado abierto
  successThreshold: 3,         // 3 éxitos para cerrar
  timeout: 5000,               // 5 segundos de timeout por llamada
  fallback: () => ({
    status: "PENDING",
    message: "Pago en cola, se procesará cuando el servicio se recupere"
  })
});

// Uso
async function procesarPago(pedidoId, monto) {
  try {
    resultado = await circuitBreaker.execute(() =>
      paymentProvider.charge(pedidoId, monto)
    );
    return resultado;
  } catch (error) {
    if (error instanceof CircuitOpenError) {
      // El circuito está abierto, se ejecutó el fallback
      encolarPagoParaReintento(pedidoId, monto);
      return error.fallbackResult;
    }
    throw error;
  }
}

Monitoring the Circuit Breaker

Circuit state changes are valuable signals for the operations team:

// Métricas a exponer
circuit_breaker_state{name="payment-provider"} = "closed" | "open" | "half_open"
circuit_breaker_calls_total{name="payment-provider", result="success"}
circuit_breaker_calls_total{name="payment-provider", result="failure"}
circuit_breaker_calls_total{name="payment-provider", result="rejected"}
circuit_breaker_state_transitions_total{name="payment-provider", from="closed", to="open"}

// Alertas recomendadas
ALERT CircuitBreakerOpen
  IF circuit_breaker_state == "open"
  FOR 1m
  LABELS { severity = "warning" }
  ANNOTATIONS { summary = "Circuit breaker abierto para payment-provider" }

Advantages

• Prevents cascading failures: A downed service does not drag the rest of the system down
• Fail fast: Requests fail immediately instead of waiting for long timeouts
• Resource release: No threads or connections are wasted on calls that are going to fail
• Automatic recovery: The half-open state detects when the service recovers
• Improved user experience: Fallbacks offer degraded responses instead of errors
• Observability: State changes are clear signals for alerting and monitoring

Trade-offs / Disadvantages

• Delicate configuration: Incorrect thresholds can open the circuit too early or too late
• Complexity in fallbacks: Designing useful fallbacks for each dependency takes effort
• Stale data: Cache-based fallbacks may return outdated information
• Complex testing: Simulating the three states and the transitions requires infrastructure
• Recovery latency: The sleep window introduces a delay between the actual recovery and its detection
• Does not fix the root cause: The circuit breaker protects the system, but the downstream service still needs fixing

When to use

• Calls to external services or microservices that can fail or become slow
• Systems where a failure in one dependency must not affect overall availability
• When you need to protect shared resources (thread pools, connection pools)
• APIs that call third-party services with variable SLAs
• Systems with high-availability requirements where graceful degradation is preferable
• When you already have metrics and alerts to monitor the state of the circuits

When to avoid

• Calls to local resources (local database, files) where the overhead is not justified
• Operations that must complete no matter what (with no possible fallback alternative)
• Simple systems with few external dependencies
• When the additional latency of the proxy is unacceptable
• Asynchronous calls where you already have queues with retries and dead letter queues
• Prototypes or MVPs where resilience is not a priority

Common technologies and implementations

Relationship with other patterns

The Circuit Breaker is rarely used on its own. It works best in combination with other resilience patterns:

┌─────────────────────────────────────────┐
│           Cadena de resiliencia         │
│                                         │
│  Solicitud                              │
│     │                                   │
│     ▼                                   │
│  ┌──────────┐                           │
│  │ Timeout  │  Limita tiempo de espera  │
│  └────┬─────┘                           │
│       ▼                                 │
│  ┌──────────┐                           │
│  │  Retry   │  Reintenta con backoff    │
│  └────┬─────┘                           │
│       ▼                                 │
│  ┌──────────┐                           │
│  │ Circuit  │  Corta si hay muchos      │
│  │ Breaker  │  fallos consecutivos      │
│  └────┬─────┘                           │
│       ▼                                 │
│  ┌──────────┐                           │
│  │ Bulkhead │  Aísla recursos por       │
│  │          │  dependencia              │
│  └────┬─────┘                           │
│       ▼                                 │
│  Servicio downstream                    │
└─────────────────────────────────────────┘

• Timeout Pattern: Defines how long to wait before considering a call as failed. The timeout feeds the circuit breaker with failure data
• Retry Pattern: Retries failed calls with exponential backoff. Retries happen before the circuit opens; once open, no retry occurs
• Bulkhead: Isolates resources (threads, connections) per dependency so that a slow service does not consume all the system’s resources
• Saga Pattern: When a saga step fails and the circuit breaker is open, the saga triggers compensations instead of retrying indefinitely
• API Gateway: The gateway can implement circuit breakers for each backend service, centralizing the protection
• Idempotency Pattern: When the circuit closes and operations are retried, idempotency guarantees that effects are not duplicated

Common mistakes

Next steps

The Circuit Breaker is a fundamental piece of resilience in microservices. To round out your understanding of resilience patterns, explore the Retry Pattern for handling transient failures with backoff, the Timeout Pattern for limiting waits, and the Bulkhead Pattern for isolating resources per dependency.