Saga Pattern
How to handle distributed transactions across microservices using choreography or orchestration with compensations.
What problem it solves
In a monolith, a database transaction can span multiple tables with a simple BEGIN β COMMIT β ROLLBACK. But in a microservices architecture, each service owns its own database. There is no global transaction that can roll back across 5 different databases if something fails at step 3.
The Saga Pattern solves this problem by splitting a distributed transaction into a sequence of local transactions, where each step has a compensating action that undoes its effect if a later step fails.
Example of the problem
Imagine a checkout flow in an e-commerce system:
- Order Service: Create the order
- Inventory Service: Reserve stock
- Payment Service: Charge the customer
- Shipping Service: Schedule the shipment
If the payment fails at step 3, you need to undo the stock reservation (step 2) and cancel the order (step 1). Without a defined pattern, this turns into a mess of manual calls and inconsistent states.
Why not use distributed transactions (2PC)
The Two-Phase Commit (2PC) protocol seems like the obvious solution, but it has serious problems in microservices:
| Aspect | 2PC | Saga |
|---|---|---|
| Availability | All participants must be available simultaneously | Each step is independent |
| Latency | Locks held during the entire transaction | Each step releases resources immediately |
| Scalability | Distributed locks limit throughput | No locks; each service scales independently |
| Coupling | Requires a coordinator that knows all participants | Services communicate through events |
| Recovery | If the coordinator fails, participants stay blocked | Compensations run asynchronously |
How it works
A saga is a sequence of local transactions where:
- Each transaction updates data within a single service
- Each transaction publishes an event or message that triggers the next one
- If a transaction fails, compensations are executed in reverse order
Successful flow:
T1 β T2 β T3 β T4 β
Flow with failure at T3:
T1 β T2 β T3 β β C2 β C1
(C = compensation)
There are two main strategies for coordinating a saga:
Choreography
Each service listens for events and decides what to do. There is no central coordinator.
ββββββββββββ OrderCreated ββββββββββββββββ
β Orders β βββββββββββββββββββΊ β Inventory β
ββββββββββββ ββββββββ¬ββββββββ
β
StockReserved
β
βΌ
ββββββββββββββββ
β Payments β
ββββββββ¬ββββββββ
β
PaymentApproved
β
βΌ
ββββββββββββββββ
β Shipping β
ββββββββββββββββ
Advantages: No single point of failure, decoupled services, easy to add participants. Disadvantages: Hard to understand the full flow, risk of cycles, hard to debug.
Orchestration
A central component coordinates the entire saga. It knows which steps to run and which compensations to apply.
ββββββββββββββββββββ
β Saga β
β Orchestrator β
ββββ¬ββββ¬ββββ¬ββββ¬βββ
β β β β
Create β β β β Schedule
order β β β β shipment
βΌ β β βΌ
βββββββββββ βββββββββββ
βOrders ββ ββShippingβ
βββββββββββ βββββββββββ
Reserve β Charge
stock β β payment
βΌ βΌ
ββββββββββββββββββββ
βInvent. ββPaymentsβ
ββββββββββββββββββββ
Advantages: Centralized flow, explicit control of compensations, easy to debug. Disadvantages: Potential single point of failure, risk of a God Service, coupling.
When to choose each strategy
| Criterion | Choreography | Orchestration |
|---|---|---|
| Number of steps | 2-4 simple steps | 5+ steps or complex flows |
| Visibility | Acceptable if simple | Needed for critical flows |
| Teams | Autonomous and independent | A central team manages the flow |
| Compensations | Simple and straightforward | Complex with conditional logic |
| Monitoring | Distributed tracing | Centralized dashboard |
Compensating transactions
Compensations are not simply an βundoβ. They are actions that semantically revert the effect of a previous transaction:
| Transaction | Compensation |
|---|---|
| Create order | Cancel order (change status, do not delete) |
| Reserve stock | Release reserved stock |
| Charge payment | Issue refund |
| Schedule shipment | Cancel shipment scheduling |
Key points about compensations:
- They are not rollbacks: They do not delete data; they create new records that revert the effect
- They must be idempotent: Running them multiple times produces the same result
- They can fail: You need retries for failed compensations
- They preserve an audit trail: Everything is recorded (the original action and the compensation)
Practical example: checkout saga with orchestration
Happy path
Orchestrator Orders Inventory Payments Shipping
β β β β β
βββ CreateOrder ββββΊβ β β β
ββββ OrderCreated βββ β β β
β β β β β
βββ ReserveStock ββββββββββββββββΊβ β β
ββββ StockReserved ββββββββββββββ β β
β β β β β
βββ ChargePayment βββββββββββββββββββββββββββΊβ β
ββββ PaymentApproved ββββββββββββββββββββββββ β
β β β β β
βββ ScheduleShipment ββββββββββββββββββββββββββββββββββββΊβ
ββββ ShipmentScheduled ββββββββββββββββββββββββββββββββββ
β β β β β
β
Saga completed
Payment failure (with compensations)
Orchestrator Orders Inventory Payments
β β β β
βββ CreateOrder ββββΊβ β β
ββββ OrderCreated βββ β β
β β β β
βββ ReserveStock ββββββββββββββββΊβ β
ββββ StockReserved ββββββββββββββ β
β β β β
βββ ChargePayment βββββββββββββββββββββββββββΊβ
ββββ PaymentRejected ββββββββββββββββββββββββ β
β β β β
β βββ COMPENSATIONS ββββββββββββββββββββ β
β βββ ReleaseStock βββββββββββββΊβ β β
β ββββ StockReleased βββββββββββ β β
β βββ CancelOrder ββββββΊβ β β β
β ββββ OrderCancelled βββ β β β
β ββββββββββββββββββββββββββββββββββββββ β
β β β
β Saga compensated
Persisted orchestrator state
{
"sagaId": "saga-12345",
"type": "FullCheckout",
"state": "COMPENSATING",
"currentStep": "ReleaseStock",
"data": {
"orderId": "ord-001",
"customerId": "cust-789",
"amount": 150.00
},
"history": [
{ "step": "CreateOrder", "state": "COMPLETED" },
{ "step": "ReserveStock", "state": "COMPLETED" },
{ "step": "ChargePayment", "state": "FAILED_INSUFFICIENT_FUNDS" },
{ "step": "ReleaseStock", "state": "COMPENSATING" }
]
}
Advantages
- Eventual consistency: Guarantees that the system will reach a consistent state
- Service autonomy: Each service keeps its own database and local transactions
- Resilience: If a step fails, compensations restore consistency
- Scalability: Requires no distributed locks or synchronous coordination
- Natural audit trail: Each step and compensation is recorded as an event
- Flexibility: Steps can be added or modified without affecting the entire flow
Trade-offs / Disadvantages
- Complexity: Designing correct compensations for each step is not trivial
- Eventual consistency: The system may be temporarily inconsistent
- Hard to debug: Tracing a saga requires good observability tooling
- Imperfect compensations: It is not always possible to fully revert an action
- Execution order: In choreography, guaranteeing the correct order is complex
- Testing: Exercising every possible path takes significant effort
- Inconsistency window: During execution, data can be in a partial state
When to use it
- Transactions that span multiple microservices with independent databases
- Long-running business flows (minutes, hours, or days)
- Systems where eventual consistency is acceptable
- When you need a complete audit trail of every step of the process
- Flows with steps that can fail and need a defined compensation
- Event-driven systems where services already communicate through events
When to avoid it
- Operations that require strong, immediate consistency (ACID)
- Simple flows involving only 2 services (may be overkill)
- When compensations are impossible or make no sense
- Systems where the inconsistency window is not acceptable
- Teams without experience in distributed systems
- When a monolith with local transactions solves the problem
Common tools and implementations
| Category | Options |
|---|---|
| Orchestrators | Temporal, Camunda, AWS Step Functions, Netflix Conductor |
| Messaging | Apache Kafka, RabbitMQ, Amazon SQS/SNS, Azure Service Bus |
| Frameworks | Axon Framework (Java), MassTransit (.NET), Eventuate Tram |
| Observability | Jaeger, Zipkin, OpenTelemetry (distributed tracing) |
| Saga state | PostgreSQL, Redis, DynamoDB |
Relationship with other patterns
The Saga rarely lives alone; it relies on a small ecosystem of supporting patterns:
βββββββββββββββββββββββββββββββββββββββββββββββββββ
β Saga ecosystem β
β β
β ββββββββββββ publishes ββββββββββββββββ β
β β Saga β βββββββββββββββΊβ Outbox β β
β β (steps) β events β (reliable) β β
β ββββββ¬ββββββ ββββββββββββββββ β
β β β
β β each step is β
β β idempotent β
β βΌ β
β ββββββββββββββββ protects βββββββββββββ β
β β Idempotency β ββββββββββββββΊβ Circuit β β
β β Pattern β calls β Breaker β β
β ββββββββββββββββ βββββββββββββ β
β β
β ββββββββββββ retries βββββββββββββ β
β β Retry β ββββββββββββββββββΊβ Timeout β β
β β Pattern β with a limit β Pattern β β
β ββββββββββββ βββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββ
- Outbox Pattern: Ensures that the events of each step are published reliably
- Idempotency Pattern: Each step and compensation must be idempotent for safe retries
- Circuit Breaker: Protects the orchestrator when a participating service is down
- Retry Pattern: Retries failed steps with backoff before triggering compensations
- Timeout Pattern: Sets a bound on each step so it does not hang forever before being considered failed
- Event-Driven Architecture: The saga is built on top of an event system
- DDD: Bounded contexts define the boundaries of each participant
Common mistakes
| Mistake | Consequence | Solution |
|---|---|---|
| Non-idempotent compensations | Duplicate refunds | Use idempotency IDs |
| Not persisting saga state | Sagas lost after a restart | Store state in a durable database |
| Ignoring failures in compensations | Stuck sagas that never resolve | Retries with a dead-letter queue |
| Saga with too many steps | Unmanageable complexity | Split into sub-sagas |
| Not monitoring active sagas | Stuck sagas go undetected | Dashboard with alerts |
Next steps
Once you master this pattern, explore the Idempotency Pattern to ensure that the sagaβs steps are safe against retries.