Saga Pattern for Distributed Transactions on Serverless
Two shops share one customer order: inventory and payment. If payment succeeds but inventory fails, someone must undo the charge. That undo dance is the saga pattern — a sequence of local steps with compensations when a later step fails. On serverless you usually run it with Step Functions.
What you'll learn in this chapter
- Explain saga vs two-phase commit in plain words
- Draw happy path and compensate path
- Choose orchestration saga (SFN) vs choreography saga
- Name idempotent compensations
- Link order-workflow and DynamoDB transactions chapters
1. Everyday example: booking a trip across two counters
Fatima in Casablanca books a train seat (MAD 420) then a hotel. The train counter stamps the ticket. The hotel is full. She must cancel the train — that cancel is a compensation. Nobody holds a global lock across both companies for hours. Same idea when Lambda A writes payment and Lambda B reserves stock.
You already orchestrated steps in order workflow project. Single-table atomic multi-item writes live in DynamoDB transactions. Saga covers cross-service work those two alone cannot.
2. Why saga?
Why do we need this?
Distributed ACID across DynamoDB, Stripe, and an email API does not exist. Waiting with locks burns Lambda time and money. Saga accepts eventual consistency: each step commits locally; failures trigger compensations.
What is it?
Saga = ordered local transactions + a compensation for each step that may need undo. Orchestration saga = Step Functions directs. Choreography saga = each service reacts to events (EventBridge/SNS) and emits the next.
3. Vertical diagram: orchestration saga
4. Orchestration vs choreography saga
| Style | Pros | Cons |
|---|---|---|
| Orchestration (SFN) | Visible history, timeouts, catch | Central definition to maintain |
| Choreography (events) | Loose coupling | Hard to see whole flow; poison loops |
| Hybrid | SFN for money; events for notify | Two mental models |
Ravindra Bagale's Tip
Money and inventory: prefer Step Functions. Marketing emails after success: events are fine.
5. How: design a payment+stock saga
How to do it (step by step)
- Write business steps without AWS names: reserve, charge, ship.
- For each step write the undo: release, refund, cancel-label.
- Pick idempotency keys: orderId+step.
- Model SFN: Task then Task; Catch then compensate chain.
- Emit OrderCompleted only after all forward steps.
- Alarm on executions failed and compensate failures.
6. Per-concept: commands with meaning
List state machines
aws stepfunctions list-state-machines --region ap-south-1 \
--query 'stateMachines[].{N:name,A:stateMachineArn}' --output table
Meaning: Confirms your saga machine exists in Mumbai before you debug executions.
Describe a failed execution
aws stepfunctions get-execution-history --region ap-south-1 \
--execution-arn "$EXEC_ARN" --reverse-order --max-items 20 \
--query 'events[?type==`TaskFailed` || type==`ExecutionFailed`]'
Meaning: Shows which Task failed and the error string - start compensate debugging here.
Start execution with order payload
aws stepfunctions start-execution --region ap-south-1 \
--state-machine-arn "$SM_ARN" \
--name "ord-7841-$(date +%s)" \
--input '{"orderId":"ord-7841","amountINR":2499,"sku":"MOMO-12"}'
Meaning: Name should be unique; reuse of the same name returns the prior execution.
Stop a runaway execution
aws stepfunctions stop-execution --region ap-south-1 \
--execution-arn "$EXEC_ARN" --error "ManualStop" --cause "lab cleanup"
Meaning: Use in labs when a wait state loops; production prefers alarms + runbook.
7. Compensation rules
- Compensate in reverse order of success.
- Compensations must be idempotent (refund twice = one money move).
- If compensate fails, park in a human queue - do not loop forever.
- Never compensate a step that did not succeed.
- Log correlation id = orderId across all Lambdas.
- Persist currency on charge; refund same currency (INR/USD/EUR/AED).
8. When NOT to use saga
| Smell | Better | Why |
|---|---|---|
| Single DynamoDB table multi-item | TransactWriteItems | True ACID in one request |
| All steps in one Lambda under 3s | Just call in process | No distributed boundary |
| Read-only fan-out | SNS/SQS no compensate | Nothing to undo |
9. Industries using sagas
- Fintech (Aisha, Lagos, NGN): disburse then notify - compensate freezes wallet.
- Ecommerce (Rohan, Pune, INR): reserve SKU, charge Razorpay, ship.
- Travel (Sofia, Sofia, EUR): flight hold + hotel - classic saga.
- Healthcare (Mei, Singapore, SGD): book slot then insurance pre-auth.
- Logistics (Omar, Amman, JOD): customs hold + carrier booking.
10. Lab: dry-run a saga on paper then SFN
11. Lab - sketch then inspect a state machine
Do this in ap-south-1 (Mumbai) unless your org forbids it. Set a budget alert first.
How to do it (step by step)
- On paper: 3 forward + 3 compensate steps for a 999 INR course purchase (Priya, Bengaluru).
- In console or CLI, open the order-workflow state machine from the project chapter.
- Start a test execution with a unique name; watch Graph view.
- Force a fail path if the project supports a fail flag; read history.
- Write which states would be compensations in your course saga.
Cleanup (stop billing)
- Stop test executions; do not leave loops running.
- Delete any lab-only state machine you created.
12. Same pattern across industries
- Fintech (Aisha, Lagos): payment webhooks → SQS → Lambda; never block the bank callback thread.
- Ecommerce (Rohan, Pune): checkout API on API Gateway+Lambda; inventory fan-out on SNS.
- Healthcare (Mei, Singapore): appointment SMS via SNS; PHI stays in DynamoDB with fine-grained IAM.
- EdTech (Diego, São Paulo): nightly cleanup EventBridge → Lambda on student drafts.
- Logistics (Fatima, Casablanca): package scan events on EventBridge bus; Step Functions for customs saga.
- Media (Kenji, Osaka): S3 upload → Lambda thumbnails; fan-out to moderation + CDN invalidate.
- SaaS multi-tenant (Priya, Bengaluru): Cognito groups + DynamoDB partition per tenant.
- Gov portal (Omar, Amman): strangler fig — peel one PDF form route per sprint to Lambda.
13. Troubleshooting checklist
- Confirm region (ap-south-1 vs us-east-1 for CloudFront/ACM quirks).
- Check CloudWatch Logs for the function name and request id.
- Verify IAM execution role allows the API you call.
- For async: check DLQ depth and SQS age of oldest message.
- For APIs: curl -v and compare status, CORS headers, body.
- For workflows: open Step Functions execution event history.
- Re-read the linked deep chapter before reinventing a service.
- If cost spikes: check concurrency, retries, and provisioned capacity.
14. Common errors and fixes
Each row is a real mistake students and juniors hit. Read the fix before you panic.
| Error / symptom | Fix |
|---|---|
| Task timed out; saga stuck | Raise Lambda timeout or SFN Task TimeoutSeconds; check VPC cold starts. |
| Refund ran twice then double credit | Idempotency key on refund provider; condition write in ledger. |
| Compensate ran though reserve never succeeded | Catch only after successful Task; use ResultPath carefully. |
| Execution name already exists | Use orderId+timestamp; do not reuse names. |
| Choreography loop: A to B to A | Add event version or causation id; prefer SFN for money. |
| Partial ship after refund | Compensate order strict; gate ship on status=PAID. |
| Human cannot find failed order | Index DynamoDB by orderId; alarm includes orderId. |
| INR vs USD currency mix in refund | Persist currency on charge step; refund same currency. |
| Step Functions payload over 256KB | Store blob in S3; pass pointer in state. |
15. FAQ
Q. Is saga same as 2PC?
No. 2PC locks; saga commits locally and undoes later.
Q. Do I need saga for one DynamoDB table?
Usually TransactWriteItems is enough.
Q. SFN Express vs Standard for saga?
Standard for long/audit; Express for high-volume short.
Q. Can compensate be async?
Yes but track status; do not mark OrderFailed until compensate done or parked.
Q. What about DynamoDB transactions chapter?
Use it inside a step; saga coordinates across steps/services.
Q. Where is the live project?
serverless-saga-pattern-distributed-transactions.
16. Interview Q&A (Beginner / Mid / Senior + MNC)
Q. Beginner: what is a saga?
Sequence of local tx with compensations on failure.
Q. Beginner: compensate?
Undo of a prior successful step.
Q. Mid: orch vs choreo?
SFN central vs event-driven reactions.
Q. Mid: idempotent refund?
Same refund key then one credit.
Q. Senior: failure in compensate?
Dead-letter to human ops with orderId; never infinite retry money.
Q. MNC: audit?
SFN history + ledger table + correlation ids for compliance.
17. Tips in English, Marathi and Hindi
Ravindra Bagale's Tip (EN)
Two counters, one trip: if hotel fails, cancel the train. That cancel is the saga.
Ravindra Bagale's Tip (MR)
don counters: hotel fail then train cancel = saga.
Tip (HI)
दो काउंटर, एक यात्रा: होटल फेल तो ट्रेन cancel — यही saga।
Try at home
- Write 3 forward + 3 compensate steps for a food-delivery order.
- Mark which must be Step Functions vs SNS.
In short
- Start from a real story, then name the AWS services.
- Link existing chapters — do not copy their labs.
- Design for failure: retries, DLQ, idempotency, rollback.
- Measure before you buy provisioned concurrency or multi-region.
18. Architecture decision checklist
- Write the user journey in five bullets with no AWS names.
- Mark each step sync (user waits) or async.
- Name failure mode: retry, DLQ, compensate, or alert-only.
- Name the idempotency key for every write.
- Pick data store from access patterns (link DynamoDB modeling).
- Estimate monthly cost at 1x and 10x traffic (USD and INR).
- Confirm account concurrency and service quotas in ap-south-1.
- Decide auth: Cognito, IAM, Lambda authorizer, or none.
- Name on-call and which CloudWatch alarm pages them.
- Link the deep chapter you will not rewrite.
- Only then open console or IaC.
19. Copy-paste quick commands (read-only first)
# Region and identity
aws configure get region
aws sts get-caller-identity
# Lambda landscape
aws lambda list-functions --region ap-south-1 --max-items 20 \
--query 'Functions[].{N:FunctionName,R:Runtime,T:Timeout,M:MemorySize}' --output table
aws lambda get-account-settings --region ap-south-1
# API Gateway (HTTP APIs)
aws apigatewayv2 get-apis --region ap-south-1 --query 'Items[].{N:Name,Id:ApiId,Ep:ProtocolType}' --output table
# Queues and topics
aws sqs list-queues --region ap-south-1
aws sns list-topics --region ap-south-1 --query 'Topics[].TopicArn'
# Event buses and state machines
aws events list-event-buses --region ap-south-1
aws stepfunctions list-state-machines --region ap-south-1 --query 'stateMachines[].name'
# DynamoDB tables (names only)
aws dynamodb list-tables --region ap-south-1
Run these in a sandbox account. Prefer read-only IAM while learning.
20. More industry and stack stories
- Fintech (Aisha, Lagos, NGN): bank webhooks must return 200 fast — SQS absorbs spikes; ledger Lambda is idempotent on paymentId.
- Ecommerce (Rohan, Pune, INR): Black Friday — API Gateway throttle + SQS order intake; SNS fans inventory, email, analytics.
- Healthcare (Mei, Singapore, SGD): appointment reminders via SNS SMS; PHI only in DynamoDB with condition keys.
- EdTech (Diego, São Paulo, BRL): EventBridge nightly cleanup of draft quizzes; Lambda timeout 30s, not 15 min.
- Logistics (Fatima, Casablanca, MAD): package scans on custom EventBridge bus; Step Functions customs saga.
- Media (Kenji, Osaka, JPY): S3 → Lambda thumbnails; never write thumbs to the watched prefix.
- SaaS (Priya, Bengaluru, INR): Cognito groups + DynamoDB PK tenantId; one Lambda authorizer for all routes.
- Gov (Omar, Amman, JOD): strangler — peel one PDF form route per sprint from the monolith.
- Startup (Sofia, Sofia, EUR): start with Function URL for demos; move to HTTP API when partners need JWT.
- Retail (Luis, Mexico City, MXN): store POS sync via SQS FIFO when order matters; Standard when it does not.
- Python: handlers thin; put domain logic in plain modules for unit tests without Lambda.
- Node: prefer async/await; never block the event loop on sync DNS.
- Java: SnapStart for cold starts; keep dependencies lean.
- .NET: AspNetCoreServer for low-traffic tenants; watch package size.
- Go: single static binary; great for SQS workers.
- PHP: keep Laravel queues on ECS; Lambda for S3/webhooks glue.
21. Glossary (short)
| Term | Meaning |
|---|---|
| Cold start | First invoke pays init; later invokes reuse the env |
| Idempotency | Same request twice → same effect |
| DLQ | Dead-letter queue for failed messages |
| Choreography | Services react to events; no central conductor |
| Orchestration | Central workflow (Step Functions) directs steps |
| Fan-out | One event → many consumers |
| Fan-in | Many branches join before next step |
| Strangler fig | Peel routes from monolith gradually |
| Saga | Distributed tx via steps + compensations |
| Provisioned concurrency | Keep N envs warm; you pay for ready capacity |
22. Practice: explain out loud
- 60 seconds: everyday story for saga pattern.
- 60 seconds: sync vs async with a checkout example.
- 60 seconds: when you refuse Lambda.
- 60 seconds: DLQ + idempotency in one breath.
- Record once; cut fillers; record again.
23. Safe lab hygiene
- Tag every lab resource with owner and expiry date.
- Set a monthly AWS Budget alarm in INR or USD before experiments.
- Prefer read-only commands first; mutate in a sandbox account.
- Delete queues, APIs, tables you created — DynamoDB and APIGW leave footprints.
- Never put production secrets in env vars or screenshots.
- Do not enable provisioned concurrency for a hello-world.
24. How this chapter links others
- Compute depth: Lambda.
- HTTP edge: API Gateway.
- Data modeling: DynamoDB.
- Messaging: SQS/SNS.
- Events: EventBridge.
- Workflows: Step Functions.
- Project chapters (build-*): worked assemblies — link, do not paste.
25. Myths to drop
| Myth | Reality |
|---|---|
| Serverless means zero servers | Servers exist; you do not manage them |
| Always cheaper than EC2 | Steady high load can favor EC2/Savings Plans |
| No Ops needed | You own IAM, data, alarms, cost, incidents |
| One Lambda per app is fine | Bounded contexts scale better |
| Queues are optional decoration | Queues are the load leveler and blast-radius wall |
26. Error budget thinking for serverless
If your monthly error budget is 0.1% on checkout, a fan-out that retries aggressively can burn the budget with duplicate emails while money stays correct. Separate business correctness SLOs from notification SLOs.
- Write two SLOs: money-correct and notify-delivered.
- Retries on money path: bounded + idempotent.
- Retries on notify path: best-effort + DLQ.
- Page humans on money DLQ; ticket on notify DLQ.
27. Regional notes (ap-south-1)
- Build labs in Mumbai (ap-south-1) unless your org mandates elsewhere.
- Some services launch first in us-east-1 — check feature availability.
- Data residency for INR fintech may require in-region only.
- Latency to end users in India usually favors ap-south-1.
28. Naming conventions that save incidents
# Suggested prefixes
# Lambda: pay-intake-prod, pay-worker-prod
# SQS: pay-orders.fifo / pay-orders-dlq
# SNS: pay-order-events
# EB bus: pay.domain
# SFN: pay-refund-saga
# DDB: pay-orders-prod
Include env (dev/stage/prod) and domain. Future you grepping CloudWatch will thank present you.
29. Pre-mortems (ten minutes)
- Imagine tomorrow headline: double charge in AED. What broke?
- Imagine DLQ depth 50,000 after a bad deploy. How do you drain?
- Imagine Cognito outage. Do APIs fail closed or serve read-only?
- Write answers in the runbook before go-live.
30. Currency and locale in examples
Rotate currencies in designs: INR, USD, EUR, AED, NGN, BRL, JPY, SGD. Rotate names: Aisha, Rohan, Mei, Diego, Fatima, Kenji, Priya, Omar, Sofia, Luis. This keeps mental models global — not one-country samples.
31. Worked story: Diego course checkout (BRL)
- Diego pays BRL 189 for a video course.
- ReserveSeat in DynamoDB succeeds.
- ChargeCard fails (bank decline).
- Compensate: ReleaseSeat; no email welcome.
- UI shows try another card - no orphan seat.
32. Anti-patterns
- Giant Lambda that does saga with sleeps - use SFN wait states.
- Compensate that calls the forward API again by mistake.
- Sharing one IAM role that can refund anything in the account.
- Skipping orderId in logs so support cannot replay.
33. Metrics and alarms
- ExecutionsFailed on the state machine.
- Lambda Errors on each compensate function.
- Custom metric RefundLatencyMs.
- DLQ depth for human park queue.
Revision, interview and comparison cards
Revise this lesson in a few minutes. Save the cards on your phone or download them as PNG files. The QR code on each card opens this lesson.








More cards: Ravi Sir's Cards.
Practice Tasks
Maple Leaf Books is a bookselling business in Toronto. Emily Tremblay looks after its IT and cloud work. They count money in CAD and run their AWS resources in the ca-central-1 Region. In the tasks below you do real jobs for them using what this lesson teaches. All names, account IDs, IP addresses and numbers are made up for practice.
0 / 16
Easy Saga is long-lived txn — 1
Why: Idea.
What to do: Enter 1 if a saga is a long-lived business transaction across services.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Easy Compensation undoes — 1
Why: Undo.
What to do: Enter 1 if a compensation step undoes a prior successful step.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Easy SFN fits saga — 1
Why: Fit.
What to do: Enter 1 if Step Functions fits orchestrating a saga with compensations.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Easy Avoid 2PC serverless — 1
Why: Anti.
What to do: Enter 1 if classic two-phase commit is a poor fit for serverless.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Easy List state machines
Why: CLI.
What to do: List Step Functions state machines
- Read the task and decide which AWS service it is about.
- Write aws, then the service name, then the operation (the action word).
- Add each needed flag (--name) followed by its value from the task.
- Flags can be in any order. Click Check answer.
Easy Saga parts YAML
Why: Map.
What to do: YAML: forward: reserve, undo: release.
- Start from the top-level keys the task names.
- Indent child keys with 2 spaces (no tabs).
- Fill in each property with the value from the task.
- Click Check answer. Key order does not matter.
Easy Money path needs saga — 1
Why: Fit.
What to do: Enter 1 if multi-service money moves often need a saga.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Medium Choreo saga via events — 1
Why: Style.
What to do: Enter 1 if a choreography saga advances via published events.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Medium Compensations idempotent — 1
Why: Safe.
What to do: Enter 1 if compensation handlers must be idempotent.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Medium Order then pay then ship — 1
Why: Flow.
What to do: Enter 1 if a typical order saga is reserve → charge → ship with reverse comps.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Medium Catch → Compensate — 1
Why: SFN.
What to do: Enter 1 if Catch states often route failures into compensate paths.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Medium Saga styles YAML
Why: Map.
What to do: YAML: orchestrated: sfn, choreographed: events.
- Start from the top-level keys the task names.
- Indent child keys with 2 spaces (no tabs).
- Fill in each property with the value from the task.
- Click Check answer. Key order does not matter.
Hard Store saga state — 1
Why: Ops.
What to do: Enter 1 if you persist saga state or execution history for recovery.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Hard Timeouts on waits — 1
Why: Bound.
What to do: Enter 1 if waits in a saga need timeouts so stuck steps do not hang forever.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Hard Partial success needs comps — 1
Why: Reality.
What to do: Enter 1 if partial success across services is why compensations exist.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.
Hard Delete lab same day — 1
Why: Hygiene.
What to do: Enter 1 if lab-spf-saga should be deleted same day.
- Write down the numbers the task gives you.
- Work it out step by step on paper.
- Type only the final answer in the box.
- Click Check answer.