Ravindra BagaleCourses & study guides मराठी Track your progress

CQRS and Event Sourcing Patterns with Lambda and DynamoDB

A cricket scorer writes every ball in a notebook (the event log). The scoreboard is a projection you can rebuild by replaying balls. CQRS splits write commands from read models. Event sourcing stores facts as events, not only the latest row. On AWS this often means DynamoDB for writes plus Streams to Lambda that update read tables or OpenSearch.

What you'll learn in this chapter

  • Explain CQRS with a scoreboard story
  • Separate command side from query side
  • Describe event sourcing and when to skip it
  • Wire DynamoDB Streams to projection Lambdas
  • Link CDC and ecommerce project chapters

1. Everyday example: match notebook vs scoreboard

Kenji in Osaka scores a local match. The notebook is append-only: ball 1, ball 2. The public board shows 142/3. If the board glitches, he rebuilds from the notebook. Luis in Mexico City (MXN POS) can model orders the same way: OrderPlaced events append; a read table serves the app list screen fast.

Hands-on pieces: CDC Streams project, ecommerce cart project, depth in DynamoDB Streams chapter. This chapter names the pattern and the traps.

2. Why CQRS?

Why do we need this?

One fat table serving writes and every dashboard query forces ugly scans and contention. Spiky reads (festival tickets) should not block writes. CQRS lets you optimize each side.

What is it?

CQRS (Command Query Responsibility Segregation) = commands change state; queries read models shaped for screens. Event sourcing = store the stream of domain events as source of truth; derive state by folding events.

3. Vertical diagram

API Command -> Write model -> Stream -> Projection Lambda -> Read modelQueries hit read model only
Do not write from projection back to the same stream without care
Client command / query API Gateway + command Lambda Write model (DynamoDB) DynamoDB Streams Projection Lambda(s) Read models (DDB / OpenSearch) Query path skips the write Lambda
CQRS with Streams projections.

4. CQRS without full event sourcing

Many teams do CQRS light: write to orders table; stream updates a per-user inbox table. You still have a current row as source of truth. Full event sourcing keeps every OrderItemAdded forever — powerful and heavier.

ApproachKeep whenAvoid when
CQRS lightRead shapes differ; rebuild rareTeam cannot own two models
Full event sourcingAudit/rebuild/time-travel neededSimple CRUD blog

5. How: projection checklist

How to do it (step by step)

  1. Name commands: PlaceOrder, CancelOrder — not UpdateRow.
  2. Define events: OrderPlaced{orderId,amountINR,...}.
  3. Write model enforces invariants (stock, totals).
  4. Projection is idempotent on eventId / sequence.
  5. Queries never call command Lambda.
  6. Plan rebuild: replay stream or snapshot + replay.

6. Per-concept commands

Describe stream on a table

aws dynamodb describe-table --region ap-south-1 --table-name orders-write \
  --query 'Table.StreamSpecification' 

Meaning: Confirms NEW_IMAGE or NEW_AND_OLD_IMAGES for projections.

List event source mappings

aws lambda list-event-source-mappings --region ap-south-1 \
  --query 'EventSourceMappings[?contains(EventSourceArn, `dynamodb`)].{F:FunctionArn,S:State,U:UUID}' 

Meaning: Shows which Lambdas consume which streams and if they are enabled.

Sample CloudWatch log filter for projection errors

aws logs filter-log-events --region ap-south-1 \
  --log-group-name /aws/lambda/orders-projector \
  --filter-pattern 'ERROR' --limit 20 

Meaning: Projection failures cause stale read models — watch ERROR early.

Count items in a read table (careful)

aws dynamodb scan --region ap-south-1 --table-name orders-inbox \
  --select COUNT --max-items 1 

Meaning: Rough health check in labs only; prefer metrics in prod.

7. Industries

  • Fintech (Aisha, Lagos, NGN): ledger append events; balance projection per wallet.
  • Ecommerce (Rohan, Pune, INR): OrderPlaced; seller dashboard read model.
  • EdTech (Diego, São Paulo, BRL): LessonCompleted events; streak board projection.
  • Media (Priya, Bengaluru, INR): VideoUploaded; moderation queue read model.
  • SaaS (Sofia, Sofia, EUR): SeatAssigned; per-tenant admin list.

8. Lab: sketch models then inspect a stream

9. Lab - CQRS sketch + stream check

Do this in ap-south-1 (Mumbai) unless your org forbids it. Set a budget alert first.

How to do it (step by step)

  1. On paper: one command, two events, one read model for a library loan (Omar, Amman, JOD).
  2. Enable or inspect StreamSpecification on a lab table in ap-south-1.
  3. Note iterator age metric name you would alarm on.
  4. Write how you would rebuild the read model after a bug.

Cleanup (stop billing)

  1. Disable lab event source mappings you created.
  2. Delete lab tables if empty and tagged.

10. Same pattern across industries

  • Fintech (Aisha, Lagos): payment webhooks → SQS → Lambda; never block the bank callback thread.
  • Ecommerce (Rohan, Pune): checkout API on API Gateway+Lambda; inventory fan-out on SNS.
  • Healthcare (Mei, Singapore): appointment SMS via SNS; PHI stays in DynamoDB with fine-grained IAM.
  • EdTech (Diego, São Paulo): nightly cleanup EventBridge → Lambda on student drafts.
  • Logistics (Fatima, Casablanca): package scan events on EventBridge bus; Step Functions for customs saga.
  • Media (Kenji, Osaka): S3 upload → Lambda thumbnails; fan-out to moderation + CDN invalidate.
  • SaaS multi-tenant (Priya, Bengaluru): Cognito groups + DynamoDB partition per tenant.
  • Gov portal (Omar, Amman): strangler fig — peel one PDF form route per sprint to Lambda.

11. Troubleshooting checklist

  1. Confirm region (ap-south-1 vs us-east-1 for CloudFront/ACM quirks).
  2. Check CloudWatch Logs for the function name and request id.
  3. Verify IAM execution role allows the API you call.
  4. For async: check DLQ depth and SQS age of oldest message.
  5. For APIs: curl -v and compare status, CORS headers, body.
  6. For workflows: open Step Functions execution event history.
  7. Re-read the linked deep chapter before reinventing a service.
  8. If cost spikes: check concurrency, retries, and provisioned capacity.

12. Common errors and fixes

Each row is a real mistake students and juniors hit. Read the fix before you panic.

Error / symptomFix
Read model hours behindCheck iterator age, Lambda concurrency, batch size, DLQ.
Projection double-applies eventIdempotency on eventId; conditional put.
Infinite loop write→stream→writeNever write back to same watched keys without guard.
Query still scans write tableRoute API to read table; fix IAM.
NEW_IMAGE missing old stockUse NEW_AND_OLD_IMAGES when diff needed.
OpenSearch mapping explodeVersion projections; recreate index carefully.
Team debates every field as eventEvent storm — start CQRS light.
PII in every event foreverMinimize payload; encrypt; retention policy.
Currency missing on OrderPlacedInclude currency code (INR/USD/EUR) in event.

13. FAQ

Q. Must CQRS use event sourcing?

No. CQRS light is common and enough.

Q. Is DynamoDB Streams event sourcing?

It is a change feed; full ES is a design choice on top.

Q. How do UIs stay consistent?

Read-your-writes via short sync or wait on projection.

Q. Link to CDC project?

dynamodb-project-cdc-streams-lambda-opensearch.

Q. When to avoid?

Simple CRUD with one screen shape.

14. Interview Q&A (Beginner / Mid / Senior + MNC)

Q. Beginner: CQRS?

Split commands that write from queries that read.

Q. Beginner: projection?

Read model updated from events/stream.

Q. Mid: idempotent projector?

Same event twice → same read state.

Q. Mid: when full ES?

Audit, rebuild, temporal queries justify cost.

Q. Senior: poison event?

DLQ, fix, replay with version; do not block whole shard forever.

Q. MNC: compliance?

Immutable event store + access logs + retention.

15. Tips in English, Marathi and Hindi

Ravindra Bagale's Tip (EN)

Notebook of balls rebuilds any scoreboard. Events rebuild read models.

Ravindra Bagale's Tip (MR)

चेंडू वही → scoreboard. Events → read models.

Tip (HI)

बॉल की कॉपी से स्कोअरबोर्ड: events से read models।

Try at home

  1. Name 3 events for a food order.
  2. Draw write vs read tables.

In short

  1. Start from a real story, then name the AWS services.
  2. Link existing chapters — do not copy their labs.
  3. Design for failure: retries, DLQ, idempotency, rollback.
  4. Measure before you buy provisioned concurrency or multi-region.

16. Architecture decision checklist

  1. Write the user journey in five bullets with no AWS names.
  2. Mark each step sync (user waits) or async.
  3. Name failure mode: retry, DLQ, compensate, or alert-only.
  4. Name the idempotency key for every write.
  5. Pick data store from access patterns (link DynamoDB modeling).
  6. Estimate monthly cost at 1x and 10x traffic (USD and INR).
  7. Confirm account concurrency and service quotas in ap-south-1.
  8. Decide auth: Cognito, IAM, Lambda authorizer, or none.
  9. Name on-call and which CloudWatch alarm pages them.
  10. Link the deep chapter you will not rewrite.
  11. Only then open console or IaC.

17. Copy-paste quick commands (read-only first)

# Region and identity
aws configure get region
aws sts get-caller-identity

# Lambda landscape
aws lambda list-functions --region ap-south-1 --max-items 20 \
  --query 'Functions[].{N:FunctionName,R:Runtime,T:Timeout,M:MemorySize}' --output table
aws lambda get-account-settings --region ap-south-1

# API Gateway (HTTP APIs)
aws apigatewayv2 get-apis --region ap-south-1 --query 'Items[].{N:Name,Id:ApiId,Ep:ProtocolType}' --output table

# Queues and topics
aws sqs list-queues --region ap-south-1
aws sns list-topics --region ap-south-1 --query 'Topics[].TopicArn'

# Event buses and state machines
aws events list-event-buses --region ap-south-1
aws stepfunctions list-state-machines --region ap-south-1 --query 'stateMachines[].name'

# DynamoDB tables (names only)
aws dynamodb list-tables --region ap-south-1

Run these in a sandbox account. Prefer read-only IAM while learning.

18. More industry and stack stories

  • Fintech (Aisha, Lagos, NGN): bank webhooks must return 200 fast — SQS absorbs spikes; ledger Lambda is idempotent on paymentId.
  • Ecommerce (Rohan, Pune, INR): Black Friday — API Gateway throttle + SQS order intake; SNS fans inventory, email, analytics.
  • Healthcare (Mei, Singapore, SGD): appointment reminders via SNS SMS; PHI only in DynamoDB with condition keys.
  • EdTech (Diego, São Paulo, BRL): EventBridge nightly cleanup of draft quizzes; Lambda timeout 30s, not 15 min.
  • Logistics (Fatima, Casablanca, MAD): package scans on custom EventBridge bus; Step Functions customs saga.
  • Media (Kenji, Osaka, JPY): S3 → Lambda thumbnails; never write thumbs to the watched prefix.
  • SaaS (Priya, Bengaluru, INR): Cognito groups + DynamoDB PK tenantId; one Lambda authorizer for all routes.
  • Gov (Omar, Amman, JOD): strangler — peel one PDF form route per sprint from the monolith.
  • Startup (Sofia, Sofia, EUR): start with Function URL for demos; move to HTTP API when partners need JWT.
  • Retail (Luis, Mexico City, MXN): store POS sync via SQS FIFO when order matters; Standard when it does not.
  • Python: handlers thin; put domain logic in plain modules for unit tests without Lambda.
  • Node: prefer async/await; never block the event loop on sync DNS.
  • Java: SnapStart for cold starts; keep dependencies lean.
  • .NET: AspNetCoreServer for low-traffic tenants; watch package size.
  • Go: single static binary; great for SQS workers.
  • PHP: keep Laravel queues on ECS; Lambda for S3/webhooks glue.

19. Glossary (short)

TermMeaning
Cold startFirst invoke pays init; later invokes reuse the env
IdempotencySame request twice → same effect
DLQDead-letter queue for failed messages
ChoreographyServices react to events; no central conductor
OrchestrationCentral workflow (Step Functions) directs steps
Fan-outOne event → many consumers
Fan-inMany branches join before next step
Strangler figPeel routes from monolith gradually
SagaDistributed tx via steps + compensations
Provisioned concurrencyKeep N envs warm; you pay for ready capacity

20. Practice: explain out loud

  1. 60 seconds: everyday story for CQRS and event sourcing.
  2. 60 seconds: sync vs async with a checkout example.
  3. 60 seconds: when you refuse Lambda.
  4. 60 seconds: DLQ + idempotency in one breath.
  5. Record once; cut fillers; record again.

21. Safe lab hygiene

  • Tag every lab resource with owner and expiry date.
  • Set a monthly AWS Budget alarm in INR or USD before experiments.
  • Prefer read-only commands first; mutate in a sandbox account.
  • Delete queues, APIs, tables you created — DynamoDB and APIGW leave footprints.
  • Never put production secrets in env vars or screenshots.
  • Do not enable provisioned concurrency for a hello-world.

23. Myths to drop

MythReality
Serverless means zero serversServers exist; you do not manage them
Always cheaper than EC2Steady high load can favor EC2/Savings Plans
No Ops neededYou own IAM, data, alarms, cost, incidents
One Lambda per app is fineBounded contexts scale better
Queues are optional decorationQueues are the load leveler and blast-radius wall

24. Error budget thinking for serverless

If your monthly error budget is 0.1% on checkout, a fan-out that retries aggressively can burn the budget with duplicate emails while money stays correct. Separate business correctness SLOs from notification SLOs.

  1. Write two SLOs: money-correct and notify-delivered.
  2. Retries on money path: bounded + idempotent.
  3. Retries on notify path: best-effort + DLQ.
  4. Page humans on money DLQ; ticket on notify DLQ.

25. Regional notes (ap-south-1)

  • Build labs in Mumbai (ap-south-1) unless your org mandates elsewhere.
  • Some services launch first in us-east-1 — check feature availability.
  • Data residency for INR fintech may require in-region only.
  • Latency to end users in India usually favors ap-south-1.

26. Naming conventions that save incidents

# Suggested prefixes
# Lambda:  pay-intake-prod, pay-worker-prod
# SQS:     pay-orders.fifo / pay-orders-dlq
# SNS:     pay-order-events
# EB bus:  pay.domain
# SFN:     pay-refund-saga
# DDB:     pay-orders-prod

Include env (dev/stage/prod) and domain. Future you grepping CloudWatch will thank present you.

27. Pre-mortems (ten minutes)

  1. Imagine tomorrow headline: double charge in AED. What broke?
  2. Imagine DLQ depth 50,000 after a bad deploy. How do you drain?
  3. Imagine Cognito outage. Do APIs fail closed or serve read-only?
  4. Write answers in the runbook before go-live.

28. Currency and locale in examples

Rotate currencies in designs: INR, USD, EUR, AED, NGN, BRL, JPY, SGD. Rotate names: Aisha, Rohan, Mei, Diego, Fatima, Kenji, Priya, Omar, Sofia, Luis. This keeps mental models global — not one-country samples.

29. Worked story: Mei appointment board (SGD)

  1. Command BookSlot writes AppointmentBooked to write table.
  2. Stream wakes projector; updates doctor-day read table.
  3. Clinic TV queries read table only.
  4. Cancel emits AppointmentCancelled; projector removes slot.

30. Anti-patterns

  • One mega-event GodUpdated with 80 fields.
  • Projections in the command Lambda synchronously always.
  • No replay story when projection code has a bug.

Revision, interview and comparison cards

Revise this lesson in a few minutes. Save the cards on your phone or download them as PNG files. The QR code on each card opens this lesson.

Revision card: CQRS splits read/write. One overloaded model makes both writes and dashboards slow. Commands update the write model; queries hit denormalized projections. Accept command → emit events → update projection asynchronously. Sara Patel separates order write path from seller dashboard reads.
Revision card: CQRS splits read/write · Download PNG · Open card page
Revision card: Event sourcing log. Overwriting state erases the why behind every change. Store facts as an append-only event stream; rebuild state by folding. Append OrderPlaced → replay to rebuild → snapshot for speed. Omar Hassan uses the event log as the audit source of truth.
Revision card: Event sourcing log · Download PNG · Open card page
Comparison: Single CRUD model vs CQRS. Single CRUD model: One model serves reads and writes. CQRS: Write model + read projections diverge on purpose.
Comparison: Single CRUD model vs CQRS · Download PNG · Open card page
Comparison: Current-state store vs Event-sourced log. Current-state store: Keep latest row only. Event-sourced log: Append facts; fold to state; audit/replay.
Comparison: Current-state store vs Event-sourced log · Download PNG · Open card page
Interview card: When CQRS helps. CQRS on a tiny CRUD app adds accidental complexity. Use when read shapes differ hard from write rules or scale separately. MNC: one good CQRS fit and one anti-fit.
Interview card: When CQRS helps · Download PNG · Open card page
Interview card: Streams → projections. Polling the write table for dashboards fights the write path. DynamoDB Streams or EventBridge feed async projection builders. MNC: how do you handle duplicate stream records?
Interview card: Streams → projections · Download PNG · Open card page
Interview card: Replay to rebuild. Broken projections without a rebuild story strand the read side. Replay the event log (optionally from snapshot) to rebuild views. MNC: why keep snapshots if you can always replay?
Interview card: Replay to rebuild · Download PNG · Open card page
Interview card: Version event schemas. Silent field renames break every consumer overnight. Version events; consumers tolerate unknown fields; owners publish contracts. MNC: additive vs breaking change — give one example each.
Interview card: Version event schemas · Download PNG · Open card page

More cards: Ravi Sir's Cards.

Practice Tasks

About this scenario

Vidarbha Cotton Mills is a textiles business in Nagpur. Ganesh Wankhede looks after its IT and cloud work. They count money in INR and run their AWS resources in the ap-south-1 Region. In the tasks below you do real jobs for them using what this lesson teaches. All names, account IDs, IP addresses and numbers are made up for practice.

0 / 16

Easy CQRS splits read write — 1

Why: Idea.

What to do: Enter 1 if CQRS separates the write model from the read model.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Easy ES stores events — 1

Why: Idea.

What to do: Enter 1 if event sourcing stores state as a sequence of events.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Easy Commands change state — 1

Why: Write.

What to do: Enter 1 if commands are the write-side intents that change state.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Easy Queries hit projections — 1

Why: Read.

What to do: Enter 1 if queries usually hit denormalized projections.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Easy List DynamoDB tables

Why: CLI.

What to do: List DynamoDB tables

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Easy CQRS sides YAML

Why: Map.

What to do: YAML: write: commands, read: projections.

How:
  1. Start from the top-level keys the task names.
  2. Indent child keys with 2 spaces (no tabs).
  3. Fill in each property with the value from the task.
  4. Click Check answer. Key order does not matter.

Easy ES helps audit — 1

Why: Why.

What to do: Enter 1 if an event log helps audit and replay.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Medium DDB Streams → projection — 1

Why: Pipe.

What to do: Enter 1 if DynamoDB Streams can feed CQRS projections.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Medium Replay rebuilds views — 1

Why: Rebuild.

What to do: Enter 1 if replaying the event log can rebuild read models.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Medium Read lag is normal — 1

Why: Tradeoff.

What to do: Enter 1 if eventual consistency between write and read is expected.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Medium Projector idempotent — 1

Why: Safe.

What to do: Enter 1 if projection handlers must tolerate duplicate events.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Medium Pipe YAML

Why: Flow.

What to do: YAML: source: stream, target: projection.

How:
  1. Start from the top-level keys the task names.
  2. Indent child keys with 2 spaces (no tabs).
  3. Fill in each property with the value from the task.
  4. Click Check answer. Key order does not matter.

Hard Snapshots speed fold — 1

Why: Perf.

What to do: Enter 1 if snapshots reduce fold time for long event streams.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Hard Version event schemas — 1

Why: Evolve.

What to do: Enter 1 if event schemas need versioning as the domain evolves.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Hard Not every CRUD needs ES — 1

Why: Fit.

What to do: Enter 1 if not every CRUD app needs full event sourcing.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Hard Delete lab same day — 1

Why: Hygiene.

What to do: Enter 1 if lab-spg-cqrs should be deleted same day.

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.