Ravindra BagaleCourses & study guides मराठी Track your progress

Project: Build Order Fan-Out with SNS, SQS, Lambda and DynamoDB (Source Code)

By , trainer LinkedIn Last updated:

One Swiggy order must notify email and inventory at the same time. SNS fans out to two SQS queues; each queue has its own Lambda writing DynamoDB.

What you'll learn in this chapter

  • SNS fan-out to two SQS queues
  • Publisher API Lambda
  • Two worker Lambdas
  • Cleanup topic, queues, functions

1. Everyday example

Why do we need this?

If email code crashes, inventory should still run. Separate queues isolate failures — that is fan-out.

What is it?

POST /orders → publisher Lambda → SNS topic → email-queue + inventory-queue → email_worker + inventory_worker → DynamoDB.

Fan-outAPIPOSTPublisherSNS PublishSNS topicSQS email + SQS inventoryLambdasDynamoDB rows
Subscribe queues to the topic with raw message delivery off (default JSON wrapper).

Download: order-fanout.zip

2. Lab

How to do it (step by step)

A. SNS + SQS + table

  1. SNS topic yourname-orders-topic.
  2. SQS queues yourname-email-q and yourname-inv-q. Subscribe both to the topic.
  3. DynamoDB yourname-fanout, PK pk String.

B. Workers + publisher

  1. Deploy email_worker.py and inventory_worker.py with SQS triggers. Env TABLE_NAME.
  2. Publisher.py with TOPIC_ARN. HTTP API POST /orders.
curl -s -X POST "$API_BASE/orders" -H 'content-type: application/json' \
  -d '{"sku":"TEE-RED","qty":1}'
# DynamoDB should show EMAIL#... and INV#... rows

Cleanup

  1. Delete API, publisher, workers, subscriptions, topic, queues, table, roles.

3. Interview

Q. Why not two publishes from the API Lambda?

SNS subscription list grows without redeploying the API.

Q. SNS to SQS wrapper?

Workers parse Message JSON inside the SQS body.

Q. Failure isolation?

Poison email queue does not block inventory queue.

Q. Filter policies?

SNS can route by message attributes later.

Q. Cleanup?

Topic, queues, Lambdas, table.

Try at home

  1. Add a third analytics queue.
  2. Use message attributes for VIP orders.

In short

  1. Fan-out = SNS to many SQS workers.
  2. One publish, many subscribers.
  3. Isolate failures per queue.
  4. Clean up.

Practice Tasks

About this scenario

Hyderabad Biryani Hub is a cloud kitchen business in Hyderabad. Syed Imran looks after its IT and cloud work. They count money in INR and run their AWS resources in the ap-south-2 Region. In the tasks below you do real jobs for them using what this lesson teaches. All names, account IDs, IP addresses and numbers are made up for practice.

0 / 12

Easy List functions

Why: See which functions exist in this Region.

What to do: List the Lambda functions in ap-south-2.

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Medium Invoke a function

Why: Invoke runs the function now and saves its answer in a file.

What to do: Invoke hydbiryani-orders-fn and save the response to out.json.

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Medium Trust policy for a Lambda role

Why: The trust policy says WHO may take the role.

What to do: Write a trust policy that lets lambda.amazonaws.com assume the role.

How:
  1. Start with { "Version": "2012-10-17", "Statement": [ ... ] }.
  2. In each statement write Effect, Action and Resource.
  3. Give only what the task needs (least privilege).
  4. Use double quotes and commas correctly. Click Check answer.

Medium Put an item

Why: DynamoDB JSON shows the type of each value: S for string, N for number.

What to do: Put an item into hydbiryani-orders with orderId "o-101" (string) and amount 2500 (number).

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Medium Change timeout and memory

Why: More memory also gives more CPU; timeout stops runaway runs.

What to do: Set hydbiryani-orders-fn to timeout 30 seconds and memory 512 MB.

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Hard Connect an SQS queue to Lambda

Why: An event source mapping makes Lambda poll the queue and process messages in batches.

What to do: Create an event source mapping from queue arn:aws:sqs:ap-south-2:130300927383:hydbiryani-orders to hydbiryani-orders-fn with batch size 10.

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Hard Create a Lambda function

Why: Lambda needs the code zip, a runtime, a handler name and an execution role.

What to do: Create function hydbiryani-orders-fn: runtime python3.12, handler app.handler, role arn:aws:iam::130300927383:role/lambda-basic, code in function.zip.

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Hard Set environment variables

Why: Keep settings like table names out of the code.

What to do: Set the environment variable TABLE=hydbiryani-orders on hydbiryani-orders-fn.

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Hard Invoke with a JSON payload

Why: In CLI v2, raw JSON needs --cli-binary-format raw-in-base64-out.

What to do: Invoke hydbiryani-orders-fn with payload {"orderId":"o-101"} (raw JSON) and save the reply in out.json.

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

Hard Lambda GB-seconds

Why: Lambda bills memory × time.

What to do: 1,000,000 invocations, 512 MB memory, 200 ms each. How many GB-seconds?

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Hard Read capacity units

Why: Provisioned tables need the right RCU.

What to do: Strongly consistent reads of 6 KB items, 10 reads per second. How many RCUs?

How:
  1. Write down the numbers the task gives you.
  2. Work it out step by step on paper.
  3. Type only the final answer in the box.
  4. Click Check answer.

Hard Let S3 call your function

Why: A resource-based permission is needed before S3 can invoke the function.

What to do: Add permission statement s3invoke to hydbiryani-orders-fn so s3.amazonaws.com can call lambda:InvokeFunction from bucket arn:aws:s3:::hydbiryani-uploads.

How:
  1. Read the task and decide which AWS service it is about.
  2. Write aws, then the service name, then the operation (the action word).
  3. Add each needed flag (--name) followed by its value from the task.
  4. Flags can be in any order. Click Check answer.

See all practice tasks in the Task Hub →

Revision, comparison and interview cards

Revise this lesson in a few minutes. Save the cards on your phone or download them as PNG files. The QR code on each card opens this lesson.

Revision card: Order fan-out with SNS+SQS. One Swiggy-style order must notify email and inventory at the same time without coupling those teams. API → Lambda → SNS topic → two SQS subscriptions (email-q, inventory-q) → each queue has its own Lambda → DynamoDB writes. Create topic, two queues, subscribe each queue to the topic, two worker roles, intake Publish, POST /orders. One POST creates two DynamoDB trails (email-sent row and inventory-reserved row) from separate workers.
Revision card: Order fan-out with SNS+SQS · Download PNG · Open card page
Revision card: Why SNS then SQS?. Calling two Lambdas from the API ties failures together and blocks the client on both. SNS fans out one message to many subscribers. SQS buffers each consumer so a slow email worker does not block inventory. Subscribe queue ARNs to the topic (SNS → SQS). Grant SNS permission to SendMessage. Workers use event source mappings. Pause the email worker; inventory still drains its queue. Restart email; backlog clears.
Revision card: Why SNS then SQS? · Download PNG · Open card page
Revision card: Failure isolation. If email code throws, inventory must still update. Shared one-Lambda fan-out does not isolate well. Separate queues and Lambdas isolate failures. Each queue can have its own DLQ and retry count. Set maxReceiveCount 3 → DLQ on each queue. Idempotent PutItem keyed by orderId+worker type. Poison email message lands in email-DLQ; inventory table still shows reserved for that orderId.
Revision card: Failure isolation · Download PNG · Open card page
Comparison: Amazon SNS vs Amazon SQS. Amazon SNS: Topic broadcasts one event to many subscribers. Amazon SQS: Queue buffers work for one consumer group.
Comparison: Amazon SNS vs Amazon SQS · Download PNG · Open card page
Interview card: Why not invoke two Lambdas from the API?. The client waits on both; one failure can fail the whole request; scaling couples the workers. SNS+SQS decouples producers from consumers and isolates failures per queue. API Publish once to SNS; email-q and inventory-q each trigger their Lambda.
Interview card: Why not invoke two Lambdas from the API? · Download PNG · Open card page
Interview card: How do you isolate email vs inventory failures?. Separate SQS queues, Lambdas, IAM roles, and optional DLQs. A poison email message does not block inventory drains. maxReceiveCount to email-DLQ; inventory queue continues independently.
Interview card: How do you isolate email vs inventory failures? · Download PNG · Open card page

More cards: Ravi Sir's Cards.