Design Patterns with AWS SQS
Key use cases, best practices, and when to combine SQS with SNS
Queues are one of the ways to decouple your software architecture and make it more resilient.
But before we start… I just want to let you know that I’m starting another newsletter AI Native Compass.
In my current position, I’m starting to transform the way to develop software at company level by embracing AI Native, as I did in the past with Cloud Native. This is not another newsletter about LLM models, but how we, as software developers and architects, adapt our systems to onboard on the AI Native world.
If you are on the same journey, this newsletter is for you 👇🏻
And for my Spanish audience, you can find the replicate in your language Brújula AI Native 👇🏻
Now, let’s continue with today’s topic.
In today’s issue, I get you back to technical leadership by understanding the managed service AWS SQS. In detail, we cover:
Main Use Cases.
Working with SNS.
Do you like what you read? Share it with your friends via 👇🏻
First, a brief introduction for those who don’t know SQS.
👉🏼 Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale distributed systems reliably and securely.
The main features are:
Queue Types
Standard Queues: virtually unlimited throughput, “at‑least‑once” delivery (duplicates possible), and best‑effort ordering.
FIFO Queues: strict first‑in, first‑out delivery with exactly‑once processing and no duplicates.
Message Retention
Configurable retention period.
Visibility Timeout
After a consumer retrieves a message, it becomes invisible to others for a configurable interval; if not deleted within that window, it returns to the queue.
Dead‑Letter Queues (DLQ)
Automatically move messages that fail processing (after configurable retry attempts) into a separate queue for debugging and analysis.
Long Polling
Consumers can wait up to X seconds for messages to arrive before returning an empty response, reducing empty receives and lowering cost.
Delivery Delay & Message Timers
You can delay delivery of new messages at the queue level (up to 15 minutes) or set individual message delay timers.
Batch Operations
Send, receive, or delete multiple messages in a single API call to improve throughput and reduce API overhead.
Security & Access Control
Encryption in transit (TLS) and at rest (SSE‑KMS).
Integration with AWS Ecosystem
Native integration with SNS (pub/sub), AWS Lambda triggers, CloudWatch metrics and alarms, CloudTrail auditing, EventBridge events, and more.
Scalability & Durability
Stores messages redundantly across multiple Availability Zones.
Automatically scales to handle tens of thousands of transactions per second (standard queues) or hundreds of TPS (FIFO).
In the paper, this looks good enough.
However, for you as a software engineer, it’s important to understand when this service fits better in your stack.
🎯 What Are The Main Use Cases?
Use SQS when you have a single type of consumer (an ImageProcessor, for example) and want to ensure delivery and decoupling, but you don’t need fan-out or filtering.
1. Background task processing
Example: an API receives file upload requests and queues them to be processed (resized, transcoded, etc.).
A single service consumes the queue at its own pace.
👉🏼 SQS provides durability, retries, and decoupling.
2. Decoupling a producer from a consumer
Example: an order microservice generates messages that need to be processed by a billing system.
There is only one destination for the message.
👉🏼 SQS acts as a buffer between services, preventing bottlenecks.
3. Tasks that require controlled retries
Example: sending emails that might fail due to quota limits or SMTP downtime.
You configure a DLQ (Dead Letter Queue) and a redrive policy.
👉🏼 SNS is not needed, since fan-out is not required.
🎁
If you reached here, you deserve a gift.
I leave you, at the end of the email, another strong use case related to Quality of Service.
Even though SQS can play along in your architecture, it has a very good friend.
👯♂️ SQS Working With SNS
The way that SQS better works with SNS is: SNS to SQS. An SNS in front of multiple SQS instances.
👉🏼 Here we have the Pub/Sub Pattern.
What it solves:
Fan-out of the same message to multiple systems.
Event type filtering at the subscription level.
Decoupling by the book; The producer doesn’t know how many or who will consume.
Typical use cases:
Shared domain events:
A microservice “publishes” the
OrderCreatedevent to SNS.Three subscribed SQS queues handle billing, inventory, and notifications.
Multiple integrations: You publish user changes to an SNS topic, and several partners/external systems read from their own SQS queues.
Isolated environments: You produce to a single topic and have queues for prod, staging, and QA subscribed to the same topic.
Remember that SQS is not the same as SQS. Maybe this helps to consolidate the idea:
SQS → 📥 Message Queue.
SNS → 📣 Dissemination of Notifications.
🎁 Bonus Use Case
As I wrote you above, I have a bonus use case for you, from the real world, on which the SQS will help you with the aspect of Quality of Service.
See this real scenario:
A service is listening to a Kafka topic for requests to execute against a REST API, like an LLM.
When a request is received, the service calls the LLM REST API and gets the response.
And the response from the LLM is produced into another Kafka topic.
Pay attention to point 2.
When a request is received, the service calls the LLM REST API and gets the response.
This implies a blocking call to the REST API. The service will have to wait until the REST API provides the response, which will cause the processing of other requests to wait.
Now, you can tell me:
Marcos, from what I read in your email, this is a great use case for an SQS, what are the news here?
And that’s true, but what if I tell you that you could use an SQS per client?
👉🏼 You could have a separate SQS queue for each client that sends requests to your service, allowing you:
To configure different quality‑of‑service levels per client and
Balance your service’s workload accordingly.
For example, for premium clients, you process as you receive, but for others, you can decide to move back a message to the end of the queue in case the REST API starts to throttle.
Hope you enjoyed this bonus use case around SQS.
Now, let’s wrap up for today.
✨ Takeaways
As always, let’s finish with the important points you have to get into your toolbelt.
Decouple with SQS: Use SQS as a buffer between producers and a single logical consumer to ensure reliable delivery, automatic retries, and load smoothing.
Choose the right queue:
– Standard for maximum throughput with relaxed ordering and duplicate tolerance.
– FIFO for strict ordering and exactly‑once processing.Leverage SNS for fan‑out: When you need to broadcast a single event to multiple systems, place SNS in front of multiple SQS queues (SNS → SQS) and use subscription filters to route messages appropriately.
We are more than ✨1440 Optimist Engineers✨!! 🚀
Thanks for your support and feedback, I really appreciate it!
You’re the best! 🖖🏼
𝘐𝘧 𝘺𝘰𝘶 𝘦𝘯𝘫𝘰𝘺𝘦𝘥 𝘵𝘩𝘪𝘴 𝘱𝘰𝘴𝘵, 𝘵𝘩𝘦𝘯 𝘤𝘭𝘪𝘤𝘬 𝘵𝘩𝘦 💜. 𝘐𝘵 𝘩𝘦𝘭𝘱𝘴!
𝘐𝘧 𝘺𝘰𝘶 𝘬𝘯𝘰𝘸 𝘴𝘰𝘮𝘦𝘰𝘯𝘦 𝘦𝘭𝘴𝘦 𝘸𝘪𝘭𝘭 𝘣𝘦𝘯𝘦𝘧𝘪𝘵 𝘧𝘳𝘰𝘮 𝘵𝘩𝘪𝘴, ♻️ 𝘴𝘩𝘢𝘳𝘦 𝘵𝘩𝘪𝘴