Microservices Communication Using AWS SNS & SQS
When we have several microservices that needs to communicate between them it’s recommended that this communication it’s asynchronous as much as we can to avoid having failures in several services in case one of them goes down.
Let’s gonna review the different use cases we might face.
Reacting to domain events
Most of the times our services will be reacting over other services domain events. This approach will keep our services decoupled from the other services.
We can also make one service react on their own domain events. It’s very used in cases where we want to decouple two resources inside the same service.
Sending asynchronous commands
Sometimes we’ll face situations where we don’t have any domain event to react from so we’ll need to create an imperative command to execute something in another service and to avoid doing a simple HTTP call we could use an asynchronous command.
Now we are gonna review which infrastructure is needed to be able to do that in AWS using SQS and SNS.
An SNS topic is the way to broadcast messages to different subscribers. A good approach is having one topic per service where we’ll be publishing all the domain events coming from that service.
Let’s say for example that we’d have
users-domain-events topic for users service and
notifications-domain-events for notifications service.
SQS queue subscribed to the SNS topic
Then we’ll need a way to get domain events from topics from another service. For that we’ll use SQS queues subscribed to the SNS topic.
It’s important that we use one SQS queue per consumer and that this consumer handles just one use case. That will ease the process of handling failures in case one use case starts failing.
Also, each queue will have one dead letter where events will be places after failing for X times.
To improve performance and reduce costs we can filter the incoming domain events by event type in the SNS subscription and we can avoid using FIFO queues for now. If we do so we’ll have to handle duplicated events or unsorted ones, this is easy to fix with some memory system like Redis.
Following the previous example we could say that we could have a SQS queue named
send-email-on-user-registered. This queue would be subscribed to the
users-domain-events topic from users service, and would be consumed by notifications service.
SQS FIFO queue for asynchronous commands
For the last use case that it’s about publishing asynchronous commands and consuming them from the same service or from another one we could use a FIFO SQS queue.
It’s important that this queue is FIFO because in case of the commands the order is very important since could affect the state or the behaviour of the application.
For example we could have
users-async-commands for user service and
notifications-async-commands for notifications service.
aws sqs sns microservices event-driven