Message-Driven Architecture is the foundation for building reactive microservices (responsive, resilient, elastic). See why in this presentation and how to do it on AWS.

1. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Jerome Van Der Linden (@jeromevdl)
Message-Driven Architecture with AWS
Worldwide Software Architecture Summit
26th
of January 2021

2. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Who am I ?
Jérôme Van Der Linden
Solutions Architect @ AWS (Geneva, Switzerland)
Former consultant, developer, tech lead, agile &
devops coach, architect @ OCTO Technology
Husband and Dad of 3
@jeromevdl

3. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Agenda
Monolith to Microservices
Microservices to Reactive Microservices
Message-Driven Architecture on AWS
Wrap-up

4. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Monoliths to Microservices

5. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Monoliths
Do not throw the baby out with the bath water…
- Simple to develop and refactor
- Simple to test (integration, end-to-end)
- Single deployment process
- Single application to monitor
- Consistency
https://martinfowler.com/bliki/MonolithFirst.html

6. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Monoliths
… but definitely some limits!
- Strong coupling: less flexibility / agility
- High impact of change
- Redeploy everything
- Risks of blast radius
- Reliability
- One failure can bring down the whole application
- Rigid scaling
- Often stateful
- Limited to the database scalability
- Development lifecycle is typically slow
- Continuous deployment is difficult
- Barrier to adopting new technologies

7. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Microservices
Microservices to the rescue…
- Reduced coupling
- Smaller impact of changes
- Independent deployments
- Better reliability
- Independent scaling
- Smaller unit of development:
- Humanely understandable
- More suitable for adopting new technologies

8. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Microservices
… but it is not that magic!
- Reduced coupling
- Smaller impact of changes
- Independent deployments
- Better reliability
- Independent scaling
- Smaller unit of development:
- Humanely understandable
- More suitable for adopting new technologies
Are you sure?

9. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Microservices to Reac8ve
Microservices

10. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Microservices communica=on
Order
Service
Invoice
Service

11. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Synchronous communica=on
Order
Service
Invoice
Service
🙂 201
created
201
created
😀

12. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
🙂
Synchronous communica=on
Order
Service
Invoice
Service
500
internal
server
error
503
service
unavailable
☹

13. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Synchronous communica=on
What’s the problem ?
- No isolation between services è still coupled
- Risk of timeout or cascading failures è still unreliable
- What if the signature of invoice API change ? è still dependent

14. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Reac=ve Principles
Responsive
Resilient
Elastic
Message-Driven
https://www.reactivemanifesto.org/

15. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Message-Driven (Asynchronous) Communica=on
Order
Service
Invoice
Service
🙂 202
accepted
😀
order
created

16. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Message-Driven (Asynchronous) Communica=on
Order
Service
Invoice
Service
🙂 202
accepted
😀
order
created

17. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Message-Driven (Asynchronous) Communica=on
Isolation between services:
- Less risk of timeout è more responsive
- No more cascading failure è more resilient è more responsive
- What if the signature of invoice API change ? è No problem 😀 !

18. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Message-Driven Architecture on AWS

19. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
(Serverless) Messaging services on AWS
Amazon SNS Amazon
EventBridge
Amazon SQS

20. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple Queue Service (SQS)
Producer SQS Queue
Decouple
producers from
consumers.
Messages polled
by consumers
Amazon SQS
Fully managed
Message Queuing
service
Consumers
AWS Lambda
Container
Instance / VM
Standard
• At-least once delivery
• Best-effort ordering
Messages are stored
up to 14 days.

21. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple Queue Service (SQS)
Producer Consumers
AWS Lambda
Amazon SQS
Fully managed
Message Queuing
service
SQS Queue
Decouple
producers from
consumers
Messages polled
by consumers
Container
Instance / VM
FIFO
• Exactly once delivery
• Keep ordering
1
2
3

22. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple Queue Service (SQS)
Producer Consumers
AWS Lambda
Amazon SQS
Fully managed
Message Queuing
service
SQS Queue
Decouple
producers from
consumers
Long Polling
Container
Instance / VM

23. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple Queue Service (SQS)
Producer Consumers
AWS Lambda
Amazon SQS
Fully managed
Message Queuing
service
SQS Queue
Decouple
producers from
consumers
Messages polled
by consumers
Container
Instance / VM
Dead-letter Queue
If a message can’t be processed successfully
it can be held in a queue

24. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Common paIern with SQS: Storage-first
Amazon API Gateway Amazon SQS Queue
Dead letter Queue
AWS Lambda
Client
…

25. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple No=fica=on Service (SNS)
Amazon SNS
Fully managed
publish & subscribe
messaging service
SNS Topic
Decouple message
publisher from
subscribers with topics
Publisher
Messages pushed to
subscribers
Subscribers
AWS Lambda
Amazon SQS
HTTP/S
Email / SMS
Amazon Kinesis
Data Firehose

26. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple No=fica=on Service (SNS)
Amazon SNS
Fully managed
publish & subscribe
messaging service
SNS Topic
Decouple message
publisher from
subscribers with topics
Publisher
FIFO
Subscribers
AWS Lambda
Amazon SQS
FIFO
HTTP/S
Email / SMS
Amazon Kinesis
Data Firehose
1
2

27. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple No=fica=on Service (SNS)
Fan Out
Publisher Amazon SNS
Fully managed
publish & subscribe
messaging service
SNS Topic
Decouple message
publisher from
subscribers with topics
Subscribers
AWS Lambda
Amazon SQS
HTTP/S
Email / SMS
Amazon Kinesis
Data Firehose

28. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple No=fica=on Service (SNS)
Publisher Amazon SNS
Fully managed
publish & subscribe
messaging service
SNS Topic
Decouple message
publisher from
subscribers with topics
Message
Filtering
Filter messages
according to
subscription
filter policies
Subscribers
AWS Lambda
Amazon SQS
HTTP/S
Email / SMS
Amazon Kinesis
Data Firehose

29. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Simple No=fica=on Service (SNS)
Publisher Amazon SNS
Fully managed
publish & subscribe
messaging service
SNS Topic
Decouple message
publisher from
subscribers with topics
Message
Filtering
Filter messages
according to
subscription
filter policies
Dead-letter Queue
If a subscriber is unavailable,
messages can be held in a queue
Subscribers
AWS Lambda
Amazon SQS
HTTP/S
Email / SMS
Amazon Kinesis
Data Firehose

30. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Common paIern with SNS & SQS: Topic-Queue Chaining
Amazon SQS
Queue A
Backend A
Publisher Amazon SNS
Topic
Amazon SQS
Queue B
Backend B
Amazon SQS
Queue C
Backend C

31. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon EventBridge
Amazon
EventBridge
Fully managed
event bus to
connect applications
Event Sources
AWS Services
Custom events
SaaS events Events pushed
to targets
Rules
Rules filter
and send events
to target
Event Bus
Default event bus
Custom event bus
SaaS event bus
Decouple event
sources and targets
Targets (20+)
AWS Lambda
Amazon SQS
Amazon SNS
AWS Step
Functions
Amazon Kinesis
AWS Batch
Amazon API
Gateway
Amazon EC2
Instance
Amazon ECS
Task

32. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon EventBridge
Event Sources
Amazon
EventBridge
Fully managed
event bus to
connect applications
Event Bus
Targets (20+)
AWS Lambda
Amazon SQS
AWS Services
Custom events
SaaS events
Default event bus
Custom event bus
SaaS event bus
Rules
Amazon SNS
AWS Step
Functions
Amazon Kinesis
AWS Batch
Amazon API
Gateway
Amazon EC2
Instance
Amazon ECS
Task
Dead-letter
Queue
If a target is unavailable,
events can be held in a queue

33. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon EventBridge
Event Sources
Amazon
EventBridge
Fully managed
event bus to
connect applications
Event Bus
Targets (20+)
AWS Lambda
Amazon SQS
AWS Services
Custom events
SaaS events
Default event bus
Custom event bus
SaaS event bus
Rules
Amazon SNS
AWS Step
Functions
Amazon Kinesis
AWS Batch
Amazon API
Gateway
Amazon EC2
Instance
Amazon ECS
Task
Event Archive
Store all events
and enables replays

34. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon EventBridge
Event Sources
Amazon
EventBridge
Fully managed
event bus to
connect applications
Event Bus
Targets (20+)
AWS Lambda
AWS Services
Custom events
SaaS events
Default event bus
Custom event bus
SaaS event bus
Rules
AWS Step
Functions
AWS Batch
Amazon API
Gateway
Amazon EC2
Instance
Amazon ECS
Task
Schema Registry
Collection of event
structures (schema)
…
Code Binding
Schema
Discovery
Schema
Publication

35. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Common paIern with EventBridge: Microservices Choreography

36. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Wrap-up

37. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Services Comparison
Amazon SQS Amazon SNS
Amazon
EventBridge
Type Queue (Pull) Publish / Subscribe (Push) Publish / Subscribe (Push)
Persistence
Yes
(Up to 14 days, default: 4 days)
No
(retry up to 23 days + DLQ)
No
(retry up to 24h + DLQ + Archives)
Ordering FIFO FIFO No
Delivery Standard: at least once
FIFO: exactly once
Standard: at least once
FIFO: exactly once
At least once
Filtering Yes Yes
No
Consumers Millions 5 per rule
/
Target services Lambda (+ compute & SDK) 2 (+ HTTP/SMS/Email/Push) 20+
Throughput *
Price *
Standard: Almost unlimited
FIFO: 3 000 TPS (soft)
Standard: 30 000 TPS (soft)
FIFO: 300 TPS (soft)
10 000 TPS (soft)
$ per million API requests
Standard ≤ 0.4 / FIFO ≤ 0.5
$ per million API requests: 0.5 $ per million events: 1
* depends on the region (us-east-1)

38. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Conclusion
Message-Driven architectures drive resilience and elas3city è responsiveness
Asynchronous Events
Improve responsiveness
and reduce dependencies
Event Stores
Buffer messages until services
are available to process
Event Routers
Abstract producers and
consumers from each other

39. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Resources

40. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
• Building event-driven applications with Amazon EventBridge
• https://virtual.awsevents.com/media/1_w2mhd582
• Scalable serverless event-driven architectures with SNS, SQS & Lambda
• https://virtual.awsevents.com/media/1_o5hp8egy
• Decoupling serverless workloads with Amazon EventBridge
• https://virtual.awsevents.com/media/1_upe6mtew
• Choosing the right event-routing service for serverless: EventBridge, SNS, or SQS
• https://lumigo.io/blog/choosing-the-right-event-routing-on-aws-eventbridge-sns-or-sqs

41. © 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Thank you
@jeromevdl