Event-driven architectures can help decouple services and simplify dependencies as your application grows. There are now new options for developers looking to benefit from this approach with the release of AWS Lambda controls for asynchronous and stream processing events, Lambda event destinations, and Amazon EventBridge, a serverless event bus service. In this tech talk, we'll demonstrate strategies for managing events to help you build more resilient event-driven applications. You will learn how to customize responses to Lambda function results with new service features to handle retries, failures, and prevent “poison pill” events. We will also demonstrate how to configure Lambda event destinations to route execution results to other AWS services, and how to set up routing rules in Amazon EventBridge that react to all of your data sources.

1. © 2020, Amazon Web Services, Inc. or its Affiliates.
Controls for Managing Events in
Serverless Applications
Ben Smith
Senior Developer Advocate, AWS Serverless
@benjamin_l_s

2. © 2020, Amazon Web Services, Inc. or its Affiliates.
Agenda
Asynchronous invocations controls
Stream processing invocation controls

3. © 2020, Amazon Web Services, Inc. or its Affiliates.
Monoliths and microservices
Monolith
Does everything
Microservices
Does one thing

4. © 2020, Amazon Web Services, Inc. or its Affiliates.
The end goal
And do it faster

5. © 2020, Amazon Web Services, Inc. or its Affiliates.
Architecture v1
This is fine for a simple command orientated design

6. © 2020, Amazon Web Services, Inc. or its Affiliates.
Architecture v2
But the Order service needs additional logic
for each additional downstream service

7. © 2020, Amazon Web Services, Inc. or its Affiliates.
Long-term architecture
The Order service soon becomes a blocker
when integrating new services

8. © 2020, Amazon Web Services, Inc. or its Affiliates.
Events to the rescue

9. © 2020, Amazon Web Services, Inc. or its Affiliates.
[i-’vent]

10. © 2020, Amazon Web Services, Inc. or its Affiliates.
Events are observable, not directed

11. © 2020, Amazon Web Services, Inc. or its Affiliates.
Long-term architecture
Hooking services together this way
Prevents teams from moving fast

12. © 2020, Amazon Web Services, Inc. or its Affiliates.
Event routers for fan out
Event router
Router determines which downstream
systems receive each event

13. © 2020, Amazon Web Services, Inc. or its Affiliates.
Amazon EventBridge
schema registry

14. © 2020, Amazon Web Services, Inc. or its Affiliates.
A serverless event bus service for
SaaS and AWS services
• Fully managed, pay-as-you-go
• Native integration with SaaS
providers
• 17 target services
• Easily build event-driven
architectures

15. © 2020, Amazon Web Services, Inc. or its Affiliates.
• Source of truth for sharing
schema
• Explicitly published and auto-
discovered
• Integrations forVS Code and
JetBrains
• Language bindings for Java,
Python, andTypeScript
Schema registry

16. © 2020, Amazon Web Services, Inc. or its Affiliates.
Serverless

17. © 2020, Amazon Web Services, Inc. or its Affiliates.
What is serverless?
No infrastructure provisioning,
no management
Automatic scaling
Pay for value Highly available and secure

18. © 2020, Amazon Web Services, Inc. or its Affiliates.
Event-driven compute
Functions as a service
Serverless FaaS

19. © 2020, Amazon Web Services, Inc. or its Affiliates.
Serverless applications
Event source / invocation Services
Changes in
data state
Requests to
endpoints
Changes in
Resource state
Function
Node.js
Python
Java
C#
Go
Ruby
Runtime API

20. © 2020, Amazon Web Services, Inc. or its Affiliates.
Lambda execution model
Synchronous (push) Asynchronous (event) Stream
(Poll-based)
Amazon
DynamoDB
Amazon
SNS
/order
Amazon
S3
reqs
Amazon
Kinesis
changes
AWS Lambda
service
Amazon API
Gateway
Lambda
function
Lambda
function
function

21. © 2020, Amazon Web Services, Inc. or its Affiliates.
Asynchronous Invocations

22. © 2020, Amazon Web Services, Inc. or its Affiliates.
Async under
the hood
• Lambda polls one event at a time and invokes a function
• Lambda only deletes a message/request after processing successfully
Events Event Queue Lambda function

23. © 2020, Amazon Web Services, Inc. or its Affiliates.
What could
possibly go
wrong?
• Function errors, retried twice
• Throttling and system errors, retried for
up to six hours
• Duplicated processing

24. © 2020, Amazon Web Services, Inc. or its Affiliates.
When retries
are not
meaningful
Use these new controls:
• Event age
(Default/Maximum: 6 hours, Minimum:
60s)
• Retry attempts
(Default/Maximum: 2, Minimum: 0)

25. © 2020, Amazon Web Services, Inc. or its Affiliates.
How to
configure?
In the AWS Management Console
Using the AWS Serverless Application Model (SAM)

26. © 2020, Amazon Web Services, Inc. or its Affiliates.
Async under the
hood
• Lambda returns 2xx status code to confirm event received
Events Event Queue Lambda function
202 response

27. © 2020, Amazon Web Services, Inc. or its Affiliates.
Do more with
less code
using Lambda
Destinations
Lambda
SNS
SQS
EventBridge
Lambda
Success/Failure
"version":
"timestamp":
"requestContext": {
"requestId":
"functionArn":
"condition":
"approximateInvokeCount”:},
"requestPayload": {}, "responseContext": {
"statusCode": "executedVersion":},
"responsePayload": {}
Execution Result being sent to
downstream

28. © 2020, Amazon Web Services, Inc. or its Affiliates.
Before
Lambda
Destinations
Response
Total running time ms
Idle running time
Idle running time
Synchronous Lambda chaining means paying for
Idle wait times.

29. © 2020, Amazon Web Services, Inc. or its Affiliates.
Lambda
Destinations
On Success
Response
Total running time ms
Using Lambda Destinations OnSuccess, you can
chain functions asynchronously without paying for
idle wait time.

30. © 2020, Amazon Web Services, Inc. or its Affiliates.
Lambda
Destinations
On Failure
S3 Identify EventBridge
SQS
S3
S3
Transform
Transform
Process Error
On Failure
Handle function failures without coding extra logic
using Lambda Destinations OnFailure.

31. © 2020, Amazon Web Services, Inc. or its Affiliates.
Use
Destinations
for:
• Building simple workflow by chaining
Lambda functions
• Handling failures with additional context
• Visibility and notification using
EventBridge or SNS
• Batching events to process with SQS or
EventBridge

32. © 2020, Amazon Web Services, Inc. or its Affiliates.
Things to
know…
• Lambda execution role must have
permission to access the destination
service.
• Execution result size shouldn’t exceed
the payload size limit of downstream
services
• On-failure destination delivers more
context than dead-letter queue.

33. © 2020, Amazon Web Services, Inc. or its Affiliates.
How to
configure?
In the AWS Management Console
Using the AWS SAM

34. © 2020, Amazon Web Services, Inc. or its Affiliates.
Stream processing invocations
(from Amazon Kinesis and Amazon DynamoDB)

35. © 2020, Amazon Web Services, Inc. or its Affiliates.
Processing
Kinesis/
DynamoDB
data streams
Data Streams
shard 1
shard 2
shard 3
shard 4
Data
Producer
Lambda as Data Consumer
functionA
(instance 1)
functionA
(instance 2)
functionA
(instance 3)
functionA
(instance 4)
• By default Lambda processes one batch at a time per shard
• Max concurrency = number of shards

36. © 2020, Amazon Web Services, Inc. or its Affiliates.
Sparse data off
peak
• By default, Lambda poller fetches data every second
• Could result in batches with sparse data
1 second poller window

37. © 2020, Amazon Web Services, Inc. or its Affiliates.
Dealing with
sparse data off
peak with
Batch Window
• Batch Window allows to batch up to 300s before invocation.
• New invocation criteria: 6MB payload, Maximum Batch
Window, Maximum Batch Size
Wait up to 300 seconds

38. © 2020, Amazon Web Services, Inc. or its Affiliates.
Dealing with
data spikes with
Parallelization
Factor
• Parallelization Factor (PF) allows you to have up to 10
batches per shard. Order is guaranteed at partition key
level.
• Faster stream processing without over-scaling
Amazon Kinesis
Shard
1213
Record
Processor
Consistent hashing
1
2
1
3
Batcher 1
Batcher 2
Batcher 3
--parallelization-factor 3

39. © 2020, Amazon Web Services, Inc. or its Affiliates.
Failures on
stream
processing
• Malformed data leads to stuck shard
and infinite (~10000) retries.
• Records are always retried in batches,
which made it hard to identify the bad
record.
• 1 malformed record can block the entire
stream, the “poison pill”

40. © 2020, Amazon Web Services, Inc. or its Affiliates.
No more
stuck shards
• Skip processing old records using
MaximumRecordAgeInSeconds (60s to 7 days).
• Specify how many retries on a record with
MaximumRetryAttempts (0 to 10,000).
• Send metadata of failed records to SNS/SQS
using on-failure destination.

41. © 2020, Amazon Web Services, Inc. or its Affiliates.
Trace down
the bad
record with
Bisect On
Function Error
Batch size = 9, Bisect = true, MaxRetryAttempt = 1
on-failure destination configured
1st invoke
2nd invoke
3rd invoke
4th invoke
Error, Request ID 1, Retry = 0
5th invoke
6th invoke
7th invoke
Error, Request ID 2, Retry = 0
Success, Request ID 3, Retry = 0
Error, Request ID 4, Retry = 0
Success, Request ID 5, Retry = 0
Error, Request ID 6, Retry = 0
Error, Request ID 6, Retry = 1, sent
to on-failure destination
Record 1 -> 9
8th invoke Success, Request ID 7, Retry = 0

42. © 2020, Amazon Web Services, Inc. or its Affiliates.
Setting the new stream processing controls

43. © 2020, Amazon Web Services, Inc. or its Affiliates.
Summary
Asynchronous Invocations
Manage event schemas with
Schema Registry
Schema Discovery
Manage retries with
MaximumEventAgeInSeconds
MaximumRetryAttempts
Do more with less code
Lambda Destinations
OnSuccess
OnFailure
Stream Processing Invocations
Handle sparse data with
Batch Window
Handle spikes with
Parallelization Factor
Eliminate ”poison pill” with
BisectOnFunctionError
Manage failures with
MaximumRecordAgeInSeconds
MaximumRetryAttempts
OnFailureDestination
Use these new controls to manage events in your serverless applications

44. © 2020, Amazon Web Services, Inc. or its Affiliates.
Thank you
Ben Smith
Senior Developer Advocate, AWS Serverless
@benjamin_l_s