Scalable serverless architectures using event-driven design - MAD308 - New York AWS Summit

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Scalable serverless architectures using
event-driven design
Sam Dengler
Principal serverless solutions architect
Amazon Web Services
M A D 3 0 8

Agenda
Serverless
Event-driven APIs
Messaging patterns
Serverless applications

What is serverless?
No infrastructure provisioning,
no management
Automatic scaling
Pay for value Highly available and secure

AWS
Lambda
AWS
Fargate
Amazon
API Gateway
Amazon Simple
Notification
Service
(Amazon SNS)
Amazon Simple
Queue Service
(Amazon SQS)
Compute
Data stores
Integration
Amazon Aurora
Serverless
Amazon Simple
Storage Service
(Amazon S3)
Amazon
DynamoDB
AWS
AppSync
AWS
Step Functions

Lambda is event-driven by nature
Services (anything)
Changes in
data state
Requests to
endpoints
Changes in
resource state
Event source Function
Node.js
Python
Java
C#
Go
Ruby
BYOR (bring your own runtime)

Enterprise integration patterns
“A technology-independent vocabulary
and a visual notation to design and
document integration solutions.”
• Messages
• Filter
• Router
• Return address
• Claim check
https://www.enterpriseintegrationpatterns.com/

Lambda Fargate
API Gateway
Amazon
SNS
Amazon
SQS
Compute
Data stores
Integration
Amazon Aurora
Serverless
Amazon
S3
DynamoDB
AWS
AppSync
Step Functions

Manage APIs with API Gateway
Mobile apps
Websites
Services
Internet Amazon
CloudFront
Amazon CloudWatch
monitoring
API
Gateway
cache
Any other
AWS service
All publicly
accessible
endpoints
Lambda
functions
Endpoints
in your VPC
Regional API endpoints
Lambda functions
Endpoints on Amazon
Elastic Cloud Compute
(Amazon EC2)
Your virtual private
cloud (VPC)
AWS

Challenges with synchronous serverless APIs
Client
API Gateway DynamoDBLambda
Timeouts
• API Gateway – 30 seconds
• Lambda – 15 minutes
Scale downstream resources
• Lambda – concurrent executions
• DynamoDB – read/write capacity units
Error handling
• Client responsibility
• Backoff/retry logic

API Gateway
Integration timeout: 30 seconds
Approaches
• Polling
• Webhooks
• WebSockets

Amazon S3 presigned URLs (claim check pattern)
• Uses permissions of the AWS Identity and Access
Management (IAM) user or role that creates the
URL
• To generate the URL, provide your security
credentials, a bucket name, an object key, HTTP
method (GET or PUT) and expiration date and
time
• Only valid until expiration time of the AWS
credentials or specified in the presign request
• Caution: Anyone with the URL can perform those
actions
Availability
Zone #1
Amazon EC2
instance
Generates
URL
Amazon
S3
Request access
Get or put
object

3. /status
Async: Polling
Flow
1. Client submits request and receives requestID
2. Backing service does work asynchronously, updating
job status
3. Client polls for status of request
4. Client fetches results when work is complete
Execution time
• < 15 mins – Step Functions (Lambda)
• >= 15 minutes – Step Functions (AWS Batch)
Throughput
• < 200 RPS (1K burst) – Direct SFn integration
• > 200 RPS – Lambda and Amazon SQS in front of SFn
Response payload
• > 10 MB – Amazon S3 presigned URL
Client
Amazon S3
1. /doWork
2
4. /getResults
API Gateway
API Gateway
API Gateway
Step Functions

Async: Polling
Pros
• Minimal changes for client
• Can wrap existing backend
Cons
• Delayed response due to polling schedule
• Excess compute work for client and server
Client
Step FunctionsAPI Gateway
API Gateway
API Gateway Amazon S3

Amazon SNS
Async: Webhooks Flow
0. (Optional) Trusted client configured with service
1. Client submits request; APIGW returns once request is
stored
2. Backing service does work asynchronously
3. Backing service calls back to client when work is complete
Client trust
• Trusted – Amazon SNS subscriptions with clientID filters
• Untrusted – Self sign-up endpoints or client provides
callback URL in request
Execution time
• < 15 minutes – Amazon SQS to Lambda
• >= 15 minutes – Step Functions or AWS Batch
Response payload
• <= 256 KB – Amazon SNS
• > 256 KB – Amazon SNS + Amazon S3 presigned URL
Client
1
3
2
Lambda
API Gateway
Amazon SQS

Async: Webhooks
Pros
• No polling; less resource-intensive for client and
server
• Amazon SNS handles delivery retries
Cons
• Client needs to host a web endpoint
• Need a documented retry policy
• For untrusted clients, you’re responsible for clients
demonstrating trust and retries
Amazon SNS
Amazon SQSClient
API Gateway
Lambda

WebSockets support in API Gateway
Real-time two-way
communications
Managed persistence
Event-based triggers
Mobile apps
Chat
Dashboards
Internet of
Things (IoT)
devices
API Gateway
WebSockets API
Stateful connection Stateless
connection
Lambda
functions
Public endpoints
on Amazon EC2
Amazon
Kinesis
Any other
AWS service
All publicly
accessible
endpoints

Async: WebSockets
Flow
1. Client submits request and receives SFn execution ARN,
task token for OpenConn worker, and WebSocket
endpoint
2. Client opens connection to WebSocket endpoint with SFn
ARN and task token; Lambda completes OpenConn task
3. When DoWork is done, SFn parallel state completes, and
we send callback
4. Client receives update over WebSockets
Throughput
• < 200 RPS (1K burst) – Sync-SFn integration
• > 200 RPS – Lambda and Amazon SQS in front of SFn
Response payload
• > 128K (32K frame) – Amazon S3 presigned URL
Client
API Gateway
(WebSockets)
Step Functions1
2
3
4
LambdaAPI Gateway (REST)
onConnect
Step Functions workflow

Async: WebSockets
Pros
• Direct client notification over open connection
• Richer eventing for client
• Web- and mobile-client friendly
Cons
• Requires familiarity with WebSocket protocol and related
libraries
• WebSockets not supported by all browsersClient Step Functions workflow
onConnect
http://bit.ly/aws-poll-to-push
API Gateway
(WebSockets)
Step FunctionsLambdaAPI Gateway (REST)

Lambda
Execution timeout: 15 minutes
Approaches
• Lambda Power Tuning1
• Step Functions
• Fargate/AWS Batch
https://github.com/alexcasalboni/aws-lambda-power-tuning

Decouple state from code using messaging
Orchestration
Sequencing
Parallel execution
State management
Step Functions
Eventing
Performance at scale
Fully managed
Enterprise-ready
Amazon SNS
Messaging
Durable and scalable
Fully managed
Comprehensive security
Amazon SQS

Using Lambda with Amazon SQS
1. Scheduled time
occurs
2. Lambda invoked
CloudWatch Events
(time-based)
Queue
3. Pull messages from
queue
CloudWatch Events
(time-based)
4. Pull message from
queue
3. Lambda invoked
n times
2. Execute step function every
minute
1. Scheduled time
occurs
Queue Queue
Lambda
Lambda

Amazon SQS as an event source (SQSaaES)
• Lambda polls the queue, and when it detects new
messages, it invokes your Lambda function by passing
the messages as a parameter
• Full control over the queue-processing settings, such
as visibility timeouts, delay queues, redrive policy,
etc.; supports batch processing
• Lambda gradually increases the queue polling rate as
long as queue polling results in fetching new
messages
• Supports identity- and resource-based policy
3. Lambda removes
message from
queue
2. Lambda polls queue and
invokes function
1. Message inserted into to a
queue
Queue
Lambda
Amazon SQS
Lambda function

Publish/subscribe
Subscriber A
Subscriber C
Subscriber B
• One logical publisher for a given domain event – fully enforces a consistent boundary
• Each subscriber can react to domain
events in their own bounded context
• Amazon SNS supports multiple
subscription channels – Lambda,
Amazon SQS, HTTP/S, email, SMS,
mobile push Topic
Publisher

Service A
Service D
Service C
Subscriber B
Mapping events to topics
Grey
topic
Blue
topic
Blue
Subscribers
Grey
Each message type is mapped to logical destination
Blue
Blue
Grey
Grey
Lambda function
Publisher

Amazon SNS message filters
• Publishers do not need to route message
• Subscribers do not need to filter for message of interest
• Lowers cost
Color
topicPublisher
Subscribers
BlueGrey
Attr=
Grey
Grey
Attr=
Blue
Blue
Filter policy
Attr=Grey
Filter policy
Attr=Blue
Service A
Service B
Service C

Coordination by function chaining
Lambda function Lambda function Lambda function
Lambda function Lambda function
Lambda function

Step Functions
• Coordinate the components of distributed
applications and microservices using visual
workflows
• Keeps the orchestration out of the code!
• Ideal for long-running processes and implementing
failure management patterns and distributed
transaction processing
• Automatically triggers and tracks each step, and
retries when there are errors; support execution
logging
• Logs the state of each step, so when things do go
wrong, you can diagnose and debug problems
quickly
“Serverless” workflow management with zero
administration
"ProcessOrder": {
"Comment": ”t1”
"Type": "Task",
"Resource": "${NewOrderFunction.Arn}",
"TimeoutSeconds": 10,
"Catch": [{
"ErrorEquals": ["ErrProcessOrder"],
"ResultPath": "$.error",
"Next": "UpdateOrderStatus"
}],
"Next": "ProcessPayment"
},

Step Functions
A
B C
A
?
CBA
BA
“I want to retry failed tasks”“I want to sequence tasks” “I want try/catch/finally”
“I want to select tasks based on data” “I want to run tasks in parallel”

Richer workflows
Simplify building workloads such as order processing,
report generation, and data analysis
Write and maintain less code; add services in minutes
More service integrations
Step Functions
Amazon SNS Amazon SQS Amazon
SageMaker
AWS Glue AWS Batch Amazon ECS Fargate

Simpler integration, less code
With serverless
polling
With new
service integration
Start
End
Lambda
functions
Start
End
No
Lambda
functions

Share reusable serverless applications using the
AWS Serverless Application Repository

Nested apps using AWS Serverless Application Repository
Compose application architectures from
reusable building blocks
• Serverless architectures deployed as a set of nested
applications
• Foster best organizational practices and reduce
duplication of effort
• Share components, modules, and full applications
privately with teams or publicly with others to improve
agility

Common patterns emerge
Storage and backup (nested serverless application)
Fork-pattern-
backup-queue
Fork-pattern-backup-
function
Fork-pattern-
backup-bucket
Search and analytics (nested serverless application)
Fork-pattern-
index-queue
Fork-pattern-
index-function
Fork-pattern-index-
table
Polls Saves
Polls Saves
Serverless application
Amazon
SNS

Messaging and integration patterns
Event store (nested app)
Polls Saves
Publisher Amazon SNS
Publishes
Subscriber B
Amazon SQS WorkerClient
DLQ
Saves
Decoupled messaging Publish/subscribe
Service A (bounded context)
Publishes
Queue
DynamoDB

Awareness of messaging-payload size limits
Lambda
Sync: 6 MB
Async: 256 KB
Amazon
SQS
256 KB
Amazon
SNS
256 KB
(SMS) 1,600 b
Step Functions
32 KB
API Gateway
HTTP: 10 MB
WebSockets: 128 KB (32-MB frames)

Conclusion

Benefits of event-driven serverless architectures
• Asynchronous communication decouples components to accommodate long-running
processing and avoid timeouts
• Queues buffer messages to decouple producers and consumers to prevent downstream
resources from being overwhelmed
• Topics scale many-to-many communication using pub/sub messaging
• No infrastructure to provision, manage, secure, or scale
• Cost is driven by message volume and payload size – No messages == No cost

Scalable serverless architectures using event-driven design - MAD308 - New York AWS Summit

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Scalable serverless architectures using event-driven design - MAD308 - New York AWS Summit

Similar to Scalable serverless architectures using event-driven design - MAD308 - New York AWS Summit (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Scalable serverless architectures using event-driven design - MAD308 - New York AWS Summit