Building Massively Parallel Event-Driven Architectures (SRV373-R1) - AWS re:Invent 2018

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building Massively Parallel Event-
Driven Architectures
Amit Kulkarni
Principal Product Manager
AWS Serverless
S R V 3 7 3 - R
Mick Thompson
Data Engineer
Mapbox

Agenda
Modern Serverless Applications
Event Delivery Concepts
Event Generation, Routing, and Processing
Real World High Scale Serverless Application
Wrap

THE SERVERLESS OPERATIONAL MODEL
No provisioning,
no management
Pay for value
Automatic
scaling
Highly available
and secure

processes 4,000 requests
per second
ingests, analyzes and
stores 17+ petabytes of
data per season
processes half a trillion
validations of stock
trades daily
executes 16 million
requests a month
processes tens of
billions of data
points monthly
SERVERLESS AT SCALE IS THE NEW NORM
API traffic to register and license
more than 47 million driver
records in Great Britain.

High scale and resiliency at very low cost
Financial Engines AWS Lambda to power the core of its
computational “engines” which automate portfolio management
combined with access to human advisors.
Benefits of AWS Serverless
• Handles 200–300 million server requests per
month at rates of up to 60,000 per minute
• Reduced infrastructure costs by more than 90
percent
• Zero server downtime
Using AWS Lambda will
enable us to scale to serve
millions more who need
financial help.
Paul Gibson
Principal System Architect

SERVICES (ANYTHING)
Changes in
data state
Requests to
endpoints
Changes in
resource state
EVENT SOURCE FUNCTION
Node.js
Python
Java
C#
Go
PowerShell
Event-Driven Architecture

To grow really big in the future, you
need to think big now!

Core Concepts
Event
Generation
Event
Routing
Event
Processing
Output
Getting the event
details to processor
Emitting information
about the event
Take action on the
event

Event Delivery – AWS Services
Event
Generation
Event
Routing
Event
Processing
Output

Key Considerations
Event
Generation
Event
Routing
Event
Processing
Output
Payload
Persistence
Duplicates
Ordering
Retry semantics
Batching
Throughput
Concurrency
Velocity
Duration
On success / failure
Transport vs. Transform
Cost $$$

How do I choose?
If you know your enemies and know yourself, you will not be
imperiled in a hundred battles. - Sun Tzu, The Art of War

Event Payload
potential_customer = {
"name":"John",
"age":30,
"cars": {
"car1":"Ford",
"car2":"BMW",
"car3":"Fiat"
}
}
Payload
Persistence

Event Persistence
Availability
Events accessible even if source is down
Retention
Events accessible till processed
Cost
Storage, data transfer, and access
Complexity
Dependency, operational, scale
Payload
Persistence

No routing / default routing
Lambda
Function
Synchronous
Request-Response
Immediate access to results
You manage:
- Retries
- Ordering
- Error Handling
- Blocking and Threads
- Timeouts
Asynchronous
Fire (and forget)
Default behavior
- Retries
- Ordering
- DLQ errors
- Throughput
You manage:
- Error Response
- Duplicates Lambda
Function
S3 bucket
Duplicates
Ordering
Retry semantics
Batching
Throughput

Async Defaults
Retry Behavior
Automatic retry twice
System defined delay
Discard if all retries fail. Unless ….
Dead Letter Queues (DLQ)
Send error events to an SQS queue or SNS topic
Turn this on!
Amazon SQS: Monitor via queue length metric or alarm
Amazon SNS: Send to persistent storage or reliable processor (e.g. error processing Lambda function)
Duplicates
At least once semantics. Be prepared for dupes.
Duplicates
Ordering
Retry semantics
Batching
Throughput

Amazon SQS
Automatically polled. Ordering not guaranteed.
At least once delivery
Set the right batch size
On success: All messages in batch deleted
On failure: All messages in batch returned to queue
Failed messages retried in subsequent batches
Set Dead-Letter Queue (redrive policy on SQS)
Partially succeeded batches: Delete successful
messages in your function
Producers: Horizontally scaled - nearly unlimited
TPS
Polling rate: f(Batch size, duration, concurrency)
Lambda
Function
SQS Queue
Duplicates
Ordering
Retry semantics
Batching
Throughput

Amazon SNS
Lambda
Function
Simple managed pub/sub
Fan out to multiple subscribers per topic - 10M+
Publish at tens of thousands of TPS
Ordering not guaranteed.
At least once delivery
Highly reliable delivery to Lambda
Amazon SNS will make 50 attempts over 13 hours before giving up
Retry: Default async behavior
One function invoke per message
SNS  SQS  Lambda if you want batches
SNS Topic
Duplicates
Ordering
Retry semantics
Batching
Throughput

Concurrency
TPS ≠ Concurrency
Concurrency = TPS x Duration
Estimating - not straightforward
TPS, duration and distributions
Multiplying large numbers  chronic
overestimation
But then…. Don’t underestimate
Start: Peak TPS x Average Duration
Then load test to refine
Concurrency and Duration – CloudWatch
metrics
Account Limit vs Per-Function
Estimation by Paul Downey. Creative Commons License 2.0
Concurrency
Velocity
Duration
Transport vs Transform

Per Function Concurrency
Limit ≠ Reservation
The difference is subtle but important
Maximum concurrency limit for that function
Protect scale-limited targets
Kill switch  Throttle down to 0
Reduces effective limit available for other functions
Prevent noisy neighbors in account
DO: Set on SLA-bound functions in your account
Predictable concurrency for functions on which configured
DO: Evaluate higher account limit if you need headroom
Concurrency
Velocity
Duration

Velocity
Know your application targets
Throughput goals
Event to processed latency
Scale behavior
Token bucket
Initial concurrency increase
Per minute increase
Slower Ramp – Application impact
Retry on throttles
Implement with back off
Concurrency
Velocity
Duration

Duration
Use code to transform not
transport
Batch size matters
15 minute functions
Optimum memory size
Controls CPU as well
>1.8 GB multi-core
Time = Money
Don’t spend $$$ just waiting
No orchestration in code
AWS Step Functions
Concurrency
Velocity
Duration

Maps, Search, Directions
420 million+ active users
Billions of requests per day

Mapbox <3 Lambda
Early adopters of Lambda
• Stream processing
• Amazon API Gateway
• Amazon S3 notification events
• AWS CloudFormation custom resources
• Scheduled events

Lambda usage

Thinking in Lambda
Start with Lambda
Take on more operational work only as needed
Trade offs in overhead

Logging processing
To have insight into the edge of our
API, we trigger ETL of Amazon
CloudFront logs as soon as they
arrive on Amazon S3.
These are used in tracking usage,
monitoring issues, planning &
analysis, and debugging.

Amazon Athena
We need to be able to run
interactive queries over large
volume of logs.
Athena is a hosted PrestoDB
service, which allows you to write
SQL to query files on Amazon S3.
The speed and price of queries is
greatly impacted by how the data
structured and format its stored
in.

CloudFront logs
Billions of request per day
20 TB+ of compressed request logs per month
Delivered to Amazon S3 in 1 path. gzip’d TSVs
Log volume varies a lot throughout the day with traffic. Hourly spikes of 3x.

Log Enrichment
Parse complex requests
Normalize values
Lookups (geoip, useragent,
etc)

Putting it together

Deployment
Bundled by Lambda + AWS CodeBuild
AWS CloudFormation Lambda::Function
Amazon S3 notification event trigger

Lambda code
99% business logic
Amazon S3 event on new
objects
Load file, parse, write to
Amazon S3

When stuff goes wrong
Retries
DLQ
Monitoring / Alarms

Cost
Charged by 100ms * amount of memory.
Make function faster or use less memory.
Amount of memory also controls CPU.

Scaling & Concurrency
Scaling done for you. Thanks AWS.
Concurrency is shared account wide.
Use concurrency reservations

Caching
Lambda reuse the same node
process
(although no guarantees)

Amazon SQS
Concurrency, deal with event spikes
Retries / redrive policy (DLQ)
Control over batching

Wrap up
Lambda scaling to the exact workload
Your code is focused on your business logic
Monitor and recover from failures
Control cost with memory and time
Easy deployment with AWS CodeBuild + AWS CloudFormation

Thanks!

In closing
Rich Serverless Service Portfolio
A world full of choices
Considerations for Event Generation,
Routing, and Processing
Choose the right tool for the job
Serverless pricing makes it easy
Iterate Fast: Try – Tweak – Deploy
AWS
SAM
Compliance
services

Breakout repeats
Wednesday, November 26
Building Massively Parallel Event-Driven Architectures
6:15 pm – 7:15 pm | Venetian, Level 3, Murano 3205

Related breakouts
Thursday, November 29
Leadership Session: Using DevOps, Microservices, and Serverless
to Accelerate Innovation (SRV325)
12:15 – 1:15 PM | Venetian Theatre (Level 2)

Thank you!
Amit Kulkarni
amikul@amazon.com
@kulkarni_amit
Mick Thompson
mick@mapbox.com

Building Massively Parallel Event-Driven Architectures (SRV373-R1) - AWS re:Invent 2018

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Building Massively Parallel Event-Driven Architectures (SRV373-R1) - AWS re:Invent 2018

Similar to Building Massively Parallel Event-Driven Architectures (SRV373-R1) - AWS re:Invent 2018 (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Building Massively Parallel Event-Driven Architectures (SRV373-R1) - AWS re:Invent 2018