Getting Started with Serverless Architectures

P U B L I C S E C T O R
S U M M I T
S U M M I T
WASHINGTON, D C

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
S U M M I T
Getting Started With Serverless
Architectures
Ray Chang
Solutions Architect
Amazon Web Services
S e s s i o n 2 9 5 4 2 1
Sandy Hider
Lead Developer, bossDB
Johns Hopkins University – Applied
Physics Laboratory
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

S U M M I T
S U M M I T
Agenda
Fundamentals of serverless computing on AWS
Real world serverless computing application: bossDB

S U M M I T
S U M M I T

S U M M I T
S U M M I T
What is serverless?
No infrastructure provisioning,
no management
Automatic scaling
Pay for value Highly available and secure

S U M M I T
S U M M I T
Managed Serverless
Microsoft SQL
Server
“On Amazon Elastic
Compute Cloud
(Amazon EC2)”
Amazon EC2
AWS LambdaAmazon Elastic Container
Service for Kubernetes
(EKS)
AWS FargateAmazon Elastic Container
Service (ECS)
Amazon EMR Amazon Elasticsearch
Service
Amazon Redshift
Amazon Kinesis
Amazon Simple
Queue Service
(SQS)
AWS
Step Functions
Amazon
DynamoDB
Amazon ElastiCache
Amazon Relational
Database Service (RDS)
AWS IoT CoreAmazon API
Gateway
Amazon Cognito
Amazon
Simple
Storage
Service (S3)
Spectrum of AWS Offerings

S U M M I T
S U M M I T
Comparison of operational responsibility
AWS Lambda
Serverless functions
AWS Fargate
Serverless containers
Serverless
Infrastructure
AWS manages Customer manages
• Data source integrations
• Physical hardware, software, networking,
and facilities
• Provisioning
• Application code
• Container orchestration, provisioning
• Cluster scaling
• Physical hardware, host OS/kernel,
networking, and facilities
• Security config and updates, network config,
management tasks
• Container orchestration control plane
• Physical hardware, software,
• Work clusters
firewall, management tasks
• Physical hardware, software,
• Scaling
management tasks
• Provisioning, managing, scaling, and
patching of servers
Amazon ECS/EKS
Container management as a service
Amazon EC2
Infrastructure as a service

S U M M I T
S U M M I T
Serverless event-driven
code execution
Short-lived
All language runtimes
Data source integrations

S U M M I T
S U M M I T
Making development easier with AWS Lambda
Trillions of executions every month for hundreds of thousands of active customers
Accessible for
all developers
Enable new
application patterns
Greater
productivity
Support for all runtimes
with Lambda Layers and Runtime API
ISO, PCI, HIPAA, SOC, GDPR,
and FedRamp compliances
15 minute functions
Amazon SQS for Lambda
Automatic Load Balancing for Lambda
Support for Kinesis Data Streams Enhanced
Fan-Out and HTTP/2
Toolkits for popular IDEs:
VSCode, IntelliJ, and PyCharm
Simplified deployment
with nested apps

S U M M I T
S U M M I T
DATA STORES ENDPOINTS
DEVELOPMENT AND MANAGEMENT TOOLS EVENT/MESSAGE SERVICES
Event sources that trigger AWS Lambda
…and more!
Amazon S3 Amazon
DynamoDB
Amazon Kinesis Amazon Cognito AWS Step Functions AWS IoT CoreAmazon API
Gateway
Alexa
Amazon SQS Amazon
Simple
Notification
Service
(SNS)
Amazon Simple
Email Service
(SES)
AWS
CloudFormation
AWS
CloudTrail
AWS
CodePipeline
Amazon
CloudWatch

S U M M I T
S U M M I T
'use strict'
exports.handler = function (event, context, callback) {
console.log(“Event:”, JSON.stringify(event))
callback(null, ”Hello World!")
}
{
“Records”:[
“s3”: {
“object”:” { … }
}
]
}
JSON Context
Basic Function Code

S U M M I T
S U M M I T
Author and debug Lambda applications on AWS
using your favorite IDEs
AWS
Cloud9
Python, Node
AWS Toolkit
for PyCharm
Python
AWS Toolkit
for IntelliJ
Developer
Preview
Java, Python
AWS Toolkit
for Visual Studio
Code
Developer
Preview
.NET, Node
AWS Toolkit
for Eclipse
Java

S U M M I T
S U M M I T
Lambda Layers
Lets functions easily share code: Upload layer
once, reference within any function
Promote separation of responsibilities, lets
developers iterate faster on writing business logic
Built in support for secure sharing by ecosystem

S U M M I T
S U M M I T
Create a unified
API front end for
multiple
microservices
Authenticate and
authorize
requests to a
backend
DDoS protection
and throttling
for your backend
Throttle, meter,
and monetize API
usage by third-
party developers
Amazon API Gateway

S U M M I T
S U M M I T
Internet
Mobile Apps
Websites
Services
All publicly
accessible endpoints
Amazon
CloudFront
Amazon
CloudWatch
Amazon API
Gateway
AWS Step Functions
Amazon EC2
AWS Lambda
Other
AWS
services
Amazon API Gateway

S U M M I T
S U M M I T
“Serverless” workflow management
with zero administration:
• Coordinates distributed
applications using visual workflows
• Automatically triggers, tracks, and
logs each step
AWS Step Functions

S U M M I T
S U M M I T
• User pools for secure, managed
user directories
• Identity pools for federation and
role-based access control
Amazon Cognito
• Fine-grained access control to
AWS resources
AWS Identity and Access
Management (IAM)
Security and Identity

S U M M I T
S U M M I T
AWS X-RayAmazon CloudWatch
Logging and Monitoring

S U M M I T
S U M M I T
bossDB
• Neuroscience data storage service
• Resilient, multi-tier cloud data storage
& data caching
• Scalable, highly available RESTful
interfaces; load balancing
• User authentication / authorization
through SSO
• API with numerous core services,
supported clients, integrated tools
• Data Annotation (e.g., supervoxel objects)
• Data Visualization
• Data Access Services (e.g., images, objects,
volumetric, metadata)
• Data Ingest (e.g., image stack, volumetric)

S U M M I T
S U M M I T
bossDB Sponsors
IARPA MICrONS
Machine Intelligence from
Cortical Networks
• Seeks to revolutionize
machine learning
• Reverse-engineering the
algorithms of the brain
NIH bossDB
Neuroscience Community
Data Ecosystem
• A cloud-based data
ecosystem
• Electron microscopy,
X-ray microtomography

S U M M I T
S U M M I T
Johns Hopkins University / Applied Physics Laboratory
• Division of Johns Hopkins University
• University Affiliated Research Center
• Founded in 1942
• Laurel, MD Campus
• > 6,000 staff
• Government sponsors:

S U M M I T
S U M M I T
bossDB - Blocks
• Block and Object Storage Service (“boss”)
• “Block” is multi-dimensional data (cuboid) created from several image tiles

S U M M I T
S U M M I T
bossDB - Objects
• bossDB stores annotation data co-registered to image data
• An annotation is a unique 64-bit identifier applied to a set of voxels, representing
its spatial distribution or “Object”
ID: 1267
ID: 345345
ID: 534534799

S U M M I T
S U M M I T
How we use serverless technologies
• Tasks that only run periodically
• Error Reporting
• Managing Servers
Lambda
DynamoDB
Amazon SQS Step Function
Amazon S3
Amazon SNS

S U M M I T
S U M M I T
Specific examples
• High speed ingest
• Downsample
• Management of HashiCorp’s Vault and Consul Servers

S U M M I T
S U M M I T
Generic high speed ingest
• 16 Images (tiles) are converted to 3D Blocks of data called cuboids
DynamoDB
Cuboid Index
S3 Cuboid

S U M M I T
S U M M I T
High speed ingest by the numbers
• Ingest can be started at any hour by use
• Scale up between 2GB to 2PB of data for an Ingest
• Clients bandwidth should be limiting factor

S U M M I T
S U M M I T
1st attempt at high speed ingest
Client Cuboid
Bucket
Cuboid
Index
• Goal - populate cuboids into S3
and DynamoDB

S U M M I T
S U M M I T
Client
Tile
Index
Tile
Bucket
Cuboid
Bucket
Cuboid
Index
• Use temporary storage / index
• Need 16 consecutive tile images

S U M M I T
S U M M I T
Client Images
to Upload
Tile
Index
Tile
Bucket
Step Function
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• At start of ingest job, create SQS
Queues and Step Function

S U M M I T
S U M M I T
Client
Populate
Images
to Upload
Tile
Index
Tile
Bucket
Step Function
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• Step Function launches Lambdas
• Populates SQS with all tiles

S U M M I T
S U M M I T
Client
Populate
Images
to Upload
Tile
Index
Tile
Bucket
Step Function
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• Client gets message
• Uploads requested tile

S U M M I T
S U M M I T
Client
Populate
Images
to Upload
Tile
Index
Tile
Bucket
Step Function
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• Client is multithreaded
• Multiple clients can join

S U M M I T
S U M M I T
Client
Populate
Images
to Upload
Tile
Index
Tile
Bucket
Step Function
Object
Triggered
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• New S3 Object Triggers Lambda
• If 16 tiles – Create Cuboid Ready

S U M M I T
S U M M I T
Client
Populate
Images
to Upload
Tile
Index
Tile
Bucket
Step Function
Object
Triggered
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• Delete tile message from SQS

S U M M I T
S U M M I T
Client
Populate
Images
to Upload
Tile
Index
Tile
Bucket
Step Function
Object
Triggered
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
Create
Cuboid
• SQS triggers Lambda to create
cuboid

S U M M I T
S U M M I T
Client
Populate
Images
to Upload
Tile
Index
Tile
Bucket
Step Function
Object
Triggered
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
Create
Cuboid
• Lambda is given list of 16 tiles
• Creates cuboid

S U M M I T
S U M M I T
Problem with triggered lambda deleting messages in SQS
• Concurrent lambda limit can
be reached
• Lambdas throttle and fail to
delete image message
Images
to upload
Tile
Bucket
Object
Triggered
Client

S U M M I T
S U M M I T
Problem with triggered lambda deleting messages in SQS
• Concurrent lambda limit can
be reached
• Lambdas throttle and fail to
delete image message
• Message becomes visible
• Client repeats work!
Images
to upload
Tile
Bucket
Object
Triggered
Client

S U M M I T
S U M M I T
Additional problem
• Created DynamoDB Global Secondary Index of Ingest Job ID
• Caused hot-spot in single partition
• Solution append random number 001-100 to the ID
Client Images
to upload
Tile
Index
Tile
Bucket
Object
Triggered

S U M M I T
S U M M I T
Final solution to high speed ingest
Client
Tile
Index
Tile
Bucket
Cuboid
Bucket
Cuboid
Index

S U M M I T
S U M M I T
Client Images
to Upload
Tile
Index
Tile
Bucket
Tile
Queue
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• New SQS Tile Queue

S U M M I T
S U M M I T
Client Images
to Upload
Tile
Index
Tile
Bucket
Tile
Queue
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• Client gets message
• Uploads requested tile

S U M M I T
S U M M I T
Client Images
to Upload
Tile
Index
Tile
Bucket
Tile
Queue
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• Client creates corresponding
message in Tile Queue

S U M M I T
S U M M I T
Client Images
to Upload
Tile
Index
Tile
Bucket
Tile
Queue
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• Client deletes message

S U M M I T
S U M M I T
Client Message
Triggered
Images
to Upload
Tile
Index
Tile
Bucket
Tile
Queue
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
• Message triggers Lambda

S U M M I T
S U M M I T
Client Message
Triggered
Images
to Upload
Tile
Index
Tile
Bucket
Tile
Queue
Cuboid
Ready
Cuboid
Bucket
Cuboid
Index
Create
Cuboid
• No other changes
• Works when lambdas throttle

S U M M I T
S U M M I T
Ingest Speeds
Ingest speeds
over
200 GB / min

S U M M I T
S U M M I T
Heaviside
Python Library and DSL for working with AWS Step Functions
• The Step Function state machine language, while flexible, is hard to
write and maintain
• Heaviside is a Python package that provides several components to
make Step Functions easier to use
• DSL and Compiler – Greatly simplifies writing and maintaining Step Functions
in AWS
• Framework for running Activities
https://github.com/jhuapl-boss/heaviside

S U M M I T
S U M M I T
Heaviside Example
"""Delete Cuboid
Removes all of the different data related to a given
cuboid,
removes the actual cuboid data, and then cleans up
the final
bookkeeping for the cuboid
"""
parallel:
Activity('delete_metadata')
"""delete_metadata
deletes metadata"""
retry [] 60 4 2.0
parallel:
Activity('delete_id_count')
"""delete_id_count
deletes from dynamodb table idcount"""
retry [] 60 4 2.0
parallel:
Activity('delete_id_index')
"""delete_id_index
deletes from dyanmodb table idindex"""
retry [] 60 4 2.0
Compiles
48 lines
154 lines
Step Function JSON

S U M M I T
S U M M I T
Downsampling

S U M M I T
S U M M I T
Downsampled tiles
~2 PB
~512 TB
~128 TB
~32 TB
~8 TB
~2 TB
~0.5 TB
125 GB
Can have 6 – 10 resolution levels

S U M M I T
S U M M I T
Downsampling by the numbers
• Only downsample 1% of the time
• For a 2PB image dataset downsample could require
• 37 million S3 and DynamoDB writes
• 148 million S3 and DynamoDB reads
• Users can kick these off at any hour of the day
• Serverless Benefit
• Don’t need to keep servers up when workflow is not running
• Can massively scale processing for short periods of time without administrator in the loop
• Don’t need to worry about high availability

S U M M I T
S U M M I T
Our solution to downsampling
Cuboid
Bucket
Cuboid
Index
AWS Step Functions workflow
Populate
Activity
Downsample
Sets
Downsample
Lambda DLQ
• Downsample Sets
4 -> 1 Populated

S U M M I T
S U M M I T
Downsample
Cuboid
Bucket
Cuboid
Index
Populate
Activity
• Step Function
launches Lambdas in
parallel
Downsample
Sets
Downsample
Lambda DLQ
Lambda checker
Activity

S U M M I T
S U M M I T
Downsample
Cuboid
Bucket
Cuboid
Index
Populate
Activity
• Lambda gets message
• Creates new cuboid,
merging 4 together
Downsample
Sets
Downsample
Lambda DLQ
Lambda checker
Activity

S U M M I T
S U M M I T
Downsample
Cuboid
Bucket
Cuboid
Index
Populate
Activity
• Uses Lambda DLQ to
find errors
Downsample
Sets
Downsample
Lambda DLQ
Lambda checker
Activity

S U M M I T
S U M M I T
Downsample
Cuboid
Bucket
Cuboid
Index
Populate
Activity
• Repeat for every
downsample
resolution level
Downsample
Sets
Downsample
Lambda DLQ
Lambda checker
Activity

S U M M I T
S U M M I T
If I were to redesign downsample now
Downsample
Cuboid
Bucket
Cuboid
Index
Populate
Activity
Lambda checker
Activity
• Use Event Source
• Can limit concurrent
Lambdas
Event SourceDownsample
Sets
Downsample
Lambda DLQ

S U M M I T
S U M M I T
Management of HashiCorp’s Vault and Consul servers
• Use Vault and Consul to manage system
secrets
• Consul distributed database need 3 of
the 5 servers up syncing data to
function
• In memory only database
• Cannot allow servers to terminate as could trash
in memory database
• New Vault Servers need to be manual
unsealed

S U M M I T
S U M M I T
Solution to manage Vault and Consul
VPC

S U M M I T
S U M M I T
Serverless architecture takeaways
• Easy to connect and coordinate distributed components
and microservices
• Don’t waste resources for task that only run periodically
• High availability without extra resources
• Massively scale on demand
• Diagnose and debug problems faster

S U M M I T
S U M M I T
Acknowledgements
• Caitlyn Bishop
• Joe Downs
• Conrad Fernandes
• Tim Gion
• Dean Kleissas
• Jordan Matelsky
• Derek Pryor
• Luis Rodiguez
• Will Gray Roncal
• Brock Wester
• Miller Wilt

S U M M I T
S U M M I T
Project Links
bossDB: github.com/jhuapl-boss
bossDB.org
Heaviside: github.com/jhuapl-boss/Heaviside

S U M M I T
S U M M I T
Thank you!
S U M M I T
Thank you!
S U M M I T
Sandy Hider
Sandy.hider@jhuapl.edu
Ray Chang
rchang@amazon.com

S U M M I T
S U M M I T
S U M M I T

Getting Started with Serverless Architectures

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Getting Started with Serverless Architectures

Similar to Getting Started with Serverless Architectures (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Getting Started with Serverless Architectures