© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chris Munns – Senior Developer Advocate – AWS
Serverless
Become a
Serverless Black
Belt

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About me:
Chris Munns - munns@amazon.com, @chrismunns
• Senior Developer Advocate - Serverless
• New Yorker
• Previously:
• AWS Business Development Manager – DevOps, July ’15 - Feb ‘17
• AWS Solutions Architect Nov, 2011- Dec 2014
• Formerly on operations teams @Etsy and @Meetup
• Little time at a hedge fund, Xerox and a few other startups
• Rochester Institute of Technology: Applied Networking and Systems
Administration ’05
• Internet infrastructure geek

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
https://secure.flickr.com/photos/mgifford/4525333972
Why are we
here today?

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
No servers to provision
or manage
Scales with usage
Never pay for idle Availability and fault
tolerance built in
Serverless means…

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SERVICES (ANYTHING)
Changes in
data state
Requests to
endpoints
Changes in
resource state
EVENT SOURCE FUNCTION
Node.js
Python
Java
C#
Go
Serverless applications

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Serverless is about maximizing elasticity,
cost savings, and agility.

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Multiple points to optimize
Amazon
API
Gateway
Amazon
Alexa
AWS
IoT Amazon
Kinesis
Amazon
SNS
Amazon
SES
AWS Step
Functions 2
Invocations
1
Functions
3
Interactions
Amazon
S3
Amazon
DynamoDB
Custom
endpoints
Amazon
Cloudwatch
Amazon
Elasticsearch
EC2
instance

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Optimization Katas
1. THE LEAN FUNCTION
2. EVENTFUL INVOCATIONS
3. COORDINATED CALLS
4. SERVICEFUL OPERATIONS

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Goal today
Repeatable regimen for building highly
resilient, high-performance serverless
applications.

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THE LEAN FUNCTION
K A T A # 1
Concise logic, efficient/single purpose code, ephemeral environment

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Anatomy of a function
Your
function
Language
runtime
Execution
Environment
Compute
substrate

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The function lifecycle
Bootstrap
the runtime
Start your
code
Full
cold start
Partial
cold start
Warm
start
Download
your code
Start new
container
AWS optimization Your optimization

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Same view in AWS X-Ray

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The function lifecycle
Bootstrap
the runtime
Start your
code
Full
cold start
Partial
cold start
Warm
start
Download
your code
Start new
container
AWS optimization Your optimization

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Concise function logic
• Separate Lambda handler (entry point) from core
logic
• Use functions to TRANSFORM, not
TRANSPORT
• Read only what you need. For example:
• Properly indexed databases
• Query filters in Aurora
• Use S3 select

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Small changes, big difference
200 seconds and 11.2 cents
# Download and process all keys
for key in src_keys:
response =
s3_client.get_object(Bucket=src_bucket,
Key=key)
contents = response['Body'].read()
for line in contents.split('n')[:-1]:
line_count +=1
try:
data = line.split(',')
srcIp = data[0][:8]
….
95 seconds and costs 2.8 cents
# Select IP Address and Keys
for key in src_keys:
response =
s3_client.select_object_content
(Bucket=src_bucket, Key=key,
expression =
SELECT SUBSTR(obj._1, 1, 8), obj._2
FROM s3object as obj)
contents = response['Body'].read()
for line in contents:
line_count +=1
try:
AfterBefore
(https://github.com/awslabs/lambda-refarch-mapreduce)

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Efficient function code
• Avoid “fat”/monolithic functions
• Control the dependencies in your
function's deployment package
• Optimize for your language
• Node – Browserfy, Minify

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Java – Scope your POM file
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bom</artifactId>
<version>2.10.10</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.10.5</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-dynamodb</artifactId>
<version>1.10.10</version>
</dependency>
<dependencies>
Maven Bill Of Materials
(BOM) module for AWS
SDK
Only use what you need from the aws-java-sdk!
Select service
dependencies only

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ephemeral function environment
• Lambda processes a single event
per-container
• No need for non-blocking execution
on the frontend
• REMEMBER – containers are reused
• Lazily load variables in the global
scope
• Don’t load it if you don’t need it – cold
starts are affected
import boto3
client = None
def my_handler(event, context):
global client
if not client:
client =
boto3.client("s3")
# process

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Smart resource allocation
Match resource allocation (up to 3 GB!) to logic
Stats for Lambda function that calculates 1000 times all prime numbers
<= 1000000
128 MB 11.722965sec $0.024628
256 MB 6.678945sec $0.028035
512 MB 3.194954sec $0.026830
1024 MB 1.465984sec $0.024638

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Impact of a memory change
50% increase
in memory
95th percentile
changes from
3s to 2.1s
https://blog.newrelic.com/2017/06/20/lambda-functions-xray-traces-custom-serverless-metrics/

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Don’t guestimate
alexcasalboni
aws-lambda-power-tuning

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Multithreading? Maybe!
• <1.8GB is still single core
• CPU bound workloads won’t see gains – processes share
same resources
• >1.8GB is multi core
• CPU bound workloads will gains, but need to multi thread
• I/O bound workloads WILL likely see gains
• e.g. parallel calculations to return

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
No orchestration in code
STARTJOB
JOB#XSTARTED
HTTPPOST
HTTPPOST
AREWETHEREYET?
NOPE!
WE’REDONE!
ZzZz
OR
time.sleep(10)

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
No orchestration in code

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THE LEAN FUNCTION
K A T A # 1
Concise logic, efficient/single purpose code, ephemeral environment

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EVENTFUL INVOCATIONS
K A T A # 2
Succinct payloads, resilient routing, concurrent execution

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Invocation paths
Amazon
API
Gateway
Amazon
Alexa
AWS
IoT Amazon
Kinesis
Amazon
SNS
Amazon
SES
AWS Step
Functions
Amazon
S3
Amazon
DynamoDB
Custom
endpoints
Amazon
Cloudwatch
Amazon
Elasticsearch
EC2
instance

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Gateways and routers
• Choose suitable entry point for client
applications
• Single, custom client? Use the AWS
SDK
• Not end user facing? use regional
endpoints on API Gateway
• Discard uninteresting events ASAP
• S3 – Event prefix
• SNS – Message filtering

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Succinct invocations
• Scrutinize the event
• Must have provenance i.e. “What happened for this notification
to occur?”
• Additional content – identifier or payload
• Remember payload constraints
• Async invocation is only 128K
• Avoid large responses like an image

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
http://theburningmonk.com/2017/09/using-protocol-buffers-with-api-gateway-and-aws-lambda/
The same
response in
Protocol Buffers
is nearly 40%
smaller
compared to
default JSON
Evaluate appropriate protocol
'use strict’;
const co = require('co');
const Promise = require('bluebird');
const protobuf = Promise.prmisifyAll(require("protobufjs"));
const lib = require('./lib');
const fs = require('fs’);
module.exports.handler = co.wrap(function* (event, context, callback)
{
console.log(JSON.stringify(event));
let players = lib.genPlayers();
let root = yield protobuf.loadAsync("functions/player.proto");
let Players = root.lookupType("protodemo.Players");
let message = Players.create(players);
let buffer = Players.encode(message).finish();
const response = {
statusCode: 200,
headers: { 'Content-Type': 'application/x-protobuf' },
body: buffer.toString('base64’),
isBase64Encoded: true
};
callback(null, response);
});

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VS.
Resilient: Use an event store

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Think concurrent, not TPS
Simple
No event store
Queue based
Stream based

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Concurrency vs. Latency
Streams
• Maximum theoretical throughput:
# shards * 2 MB / (s)
• Effective theoretical throughput:
( # shards * batch size (MB) ) /
( function duration (s) * retries until expiry)
• If put / ingestion rate is greater than
the theoretical throughput, consider
increasing number of shards while
optimizing function duration to
increase throughput
Everything else
• Maximum Processing rate :
Maximum concurrency / average
duration (events per second)
• Effective Processing rate :
Effective concurrency / average
duration (events per second)
• Use concurrency metric and duration
metric to estimate processing time

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Resilient: Retry policies
• Understand retry policies
• Sync never retried
• Async retried 2 times
• Streams retried all the time
• Leverage Dead Letter Queues
• SQS or SNS for replays
• REMEMBER: Retries count as invokes

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EVENTFUL INVOCATIONS
K A T A # 2
Succinct payloads, resilient routing, concurrent execution

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
COORDINATED CALLS
K A T A # 3
Decoupled via APIs, scale-matched downstream, secured

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ingestion service
Ingestion
API
ingest
&
sanitize()
Metadata service
CRUD
API
read & write
metadata()
Frontend service
Frontend
API
express()
Decoupled: APIs as contracts

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scale-matched: Concurrency controls
• Concurrency a shared pool by default
• Separate using per function concurrency settings
• Acts as reservation
• Also acts as max concurrency per function
• Especially critical for data sources like RDS
• “Kill switch” – set per function concurrency to zero

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Secured: Do I need a VPC?
Should my
Lambda
function be
in a VPC?
Does my function
need to access
any specific
resources in a
VPC?
Does it also need to
access resources or
services in the
public internet?
Don’t put the
function in a
VPC
Put the
function in a
private subnet
Put the
function in a
subnet with a
NAT’d route to
the internet
Yes Yes
No No

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Secured: Function lifestyle with VPC
Download
your code
Start new
container
Start your
code
Create
VPC ENI
Attach
VPC ENI
Full
cold start
Warm
start
Bootstrap
runtime
Partial
cold start
AWS optimization Your optimization

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Secured: VPC vs. Resilience
• ALWAYS configure a minimum of 2
Availability Zones
• Give your Lambda functions their own
subnets
• Give your Lambda subnets a large IP range
to handle potential scale
• If your functions need to talk to a resource on
the internet, you need a NAT!

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
COORDINATED CALLS
K A T A # 3
Decoupled via APIs, scale-matched downstream, secured

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SERVICEFUL OPERATIONS
K A T A # 4
Automated operations, Monitored applications, Innovation mindset
In the end, it’s about the people

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Automate development: Start with a framework
Chalice

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ClaudiaJS
Node.js framework for deploying projects
to AWS Lambda and Amazon API
Gateway
• Has sub projects for microservices,
chat bots and APIs
• Simplified deployment with a single
command
• Use standard NPM packages, no need
to learn swagger
• Manage multiple versions
https://claudiajs.com
https://github.com/claudiajs/claudia
app.js:
var ApiBuilder = require('claudia-api-
builder')
var api = new ApiBuilder();
module.exports = api;
api.get('/hello', function () {
return 'hello world';
});
$ claudia create --region us-east-1 --api-module app

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chalice
Python serverless “microframework” for
AWS Lambda and Amazon API Gateway
• A command line tool for creating,
deploying, and managing your app
• A familiar and easy to use API for
declaring views in python code
• Automatic Amazon IAM policy
generation
https://github.com/aws/chalice
https://chalice.readthedocs.io
app.py:
from chalice import Chalice
app = Chalice(app_name="helloworld")
@app.route("/")
def index():
return {"hello": "world"}
$chalice deploy
Chalice

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
from chalice import Chalice
from chalice import BadRequestError
app = Chalice(app_name='apiworld-hot')
FOOD_STOCK = {
'hamburger': 'yes’,
'hotdog': 'no'
}
@app.route('/')
def index():
return {'hello': 'world'}
@app.route('/list_foods')
def list_foods():
return FOOD_STOCK.keys()
@app.route('/check_stock/{food}')
def check_stock(food):
try:
return {'in_stock': FOOD_STOCK[food]}
except KeyError:
raise BadRequestError("Unknown food '%s', valid choices are: %s" % (food, ', '.join(FOOD_STOCK.keys())))
@app.route('/add_food/{food}', methods=['PUT'])
def add_food(food):
return {"value": food}
Chalice – a bit deeper
Chalice
application routes
error handling
http method support

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Meet
SAM!

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Serverless Application Model (SAM)
CloudFormation extension optimized for
serverless
New serverless resource types: functions, APIs,
and tables
Supports anything CloudFormation supports
Open specification (Apache 2.0)
https://github.com/awslabs/serverless-application-model

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Automated operations: Deployment
Source
Source
CodeCommit
MyApplication
Build
test-build-source
CodeBuild
Deploy Testing
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Run-stubs
AWS Lambda
Deploy Staging
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Run-API-test
Runscope
QA-Sign-off
Manual Approval
Review
Deploy Prod
create-changeset
AWS
CloudFormation
execute-changeset
AWS
CloudFormation
Post-Deploy-Slack
AWS Lambda
This CodePipeline pipeline:
• Five Stages
• Builds code artifact w/ CodeBuild
• Three deployed to “Environments”
• Uses SAM/CloudFormation to
deploy artifact and other AWS
resources
• Has Lambda custom actions for
running my own testing functions
• Integrates with a 3rd party
tool/service
• Has a manual approval before
deploying to production

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CloudWatch Metrics
• 7 Built in metrics for Lambda
• Can call “put-metric-data”
from your function code for
custom metrics
• 7 Built in metrics for API-
Gateway
Monitored: Metrics and logging are a universal right!
CloudWatch Logs
• Lambda Logging
• Custom logging from your
code with your language’s
equivalent of console.log()
• API Gateway Logging
• Custom formats
• Log Pivots
• Build metrics based on log
filters

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS X-Ray

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Application instrumentation (Node.js)

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Innovate: Incremental infrastructure
Serverless
Monolith
Serverless
Microservices

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Innovate: Get creative!

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SERVICEFUL OPERATIONS
K A T A # 4
Automated operations, Monitored applications, Innovation mindset
In the end, it’s about the people

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Optimization Katas
1. THE LEAN FUNCTION –
CONCISE. EFFICIENT. EPHEMERAL.
2. EVENTFUL INVOCATIONS –
SUCCINT. RESILIENT. CONCURRENT.
3. COORDINATED CALLS –
DECOUPLED. SCALE MATCHED. SECURED.
4. SERVICEFUL OPERATIONS –
AUTOMATE. MONITOR. INNOVATE.

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
aws.amazon.com/serverless

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chris Munns
munns@amazon.com
@chrismunnshttps://www.flickr.com/photos/theredproject/3302110152/

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
?
https://secure.flickr.com/photos/dullhunk/202872717/