Beware the potholes on the road to serverless

MIND
THE
POTHOLES
MIND
THE
POTHOLES

@theburningmonk theburningmonk.com
Jan 24th, 2007

we’re getting a new server
in 3 months time

yay!
about time!
hooray!!
ﬁnally!

but we have to decide
what dependencies to
install on it now..

on premise VMs in the cloud

abstraction level

abstraction levelproductivity

less is more

right?

“why are we failing at this?”

hidden dangers

monolith microservices serverless

observability
distributed
systems
bounded
context

observability
distributed
systems
bounded
context
event driven

monolith serverless
missing learnings
from microservices

monolith serverless
missing learnings
from microservices
poor decisions

Yan Cui
http://theburningmonk.com
@theburningmonk
AWS user for 10 years

Yan Cui
@theburningmonk
Developer Advocate @

enter the code “belfast60”
for 60 days free access to Lumigo

Yan Cui
@theburningmonk
Independent Consultant
advisetraining delivery

https://theburningmonk.com/courses

#1
not letting go of legacy
thinking

“we’re doing serverless,
but why aren’t thing
going faster?”

Socio Technical

there are no silver bullets

centralised team
Team A Team B Team C Team D …

“but the developers don’t understand AWS and how
our infrastructure is set up”

“but the developers don’t understand AWS and how
our infrastructure is set up”
let’s solve this
problem instead!

what got you here won’t get you there

if (path == “/user” && method == “GET”) {
return getUser(…);
} else if (path == “/user” && method == “DELETE”) {
return deleteUser(…);
} else if (path == “/user” && method == “POST”) {
return createUser(…);
} else if ….
Monolithic Functions

GET /user
POST /user
DELETE /user
Single-Purposed Functions

author: yan.cui
feature: user-api
user-api-dev
Monolithic Single-Purposed
author: yan.cui
feature: user-api
user-api-dev-get-user
author: yan.cui
feature: user-api
user-api-dev-create-user
author: yan.cui
feature: user-api
user-api-dev-delete-user

author: yan.cui
feature: user-api
user-api-dev
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
ﬁnd related
functions by preﬁx

author: yan.cui
feature: user-api
user-api-dev
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
discoverability
(without having to dig into the code)

author: yan.cui
feature: user-api
user-api-dev
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
what does it do?

author: yan.cui
feature: user-api
user-api-dev
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
dynamodb:GetItem
dynamodb:PutItem
dynamodb:DeleteItem

author: yan.cui
feature: user-api
user-api-dev
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
dynamodb:GetItem
dynamodb:PutItem
dynamodb:DeleteItem
no least privilege…

author: yan.cui
feature: user-api
user-api-dev
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
require(x)
require(y)
require(z)

more dependecies equals
slower cold start

author: yan.cui
feature: user-api
user-api-dev
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
author: yan.cui
feature: user-api
require(x)
require(y)
require(z)
worse cold start
performance

keep functions simple, and single-purposed

#2
one account that rules
them all

mind the shared limits

no. of DynamoDB tables
no. of API Gateway regional APIs
no. of API Gateway edge-optimized APIs
no. of Kinesis shards
no. of IAM roles
no. of S3 buckets
no. of CloudFormation stacks
no. of SNS subscription ﬁlters
no. of SSM parameters
…
Resource Limits

DynamoDB read & write
API Gateway requests/second
Lambda concurrent executions
SSM parameter ops/second
…
Throughput Limits

compartmentalise security breaches

One account per Team per Environment

https://github.com/OlafConijn/AwsOrganizationFormation

https://github.com/OlafConijn/AwsOrganizationFormation
Accounts
Org Units
SCPs
Pwd Policies
Multi-Region
Pseudo-Funs
Init & Validate
CI/CD

https://einaregilsson.com/serverless-15-percent-slower-and-eight-times-more-expensive/

the platforms need to do better at educating users on
how to choose between different services

SNS vs SQS vs Kinesis vs MKS?
the platforms need to do better at educating users on
how to choose between different services

ordering
replay events
Kinesis SQS SNS
by shard
none (standard)
global (FIFO)
none
up to 7 days none none
mode
retry
batched batched (up to 10) singular
retried until
success
(customizable)
retry + DLQ retry + DLQ
concurrency 1 per shard auto-scaled fan-out!!!
subscribers many one-to-one many
EventBridge
many
none
none
singular
retry + DLQ
fan-out!!!

https://medium.com/theburningmonk-com/all-my-posts-on-serverless-aws-lambda-43c17a147f91

https://www.jeremydaly.com/newsletter/

#4
not using a deployment
toolkit

https://lumigo.io/blog/comparison-of-lambda-deployment-frameworks

don’t write your own deployment framework

github
repo
github
repo
github
repo
github
repo
github
repo
github
repo
github
repo
github
repo
github
repo

monorepo?

github
repo

one repo per service?

github
repo
github
repo
github
repo
github
repo
user-api
timeline-api
relationship-api
search-api

https://lumigo.io/blog/mono-repo-vs-one-per-service/

one CI/CD pipeline per service

functions are deployed together, as a stack

unencrypted secrets
in env vars
#6

secrets should NEVER be in plain text in env variables

SSM Parameter Store
Secret 1
Secret 2
IAM
Environment:
SECRET_1: …
SECRET_2: …
Environment:
SECRET_1: …
SECRET_2: …

SSM Parameter Store
Secret 1
Secret 2
IAM
Environment:
SECRET_1: …
SECRET_2: …
Environment:
SECRET_1: …
SECRET_2: …
yay!

SSM Parameter Store
Secret 1
Secret 2
IAM
fetch at cold start,
cache,
invalidate every x mins

https://github.com/middyjs/middy

SSM Parameter Store
Secret 1
Secret 2
IAM
switch to Higher
Throughput if you need
more than 40 ops/s

not following least
privilege principle
#7

too much/too little
concurrency
#8

“Lambda generates too much load for the downstream system”

one invocation
per message
SNS
Lambda

Downstream
System
SNS
Lambda

ordering
replay events
Kinesis SQS SNS
by shard
none (standard)
global (FIFO)
none
up to 7 days none none
mode
retry
batched batched (up to 10) singular
retried until
success
(customizable)
retry + DLQ retry + DLQ
concurrency 1 per shard auto-scaled fan-out!!!
subscribers many one-to-one many
EventBridge
many
none
none
singular
retry + DLQ
fan-out!!!

if you want…
maximum
throughput
SNS
precise control
over throughput
Kinesis

if you want…
maximum
throughput
SNS
precise control
over throughput
Kinesis
how quickly it scales out

if you want…
maximum
throughput
SNS
precise control
over throughput
Kinesis
how quickly it scales out
SQS DynamoDB
Streams

#1 not letting go of legacy thinking
#2 one account that rules them all
#3 do ﬁrst, research later
#4 not using a deployment framework
#5 one repo per function
#6 unencrypted secrets in env vars
#7 not following least privilege principle
#8 too much/too little concurrency

https://theburningmonk.com/hire-me
AdviseTraining Delivery
“Fundamentally, Yan has improved our team by increasing our
ability to derive value from AWS and Lambda in particular.”
Nick Blair
Tech Lead

https://theburningmonk.com/workshops
Amsterdam, March 19-20 Helsinki, May 4-5 Stockholm, May 14-15
Dublin, June 16-17 London, September 24-25 Berlin, October 8-9

Production-Ready Serverless

bit.ly/prod-ready-sls-dublin-2020
20% off with
sls-belfast-0124

@theburningmonk
theburningmonk.com
github.com/theburningmonk

Beware the potholes on the road to serverless

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Beware the potholes on the road to serverless

Similar to Beware the potholes on the road to serverless (20)

More from Yan Cui

More from Yan Cui (20)

Recently uploaded

Recently uploaded (20)

Beware the potholes on the road to serverless