The Lambda service from Amazon, and similar services from Microsoft and Google, signals the arrival of the Actor/CSP model in the cloud. These services are the beginning of an evolutionary path that will ultimately supplant today’s VM-based cloud usage model. In this talk Jack Mills will give a tutorial on the Lambda usage model, its pricing behavior, some details on the internal infrastructure Amazon uses to implement Lambda, and describe potential innovations for Lambda-like services to transform cloud computing in the future.

1. 1Jack D. Mills
Amazon Lambda
and
the Transition to Cloud2.0

2. 2Jack D. Mills
introduction
 “cloud1.0” was based on VM’s
» enabled “lift-and-shift” of existing binaries which in turn enabled
rapid market ramp
» transformed customer’s fixed cost capex to variable cost opex
 lambda is the first “born-in-the-cloud” software architecture
» will transform both customer experience and vendor
infrastructure
» has a long road of evolution ahead of it
 the cloud is as much an economic revolution as a
technological one

3. 3Jack D. Mills
what is a lambda?
 a function/method with a particular interface…
» foo( eventObject, contextObject )
 …invoked by a specified event:
» HTTP message arrival (API gateway)
» storage change (S3, dynamoDB)
» message/email arrival (SQS, SES, SNS)
» record arrival on a stream (kinesis)
» programmatic invocation (execute invoke())
» deployment/behavior change (cloudFormation, cloudWatch)

4. 4Jack D. Mills
configuration
 user does not deal with a virtual server or a full stack binary
» a lambda-based system is sometimes called “serverless”
 enter/upload code for the lambda function/method
» can use most any standard linux service (processes, threads,
etc.) and most any library from the language ecosystem
» must include all dependencies (a “deployment package”)
 specify trigger event, max memory, and max execution time
» memory ranges from 128 MB to 3 GB in 64 MB increments
» CPU capacity proportional to memory size (just like VM’s)

5. 5Jack D. Mills
execution context
 environment variables
» key-value pairs defined at configuration time or by CloudFormation
 /tmp directory in local filesystem
 lambda’s are stateless
» memory-resident state and /tmp are not saved on termination
» state that crosses a l lifetime must be saved in persistent storage
 two invocation types:
» async: value not returned to invoking service
» sync: value returned to invoking service

6. 6Jack D. Mills
constraints
 languages: javascript (node), java, python, go, C#
 runs only on amazon linux (not sure about C#)
 max execution time: 300 seconds
» will probably increase over time
 max concurrently executing lambda’s: 1,000 by default
» can be increased on request
 max /tmp directory size: 512 MB
 cold start vs. warm start
» a lambda that has not been invoked for > ~4 minutes will
experience longer startup time

7. 7Jack D. Mills
API gateway
lambda vs. VM: design
vm
endpoint l
endpoint
endpoint
l
l
endpoint l
l
l
l
vm
SQS,
SNS
S3,
dynamoDB
cloudWatch l
internet
l’s invoked on message
arrival at REST endpoints
l’s invoked
by other l’s
l’s can put/get messages on
queueing/messaging services
l’s invoked on
a change in
persistent state
l’s invoked when perf
metrics cross threshold
a system can be composed
of a mix of l’s and VM’s

8. 8Jack D. Mills
pricing
 total cost = $/invocation + $/executeTime
» $/ invocation independent of memory size
» $/executeTime scales linearly with memory size (just like VM’s)
 invocation cost: $0.20 per 1M invocations
» 1M free invocations per month
 execution cost: 0.00001667 $/GB-sec (1.667E-5)
» 400K free GB-seconds per month
» execution time measured in 100 msec granularity

9. 9Jack D. Mills
lambda vs. VM: cost
0.0E+00
2.0E-06
4.0E-06
6.0E-06
8.0E-06
1.0E-05
1.2E-05
1.4E-05
1.6E-05
1.8E-05
m4 m5 lambda
$/GB-sec
1.0X1.04X
5.0X
US West, Linux
 because VM
instance cost
scales linearly
with memory…
 …it can be
expressed in
$/GB-sec for
comparison to
lambda

10. 10Jack D. Mills
lambda vs. VM: cost
time
thread1
threadN
time
vm lambda
pay for wall clock time
independent work load
pay only for time to actually
perform work plus invocation
 at some level
of activity
VM’s will be
cheaper than
lambda’s
 performance
is a separate
matter

11. 11Jack D. Mills
lambda vs. VM: cost
0
100
200
300
400
500
600
0 50 100 150 200 250 300
averagemsec/invocation
average invocations/second
m4.large (8GB) vs. 128M lambdas
m4.xlarge (16GB) vs. 256M lambdas
m4.2xlarge (32GB) vs. 512M lambdas
m4.4xlarge (64GB) vs. 1024M lambdas
 X/Y axes delineate
l activity factors
 VM cost constant
over this space
 l cost is zero at
origin, highest at
upper right corner
 lines are activity
levels where l cost
equals VM cost

12. 12Jack D. Mills
lambda vs. VM: performance
events
perf
per
event
l
vm
time
thread1
threadN
time
vm lambda
threads/closures executing on a VM compete
for a common pool of CPU and memory
but have larger maximum CPU and memory
active
idle or
suspended
each l gets a dedicated pool of CPU
and memory but with 3GB maximum

13. 13Jack D. Mills
precursors and contemporaries
 actors & communicating sequential processes (CSP)
» two approaches to concurrency first proposed in the 70’s
» concurrently executing modules with no shared state
» interaction only via message passing
 microservices
» decomposition of a software system into smaller modules
» typically have a REST interface (HTTP method + URL)
» typically stateless
 these are on the same continuum – SW has always been modular
» what varies are module boundaries & inter-module communication

14. 14Jack D. Mills
linux linux…
hypervisor
vmvm …
l…
customer A customer B
lllll
 each lambda executes in it’s own
container…
» not Docker-compatible
» not visible to the user
 …which execute on linux…
» which can run multiple lambda’s
 …which runs in a VM
 all lambda’s on the same VM are
from the same customer
» greater isolation
a peek under the hood

15. 15Jack D. Mills
linux
hypervisor
vm
lll
 containers are kept in memory for
minutes after execution
completes
» l will warm-start if subsequent
event arrives during this period
» lambda code should not assume
a warm-start
a peek under the hood

16. 16Jack D. Mills
linux linux
hypervisor
vmvm
ll
 VM’s take seconds to start up
» far too long for lambda’s which
may only execute for msec’s
 maintain a pool of pre-built VM’s
to hide VM startup latency
» pre-built linux containers
» pre-built language-specific
containers?
a peek under the hood
l

17. 17Jack D. Mills
evolving to cloud2.0

18. 18Jack D. Mills
spot lambdas
 implement mechanism for user to
convey flexibility when invoking l’s
» within specified time after event
» by a specified calendar time
» if dynamic pricing: when price
drops below a threshold
 l scheduling flexibility allows cloud
vendor to increase capex ROI
» like spot VM’s, share cost savings
with customers via lower price

19. 19Jack D. Mills
l osl os
…
l
l
l
…
hypervisor
vmvm
customer A
l
l
l
customer B
…
lambda graph scheduling
 lambda’s are not disassociated
» behavior captured as a graph
 convey knowledge of graph to OS
» from thread-centric scheduling to
graph-centric scheduling
 predict lambda to invoke next
» reduce startup latency
 schedule lambdas more efficiently
» on same physical server/rack/
subnet to reduce network traffic
» near common services

20. 20Jack D. Mills
l os l os
…
l
l
l
…
hypervisor
vmvm
customer A
l
l
l
customer B
…
lambda graph scheduling
 improve efficiency by adding
history & trace data to graph
 infer graph from configuration,
static code, & dynamic monitoring
» don’t need to know entire graph to
gain advantage
 lambda decomposes the black box
of full stack binaries (AMI’s)
» allows cloud vendor’s infrastructure
to have finer-grained visibility into
customer’s workload

21. 21Jack D. Mills
…
physical server
vmvm
l osl os
l
l
l
…
physical server
vm
l os
l
l
l
l
l
l
… …
…
…
lambda graph scheduling
persistent storage
S2S1
 other cloud
services can be
considered
nodes in lambda
graph
 l graph extends
over multiple
virtual & physical
machines
 graph scheduling
is distributed

22. 22Jack D. Mills
pricing models
 allow customers to pay for lower startup time for lambda’s
» consumes memory, not CPU
» variable, potentially long, startup time limits range of l application
 allow customers to pay for persistent lambda’s
» eliminates need for lambda to re-do work on startup
» need load/unload events/API’s
» store in local NVMe flash
 dynamic pricing
» e.g. priceline model: customers specify cost below which a lambda
should be invoked
» perhaps coupled with timeout when l is invoked at market price

23. 23Jack D. Mills
transparent persistence
 accessing storage is a common pattern for stateless lambda’s
 implement mechanism to specify objects as persistent
» auto-restore before & auto-save after lambda execution
» use lambda graph knowledge to prefetch objects before start
» specify persistence via configuration, statically, or dynamically
 reduces developer cognitive load, time to market, and bugs
 implement access control mechanism between lambdas and
the persistent objects they are allowed to access

24. 24Jack D. Mills
l l l
l l
l l l
reify customer economics
 customers are spending their limited IT $ on
cloud resources for output of max value
» implement mechanism for customers to
map l’s, or paths through the l graph,
to business outcomes or processes
 will allow customers to quantify what they’re spending
and what they’re getting for it
» make business processes first-class citizens
 allow encapsulation of lambda’s for mapping to business
processes or IT spending accounts (and re-use)
» make lambda graphs hierarchical

25. 25Jack D. Mills
l l l
l l
reify customer economics
l l l
l l l
l l
l l l
 capture cost & performance metrics
per l and per encapsulated l’s
» Amazon has started down this path
with X-Ray
 capture cost & performance metrics
per specified paths through l graph
» paths indicated by one or more
starting and ending lambda’s

26. 26Jack D. Mills
l
l
l
l
l
l
storage
engine
storage
service
 databases are a locus of state
access
» therefore have knowledge of
inter-lambda dependencies
 integrate storage engine’s view
of data dependencies with l
graph scheduler’s view of control
dependencies
unify persistence and compute