Serverless computing is increasingly popular because of the promise of lower cost and the convenience it provides to users who do not need to focus on server management. This has resulted in the availability of a number of proprietary and open-source serverless solutions. We seek to understand how the performance of serverless computing depends on a number of design issues using several popular open-source serverless platforms. We identify the idiosyncrasies affecting performance (throughput and latency) for different open-source serverless platforms. Further, we observe that just having either resource-based (CPU and memory) or workload-based (request per second (RPS) or concurrent requests) auto-scaling is inadequate to address the needs of the serverless platforms.
Unlocking the Potential of the Cloud for IBM Power Systems
Understanding Open Source Serverless Platforms: Design Considerations and Performance
1. 1
Understanding Open Source Serverless Platforms:
Design Considerations and Performance
Junfeng Li, Sameer G. Kulkarni,
K. K. Ramakrishnan, Dan Li
hotjunfeng@163.com
3. 3
Motivation and Goals
❖ To develop an understanding on the open source
serverless platforms:
➢Do measurements to understand the impact of key
configuration parameters of different components
(platform, gateway, controller and function)
❖ Evaluate and compare the performance of open source
serverless platforms:
➢Different workloads
➢Different auto-scaling modes
6. 6
Service Exporting and Routing
Public
NodePort
⇕
Internal
Pod_IP
Encapsulate
Execute
functions
Configure mapping
rules for exporting
service (Netfilter
NAT rules)
Get events and
push to a worker
Route &
Load balance
Decapsulate
7. 7
❖ To develop an understanding on the open source
serverless platforms:
➢Do measurements to understand the impact of key
configuration parameters of different components
(gateway, controller and function)
❖ Evaluate and compare the performance of open source
serverless platforms:
➢Different workloads
➢Different auto-scaling modes
Motivation and Goals
v1.1.16 Gateway: v0.17.0
Faas-netes: v0.8.6
Faas-cli: v0.9.2
v0.8 v1.0.4
8. 8
❖ Topology: Kubernetes cluster (1 master, 2 workers) on CloudLab1
➢Hardware: Intel Xeon E5-2640 v4 @ 20 Hyperthread cores.
➢Operating System: Ubuntu 16.04.1 LTS
➢Kubernetes v1.16.1, Docker v18.09.2
❖ Functions and Workload:
➢Python ‘Hello-world’ function
➢Python ‘HTTP’ function
➢Workload Generator: wrk
Experiment Setup
[1] Duplyakin, Dmitry, et al. "The design and operation of CloudLab." 2019 USENIX Annual Technical
Conference (USENIX ATC 19). 2019.
12. 12
❖ Salient parameter: the number of workers within one pod
Knative
Performance improves, but relatively
lower than Nuclio.
Fig. Throughput of Knative. Fig. Throughput of Nuclio.
13. 13
❖ Working model
OpenFaaS
Multiple Modes for Of-Watchdog:
(1) Fork-per-request:
Cold start for every request;
(2) Pre-fork:
Start the function once and keep warm.
Get events and
invoke function
One Worker
18. 18
Kubeless
Fig. Throughput of Kubeless. Fig. Throughput of OpenFaaS.
Fork-per-request
The performance of Kubeless is similar to that of
OpenFaaS in “Fork-per-request” mode.
19. 19
❖ To develop an understanding on the open source
serverless platforms:
➢Describe how different components work
➢Do measurements to understand the impact of key
configuration parameters of different components
(platform, gateway, controller and function)
❖ Evaluate and compare the performance of open source
serverless platforms:
➢Different workloads
➢Different auto-scaling modes
Motivation and Goals
23. 23
Performance
Latency breakdown of helloworld function:
Fork-Per-Request
(Cold Start All the Time)
Same Python Runtime
⇒ Same execution time
Platform Specific
Platform Specific
25. 25
Performance
HTTP Workload: fetch a web page (5 Byte) from a local server
Nuclio:
No ingress controller
⇒ Bypass the queue of
ingress controller
⇒ Highest throughput
27. 27
Performance
Different modes of exporting services:
Direct call to
func pod
(NodePort)
Invoke through IC/GW
IC/GW: Overhead of Ingress
Controller/API Gateway.
28. 28
Performance: Auto-scaling
Resource-based auto-scaling:
Resource-based auto-scaling
depends on Kubernetes HPA
(Horizontal Pod Autoscaler)
In spite of the same function and
HPA, platform characteristics
govern auto-scaling.
(Different performance
⇒ Different resource utilization
⇒ Different auto-scaling rate)
32. 32
Performance: Auto-scaling
Issues about load balancing for OpenFaaS:
Load-balancing Issue!
If client enables keep-alive,
OpenFaaS does not set up
connections with newly
created function pods, which
hinders performance
improvement.
Behavior: Auto-scaling happens
but NO performance improvement!
Fig. RPS-based auto-scaling
in OpenFaaS
No improvement
Auto-scale
34. 34
Performance: Auto-scaling
Issues about Concurrent-based auto-scaling:
Misconfiguration inhibits auto-
scaling.
(Conc. does not exceed threshold.
⇒ No auto-scaling with workload of
low concurrency but high RPS.)
Traffic: Conc=9, RPS=400
Configuration: Conc_Threshold=10
Fig. Conc-based auto-scaling in Knative
No improvement
No Auto-scale
Behavior: No Auto-scaling!
No able to scale to 400 RPS (Actual
RPS=~220)
35. 35
❖ Function processing:
➢Multiple workers within one function pod contribute to
performance improvement.
➢Pre-fork mode (warm worker) increases the throughput
and reduces the latency.
❖ Load balancing:
➢Plays an important role in the performance and
scalability.
➢Coupling routing with load balancing can adversely
affect the performance -- Needs greater attention!
Key Observations
36. 36
❖ Autoscaling:
➢For resource-based auto-scaling, in spite of the same
function and HPA, platform characteristics govern
auto-scaling.
➢Misconfiguration of auto-scaling rules can severely
degrade the performance and system utilization.
➢Current Auto-scaling approaches are based only on
the total processed requests, while the dropped
requests are missed out. -- Incoming request rate
needs to be accounted for.
Key Observations
37. 37
❖ Paper title:
➢Understanding Open Source Serverless Platforms:
Design Considerations and Performance.
❖ Paper links:
➢https://dl.acm.org/doi/10.1145/3366623.3368139
➢https://arxiv.org/abs/1911.07449
➢https://www.researchgate.net/publication/337362475_
Understanding_Open_Source_Serverless_Platforms_
Design_Considerations_and_Performance
More details
39. 39
❖ Load balancing:
➢Improper load balancing results in poor performance
improvement -- Needs greater attention!
❖ Autoscaling:
➢Misconfiguration of auto-scaling rules can severely
degrade the performance and system utilization.
➢Current Auto-scaling approaches are based only on the
total processed requests, while the dropped requests are
missed out. -- Incoming request rate needs to be
accounted for.
Serverless 2020 and Beyond
40. 40
Motivation
❖ To understand how the serverless platforms work?
❖ What is the impact of configuration parameters?
❖ What is the performance of serverless platforms?
❖ What is the behavior of auto-scaling?