Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Sixth International Workshop on Serverless Computing (WoSC6) 2020 Dec 7, 2020
Temporal Performance Modeling of Serverless ...
Sixth International Workshop on Serverless Computing (WoSC6) 2020 Dec 7, 2020
TOC
Introduction
System Description
Analytic...
Introduction
3
Serverless Computing
● Runtime operation and management done by the provider
○ Reduces the overhead for the software owner...
Serverless Computing
Image source: https://aws.amazon.com/lambda/
5
The Need for a Performance Model
● No previous work has been done for performance modelling of Serverless Computing
platfo...
System Description
7
Function States, Cold Starts, and Warm Starts
● Function States:
○ Initializing: Performing initialization tasks to prepar...
Autoscaling
Expiration Threshold
Scaling Out:
Scaling In:
9
Other Important Characteristics
● Initialization Time: The amount of time instance spends in the initializing state.
● Res...
Analytical Modelling
11
Overview
Warm Pool Model:
12
● Cold Start Rate
○ The system behaves like an Erlang Loss System (M/G/m/m).
○ The blocked requests are either rejected (r...
● For each state, we can also calculate:
○ Probability of Rejection: Probability of being blocked when reaching
maximum co...
Experimental Validation
15
Experimental Setup
● Experiments done on AWS Lambda
○ Python 3.6 runtime with 128MB of RAM on us-east-1 region
○ A mixture...
Sample Workload
17
Experimental Results
18
Experimental Results (2)
19
Conclusion
20
Conclusion
● Accurate and tractable analytical performance model
● Ability to predict important performance/cost related m...
Summary
Website: nima-dev.com
Twitter: @nima_mahmoudi
22
Upcoming SlideShare
Loading in …5
×

of

Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 1 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 2 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 3 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 4 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 5 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 6 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 7 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 8 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 9 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 10 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 11 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 12 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 13 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 14 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 15 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 16 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 17 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 18 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 19 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 20 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 21 Temporal Performance Modelling of Serverless Computing Platforms - WoSC6 Slide 22
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

Temporal Performance Modelling of Serverless Computing Platforms - WoSC6

Download to read offline

This presentation is an overview of the "Temporal Performance Modeling of Serverless Computing Platforms" paper published in Sixth International Workshop on Serverless Computing (WoSC6) 2020 as part of IEEE Middleware conference.

Authors: Nima Mahmoudi and Hamzeh Khazaei

Paper: https://www.serverlesscomputing.org/wosc6/#p1
Preprint and Artifacts: https://research.nima-dev.com/publication/mahmoudi-2020-tempperf/
Full Presentation: https://youtu.be/9r3j_1B5t8c
Lightning Talk (1 min): https://youtu.be/E5KigIq0Z1E
PACS Lab: https://pacs.eecs.yorku.ca/

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Temporal Performance Modelling of Serverless Computing Platforms - WoSC6

  1. 1. Sixth International Workshop on Serverless Computing (WoSC6) 2020 Dec 7, 2020 Temporal Performance Modeling of Serverless Computing Platforms N. Mahmoudi and H. Khazaei, “Temporal Performance Modelling of Serverless Computing Platforms,” in Proceedings of the 6th International Workshop on Serverless Computing, 2020, p. 1–6., doi: 10.10.1145/3429880.3430092. https://www.serverlesscomputing.org/wosc6/#p1 Hamzeh Khazaei hkh@yorku.ca Performant and Available Computing Systems (PACS) Lab Nima Mahmoudi nmahmoud@ualberta.ca
  2. 2. Sixth International Workshop on Serverless Computing (WoSC6) 2020 Dec 7, 2020 TOC Introduction System Description Analytical Modelling Experimental Validation Conclusion 2
  3. 3. Introduction 3
  4. 4. Serverless Computing ● Runtime operation and management done by the provider ○ Reduces the overhead for the software owner ○ Provisioning ○ Scaling resources ● Software is developed by writing functions ○ Well-defined interface ○ Functions deployed separately 4
  5. 5. Serverless Computing Image source: https://aws.amazon.com/lambda/ 5
  6. 6. The Need for a Performance Model ● No previous work has been done for performance modelling of Serverless Computing platforms ● Accurate performance modelling can beneficial in many ways: ○ Ensure the Quality of Service (QoS) ○ Improve performance metrics ○ Predict/optimize infrastructure cost ○ Move from best-effort to performance guarantees ● It can benefit both serverless provider and application developer 6
  7. 7. System Description 7
  8. 8. Function States, Cold Starts, and Warm Starts ● Function States: ○ Initializing: Performing initialization tasks to prepare the function for incoming requests. Includes infrastructure initialization and application initialization. ○ Running: Running the tasks required to process a request. ○ Idle: Provisioned instance that is not running any workloads. The instances in this state are not billed. ● Cold Start Requests: ○ A request that needs to go through initialization steps due to lack of provisioned capacity. ● Warm Start Requests: ○ Only includes request processing time since idle instance was available 8
  9. 9. Autoscaling Expiration Threshold Scaling Out: Scaling In: 9
  10. 10. Other Important Characteristics ● Initialization Time: The amount of time instance spends in the initializing state. ● Response Time: No queuing, so it is equal to service time. It remains stable throughout time for cold and warm start requests. ● Maximum Concurrency Level: Maximum number of instances that can be in the state running in parallel. ● Request Routing: To facilitate scaling in, requests are routed to recently created instances first. 10
  11. 11. Analytical Modelling 11
  12. 12. Overview Warm Pool Model: 12
  13. 13. ● Cold Start Rate ○ The system behaves like an Erlang Loss System (M/G/m/m). ○ The blocked requests are either rejected (reached maximum concurrency level) or cause a cold start (and thus the creation of a new instance) ● Arrival Rate for Each Instance ○ Requests blocked by instance n are either processed by instance n+1 or blocked by it. ○ The difference between blocked rates gives us individual arrival rates. ● Server Expiration Rate ○ Can be calculated knowing individual arrival rates and expiration threshold. ● Warm Pool Model ○ Each state represents the number of instances in the warm pool. 13
  14. 14. ● For each state, we can also calculate: ○ Probability of Rejection: Probability of being blocked when reaching maximum concurrency. ○ Probability of Cold Start: Probability of being blocked in other states. ○ Average Response Time: ○ Mean Number of Instances in Warm Pool: ■ Running ■ Idle ○ Utilization: Ratio of instances in running state over all instances. ● All predictions can be found in a time-bounded fashion (e.g., answers the question “what happens to my QoS in the next 5 minutes?”) 14
  15. 15. Experimental Validation 15
  16. 16. Experimental Setup ● Experiments done on AWS Lambda ○ Python 3.6 runtime with 128MB of RAM on us-east-1 region ○ A mixture of CPU and IO intensive tasks ● Client was a virtual machine on Compute Canada Arbutus ○ 8 vCPUs, 16GB of RAM, 1000Mbps connectivity, single-digit milliseconds latency to AWS servers ○ Python with in-house workload generation tool pacswg ○ Official boto3 library for API communication ○ Communicated directly with Lambda API, no intermediary interfaces like API Gateway ● Predictions are made 5 minutes into the future ● Assumed oracle request pattern prediction 16
  17. 17. Sample Workload 17
  18. 18. Experimental Results 18
  19. 19. Experimental Results (2) 19
  20. 20. Conclusion 20
  21. 21. Conclusion ● Accurate and tractable analytical performance model ● Ability to predict important performance/cost related metrics ● Can predict QoS ● Can benefit serverless providers ○ Ability to predict QoS under different loads on deployment time ○ Can be used in the management to prevent performance degradation ● Could be useful to application developers ○ Predict how their system will perform in the immediate future ○ Help them optimize their memory configuration to occur minimal cost that satisfies performance requirements throughout their daily request patterns ● Can be used in the management systems to warm-up instances to prevent SLA violations 21
  22. 22. Summary Website: nima-dev.com Twitter: @nima_mahmoudi 22

This presentation is an overview of the "Temporal Performance Modeling of Serverless Computing Platforms" paper published in Sixth International Workshop on Serverless Computing (WoSC6) 2020 as part of IEEE Middleware conference. Authors: Nima Mahmoudi and Hamzeh Khazaei Paper: https://www.serverlesscomputing.org/wosc6/#p1 Preprint and Artifacts: https://research.nima-dev.com/publication/mahmoudi-2020-tempperf/ Full Presentation: https://youtu.be/9r3j_1B5t8c Lightning Talk (1 min): https://youtu.be/E5KigIq0Z1E PACS Lab: https://pacs.eecs.yorku.ca/

Views

Total views

34

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

1

Shares

0

Comments

0

Likes

0

×