SlideShare a Scribd company logo
1 of 24
Download to read offline
CASCON 2020 Nov 12, 2020
Performance Modeling of Serverless Computing Platforms
N. Mahmoudi and H. Khazaei, "Performance Modeling of
Serverless Computing Platforms," in IEEE Transactions on
Cloud Computing, doi: 10.1109/TCC.2020.3033373.
Hamzeh Khazaei
hkh@yorku.ca
Performant and Available
Computing Systems (PACS) Lab
Nima Mahmoudi
nmahmoud@ualberta.ca
CASCON 2020 Nov 12, 2020
TOC
Introduction
System Description
Analytical Modelling
Experimental Validation
Conclusion
2
Introduction
3
Serverless Computing
● Runtime operation and management done by the provider
○ Reduces the overhead for the software owner
○ Provisioning
○ Scaling resources
● Software is developed by writing functions
○ Well-defined interface
○ Functions deployed separately
4
Serverless Computing
Image source: https://aws.amazon.com/lambda/
5
The Need for a Performance Model
● No previous work has been done for performance modelling of Serverless Computing
platforms
● Accurate performance modelling can beneficial in many ways:
○ Ensure the Quality of Service (QoS)
○ Improve performance metrics
○ Predict/optimize infrastructure cost
○ Move from best-effort to performance guarantees
● It can benefit both serverless provider and application developer
6
System Description
7
Function States, Cold Starts, and Warm Starts
● Function States:
○ Initializing: Performing initialization tasks to prepare the function for incoming requests. Includes
infrastructure initialization and application initialization.
○ Running: Running the tasks required to process a request.
○ Idle: Provisioned instance that is not running any workloads. The instances in this state are not billed.
● Cold Start Requests:
○ A request that needs to go through initialization steps due to lack of provisioned capacity.
● Warm Start Requests:
○ Only includes request processing time since idle instance was available
8
Autoscaling
Expiration Threshold
Scaling Out:
Scaling In:
9
Other Important Characteristics
● Initialization Time: The amount of time instance spends in the initializing state.
● Response Time: No queuing, so it is equal to service time. It remains stable
throughout time for cold and warm start requests.
● Maximum Concurrency Level: Maximum number of instances that can be in the state
running in parallel.
● Request Routing: To facilitate scaling in, requests are routed to recently created
instances first.
10
Analytical Modelling
11
Overview
Warm Pool Model:
12
● Cold Start Rate
○ The system behaves like an Erlang Loss System (M/G/m/m).
○ The blocked requests are either rejected (reached maximum concurrency level) or cause a
cold start (and thus the creation of a new instance)
● Arrival Rate for Each Instance
○ Requests blocked by instance n are either processed by instance n+1 or blocked by it.
○ The difference between blocked rates gives us individual arrival rates.
● Server Expiration Rate
○ Can be calculated knowing individual arrival rates and expiration threshold.
● Warm Pool Model
○ Each state represents the number of instances in the warm pool.
13
● For each state, we can also calculate:
○ Probability of Rejection: Probability of being blocked when reaching
maximum concurrency.
○ Probability of Cold Start: Probability of being blocked in other states.
○ Average Response Time:
○ Mean Number of Instances in Warm Pool:
■ Running
■ Idle
○ Utilization: Ratio of instances in running state over all instances.
14
15
Experimental Validation
16
Experimental Setup
● Experiments done on AWS Lambda
○ Python 3.6 runtime with 128MB of RAM on us-east-1 region
○ A mixture of CPU and IO intensive tasks
● Client was a virtual machine on Compute Canada Arbutus
○ 8 vCPUs, 16GB of RAM, 1000Mbps connectivity, single-digit milliseconds latency to AWS
servers
○ Python with in-house workload generation tool pacswg
○ Official boto3 library for API communication
○ Communicated directly with Lambda API, no intermediary interfaces like API Gateway
17
Experimental Results
18
Sample Workloads
Name Application Warm (ms) Cold (ms)
W1 CPU and disk intensive benchmark 2000 2200
W2 A range of benchmarks with different configurations 300 10000
W3 Startup test with echo on Apache OpenWhisk 20 1000
W4 Fibonacci Calculation on AWS Lambda 4211 5961
W5 Fibonacci calculation on Azure Functions 1809 26681
19
Probability of Rejection
20
What-If Analysis Results
21
Conclusion
22
Conclusion
● Accurate and tractable analytical performance model
● Ability to predict important performance/cost related metrics
● Can predict QoS
● Can benefit serverless providers
○ Ability to predict QoS under different loads on deployment time
○ Optimize infrastructure cost, knowing how it would affect performance
● Could be useful to application developers
○ Predict how their system will react under different loads
○ Help them optimize their memory configuration to occur minimal cost that satisfies
performance requirements
● Making the expiration threshold adaptive would allow better cost-performance tradeoff
23
Summary
Website: research.nima-dev.com
Twitter: @nima_mahmoudi
24

More Related Content

What's hot

Red Hat OpenShift Operators - Operators ABC
Red Hat OpenShift Operators - Operators ABCRed Hat OpenShift Operators - Operators ABC
Red Hat OpenShift Operators - Operators ABCRobert Bohne
 
Accelerate Your ML Pipeline with AutoML and MLflow
Accelerate Your ML Pipeline with AutoML and MLflowAccelerate Your ML Pipeline with AutoML and MLflow
Accelerate Your ML Pipeline with AutoML and MLflowDatabricks
 
Camel Day Italia 2021 - Camel K
Camel Day Italia 2021 - Camel KCamel Day Italia 2021 - Camel K
Camel Day Italia 2021 - Camel KNicola Ferraro
 
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...Dataconomy Media
 
Architecture of TPU, GPU and CPU
Architecture of TPU, GPU and CPUArchitecture of TPU, GPU and CPU
Architecture of TPU, GPU and CPUGlobalLogic Ukraine
 
Estaleiro o uso de kubernetes no serpro
Estaleiro   o uso de kubernetes no serproEstaleiro   o uso de kubernetes no serpro
Estaleiro o uso de kubernetes no serproRicardo Katz
 
Red Hat multi-cluster management & what's new in OpenShift
Red Hat multi-cluster management & what's new in OpenShiftRed Hat multi-cluster management & what's new in OpenShift
Red Hat multi-cluster management & what's new in OpenShiftKangaroot
 
Quality Assurance of Machine Learning-based AI Systems
Quality Assurance of Machine Learning-based AI Systems Quality Assurance of Machine Learning-based AI Systems
Quality Assurance of Machine Learning-based AI Systems Fuyuki Ishikawa
 
From Data Science to MLOps
From Data Science to MLOpsFrom Data Science to MLOps
From Data Science to MLOpsCarl W. Handlin
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesDatabricks
 
Netflix: From Clouds to Roots
Netflix: From Clouds to RootsNetflix: From Clouds to Roots
Netflix: From Clouds to RootsBrendan Gregg
 
Strata - Scaling Jupyter with Jupyter Enterprise Gateway
Strata - Scaling Jupyter with Jupyter Enterprise GatewayStrata - Scaling Jupyter with Jupyter Enterprise Gateway
Strata - Scaling Jupyter with Jupyter Enterprise GatewayLuciano Resende
 
Integrating Apache Kafka and Elastic Using the Connect Framework
Integrating Apache Kafka and Elastic Using the Connect FrameworkIntegrating Apache Kafka and Elastic Using the Connect Framework
Integrating Apache Kafka and Elastic Using the Connect Frameworkconfluent
 
22nd Athens Big Data Meetup - 1st Talk - MLOps Workshop: The Full ML Lifecycl...
22nd Athens Big Data Meetup - 1st Talk - MLOps Workshop: The Full ML Lifecycl...22nd Athens Big Data Meetup - 1st Talk - MLOps Workshop: The Full ML Lifecycl...
22nd Athens Big Data Meetup - 1st Talk - MLOps Workshop: The Full ML Lifecycl...Athens Big Data
 
Method of NUMA-Aware Resource Management for Kubernetes 5G NFV Cluster
Method of NUMA-Aware Resource Management for Kubernetes 5G NFV ClusterMethod of NUMA-Aware Resource Management for Kubernetes 5G NFV Cluster
Method of NUMA-Aware Resource Management for Kubernetes 5G NFV Clusterbyonggon chun
 
GT.M: A Tried and Tested Open-Source NoSQL Database
GT.M: A Tried and Tested Open-Source NoSQL DatabaseGT.M: A Tried and Tested Open-Source NoSQL Database
GT.M: A Tried and Tested Open-Source NoSQL DatabaseRob Tweed
 
Understanding LLM LLMOps & MLOps_open version.pdf
Understanding LLM LLMOps & MLOps_open version.pdfUnderstanding LLM LLMOps & MLOps_open version.pdf
Understanding LLM LLMOps & MLOps_open version.pdfChun Myung Kyu
 
create auto scale jboss cluster with openshift
create auto scale jboss cluster with openshiftcreate auto scale jboss cluster with openshift
create auto scale jboss cluster with openshiftYusuf Hadiwinata Sutandar
 

What's hot (20)

Red Hat OpenShift Operators - Operators ABC
Red Hat OpenShift Operators - Operators ABCRed Hat OpenShift Operators - Operators ABC
Red Hat OpenShift Operators - Operators ABC
 
What is MLOps
What is MLOpsWhat is MLOps
What is MLOps
 
Accelerate Your ML Pipeline with AutoML and MLflow
Accelerate Your ML Pipeline with AutoML and MLflowAccelerate Your ML Pipeline with AutoML and MLflow
Accelerate Your ML Pipeline with AutoML and MLflow
 
Camel Day Italia 2021 - Camel K
Camel Day Italia 2021 - Camel KCamel Day Italia 2021 - Camel K
Camel Day Italia 2021 - Camel K
 
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
 
Architecture of TPU, GPU and CPU
Architecture of TPU, GPU and CPUArchitecture of TPU, GPU and CPU
Architecture of TPU, GPU and CPU
 
Estaleiro o uso de kubernetes no serpro
Estaleiro   o uso de kubernetes no serproEstaleiro   o uso de kubernetes no serpro
Estaleiro o uso de kubernetes no serpro
 
Red Hat multi-cluster management & what's new in OpenShift
Red Hat multi-cluster management & what's new in OpenShiftRed Hat multi-cluster management & what's new in OpenShift
Red Hat multi-cluster management & what's new in OpenShift
 
Quality Assurance of Machine Learning-based AI Systems
Quality Assurance of Machine Learning-based AI Systems Quality Assurance of Machine Learning-based AI Systems
Quality Assurance of Machine Learning-based AI Systems
 
MLOps with Kubeflow
MLOps with Kubeflow MLOps with Kubeflow
MLOps with Kubeflow
 
From Data Science to MLOps
From Data Science to MLOpsFrom Data Science to MLOps
From Data Science to MLOps
 
Scaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on KubernetesScaling your Data Pipelines with Apache Spark on Kubernetes
Scaling your Data Pipelines with Apache Spark on Kubernetes
 
Netflix: From Clouds to Roots
Netflix: From Clouds to RootsNetflix: From Clouds to Roots
Netflix: From Clouds to Roots
 
Strata - Scaling Jupyter with Jupyter Enterprise Gateway
Strata - Scaling Jupyter with Jupyter Enterprise GatewayStrata - Scaling Jupyter with Jupyter Enterprise Gateway
Strata - Scaling Jupyter with Jupyter Enterprise Gateway
 
Integrating Apache Kafka and Elastic Using the Connect Framework
Integrating Apache Kafka and Elastic Using the Connect FrameworkIntegrating Apache Kafka and Elastic Using the Connect Framework
Integrating Apache Kafka and Elastic Using the Connect Framework
 
22nd Athens Big Data Meetup - 1st Talk - MLOps Workshop: The Full ML Lifecycl...
22nd Athens Big Data Meetup - 1st Talk - MLOps Workshop: The Full ML Lifecycl...22nd Athens Big Data Meetup - 1st Talk - MLOps Workshop: The Full ML Lifecycl...
22nd Athens Big Data Meetup - 1st Talk - MLOps Workshop: The Full ML Lifecycl...
 
Method of NUMA-Aware Resource Management for Kubernetes 5G NFV Cluster
Method of NUMA-Aware Resource Management for Kubernetes 5G NFV ClusterMethod of NUMA-Aware Resource Management for Kubernetes 5G NFV Cluster
Method of NUMA-Aware Resource Management for Kubernetes 5G NFV Cluster
 
GT.M: A Tried and Tested Open-Source NoSQL Database
GT.M: A Tried and Tested Open-Source NoSQL DatabaseGT.M: A Tried and Tested Open-Source NoSQL Database
GT.M: A Tried and Tested Open-Source NoSQL Database
 
Understanding LLM LLMOps & MLOps_open version.pdf
Understanding LLM LLMOps & MLOps_open version.pdfUnderstanding LLM LLMOps & MLOps_open version.pdf
Understanding LLM LLMOps & MLOps_open version.pdf
 
create auto scale jboss cluster with openshift
create auto scale jboss cluster with openshiftcreate auto scale jboss cluster with openshift
create auto scale jboss cluster with openshift
 

Similar to Performance Modeling of Serverless Computing Platforms - CASCON2020 Workshop on Cloud Computing

Temporal Performance Modelling of Serverless Computing Platforms - WoSC6
Temporal Performance Modelling of Serverless Computing Platforms - WoSC6Temporal Performance Modelling of Serverless Computing Platforms - WoSC6
Temporal Performance Modelling of Serverless Computing Platforms - WoSC6Nima Mahmoudi
 
Serving deep learning models in a serverless platform (IC2E 2018)
Serving deep learning models in a serverless platform (IC2E 2018)Serving deep learning models in a serverless platform (IC2E 2018)
Serving deep learning models in a serverless platform (IC2E 2018)alekn
 
Zero Downtime JEE Architectures
Zero Downtime JEE ArchitecturesZero Downtime JEE Architectures
Zero Downtime JEE ArchitecturesAlexander Penev
 
Container world 2019 Canary Release
Container world 2019 Canary ReleaseContainer world 2019 Canary Release
Container world 2019 Canary ReleaseBilly Yuen
 
How Netflix Uses Amazon Kinesis Streams to Monitor and Optimize Large-scale N...
How Netflix Uses Amazon Kinesis Streams to Monitor and Optimize Large-scale N...How Netflix Uses Amazon Kinesis Streams to Monitor and Optimize Large-scale N...
How Netflix Uses Amazon Kinesis Streams to Monitor and Optimize Large-scale N...Amazon Web Services
 
Serverless GraphQL. AppSync 101
Serverless GraphQL. AppSync 101Serverless GraphQL. AppSync 101
Serverless GraphQL. AppSync 101Marcin Sodkiewicz
 
Machine learning at scale with aws sage maker
Machine learning at scale with aws sage makerMachine learning at scale with aws sage maker
Machine learning at scale with aws sage makerPhilipBasford
 
Kafka Summit NYC 2017 - Scalable Real-Time Complex Event Processing @ Uber
Kafka Summit NYC 2017 - Scalable Real-Time Complex Event Processing @ UberKafka Summit NYC 2017 - Scalable Real-Time Complex Event Processing @ Uber
Kafka Summit NYC 2017 - Scalable Real-Time Complex Event Processing @ Uberconfluent
 
Stream and Batch Processing in the Cloud with Data Microservices
Stream and Batch Processing in the Cloud with Data MicroservicesStream and Batch Processing in the Cloud with Data Microservices
Stream and Batch Processing in the Cloud with Data Microservicesmarius_bogoevici
 
Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3LibbySchulze
 
SimScale: Unparalleled CFD Speeds with Parallel Computing
SimScale: Unparalleled CFD Speeds with Parallel ComputingSimScale: Unparalleled CFD Speeds with Parallel Computing
SimScale: Unparalleled CFD Speeds with Parallel ComputingSimScale
 
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...InfluxData
 
Phil Basford - machine learning at scale with aws sage maker
Phil Basford - machine learning at scale with aws sage makerPhil Basford - machine learning at scale with aws sage maker
Phil Basford - machine learning at scale with aws sage makerAWSCOMSUM
 
Test strategies for data processing pipelines
Test strategies for data processing pipelinesTest strategies for data processing pipelines
Test strategies for data processing pipelinesLars Albertsson
 
TechTalk_Cloud Performance Testing_0.6
TechTalk_Cloud Performance Testing_0.6TechTalk_Cloud Performance Testing_0.6
TechTalk_Cloud Performance Testing_0.6Sravanthi N
 
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per Day
Hadoop summit - Scaling Uber’s Real-Time Infra for  Trillion Events per DayHadoop summit - Scaling Uber’s Real-Time Infra for  Trillion Events per Day
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per DayAnkur Bansal
 
How Uber scaled its Real Time Infrastructure to Trillion events per day
How Uber scaled its Real Time Infrastructure to Trillion events per dayHow Uber scaled its Real Time Infrastructure to Trillion events per day
How Uber scaled its Real Time Infrastructure to Trillion events per dayDataWorks Summit
 
HKG15-204: OpenStack: 3rd party testing and performance benchmarking
HKG15-204: OpenStack: 3rd party testing and performance benchmarkingHKG15-204: OpenStack: 3rd party testing and performance benchmarking
HKG15-204: OpenStack: 3rd party testing and performance benchmarkingLinaro
 
AppRunner DeepDive
AppRunner DeepDiveAppRunner DeepDive
AppRunner DeepDiveDhaval Nagar
 

Similar to Performance Modeling of Serverless Computing Platforms - CASCON2020 Workshop on Cloud Computing (20)

Temporal Performance Modelling of Serverless Computing Platforms - WoSC6
Temporal Performance Modelling of Serverless Computing Platforms - WoSC6Temporal Performance Modelling of Serverless Computing Platforms - WoSC6
Temporal Performance Modelling of Serverless Computing Platforms - WoSC6
 
Nexmark with beam
Nexmark with beamNexmark with beam
Nexmark with beam
 
Serving deep learning models in a serverless platform (IC2E 2018)
Serving deep learning models in a serverless platform (IC2E 2018)Serving deep learning models in a serverless platform (IC2E 2018)
Serving deep learning models in a serverless platform (IC2E 2018)
 
Zero Downtime JEE Architectures
Zero Downtime JEE ArchitecturesZero Downtime JEE Architectures
Zero Downtime JEE Architectures
 
Container world 2019 Canary Release
Container world 2019 Canary ReleaseContainer world 2019 Canary Release
Container world 2019 Canary Release
 
How Netflix Uses Amazon Kinesis Streams to Monitor and Optimize Large-scale N...
How Netflix Uses Amazon Kinesis Streams to Monitor and Optimize Large-scale N...How Netflix Uses Amazon Kinesis Streams to Monitor and Optimize Large-scale N...
How Netflix Uses Amazon Kinesis Streams to Monitor and Optimize Large-scale N...
 
Serverless GraphQL. AppSync 101
Serverless GraphQL. AppSync 101Serverless GraphQL. AppSync 101
Serverless GraphQL. AppSync 101
 
Machine learning at scale with aws sage maker
Machine learning at scale with aws sage makerMachine learning at scale with aws sage maker
Machine learning at scale with aws sage maker
 
Kafka Summit NYC 2017 - Scalable Real-Time Complex Event Processing @ Uber
Kafka Summit NYC 2017 - Scalable Real-Time Complex Event Processing @ UberKafka Summit NYC 2017 - Scalable Real-Time Complex Event Processing @ Uber
Kafka Summit NYC 2017 - Scalable Real-Time Complex Event Processing @ Uber
 
Stream and Batch Processing in the Cloud with Data Microservices
Stream and Batch Processing in the Cloud with Data MicroservicesStream and Batch Processing in the Cloud with Data Microservices
Stream and Batch Processing in the Cloud with Data Microservices
 
Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3Scaling Monitoring At Databricks From Prometheus to M3
Scaling Monitoring At Databricks From Prometheus to M3
 
SimScale: Unparalleled CFD Speeds with Parallel Computing
SimScale: Unparalleled CFD Speeds with Parallel ComputingSimScale: Unparalleled CFD Speeds with Parallel Computing
SimScale: Unparalleled CFD Speeds with Parallel Computing
 
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
 
Phil Basford - machine learning at scale with aws sage maker
Phil Basford - machine learning at scale with aws sage makerPhil Basford - machine learning at scale with aws sage maker
Phil Basford - machine learning at scale with aws sage maker
 
Test strategies for data processing pipelines
Test strategies for data processing pipelinesTest strategies for data processing pipelines
Test strategies for data processing pipelines
 
TechTalk_Cloud Performance Testing_0.6
TechTalk_Cloud Performance Testing_0.6TechTalk_Cloud Performance Testing_0.6
TechTalk_Cloud Performance Testing_0.6
 
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per Day
Hadoop summit - Scaling Uber’s Real-Time Infra for  Trillion Events per DayHadoop summit - Scaling Uber’s Real-Time Infra for  Trillion Events per Day
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per Day
 
How Uber scaled its Real Time Infrastructure to Trillion events per day
How Uber scaled its Real Time Infrastructure to Trillion events per dayHow Uber scaled its Real Time Infrastructure to Trillion events per day
How Uber scaled its Real Time Infrastructure to Trillion events per day
 
HKG15-204: OpenStack: 3rd party testing and performance benchmarking
HKG15-204: OpenStack: 3rd party testing and performance benchmarkingHKG15-204: OpenStack: 3rd party testing and performance benchmarking
HKG15-204: OpenStack: 3rd party testing and performance benchmarking
 
AppRunner DeepDive
AppRunner DeepDiveAppRunner DeepDive
AppRunner DeepDive
 

Recently uploaded

Artificial Intelligence in Power System overview
Artificial Intelligence in Power System overviewArtificial Intelligence in Power System overview
Artificial Intelligence in Power System overviewsandhya757531
 
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxCurve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxRomil Mishra
 
AntColonyOptimizationManetNetworkAODV.pptx
AntColonyOptimizationManetNetworkAODV.pptxAntColonyOptimizationManetNetworkAODV.pptx
AntColonyOptimizationManetNetworkAODV.pptxLina Kadam
 
input buffering in lexical analysis in CD
input buffering in lexical analysis in CDinput buffering in lexical analysis in CD
input buffering in lexical analysis in CDHeadOfDepartmentComp1
 
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdfLivre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdfsaad175691
 
Indian Tradition, Culture & Societies.pdf
Indian Tradition, Culture & Societies.pdfIndian Tradition, Culture & Societies.pdf
Indian Tradition, Culture & Societies.pdfalokitpathak01
 
Cost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionCost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionSneha Padhiar
 
Detection&Tracking - Thermal imaging object detection and tracking
Detection&Tracking - Thermal imaging object detection and trackingDetection&Tracking - Thermal imaging object detection and tracking
Detection&Tracking - Thermal imaging object detection and trackinghadarpinhas1
 
priority interrupt computer organization
priority interrupt computer organizationpriority interrupt computer organization
priority interrupt computer organizationchnrketan
 
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Sumanth A
 
Machine Learning 5G Federated Learning.pdf
Machine Learning 5G Federated Learning.pdfMachine Learning 5G Federated Learning.pdf
Machine Learning 5G Federated Learning.pdfadeyimikaipaye
 
Secure Key Crypto - Tech Paper JET Tech Labs
Secure Key Crypto - Tech Paper JET Tech LabsSecure Key Crypto - Tech Paper JET Tech Labs
Secure Key Crypto - Tech Paper JET Tech Labsamber724300
 
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...gerogepatton
 
The Satellite applications in telecommunication
The Satellite applications in telecommunicationThe Satellite applications in telecommunication
The Satellite applications in telecommunicationnovrain7111
 
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...IJAEMSJORNAL
 
Robotics Group 10 (Control Schemes) cse.pdf
Robotics Group 10  (Control Schemes) cse.pdfRobotics Group 10  (Control Schemes) cse.pdf
Robotics Group 10 (Control Schemes) cse.pdfsahilsajad201
 
Module-1-(Building Acoustics) Noise Control (Unit-3). pdf
Module-1-(Building Acoustics) Noise Control (Unit-3). pdfModule-1-(Building Acoustics) Noise Control (Unit-3). pdf
Module-1-(Building Acoustics) Noise Control (Unit-3). pdfManish Kumar
 
Network Enhancements on BitVisor for BitVisor Summit 12
Network Enhancements on BitVisor for BitVisor Summit 12Network Enhancements on BitVisor for BitVisor Summit 12
Network Enhancements on BitVisor for BitVisor Summit 12cjchen22
 
Defining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxDefining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxAshwiniTodkar4
 
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSneha Padhiar
 

Recently uploaded (20)

Artificial Intelligence in Power System overview
Artificial Intelligence in Power System overviewArtificial Intelligence in Power System overview
Artificial Intelligence in Power System overview
 
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxCurve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
 
AntColonyOptimizationManetNetworkAODV.pptx
AntColonyOptimizationManetNetworkAODV.pptxAntColonyOptimizationManetNetworkAODV.pptx
AntColonyOptimizationManetNetworkAODV.pptx
 
input buffering in lexical analysis in CD
input buffering in lexical analysis in CDinput buffering in lexical analysis in CD
input buffering in lexical analysis in CD
 
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdfLivre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
 
Indian Tradition, Culture & Societies.pdf
Indian Tradition, Culture & Societies.pdfIndian Tradition, Culture & Societies.pdf
Indian Tradition, Culture & Societies.pdf
 
Cost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionCost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based question
 
Detection&Tracking - Thermal imaging object detection and tracking
Detection&Tracking - Thermal imaging object detection and trackingDetection&Tracking - Thermal imaging object detection and tracking
Detection&Tracking - Thermal imaging object detection and tracking
 
priority interrupt computer organization
priority interrupt computer organizationpriority interrupt computer organization
priority interrupt computer organization
 
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
 
Machine Learning 5G Federated Learning.pdf
Machine Learning 5G Federated Learning.pdfMachine Learning 5G Federated Learning.pdf
Machine Learning 5G Federated Learning.pdf
 
Secure Key Crypto - Tech Paper JET Tech Labs
Secure Key Crypto - Tech Paper JET Tech LabsSecure Key Crypto - Tech Paper JET Tech Labs
Secure Key Crypto - Tech Paper JET Tech Labs
 
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
 
The Satellite applications in telecommunication
The Satellite applications in telecommunicationThe Satellite applications in telecommunication
The Satellite applications in telecommunication
 
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
 
Robotics Group 10 (Control Schemes) cse.pdf
Robotics Group 10  (Control Schemes) cse.pdfRobotics Group 10  (Control Schemes) cse.pdf
Robotics Group 10 (Control Schemes) cse.pdf
 
Module-1-(Building Acoustics) Noise Control (Unit-3). pdf
Module-1-(Building Acoustics) Noise Control (Unit-3). pdfModule-1-(Building Acoustics) Noise Control (Unit-3). pdf
Module-1-(Building Acoustics) Noise Control (Unit-3). pdf
 
Network Enhancements on BitVisor for BitVisor Summit 12
Network Enhancements on BitVisor for BitVisor Summit 12Network Enhancements on BitVisor for BitVisor Summit 12
Network Enhancements on BitVisor for BitVisor Summit 12
 
Defining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxDefining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptx
 
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
 

Performance Modeling of Serverless Computing Platforms - CASCON2020 Workshop on Cloud Computing

  • 1. CASCON 2020 Nov 12, 2020 Performance Modeling of Serverless Computing Platforms N. Mahmoudi and H. Khazaei, "Performance Modeling of Serverless Computing Platforms," in IEEE Transactions on Cloud Computing, doi: 10.1109/TCC.2020.3033373. Hamzeh Khazaei hkh@yorku.ca Performant and Available Computing Systems (PACS) Lab Nima Mahmoudi nmahmoud@ualberta.ca
  • 2. CASCON 2020 Nov 12, 2020 TOC Introduction System Description Analytical Modelling Experimental Validation Conclusion 2
  • 4. Serverless Computing ● Runtime operation and management done by the provider ○ Reduces the overhead for the software owner ○ Provisioning ○ Scaling resources ● Software is developed by writing functions ○ Well-defined interface ○ Functions deployed separately 4
  • 5. Serverless Computing Image source: https://aws.amazon.com/lambda/ 5
  • 6. The Need for a Performance Model ● No previous work has been done for performance modelling of Serverless Computing platforms ● Accurate performance modelling can beneficial in many ways: ○ Ensure the Quality of Service (QoS) ○ Improve performance metrics ○ Predict/optimize infrastructure cost ○ Move from best-effort to performance guarantees ● It can benefit both serverless provider and application developer 6
  • 8. Function States, Cold Starts, and Warm Starts ● Function States: ○ Initializing: Performing initialization tasks to prepare the function for incoming requests. Includes infrastructure initialization and application initialization. ○ Running: Running the tasks required to process a request. ○ Idle: Provisioned instance that is not running any workloads. The instances in this state are not billed. ● Cold Start Requests: ○ A request that needs to go through initialization steps due to lack of provisioned capacity. ● Warm Start Requests: ○ Only includes request processing time since idle instance was available 8
  • 10. Other Important Characteristics ● Initialization Time: The amount of time instance spends in the initializing state. ● Response Time: No queuing, so it is equal to service time. It remains stable throughout time for cold and warm start requests. ● Maximum Concurrency Level: Maximum number of instances that can be in the state running in parallel. ● Request Routing: To facilitate scaling in, requests are routed to recently created instances first. 10
  • 13. ● Cold Start Rate ○ The system behaves like an Erlang Loss System (M/G/m/m). ○ The blocked requests are either rejected (reached maximum concurrency level) or cause a cold start (and thus the creation of a new instance) ● Arrival Rate for Each Instance ○ Requests blocked by instance n are either processed by instance n+1 or blocked by it. ○ The difference between blocked rates gives us individual arrival rates. ● Server Expiration Rate ○ Can be calculated knowing individual arrival rates and expiration threshold. ● Warm Pool Model ○ Each state represents the number of instances in the warm pool. 13
  • 14. ● For each state, we can also calculate: ○ Probability of Rejection: Probability of being blocked when reaching maximum concurrency. ○ Probability of Cold Start: Probability of being blocked in other states. ○ Average Response Time: ○ Mean Number of Instances in Warm Pool: ■ Running ■ Idle ○ Utilization: Ratio of instances in running state over all instances. 14
  • 15. 15
  • 17. Experimental Setup ● Experiments done on AWS Lambda ○ Python 3.6 runtime with 128MB of RAM on us-east-1 region ○ A mixture of CPU and IO intensive tasks ● Client was a virtual machine on Compute Canada Arbutus ○ 8 vCPUs, 16GB of RAM, 1000Mbps connectivity, single-digit milliseconds latency to AWS servers ○ Python with in-house workload generation tool pacswg ○ Official boto3 library for API communication ○ Communicated directly with Lambda API, no intermediary interfaces like API Gateway 17
  • 19. Sample Workloads Name Application Warm (ms) Cold (ms) W1 CPU and disk intensive benchmark 2000 2200 W2 A range of benchmarks with different configurations 300 10000 W3 Startup test with echo on Apache OpenWhisk 20 1000 W4 Fibonacci Calculation on AWS Lambda 4211 5961 W5 Fibonacci calculation on Azure Functions 1809 26681 19
  • 23. Conclusion ● Accurate and tractable analytical performance model ● Ability to predict important performance/cost related metrics ● Can predict QoS ● Can benefit serverless providers ○ Ability to predict QoS under different loads on deployment time ○ Optimize infrastructure cost, knowing how it would affect performance ● Could be useful to application developers ○ Predict how their system will react under different loads ○ Help them optimize their memory configuration to occur minimal cost that satisfies performance requirements ● Making the expiration threshold adaptive would allow better cost-performance tradeoff 23