This presentation was given at Serverless Architecture Conference on the topic "Courier: Delivering Serverless Functions across Multi-FaaS Cloud Deployments" With the advent of serverless computing in different domains, there is a growing need for dynamic adaption to handle diverse and heterogeneous functions. However, serverless computing is currently limited to single Function-as-a-Service (FaaS) cloud deployment (such as AWS Lambda or Google Cloud Functions). Extending serverless computing to support functions deployments across multiple FaaS cloud deployments and dynamically load balancing the invocations of the functions across these multi-cloud deployments can provide an optimal way for handling such serverless functions. We call the network of these multi-FaaS cloud deployments as "Function Delivery Network” (FDN). In this talk, we introduce FDN and present a software system: "Courier" responsible for optimally distributing the invocations of the functions (called delivering of serverless functions) across the multiple FaaS cloud deployments based on the execution time of the functions on the single FaaS cloud deployment. We developed multiple approaches which use latency-aware scheduling to deliver serverless functions. We demonstrate the functioning of our developed tool using three FaaS platforms: 1) on-premise Open-Whisk, 2) AWS Lambda, and 3) Google Cloud Functions (GCF). We show the advantages of Courier in improving the overall performance and cost of the invocations of the functions.

1. Courier: Delivering Serverless Functions
across Multi-FaaS Cloud Deployments
Anshul Jindal
anshul.jindal@tum.de
Technical University of Munich (TUM), Germany
24th March 2022

2. 2
• Currently: Ph.D. @ Technical University of Munich
• Past: 2 Years @ Samsung Semiconductors
• I write Blogs
Who am I ?
• Follow @ GoogleScholar

3. 3
Introduction and
Motivation
Problem
Statement
FDN
- Create
- Manage
- Monitor
- Schedule
Usecase
Future Scope

4. Introduction &
Motivation
4

5. 5
Introduction: Serverless Computing

6. 6
Introduction: Multi-FaaS Cloud Deployments
AWS Lambda
256MB
AWS Lambda
512MB
AWS Lambda
1024MB
AWS Lambda
us-east
AWS Lambda
Europe-central
AWS Lambda
asia-pacific
Private
Cloud
FaaS Platform
AWS Lambda Cloud Functions
MEMORY-BASED FAASD REGION-BASED FAASD PLATFORM-BASED FAASD

7. 7
Using specialized architectures and hardware
Overcoming limitations of a Serverless platform and cloudbursting
Data-locality, data-privacy and not all cloud providers have data centers
in all regions
Introduction: Why Multi-FaaS Deployments?

8. Problem
Statement
8

9. 9
??
Problem: How to Deliver Functions Invocations?

10. Function
Delivery
Network (FDN)
10

11. 11
Edge Devices
Cluster
FDN Control
Plane (Courier)
Clients Invocations
FaaS Platform
FPGA Devices
Cluster
FaaS Platform
HPC Nodes
Cluster
FaaS Platform
Private
Cloud
Private Cloud
Cluster
FaaS Platform
AWS Lambda
Cloud Functions
Public Cloud
Cluster
Public Cloud
Cluster
Function Delivery Network (FDN)
SC
SC
SC SC
SC
SC
Functions Invocations
Delivering
Results
1Jindal, A, Gerndt, M, Chadha, M, Podolskiy, V, Chen, P. Function delivery network: Extending
serverless computing for heterogeneous platforms. Softw Pract Exper. 2021; 1– 28.
FDN: Function Delivery Network

12. 12
1
How to create Multi-FaaS Clusters?
2
How to manage these Clusters?
3
How to observe/monitor these Clusters?
4 How to schedule functions?
FDN: How?

13. 13
1
How to create Multi-FaaS Clusters?
2
How to manage these Clusters?
3
How to observe/monitor these Clusters?
4 How to schedule functions?
FDN: How?

14. 14
Roles
(Infrastructure-as-
a-Code)
Terraform
Master
Workers
Cloud
Service
Provider
Create Nodes
Create
Inventory
Inventory files
Manageme-
nt Node
SSH
SSH
Playbooks
1. Install Packages
2. Install Kubernetes
3. Deploy Monitoring …
1
3
2
4
5
6
6
FDN: Create Multi-FaaS Clusters

15. 15
Edge Cluster
MinIO
OpenFaaS
Master
Workers
K8s Cluster
Public Cloud Cluster
Public Cloud Cluster
MinIO
MinIO
Lambda
Cloud
functions
Private Cloud Cluster
MinIO
OpenWhisk
Master
Workers
K8s Cluster
(Infrastructure-as-a-
Code)
Terraform
Management
Node
FDN: Multi-FaaS Clusters

16. 16
1
How to create Multi-FaaS Clusters?
2
How to manage these Clusters?
3
How to observe/monitor these Clusters?
4 How to schedule functions?
FDN: How?

17. 17
etcd
Controller
Manager
Scheduler
Kube
apiserver
kubectl
Pods
Containers
Kubelet
Kube-proxy
Pods
Containers
Kubelet
Kube-proxy
Node 2
Node 1
LB Users
Master Node
K8s
Architecture
FDN: How to Manage Clusters?

18. 18
virtual-
kubelet
FDN: How to Manage Clusters?

19. 19
Public Cloud Cluster
MinIO
Lambda
Private
Cloud
Private Cloud Cluster
Public Cloud Cluster
MinIO MinIO
Cloud
functions
OpenWhisk
…
Lower Clusters
FDN: Clusters Management

20. 20
FDN: Clusters Management

21. 21
1
How to create Multi-FaaS Clusters?
2
How to manage these Clusters?
3
How to observe/monitor these Clusters?
4 How to schedule functions?
FDN: How?

22. 22
Private
Cloud
Private Cloud Cluster
Public Cloud Cluster
Public Cloud Cluster
virtual-kubelet
node
virtual-kubelet
node
virtual-kubelet
node
MinIO
MinIO
Lambda
MinIO
Cloud
function
s
OpenWhisk
Function Delivery
Network
FDN-
Monitor
FDN-
Monitor
FDN-
Monitor
FDN
Monitoring
Clients
FDN: Monitoring Clusters

23. 23
FDN: Monitoring Clusters

24. 24
1
How to create Multi-FaaS Clusters?
2
How to manage these Clusters?
3
How to observe/monitor these Clusters?
4 How to schedule functions?
FDN: How?

25. 25
1. FDN Modeler
o Functions Performance Modeling [1]
o Functions Interactions Modeling using
Temporal Point Processes RNNs [2]
o Functions Data-access Modeling [3]
2. FDN Scheduler:
o Latency-Aware Scheduling
o Functions-interaction-Aware Scheduling
o Data-Aware Scheduling
3. Load Balancer Design
o Based on HaProxy
o Maps function requests to clusters
o Function Delivering Strategies
FDN: Functions Scheduling (Courier)

26. 26
Function Delivery
Network
FDN Backend
Sidecar Function
Delivery Network #1
Sidecar Function
Delivery Network #n
• Courier Load Balancer Design
o Decentralized load balancing
o Based on HaProxy
o Maps function requests to clusters
o Function Delivering Strategies:
• Direct
• Round Robin
• Weighted Round Robin
• Auto Weighted Round Robin
(AWRR)
• Per-function Weighted Round Robin
(PFWRR)
FDN: Functions Scheduling (Courier LB)

27. 27
FDN: Functions Scheduling (Latency-Aware)

28. 28
FDN: Functions Scheduling (Data-Aware)
Private
Cloud
Private Cloud Cluster
Public Cloud Cluster
Public Cloud Cluster
virtual-kubelet
node
virtual-kubelet
node
virtual-kubelet
node
images
models
Lambda
Cloud
functions
OpenWhisk
Function Delivery
Network
FDN-
Monitor
FDN-
Monitor
FDN-
Monitor
Functions
Invokers
HTTP request
+ “X-FDN-Bucket: images” models
images models

29. 29
1
How to create Multi-FaaS Clusters?
2
How to manage these Clusters?
3
How to observe/monitor these Clusters?
4
How to schedule functions?
FDN: How?

30. 30
FDN
Monitoring
Clients
Private
Cloud
Private Cloud Cluster
Public Cloud Cluster
Public Cloud Cluster
virtual-kubelet
node
FDN Backend Server
virtual-kubelet
node
virtual-kubelet
node
Functions
Invokers
FDN-UI
Clients
Create and
Adapt Rules
MinIO
MinIO
Courier LB
Lambda
MinIO
Cloud
function
s
OpenWhisk
Function Delivery
Network
FDN
Monitor
FDN
Monitor
FDN
Monitor
Function Creator
/ Destroyer
FDN
Scheduler
FDN
Modeler
FDN: Delivering Functions Across Multi-Clusters

31. FDN-Usecase
31

32. 32
Usecase: Federated Learning

33. 33
Andreas Grafberger, Mohak Chadha, Anshul Jindal, Jianfeng Gu, and Michael Gerndt. "FedLess: Secure and Scalable
Federated Learning using Serverless Computing". In IEEE Conference on BigData (BigData'21), 2021.
Usecase: Federated Learning (FedLess)

34. In future
34

35. 35
FDN
Conclusion

36. 36
Extension to hetereogenous devices (FPGAs) and
more cloud platforms
Support for different types of applications.
Cost-aware delivering of functions
Future Scope

37. 37
Questions?
37
anshul.jindal@tum.de
/ansjin
/Anshul_Jindal4

38. 38
[1] Anshul Jindal, Mohak Chadha, Shajulin Benedict and Michael Gerndt. "Estimating the Capacities of
Function-as-a-Service Functions". In CloudAM ’21: IEEE/ACM 10th Workshop on Cloud and Edge Computing,
and Applications Management, 2021.
[2] Markus Steinbach, Anshul Jindal, Mohak Chadha, Michael Gerndt, Shajulin Benedict (2022). TppFaaS:
Modeling Serverless Functions Invocations via Temporal Point Processes. IEEE Access 2022; 1-26
[3] Christopher Peter Smith, Anshul Jindal, Mohak Chadha, Michael Gerndt, Shajulin Benedict. "FaDO: FaaS
Functions and Data Orchestrator for Multiple Serverless Edge-Cloud Clusters" In 2022 IEEE 6th International
Conference on Fog and Edge Computing (ICFEC), pp. 1-9. IEEE, 2022.
References