Engineering Serverless Workflow Applications in Federated FaaS.pdf

1. Engineering Serverless Workflow Applications in Federated FaaS Sashko Ristov

2. Agenda • About me • Federated FaaS challenges • Research questions • Conclusion S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 1/63

3. Positions at UIBK • (Jun. 2022 - present) Assistant Professor • Quality Engineering Group • (Apr. 2016 - Apr. 2022) Postdoctoral University Assistant • Distributed and Parallel Systems Group S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 2/63

4. Recent Publications in 2022 • Journals / Magazines ×4 • S. Ristov, D. Kimovski, T. Fahringer, "FaaScinating Resilience for Serverless Function Choreographies in Federated Clouds," in IEEE Trans. on Network and Service Management, 2022. • S. Ristov, S. Pedratscher, T. Fahringer, “xAFCL: Run scalable function choreographies across multiple FaaS systems,” in IEEE Transactions on Services Computing, pp. 1–1, 2022 • S. Pedratscher, S. Ristov, T. Fahringer, "M2FaaS: Transparent and Fault Tolerant FaaSification of Node.js Monolith Code Blocks", in Elsevier Fut. Gen. Comp. Sys., vol. 135, pp. 57–71, 2022. • D. Kimovski, S. Ristov, and R. Prodan, “Decentralized machine learning for intelligent health-care systems on the computing continuum,” Computer, vol. 55, no. 10, pp. 55–65, 2022. • Top Conferences ×3 • S. Ristov, M. Hautz, C. Hollaus, R. Prodan. “Simulate Serverless Workflows and Their Twins and Siblings in Federated FaaS,” ACM Symp. on Cloud Computing (SoCC’22), San Francisco, Nov. 2022 • S. Ristov and P. Gritsch. "FaaSt: Optimize makespan of serverless workflows in federated commercial FaaS," (IEEE CLUSTER ’22). Heidelberg, Germany, 182–194, 2022. • S. Ristov, S. Brandacher, M. Felderer, R. Breu, “GoDeploy: Portable Deployment of Serverless Functions in Federated FaaS”, IEEE Cloud Summit 2022, Fairfax, Virginia, USA, Oct. 2022 • Workshops Papers on Top-tier conferences×2 • S. Ristov, C. Hollaus, and M. Hautz. "Colder Than the Warm Start and Warmer Than the Cold Start! Experience the Spawn Start in FaaS Providers," in ApPLIED ’22). Salerno, Italy, 35–39, 2022. • M. Gusev, S. Ristov, et al. „CardioHPC: Serverless Approaches for Real-Time Heart Monitoring of Thousands of Patients“, WORKS’22 Workshop (Supercomputing) , Dallas, USA, Nov. 2022 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 3/63

5. Recent Collaborations with Uni-Klu • IEEE Transactions / Magazines • S. Ristov, D. Kimovski, T. Fahringer, "FaaScinating Resilience for Serverless Function Choreographies in Federated Clouds," in IEEE Trans. on Network and Service Management, 2022. • R. Matha, S. Ristov, T. Fahringer, and R. Prodan, “Simplified workflow simulation on clouds based on computation and communication noisiness,” in IEEE Transactions on Parallel and Distributed Systems, vol. 31, no. 7, pp. 1559–1574, 2020 • D. Kimovski, S. Ristov, and R. Prodan, “Decentralized machine learning for intelligent health-care systems on the computing continuum,” Computer, vol. 55, no. 10, pp. 55–65, 2022. • IEEE Magazines • S. Ristov, M. Hautz, C. Hollaus, R. Prodan. “Simulate Serverless Workflows and Their Twins and Siblings in Federated FaaS,” ACM Symp. on Cloud Computing (SoCC’22), San Francisco, Nov. 2022 • Top tier conferences • M. Gusev, S. Ristov, A. Amza, A. Hohenegger, R. Prodan, D. Mileski, P. Gushev, G. Temelkov "CardioHPC: Serverless Approaches for Real-Time Heart Monitoring of Thousands of Patients“, WORKS’22 Workshop (Supercomputing) , Dallas, USA, Nov. 2022 • Project Collaboration - CardioHPC • European High-Performance Computing Joint Undertaking, under grant agreement 951745 (FF4EuroHPC project and CardioHPC experiment) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 4/63

7. Federated FaaS Heterogeneity IBM Google AWS Alibaba S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 6/63

8. Free at last? We can compute everywhere! S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 7/63

9. But, Are We Really Free? Hurdles in all directions S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 8/63

10. Federated FaaS Challenges Such a heterogeneous environment. How to: • Deploy? • Model? • Orchestrate? • Optimize? • Run? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 9/63

11. Federated FaaS Challenges Such a heterogeneous environment. How to: • Deploy? Ristov et al. 22 (GoDeploy - IEEE Cloud Summit BEST PAPER) • Model? • Orchestrate? • Optimize? • Run? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 9/63

12. Federated FaaS Challenges Such a heterogeneous environment. How to: • Deploy? Ristov et al. 22 (GoDeploy - IEEE Cloud Summit BEST PAPER) • Model? Ristov et al. (SimLess - ACM SoCC’22) • Orchestrate? • Optimize? • Run? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 9/63

13. Federated FaaS Challenges Such a heterogeneous environment. How to: • Deploy? Ristov et al. 22 (GoDeploy - IEEE Cloud Summit BEST PAPER) • Model? Ristov et al. (SimLess - ACM SoCC’22) • Orchestrate? Ristov et al. 2021 (AFCL - Elsev. FGCS) • Optimize? • Run? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 9/63

14. Federated FaaS Challenges Such a heterogeneous environment. How to: • Deploy? Ristov et al. 22 (GoDeploy - IEEE Cloud Summit BEST PAPER) • Model? Ristov et al. (SimLess - ACM SoCC’22) • Orchestrate? Ristov et al. 2021 (AFCL - Elsev. FGCS) • Optimize? Ristov & Gritsch. 2022 (FaaSt - IEEE Cluster ’22) • Run? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 9/63

15. Federated FaaS Challenges Such a heterogeneous environment. How to: • Deploy? Ristov et al. 22 (GoDeploy - IEEE Cloud Summit BEST PAPER) • Model? Ristov et al. (SimLess - ACM SoCC’22) • Orchestrate? Ristov et al. 2021 (AFCL - Elsev. FGCS) • Optimize? Ristov & Gritsch. 2022 (FaaSt - IEEE Cluster ’22) • Run? Ristov et al. 2021 (xAFCL - IEEE TSC), S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 9/63

16. Federated FaaS Challenges Such a heterogeneous environment. How to: • Deploy? Ristov et al. 22 (GoDeploy - IEEE Cloud Summit BEST PAPER) • Model? Ristov et al. (SimLess - ACM SoCC’22) • Orchestrate? Ristov et al. 2021 (AFCL - Elsev. FGCS) • Optimize? Ristov & Gritsch. 2022 (FaaSt - IEEE Cluster ’22) • Run? Ristov et al. 2021 (xAFCL - IEEE TSC), 2022 (rAFCL - IEEE TNSM) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 9/63

17. Problem Statement 1: Vendor Lock-in IBM Google AWS Alibaba S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 10/63

18. Problem Statement 1: Vendor Lock-in AWS S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 10/63

21. Problem Statement 1: Vendor Lock-in AWS AWS Step Functions S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 10/63

22. Problem Statement 1: Vendor Lock-in AWS AWS Step Functions S3 Storage S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 10/63

23. Freedom?! S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 11/63

24. Freedom?! Limited freedom (At Home) - The current state S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 11/63

25. Freedom?! We want full freedom S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 11/63

26. Problem Statement 2: Limited Concurrency xAFCL UIBK AWS Frankfurt 700 ms S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 12/63

27. Problem Statement 2: Limited Concurrency xAFCL UIBK AWS Frankfurt 1,400 ms 700 ms S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 12/63

28. Problem Statement 2: Limited Concurrency xAFCL UIBK AWS Frankfurt 1,400 ms 700 ms AWS North Virginia 1,000 ms S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 12/63

29. Problem Statement 2: Limited Concurrency xAFCL UIBK AWS Frankfurt 1,400 ms 700 ms AWS North Virginia 1,000 ms AWS Tokyo 1,200 ms S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 12/63

30. Problem Statement 2: Limited Concurrency Google North Virginia 300 ms xAFCL UIBK AWS Frankfurt 1,400 ms 700 ms AWS North Virginia 1,000 ms AWS Tokyo 1,200 ms S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 12/63

32. Research Goal RG1 - Abstraction Research goal RG1 - Abstraction Can we abstract the federated FaaS and develop distributed applications as they run on a single computer? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 14/63

33. Research Goal RG1 - Abstraction Research goal RG1 - Abstraction Can we abstract the federated FaaS and develop distributed applications as they run on a single computer? Research question RQ1: Portability S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 14/63

34. Research Goal RG1 - Abstraction Research goal RG1 - Abstraction Can we abstract the federated FaaS and develop distributed applications as they run on a single computer? Research question RQ1: Portability • How to abstract heterogeneous resources and hide them from developers? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 14/63

35. Research Goal RG1 - Abstraction Research goal RG1 - Abstraction Can we abstract the federated FaaS and develop distributed applications as they run on a single computer? Research question RQ1: Portability • How to abstract heterogeneous resources and hide them from developers? • Can we easily migrate and run serverless functions in federated clouds? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 14/63

36. Portability Levels S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 15/63

37. SOTA (b) vs xAFCL (c) Portable Approach (per function) AWS Lambda IBM Cloud Functions Google Cloud Functions AWS Step Functions IBM Composer Google Cloud Composer IBM FC Developer AWS FC Developer Google FC Developer AWS Lambda IBM Cloud Functions FaaS systems AFCL AFCL FC Developer FC system a) c) b) AWS Lambda IBM Cloud Functions Google Cloud Functions xAFCL xAFCL FC Developer … b) S. Ristov, S. Pedratscher, T. Fahringer: AFCL: An Abstract Function Choreography Language for serverless workflow specification. Future Generation Computer Systems 114: 368-382 (2021) c) S. Ristov, S. Pedratscher and T. Fahringer, "xAFCL: Run Scalable Function Choreographies Across Multiple FaaS Systems," in IEEE Transactions on Services Computing, 2021. doi: 10.1109/TSC.2021.3128137. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 16/63

38. Resilient AFCL (per function) - rAFCL AWS Lambda IBM Cloud Functions Google Cloud Functions FC systems FaaS systems AWS Step Functions IBM Composer Google Cloud Composer IBM FC Developer AWS FC Developer Google FC Developer AWS Lambda IBM Cloud Functions Google Cloud Functions FaaS systems ftAFCL ftAFCL FC Developer FC system a) b) x x x x x x x x x x x x x S. Ristov, D. Kimovski, T. Fahringer, "FaaScinating Resilience for Serverless Function Choreographies in Federated Clouds," in IEEE Transactions on Network and Service Management, 2022. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 17/63

39. rAFCL Architecture rAFCL Environment FCDev rScheduler resilience model AFCL Parser fRepository Log AF AFCL alternative rAFCL Multi-FaaS rEE Invoke Monitor ... AWS Monitor IBM Monitor FaaS Invoker ... S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 18/63

40. Example rAFCL Scheduler AP0 AP1 AP2 AP3 Omitted 70 80 90 100 99.89 99.11 99.01 97.8 95.45 86.67 69.95 aj = 99.5% 99.89 99.99 99.9 93.35 82.88 87.89 75.77 99.89 98.75 Availability (%) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 19/63

41. AFCL Flexible Resources and Fault Tolerance Flexible function deployment and alternative functions properties: - name: "resource" value: "ARN..." #Run on AWS # value: "https://europe-west6-xxx.cloudfunctions.net..." #Run on Google Zurich constraints: - name: "FT-Retries" #Retry 2 times alternatives of this function. Ignore provider's setup. value: "2" - name: "FT-AltPlan-0" #If AWS fails, retry two tasks on IBM Japan and London (duplicate) value: "0.9879;https://jp-tok...;https://eu-gb...;" - name: "FT-AltPlan-1" #If both tasks (jp and gb) fail, run alternatives on Fra. and Dallas value: "0.9808;https://eu-de...;https://us-south...;" S. Ristov, D. Kimovski, T. Fahringer, "FaaScinating Resilience for Serverless Function Choreographies in Federated Clouds," in IEEE Trans. on Network and Service Management, 2022. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 20/63

42. FaaS Concurrency Overhead Model 0 1 2 3 4 5 6 7 0 500 1,000 outliers load burst release Time (s) Concurrent functions S. Ristov, S. Pedratscher and T. Fahringer, "xAFCL: Run Scalable Function Choreographies Across Multiple FaaS Systems," in IEEE Transactions on Services Computing, 2021. doi: 10.1109/TSC.2021.3128137. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 21/63

43. AFCL Example - function: properties: - name: "resource" value: "arn:aws:lambda:xxxx" # (AWS) # value: "xxxx.functions.appdomain.cloud/xxxx" # (IBM) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 22/63

44. AFCL Simple Example S. Ristov, S. Pedratscher, T. Fahringer: AFCL: An Abstract Function Choreography Language for serverless workflow specification. Future Generation Computer Systems 114: 368-382 (2021) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 23/63

45. FCEditor: Complex Workflows FCEditor: almost two years before AWS S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 24/63

46. Some Experiment Setup Example with xAFCL S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 25/63

47. What about functions with BaaS f service dynamic URL Region 1 Region 2 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 26/63

48. RQ2: Interoperability Research question RQ2: Interoperability • How to abstract cloud services within a serverless function, regardless where it is hosted? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 27/63

49. RQ2: Interoperability Research question RQ2: Interoperability • How to abstract cloud services within a serverless function, regardless where it is hosted? • Use Google Cloud Storage instead of AWS S3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 27/63

50. RQ2: Interoperability Research question RQ2: Interoperability • How to abstract cloud services within a serverless function, regardless where it is hosted? • Use Google Cloud Storage instead of AWS S3 • Use Google Vision instead of AWS Rekognition, regardless where the image is located (AWS S3 or Google Cloud Storage) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 27/63

51. SOTA Approach: 4 Deployments for 2 Cloud Providers access deploy access deploy deploy access deploy access S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 28/63

52. 2 Chained Services = 8 Deployments 4 Lambda and 4 Google functions that: • use AWS S3 and AWS Rekognition • use AWS S3 and Google Vision • use GCS and AWS Rekognition • use GCS and Google Vision S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 29/63

53. Let’s start with some challenging game Exchange places of black and white knights with minimum number of moves! S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 30/63

54. Let’s start with some challenging game Exchange places of black and white knights with minimum number of moves! 40 moves are needed! S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 30/63

55. Let’s use Homothety 2 1 4 10 8 6 5 9 7 3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 31/63

58. Let’s use Homothety 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 31/63

59. Let’s use Homothety 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 31/63

60. Let’s use Homothety 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 31/63

61. Let’s use Homothety 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 31/63

62. Let’s use Homothety 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 31/63

63. Let’s use Homothety 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 31/63

64. Let’s use Homothety 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 2 1 4 10 8 6 5 9 7 3 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 31/63

65. Long-term Goal • Can we hide cloud providers, services, and resources? • Can we abstract Federated FaaS heterogeneity as the Von Neumann’s architecture? • The current state-of-the-art approaches are towards "code once, run everywhere" • I plan to go even further: Can we code once, run everywhere, but also with everything (every cloud service)? • even during runtime, without code rewriting and redeployment! S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 32/63

66. Long-term Goal • Can we hide cloud providers, services, and resources? • Can we abstract Federated FaaS heterogeneity as the Von Neumann’s architecture? • The current state-of-the-art approaches are towards "code once, start everywhere" • I plan to go even further: Can we code once, run everywhere, but also with everything (every cloud service)? • even during runtime, without code rewriting and redeployment! S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 32/63

67. Homothety: Federated FaaS to Von Neumann OS Computer App1 App2 APPs NET FS S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 33/63

68. fService Programming Model: fStorage access deploy access fStorage {"source": "https://bucket.s3.region.amazonaws.../..."} {"source": "https://storage.google.cloud.com/.../..."} Supported languages: Java, Python, Go. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 34/63

69. fService Architecture fStorage.copy(bucketUrlAWS, "/tmp/destFile"); fStorage.copy("/tmp/sourceFile", bucketUrlGoogle); fStorage.copy(bucketUrlGoogle, bucketUrlAWS); fRecognition.detectFaces("path/to/image.jgp") Unpublished work yet. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 35/63

70. fService Chained Approach: fRecognition + fStorage AWS Rekognition Google Storage AWS S3 Google Vision The red arrows show the fastest runtime. Supported languages: Java. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 36/63

71. Results with fStorage + fScheduler Europe US S3 O AWS N GCF F GCS F xAFCL UIBK AWS O S3 N Asia GCS S GCF S input files are scattered faster is to use GCF F and even AWS N, although there is no input data. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 37/63

72. Research Goal RG2 - Runtime System Optimization Research goal RG2 - Runtime System Optimization Can we create a "no-learning" unified system model and optimize serverless applications in federated clouds? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 38/63

73. Research Goal RG2 - Runtime System Optimization Research goal RG2 - Runtime System Optimization Can we create a "no-learning" unified system model and optimize serverless applications in federated clouds? Research question RQ3: FaaS system model S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 38/63

74. Research Goal RG2 - Runtime System Optimization Research goal RG2 - Runtime System Optimization Can we create a "no-learning" unified system model and optimize serverless applications in federated clouds? Research question RQ3: FaaS system model • Which parameters are important? Which features of functions, runtime systems, and the underlying resources should be formalized to estimate functions’ behavior? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 38/63

75. Research Goal RG2 - Runtime System Optimization Research goal RG2 - Runtime System Optimization Can we create a "no-learning" unified system model and optimize serverless applications in federated clouds? Research question RQ3: FaaS system model • Which parameters are important? Which features of functions, runtime systems, and the underlying resources should be formalized to estimate functions’ behavior? • Can we determine the behavior of not even deployed function in AWS Frankfurt based on the same function in AWS Tokyo? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 38/63

76. SOTA for RQ3 • Running benchmark applications • Too expensive for the computing continuum • ML approaches • Maybe cheaper, but still too expensive • Requires time series S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 39/63

77. Method: Twins and Siblings Tokyo 128MB SIBLING 512MB TWIN 128MB Frankfurt S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 40/63

78. Innovative Approach: RTT = ET + O ET: Exec. time O: Overhead RTT: Round Trip Time SO: Session AO: Authentication FO: FaaS CO: Concurrency NO: Network O: Overhead S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 41/63

79. FC Simulation Model RTT = O + ET (1) O = SO + NO + AO + FO + CO (2) AO = ( cr + 3 · NO, 2-way authentication; cr + 2 · NO, 1-way authentication. (3) CO = (k − 1) · d (4) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 42/63

80. FC Simulation Model: Twins and Siblings RTTS = (RTT-O) x s(m)/m + O RTTT = RTT - O + OT 128MB SIBLING 512MB TWIN 128MB xAFCL SimLess run Tokyo Frankfurt O ET RTT OS = O ETS = ET x s(4)/4 ETT = ET O ET = RTT - O OT > O simulate twins simulate siblings OT S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 43/63

81. Cloud Regions for Learning and Validation Goal Cont. AWS IBM Google 3*Learn EU1 AF IF GB US1 AV IW GV AP1 AT IT GT 3*Validate EU2 AL IL GL US2 AC ID GI AP2 AS IS GH AF MC10 MC1000 w Auth. no-op w Auth. no-op w/o Auth. CloudPing gcPing xAFCL NO xAFCL FO, SO, xAFCL AO, cr CO SimLess RTTT O RTT, M ~ AT AV twins siblings RTTS d ~ M S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 44/63

82. Parameter Setup Region SO NO cr AO FO O e O δO[%] AF 3*550 30 3*76 166 3*30 226 189 19.6 AV 140 496 666 676 1.48 AT 267 877 1174 1200 2.17 IF 3*152 30 3*76 136 3*57 223 204 9.31 IW 140 356 553 577 4.16 IT 267 610 934 930 0.43 GB 3*112 59 3*– 3*– 3*23 82 95 13.7 GV 131 154 131 17.6 GT 275 298 307 2.93 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 45/63

83. Overhead Accuracy AF AV AT AL AC AS IF IW IT IL ID IS GB GV GT GL GI GH Region 0 200 400 600 800 1000 1200 1400 O (ms) 0% 5% 10% 15% 20% 25% 30% δ O O f O δO S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 46/63

84. Evaluated Tools S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 47/63

85. Low Concurrency MC10 and BWA20 AF AL AV AC IF IL IW ID GB GL GV GI Region 0 5 10 15 20 25 30 RTT T / M (s) 0% 5% 10% 15% 20% 25% ± RTTT g RTTT M f M ±RTT ±M Simulated versus measured RTT and makespan of monteCarlo twins, and their inaccuracy. S p l i t I n d e x A L N 1 A L N 2 S a m p e M S p l i t I n d e x A L N 1 A L N 2 S a m p e M Functions and total Makespan 0 10 20 30 40 RTT T / M (s) 0% 5% 10% 15% 20% 25% 30% ± RTT AV from AF AT from AF RTTT, M g RTTT, f M ±RTT Simulated versus measured RTT of BWA20 twins and their inaccuracy S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 48/63

86. High Concurrency MC1,000 and BWA2,500 0 200 400 600 800 1000 functions 16 17 18 19 20 ct (s) 5 6 7 ± ct (%) f ct ct ±ct Simulated versus measured completion time of MC1,000 functions on AF and their inaccuracy. 0 500 1000 1500 2000 2500 functions 10 20 30 40 ct (s) 0 10 20 30 40 ± ct (%) f ct ct ±ct Simulated vs. measured completion time ct of BWA2,500 functions on AF and their inaccuracy. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 49/63

87. Research Question RQ4 Research question RQ4 Can serverless workflow applications benefit from executing individual functions across different FaaS systems in the federated FaaS? S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 50/63

88. SOTA1 for RQ4: Serverful Scheduling • Scheduling of serverful workflow applications that run on VMs S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 51/63

89. SOTA1 for RQ4: Serverful Scheduling • Scheduling of serverful workflow applications that run on VMs • Not applicable for FaaS • Resources are selected during deployment, not during runtime • Iterate over functions, then look for possible function deployments, rather than VMs. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 51/63

90. SOTA2 for RQ4: FaaS Scheduling • Many container-based algorithms schedule individual functions on specific container (executor) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 52/63

91. SOTA2 for RQ4: FaaS Scheduling • Many container-based algorithms schedule individual functions on specific container (executor) • within a single region on open source FaaS platforms (e.g. OpenWhisk) S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 52/63

92. SOTA3 for RQ4: FC Distribution in Federated FaaS federated clouds single region pFor N = 1000 f2 f3 f4 f1 pFor N = 600 f2 f3 f4 f1 pFor N = 400 f2 f3 f4 f1 federated clouds pFor N = 500 f2 f3 f4 f1 pFor N = 400 f2 f3 f4 f1 pFor N = 100 f2 f3 f4 f1 a) per function single region b) per FC federated clouds c) per function federated clouds • a) only f1 and f4 can run 1,000 instances concurrently, but not f2 and f3 • b) cannot exploit f1 runs faster on blue, while f4 on the left (orange). • c) schedules f1 on blue, f4 on orange. S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 53/63

93. FaaS Deployment Model • Function type - WHAT S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 54/63

94. FaaS Deployment Model • Function type - WHAT • high-level of abstraction • data inputs and outputs, semantics • FC dependencies (data-flow, control-flow) • YAML, JSON pFor N = 1000 f2 f3 f4 f1 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 54/63

95. FaaS Deployment Model • Function type - WHAT • high-level of abstraction • data inputs and outputs, semantics • FC dependencies (data-flow, control-flow) • YAML, JSON • Function deployment - HOW and WHERE • real application / code • e.g. python, for AWS, algorithm • code + resources • e.g. region, memory, handler function pFor N = 1000 f2 f3 f4 f1 pFor N = 500 f2 f3 f4 f1 pFor N = 400 f2 f3 f4 f1 pFor N = 100 f2 f3 f4 f1 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 54/63

96. FaaS Abstract Resource Model • R regions • cl - concurrency limit • a user can run cl functions, regardless of assigned memory • Abstract resource ar S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 55/63

97. FC Scheduler r1 fd11 fd21 r2 fd22 fd12 FT1 (Split) FT2 (Index) FT f f1 f2 f3 f4 f5 Split3 Split2 Split1 Index1 Index2 FDs & Regions AR AR1 ar12 ar11 ar14 f1-> fd11 f2-> fd11 f5-> fd21 ar13 AR2 ar21 ar22 f3-> fd12 f4-> fd22 ar23 S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 56/63

98. Algorithm 1: FC scheduling algorithm Output: schedFC = {(fi, sched(fi, fdij, arrk))|∀fi ∈ F}; // FC schedule 1 Function (FC, FD, AR): 2 Rank ← B-Rank(F); // order the tasks according to B-rank 3 schedFC ← ∅ ; // Initialize FC schedule 4 for i ← 1 to N do // Iterate over each fi 5 Tmin ← ∞; fdmin ← 0; ct(fi) ← ∞; // Initialization 6 for j ← 1 to Mi do // Iterate over each fdij of fi 7 r ← reg(fdij); // find the region of fdij 8 for k ← 1 to clr do // Iterate over each arrk of rr 9 Tj ← EST(fdij, arrk); // Compute EST 10 ct(fi) ← Tj + RTTij; // compute completion time 11 if ct(fi) < Tmin then 12 Tmin ← ct(fi) ; // Save min. completion time 13 fdmin ← j; // Save the fdij for fi 14 armin ← k; // Save the arrk for fdij 15 end 16 end 17 end 18 schedFC ← schedFC ∪ sched(fi, fdmin, armin); // schedule fi 19 end 20 return S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 57/63

100. Summary - Scenarios S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 59/63

101. Summary - Approaches S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 60/63

102. Summary - SOTA S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 61/63

103. Further Investigation in 2022/2023 Storage Interoperability + Data-aware Scheduler • copy(srcURL, destURL) (Java, Python, Go1 ) • Dynamically select storage during runtime, without redeployment DeployLess Model and Scheduler Functions Size AF AN AT Split, Index, AlnR1, AlnR2 0.17 MB 1.8 s 2.7 s 4.5 s Sampe, Merge, Sort 1.7 MB 2.0 s 3.1 s 5.6 s 1 https://github.com/FaaSTools/GoStorage S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 62/63

104. QUESTIONS? Email: sashko.ristov@uibk.ac.at S. Ristov: Engineering Serverless Workflow Applications in Federated FaaS | TEWI-Kolloquium, Uni-Klu | 2022-11-29 63/63

Engineering Serverless Workflow Applications in Federated FaaS.pdf

You are reading a preview.

Check these out next

Engineering Serverless Workflow Applications in Federated FaaS.pdf

More Related Content

Similar to Engineering Serverless Workflow Applications in Federated FaaS.pdf (20)

More from Förderverein Technische Fakultät (20)

Recently uploaded (20)