Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

RECAP: The Simulation Approach

27 views

Published on

This presentation was delivered by Patricia Endo (DCU) and Sergej Svorobej (DCU) at the IC4 research seminar on November 15, 2018 in Dublin, Ireland.

Published in: Software
  • Be the first to comment

  • Be the first to like this

RECAP: The Simulation Approach

  1. 1. Reliable Capacity Provisioning and Enhanced Remediation for Distributed Cloud Applications http://recap-project.eu recap2020 THIS PROJECT HAS RECEIVED FUNDING FROM THE EUROPEAN UNION’S HORIZON 2020 RESEARCH AND INNOVATION PROGRAMME UNDER GRANT AGREEMENT NUMBER 732667 RECAP (Reliable Capacity Provisioning and Enhanced Remediation for Distributed Cloud Applications): The Simulation Approach Sergej Svorobej, Patricia Takako Endo (DCU)
  2. 2. Cloud data centres 2 • Pros: • Large scale • (almost) Unlimited resources • Cons: • Too far away • Slow • High network cost
  3. 3. BT Landscape on UK map
  4. 4. • RECAP is a 3-years (2017- 2019) project composed of: ⁃ 10 partners in 6 countries • Overall budget 4.6 million € RECAP Consortium 4
  5. 5. RECAP Goals • RECAP is a EU-funded research project to develop the next generation of distributed cloud, edge and fog computing systems • RECAP is creating a novel concept for the provisioning and management of cloud services ⁃Services are automatically instantiated ⁃Services are elastically deployed close to the users ⁃Services are self-configurable 5
  6. 6. RECAP Management Orchestration Remediation Optimisation Simulation Application Orchestration RECAP Operations Infrastructure Optimisation RECAP Data RECAP Data Data Collection and Analysis RECAP Models Infrastructure Model Applications Model Workload Models IOE / IOT EDGE CORE CLOUD DATA CENTRE RECAP Approach
  7. 7. IOE / IOT EDGE CORE CLOUD DATA CENTRE RECAP Use Cases • Use Case A: Infrastructure and Network Management • Use Case B: Big Data Analytics Engine • Use Case C: Edge/Fog Computing for Smart Cities • Use Case D.1: Virtual Content Distribution Networks • Use Case D.2: Network Function Virtualization
  8. 8. RECAP Simulation Framework • Develop the RECAP Simulation Framework able to ⁃Make reproducible and controllable experimentation • Large scale and geographically distributed scenarios • Physical and virtual resource capacity • Different applications’ deployment ⁃Assist the RECAP Optimisation Framework • Evaluate different deployment and infrastructure management alternatives • Cost, energy, resource allocation and utilization 8
  9. 9. RECAP Simulation Framework 9
  10. 10. RECAP Simulation Framework • Setup of a simulation experiment ⁃Models created from monitoring data ⁃Define • Application • Infrastructure • Workload ⁃Experiment configuration 10
  11. 11. RECAP Simulation Framework • Integration ⁃API to integrate • Models • Experiment Configurations • Optimisation Framework ⁃Simulation Manager • Send experiments’ configuration for the correspondent simulation approach 11
  12. 12. RECAP Simulation Framework • DES (Discrete Event Simulation) ⁃Granularity ⁃Based on CloudSim ⁃Use cases • Use Case A: Infrastructure and Network Management (TIETO) • Use Case B: Big Data Analytics Engine (Linknovate) • Use Case D.2: Network Function Virtualization (BT) 12
  13. 13. RECAP Simulation Framework • DTS (Discrete Time Simulation) ⁃Scalability ⁃Based on CloudLightning Simulator ⁃Use cases • Use Case C: Edge/Fog Computing for Smart Cities (SATEC) • Use Case D.1: Virtual Content Delivery Networks (BT) 13
  14. 14. DES approach 14
  15. 15. RecapSim DES architecture 15 CloudSim Event Coordinator Model Mapper Simulation Manager Model Generators API RecapCloudlet RecapVE RecapHost Recap DC Broker Recap VE Allocation PolicyLVL 1 LVL 2
  16. 16. API API APPLICATION COMPONENT 1 A 300 B 500 C 1000 LB COMPONENT 1’ A 300 B 500 C 1000 Request: ID Name AppID ApiId Data amount to send LB LB broker (one for each app): 1) Get request(task) app id, component ID, and the API ID 2) Get VM utilisation from the right app with the data 3) Send task to the least utilised (or other logic) Task: 1)When request reaches the API execute task with API specified load 2) When task completed 3) Get next API call in the chain 4) Create task 5) Get where to send it from LB RequestDevice Device: ID Name Data amount to send Tuple <Time,Geolocation> Tuple <Time,Request> API API COMPONENT 2 A 500 B 450 C 100 COMPONENT 2’ A 500 B 450 C 100 Application model
  17. 17. DES Summary ⁃Discrete Even Simulation ⁃RecapSim based on CloudSim framework ⁃Pros: • Granular analysis • High level modelling (application modelling) ⁃Cons: • Hard to parallelise • Memory intense 17
  18. 18. DTS approach 18
  19. 19. DTS approach • Timestep based ⁃At each timestep, a list of requests is introduced in the simulator • Update system components’ status • Calculate metrics • Topology is a graph mapped as a sparse matrix ⁃Allow to easily run through the all components of the network • Advantages ⁃Remove the need of pre-computation and storage of events ⁃Reduce the memory requirement to execute simulations ⁃Allow the modelling and simulation of large scale scenarios 19
  20. 20. Metrics • Metrics ⁃Cache hits/misses ⁃Accepted/rejected requests ⁃Energy consumption (GWh) ⁃Active VMs/links ⁃Network utilization ⁃CPU utilization 20
  21. 21. Challenges • Most computationally demanding tasks ⁃Update of each system component status ⁃Calculation and aggregation of all metrics given the timestep • Small value: the output size increases and simulation performance decreases • Large value: results can be in under-sampling of the behavior, neglecting possible transient phenomena emerging during simulation ⁃We are improving the DTS by parallelising these tasks 21
  22. 22. BT topology and vCDN use case 22 Inbound connection Outbound connection VM 1 VM M… Node N VM 1 VM M… Node 1 … Content 1 Content C … Inner core does not have inbound connection because it retains all of the content Inner core 1 Inner core IC Outer core 1 Outer core 2 Outer core OC Metro MC Metro 1 Metro 2 MSAN M MSAN 3 MSAN 4 MSAN 2 MSAN 1 … … … … Metro 3
  23. 23. Node and VM model • Each node has predefined vCPUs, memory and storage • Each VM, i, requires part of a node resource defined as: ⁃Required number of vCPUs 𝐶𝑖 𝑉𝑀 ⁃Required amount of memory in GBytes 𝑀𝑖 𝑉𝑀 ⁃Required amount of storage in GBytes 𝑆𝑖 𝑉𝑀 • Each VM, i, has a maximum number of users it can service 𝑈𝑖 𝑉𝑀 23
  24. 24. DTS mapping 24 Simulator Sites Content Engine IC 2 OC 2 Metro 2 MSAN 2 IC 1 OC 1 Metro 1 MSAN 1 Graph Power Consumption Engine Request Creation Engine Statistic Engine Site Inboundconnection Outboundconnection Cache Node 1 VM 2 VM 1 Rq 1 Rq 1 Node 2 VM 2 VM 1 Rq 1 Rq 1 Sharedmemoryparallelization
  25. 25. Scalability results • Scenario configuration ⁃BT network topology (1132 sites) • 8 inner core sites • 12 outer core sites • 86 metro sites • 1026 MSAN sites ⁃Nodes per site: 2 ⁃Simulation time: 10000s ⁃Average request duration: 300s ~ 1000s ⁃Number of requests: from 300180 to 1602203 • Request distribution is uniform (MSAN sites) 25
  26. 26. Scalability result 26 300180 497575 702435 1001173 1190238 1602203 Thread=1 10.6654 17.1852 22.9579 29.4356 31.1693 40.1331 Thread=2 6.6926 10.2716 13.4838 17.2697 18.2595 23.4933 Thread=4 3.9186 5.9137 7.7218 9.8536 10.4059 13.2601 Thread=6 2.7837 4.1934 5.4371 6.9507 7.4059 9.6385 Thread=12 2.4256 3.3866 4.1796 4.7037 5.3789 6.7748 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 40.00 45.00 Elapsedtime(s) Number of requests Thread=1 Thread=2 Thread=4 Thread=6 Thread=12
  27. 27. Next steps 27
  28. 28. Future Work • Final RECAP Simulation Framework ⁃Improve core simulation framework capabilities • Simulation models • Parallelisation process • Simulation results ⁃Integration progression • Further model support via API • Recap central DB integration ⁃Optimisation alignment • Migration and scalability decision support 28
  29. 29. THANK YOU http://recap-project.eu recap2020 RECAP Project ■ H2020 ■ Grant Agreement #732667 Call: H2020-ICT-2016-2017 ■ Topic: ICT-06-2016 THIS PROJECT HAS RECEIVED FUNDING FROM THE EUROPEAN UNION’S HORIZON 2020 RESEARCH AND INNOVATION PROGRAMME UNDER GRANT AGREEMENT NUMBER 732667

×