Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The RECAP Project: Large Scale Simulation Framework

166 views

Published on

In this presentation, Sergej Svorobej (DCU) gave a brief overview of RECAP and introduced the large scale simulation framework used in the project. The event was held in conjunction with the National Conference on Cloud Computing and Commerce (http://2018.nc4.ie/) and took place April 10, 2018 in Dublin, Ireland.

Learn more about RECAP: https://recap-project.eu/

Published in: Technology
  • Be the first to comment

  • Be the first to like this

The RECAP Project: Large Scale Simulation Framework

  1. 1. Reliable Capacity Provisioning and Enhanced Remediation for Distributed Cloud Applications http://recap-project.eu recap2020 THIS PROJECT HAS RECEIVED FUNDING FROM THE EUROPEAN UNION’S HORIZON 2020 RESEARCH AND INNOVATION PROGRAMME UNDER GRANT AGREEMENT NUMBER 732667 REliable CApacity Provisioning for Distributed Cloud/Edge/Fog Computing Applications (RECAP) Sergej Svorobej Dublin City University
  2. 2. Agenda • Introduction • The RECAP Vision • Use Cases for RECAP • DCU - Simulation Framework • Intel 2
  3. 3. What is RECAP? • Reliable Capacity Provisioning and Enhanced Remediation for Distributed Cloud/Edge/Fog Applications • Founded under European Horizon 2020 framework • Jan 2017 – Dec 2019 • Consortium comprises many academic and industrial partners ⁃ 9 partners from 5 countries (Germany, Ireland, Spain, Sweden, UK) • http://recap-project.eu 3
  4. 4. Partner Location 4
  5. 5. Project Motivation • Large-scale systems are typically built as distributed systems • Tradeoffs in the placement of application components: ⁃ Data center High latency, high power ⁃ Fog/Edge Low latency, low power • Cloud computing capacity is provisioned using best-effort models and coarsed- grained QoS mechanisms ⁃ Not a sustainable way as the number of connected ”things” increases 5
  6. 6. Realisation Approach (1/2) • An architecture for cloud-edge computing capacity provisioning and remediation ⁃ Fine-grained and accurate models of application behaviour and deployment ⁃ Model of QoS requirements at application component-level ⁃ Model of workloads • To understand and predict the behaviour of applications (users) • To enhance the proactive remediation of systems 6
  7. 7. Realisation Approach (2/2) 7
  8. 8. Architecture • Feedback Loop ⁃ Collector ⁃ Application Modeler ⁃ Workload Modeler ⁃ Optimizer ⁃ Simulator 8
  9. 9. The RECAP Collector • Gathers, synthesizes and analyzes metrics to be monitored across the infrastructure • Acquires, characterizes, and analyzes data ⁃ Workload patterns and their relationship ⁃ Status of the infrastructure ⁃ … • Visualizes, annotates,archives and manages the collected data 9 Knowledge Discovery
  10. 10. The RECAP Application Modeler • With the knowledge provided by the Collector, the Modeler discovers and defines ⁃ The internal structure of cloud applications ⁃ The QoS requirements for each applications • To support intelligent decision making ⁃ Application/Component placement and autoscaling 10 Application Modeling
  11. 11. The RECAP Workload Modeler • Decomposes, classifies, and predicts the workloads in the network, and the load propagation in applications ⁃ CPU, memory, network traffic, … • Models the workload distribution and load propagation patterns • To improve planning decision and ensure QoS • To support the construction of an artificial workload generation tool ⁃ Validating and training the learning models 11 Workload Modeling
  12. 12. The RECAP Optimizer • With the models provided by the Modelers, the Optimizer performs optimization tasks ⁃ Application/Component placement: scaling vs. migration ⁃ Infrastructure management decisions: energy, utilization rate, load balancing • To improve the efficiency in resources utilzation while maintaining QoS 12 Optimization
  13. 13. The RECAP Simulator • Simulation is needed due to the size and complexity of the intended target systems ⁃ Simulateinteractions of distributed cloud application behaviors ⁃ Emulate data center and network systems • To assist the Optimizer with the evaluation of different deployments of applications and infrastructures • To feed back to the Data Collector with simulation results 13 Testing & Improvement
  14. 14. Use Cases from Industry • BT ⁃ NFV, QoS Management and Remediation • Linknovate ⁃ Complex Big Data Analytics Engine • Satec ⁃ Fog Computing and Large Scale IoT Scenario for supporting Smart Cities • Tieto ⁃ Infrastructure and Network Management 14
  15. 15. Reliable Capacity Provisioning and Enhanced Remediation for Distributed Cloud Applications http://recap-project.eu recap2020 THIS PROJECT HAS RECEIVED FUNDING FROM THE EUROPEAN UNION’S HORIZON 2020 RESEARCH AND INNOVATION PROGRAMME UNDER GRANT AGREEMENT NUMBER 732667 WP7 – Large Scale Simulation Framework Sergej Svorobej Dublin City University
  16. 16. 16
  17. 17. Role of simulation • Reason on implementation effectiveness • Show optimisation effects ⁃ Placement ⁃ Consolidation ⁃ Elasticity ⁃ Infrastructure and Application configuration ⁃ Cost • Provide value to the use case owners 17
  18. 18. 18 API API APPLICATION COMPONENT 1 A 300 B 500 C 1000 LB COMPONENT 1’ A 300 B 500 C 1000 LB RequestDevice API API COMPONENT 2 A 500 B 450 C 100 COMPONENT 2’ A 500 B 450 C 100 High level model DeviceDevice/ User Request Request Network Node Node Node Node Node Node Node Node Node Network Network Tier 1 Metro Core
  19. 19. Simulation challenges • Experiment model size ⁃ Network topology (Graph) ⁃ Infrastructure (Physical and Virtual) ⁃ Workload ⁃ Application • Speed and resource demand • Accuracy and granularity 19
  20. 20. Research direction • Discrete Event Simulation (DES) ⁃ Event Queue simulation engine ⁃ Java 8 ⁃ CloudSimPlus based (http://cloudsimplus.org/) • “Time step” simulation ⁃ Calculation loop ⁃ C/C++, MPI based ⁃ CLSim (https://cloudlightning.eu/) 20
  21. 21. THANK YOU http://recap-project.eu recap2020 RECAP Project ■ H2020 ■ Grant Agreement #732667 Call: H2020-ICT-2016-2017 ■Topic: ICT-06-2016 THIS PROJECT HAS RECEIVED FUNDING FROM THE EUROPEAN UNION’S HORIZON 2020 RESEARCH AND INNOVATION PROGRAMME UNDER GRANT AGREEMENT NUMBER 732667
  22. 22. Additional Slides 22
  23. 23. BT 23 • NFV infrastructure and virtual CDNs ⁃ Softwarization of network appliances ⁃ Mulitiple distributed applications per CDN operator • RECAP: ⁃ Automated decision making • Optimization of placement and scaling vCDN systems • Monitoring and Remediation ⁃ Improves resource utilization while guaranteeing SLAs
  24. 24. Linknovate • Big data analytics engine ⁃ Data acquisition, data aggregation, data processing, data visualization • RECAP: ⁃ Characterizes workload and models workload distribution ⁃ Automatically and dynamically allocates computing resources ⁃ To reduce costs and improve performance 24
  25. 25. Satec • IoT, smart city ⁃ Data management system: to collect sensory data, and to provide data (in a distributed manner) • RECAP: ⁃ Optimizes the placement of IoT resources such as computation, storage for cost and latency ⁃ Automated reallocation of resources ⁃ Automated deployment of IoT applications 25
  26. 26. Tieto 26 • Telecommunication infrastructure systems and applications • RECAP: ⁃ Automated profiling and simulating the infrastructure & VNFs ⁃ QoS and low latency

×