Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DECIDE for Dummies

DECIDE DevOps framework provides an integrated environment for multi-cloud native application developers and operators to design, develop, deploy and operate multi-cloud applications following the DevOps philosophy on continuous integration, continuous quality and continuous delivery. DECIDE for dummies provides a thorough guidance on how to use the DECIDE DevOps framework with the Sockshop microservices based application example. It includes the installation guidelines for the different components and an step by step guidance through the different components.
The DevOps framework integrates all the DECIDE components in one platform to 1) define NFRs and assign them to specific application components, 2) apply architectural patterns at different dimensions (generic, optimization, availability, performance) using ARCHITECT, 3) optimize and select the best topology for a multi-cloud application to be deployed on multiple clouds through OPTIMUS using cloud services offerings directly from CSPs or from the ACSmI, 3) define a multi-cloud SLA (MCSLA) based on the selected CSPs where the application will be deployed with the support of MCSLA editor, 4) automatically deploy the components following an application containerization approach on multiple clouds using ADAPT, 5) to monitor the behavior of the application with respect to its own MCSLA and the established NFR for the cloud resources where the application is deployed in order to (semi-)automatically re-adapt and redeploy the application to the new configuration suggested by OPTIMUS.

  • Be the first to comment

  • Be the first to like this

DECIDE for Dummies

  1. 1. DECIDE for Dummies Juncal Alonso, Leire Orue-Echevarria (TECNALIA)
  2. 2. DECIDE for Dummies  This presentation is a guide to the usage of the different components developed in the context of DECIDE H2020 Project.  A video of the demo is also available in the DECIDE H2020 webpage here.  The presentation is organized as follows:  Chapter 1: DECIDE DevOps framework access  Chapter 2: The used example: Sockshop application  Chapter 3: DECIDE DevOps framework usage • Chapter 3.1: DECIDE extended DevOps framework • Chapter 3.2: Architectural design • Chapter 3.3: Deployment optimization • Chapter 3.4: Multi-cloud SLA creation • Chapter 3.5: Resources contracting • Chapter 3.6: Components deployment • Chapter 3.7:Components and underlying resources monitoring • Chapter 3.8: Multi-cloud application adaptation • Chapter 3.9: Billing GA 731533 (c) DECIDE Consortium 2
  3. 3. DECIDE DevOps framework access DECIDE integrated framework access and installation guidelines
  4. 4. DevOps framework access  The DevOps framework is the integrated framework for all the DECIDE tools.  To follow this DECIDE for Dummies tutorial the DevOps framework is accesible here:  http://85.91.40.245:8084/  It can also be installed on premise. For that please follow the corresponding guidelines on the annex: D2.8-Final Integrated DevOps framework. GA 731533 (c) DECIDE Consortium 4
  5. 5. The used example: The Sockshop application Introduction to the used sample multi-cloud based application
  6. 6. The Sockshop application  During this tutorial the examples shown will be based on the Sockshop application.  The SockShop App1, is a loosely coupled microservices demo application. It simulates the user-facing part of an e-commerce website that sells socks. It is available as open source software and has been developed with the intention to aid in demonstrating and testing microservices and cloud native technologies.  The Sock Shop app is designed to provide as many microservices as possible. The microservices are defined by functionality required in an e-commerce site and are loosely coupled.  Sock Shop microservices are designed to have minimal expectations, using DNS to find other services. The Application uses a message broker for sending messages using queues.  All services communicate using REST over HTTP. Furthermore, the Sock Shop app is polyglot being built using Spring Boot4, Go kit5 and Node.js6 and is packaged in Docker containers.  During this tutorial the example’s will be based on the Sockshop application with 10 microservices GA 731533 (c) DECIDE Consortium 6 1 https://microservices-demo.github.io/
  7. 7. DECIDE extended DevOps framework Integrated framework to support the development and operation of multi-cloud applications
  8. 8. DevOps framework: theory GA 731533 (c) DECIDE Consortium 8 Development and Operations Agility Stability DevOps
  9. 9. DevOps framework: theory  There is not a common and agreed definition for DevOps  DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality. [Bass, Len; Weber, Ingo; Zhu, Liming. DevOps: A Software Architect's Perspective. ISBN 978-0134049847.]  DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology — especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective. [gartner, https://www.gartner.com/it-glossary/devops] GA 731533 (c) DECIDE Consortium 9 DevOps
  10. 10. DevOps framework: theory GA 731533 (c) DECIDE Consortium 10  All definitions present commom characteristics DevOps seek to bring closer the interaction between Dev and Ops, using similar tools Continuous delivery pipeline • Continuous integration (CI) • Continuous Quality (CQ) • Continuous Delivery(CD) DevOps
  11. 11. DevOps framework: theory GA 731533 (c) DECIDE Consortium 11 DevOps  DevOps is represented as an infinte loop
  12. 12. DevOps framework: theory GA 731533 (c) DECIDE Consortium 12 https://blog.rackspace.com/uk/rackspace-finds-clear-business-benefits-increasing-use-devops-rapid-software- development-deployment Efficiency Customer satisfaction Uptime of the app Agility More value: new capabilities Productivity and satisfaction of the team Conversion of new leads DevOps: benefits
  13. 13. DevOps framework: theory GA 731533 (c) DECIDE Consortium 13 DevOps: barriers Source: www.quali.com Culture Fail to automate a part of the process Legacy applications / infrastructure Application complexity No clear plan of DevOps Tools Management of environments Training Commitment of Mgmt Budget
  14. 14. DevOps framework: multi-cloud  Multi-cloud applications with strict NFR, more specifically, performance, availability, location and cost concerns GA 731533 (c) DECIDE Consortium 14 + + + + + = Multi-cloud in most literature Multi-cloud for us Multi-cloud
  15. 15. DevOps framework: multi-cloud GA 731533 (c) DECIDE Consortium 15 Same CSPs, Different location Different CSPs Different resources / services in the same CSP and location Multi-cloud
  16. 16. DECIDE Approach GA 731533 (c) DECIDE Consortium 16 Non-functional requirements gathering and prioritization Architectural Design Development, integration and testing CSPs and microservices profiling and classification Optimization (+ CSP Discovery) Application MCSLA definition Deployment CSPs service contracting Deployment configuration provisioning Application MCSLA and NFR Monitoring CSPs SLA monitoring Self-adaptation and automatic (re-)deployment Support in the SDLC and SOLC in an integrated manner ARCHITECT OPTIMUS ACSmI ADAPT DevOps FW
  17. 17. DevOps Framework: in practice  Register yourself in the DevOps framework. Click Register GA 731533 (c) DECIDE Consortium 17
  18. 18. DevOps Framework  Complete the details in the next screen and click Register GA 731533 (c) DECIDE Consortium 18
  19. 19. DevOps Framework  Login using the email and the password selected.  When you login, you will see the following screen GA 731533 (c) DECIDE Consortium 19
  20. 20. DevOps Framework  You can edit your profile at all times by clicking GA 731533 (c) DECIDE Consortium 20
  21. 21. DevOps Framework  The following screen appears when clicking  You can change here your profile and include the credentials needed for the Docker registry. This is necessary to run ADAPT DO GA 731533 (c) DECIDE Consortium 21
  22. 22. DevOps Framework  In the Credentials Tab insert the information related to the Docker registries where you have your containers  Upload also the certificate that your Docker registry needs GA 731533 (c) DECIDE Consortium 22
  23. 23. DevOps Framework  You can create a new application by clicking on the button new application GA 731533 (c) DECIDE Consortium 23
  24. 24. DevOps Framework  And you will get this screen GA 731533 (c) DECIDE Consortium 24
  25. 25. DevOps framework  Complete the following fields:  Name: insert the name of the application  Description: insert a description of the application  Git repository: include the URL of the git repository where the DECIDE.json will be stored (see next for more explanations on the DECIDE.json)  Git token: the token needed to access the git  High technological risk: if this option is selected, the automatic redeployment will not be executed Import Description from DECIDE Project: in the case you have already a DECIDE.json from a previous project, you can import it here.  When you are done, click GA 731533 (c) DECIDE Consortium 25
  26. 26. DevOps framework  Clicking on you can add the microservices.  The following data must be inserted: Name: name of the microservice Programing language: the language in which it was developed Stateful: if the microservice has state or not Public IP: if the microservice should have a public IP (e.g. in the case it is a user interface) Tags: this is used for a later stage, for the NFRs and the monitoring. By default, the tag ‘application’ is included, but you can also insert the tag that indicates one of the microservices.  When you are done, click GA 731533 (c) DECIDE Consortium 26
  27. 27. DevOps framework  Now include the NFRs.  The NFRs availability, performance location, and cost are later on monitored and used in ARCHITECT for the recommendation of the patterns  The NFR scalability and legal level are only used by ARCHITECT for the recommendation of the patterns GA 731533 (c) DECIDE Consortium 27 NFR
  28. 28. • Availability level: high, medium or low. Depending on the level selected the number of suggested patterns by ARCHITECT will vary • Availability (%): that is the percentage of availability for the whole application (by default) and in the case additional tags were inserted when creating the microservice, these values will also applied to them. This value will be used later on for monitoring purposes • Tags: select here if the NFR is defined at application level or at microservice level DevOps framework GA 731533 (c) DECIDE Consortium 28 NFR
  29. 29. DevOps framework  When clicking the details of the selected NFR will appear below GA 731533 (c) DECIDE Consortium 29 NFR
  30. 30. • Performance level: high, medium or low. Depending on the level selected the number of suggested patterns by ARCHITECT will vary • Response time (ms): the maximum time allowed for the whole application (by default) to respond. In the case additional tags were inserted when creating the microservice, these values will also applied to them. This value will be used later on for monitoring purposes • Tags: select here if the NFR is defined at application level or at microservice level DevOps framework GA 731533 (c) DECIDE Consortium 30 NFR
  31. 31. • Scalability level: high, medium or low. Depending on the level selected the number of suggested patterns by ARCHITECT will vary • Scalability value (request/sec): the maximum numbers of requests per second allowed for the whole application (by default. In the case additional tags were inserted when creating the microservice, these values will also applied to them. This value will not be used later on for monitoring purposes • Tags: select here if the NFR is defined at application level or at microservice level DevOps framework GA 731533 (c) DECIDE Consortium 31 NFR
  32. 32. • Location type: single location, single country, cross-border. This will be used later on for the selection of the cloud offerings. • Region: Europe, Asia, north America, south America. The region in which the cloud service offering should be located. This value will be used later on for monitoring purposes • Zone: the zone where the cloud service offering will be located (e.g. Germany, France, US …). This value will be used later on for monitoring purposes • Tags: select here if the NFR is defined at application level or at microservice level DevOps framework GA 731533 (c) DECIDE Consortium 32 NFR
  33. 33. • Cost level: high, medium or low. Depending on the level selected the number of suggested patterns by ARCHITECT will vary • Cost: The maximum cost you would be willing to pay. This value will be used later on for monitoring purposes • Currency: Euro, pound, dollar • Tags: select here if the NFR is defined at application level or at microservice level DevOps framework GA 731533 (c) DECIDE Consortium 33 NFR Euro, pound, dollar
  34. 34. • Legal level: tier 1, tier 2 and tier 3. This value will be used for the selection of the cloud offerings DevOps framework GA 731533 (c) DECIDE Consortium 34 NFR The explanation of the legal levels are found next
  35. 35. DevOps framework  Legal level tier 3: basic legal safeguards Basic general safeguards (company registration, presence of representative if relevant, presence of a DPO or data protection officer, DPA and transfer mechanism where relevant) Basic contractual safeguards (liability, confidentiality, exit options, ADR) Basic GDPR compliance (basic contractual guarantees are present but may be faulty depending on interpretation, conditions may not be ideal) No certification No adherence to approved self-regulatory instruments GA 731533 (c) DECIDE Consortium 35
  36. 36. DevOps framework  Legal level tier 2: substantial legal safeguards Basic general safeguards (company registration, presence of representative if relevant, presence of a DPO or data protection officer, DPA and transfer mechanism where relevant) Substantial contractual guarantees for the most relevant aspects (liability, confidentiality, exit options, ADR), basic guarantees for others Substantial GDPR compliance (balanced and substantial contractual guarantees, issues or challenges are unlikely, although conditions may not be ideal) Valid basic certification (ISO 27001 or equivalent) covering the service No adherence to approved self-regulatory instruments GA 731533 (c) DECIDE Consortium 36
  37. 37. DevOps framework  Legal level tier 1: strong legal safeguards  Basic general safeguards (company registration, presence of representative if relevant, presence of a DPO or data protection officer, DPA and transfer mechanism where relevant)  Strong contractual guarantees for the most relevant aspects (liability, confidentiality, exit options, ADR), substantial guarantees for others  Strong GDPR compliance (strong contractual guarantees with favorable conditions of the Cloud user/controller)  Valid basic certification (ISO 27001 or equivalent)  Valid Advanced cloud-specific certification that meets all of the 27 security objectives of the Cloud Certification Schemes Metaframework as defined by ENISA covering the service  Adherence to approved self-regulatory instruments GA 731533 (c) DECIDE Consortium 37
  38. 38. DevOps framework  The final set of selected NFRs are as follows. Then click ‘Save’ GA 731533 (c) DECIDE Consortium 38
  39. 39. DevOps framework  The DECIDE.json is created and stored in the git GA 731533 (c) DECIDE Consortium 39
  40. 40. Application description 40GA 731533 (c) DECIDE Consortium  Each tool shares needed information by pushing and pulling from a dedicated repository.  Information model specific to the DECIDE Application that is stored in a repository (e.g. Git) and is accessed by each tool. It is a JSON file  Interoperable mechanism
  41. 41. Application description 41GA 731533 (c) DECIDE Consortium  It looks like this
  42. 42. DevOps framework  Now the dashboard has changed and it shows the main characteristics of our application GA 731533 (c) DECIDE Consortium 42
  43. 43. DevOps framework  In the dashboard you can see:  the NFRs selected and their values  the microservices of the application and the tags  The integration with Jenkins and SonarQube … More fields GA 731533 (c) DECIDE Consortium 43
  44. 44. DevOps framework  DECIDE allows to integrate with CI tools such as Jenkins GA 731533 (c) DECIDE Consortium 44
  45. 45. DevOps framework  Now that the basic data of the application has been included, the left menu shows in green, the actions that can be carried out GA 731533 (c) DECIDE Consortium 45
  46. 46. DevOps framework  In this case, the next step is the architectural design but also the cloud services Discovery could be used.  Cloud services can be discovered at any point in the workflow. GA 731533 (c) DECIDE Consortium 46
  47. 47. Architectural design Architectural design for multi-cloud applications with relevant NFRs using ARCHITECT
  48. 48. ARCHITECT: theory  Based on the NFRs and the values inserted, DECIDE ARCHITECT recommends a set of patterns  ARCHITECT is distributed in two flavours: GA 731533 (c) DECIDE Consortium 48 Eclipse Plug inWeb application + Integrated with OPTIMUS Integrated with DevOps framework
  49. 49. ARCHITECT: theory GA 731533 (c) DECIDE Consortium 49 Pattern Classification Fundamental (4) Development (33) Deployment (20) Optimization (12) Pattern Description Name / Type / Context Problem / Solution Model NFR Impact Related Patterns DECIDE ARCHITECT  There are patterns defined for each phase of the application plus a set of fundamental ones.  All patterns are described the same way
  50. 50.  The patterns are recommended based on the NFRs values ARCHITECT: theory GA 731533 (c) DECIDE Consortium 50 DECIDE ARCHITECT: Pattern recommendation Pattern recommendation Software architecture Deployment simulation Development patterns Optimization patterns Deployment patterns Deployment requirements NFRs
  51. 51. ARCHITECT: theory GA 731533 (c) DECIDE Consortium 51 DECIDE ARCHITECT: Pattern recommendation  The inference and recommendation mechanisms are shown below:
  52. 52. ARCHITECT: in practice GA 731533 (c) DECIDE Consortium 52 Recommended patterns All patterns avaiable
  53. 53. ARCHITECT: in practice  The recommended patterns can be grouped by:  alphabetical order  non-functional requirement (general, availability, location, performance,…)  category (fundamental, development, deployment, optimization) GA 731533 (c) DECIDE Consortium 53
  54. 54. ARCHITECT: in practice  Clicking on each pattern the following information is shown: GA 731533 (c) DECIDE Consortium 54
  55. 55. ARCHITECT: in practice  Select the patterns that you consider relevant and ‘Confirm Selection’  The application description file is updated with the selected patterns (field “selected” is true) GA 731533 (c) DECIDE Consortium 55
  56. 56. ARCHITECT: in practice GA 731533 (c) DECIDE Consortium 56
  57. 57.  ARCHITECT is also offered as an Eclipse plugin. The way it Works is the same as the one integrated in the DevOps framework DECIDE ARCHITECT: in practice GA 731533 (c) DECIDE Consortium 57
  58. 58. Deployment optimization Deployment configuration optimization for specific NFRs and cloud resources classification using OPTIMUS
  59. 59. DECIDE OPTIMUS: theory  OPTIMUS has the following functionalities:  Classifies the components of the multi-cloud application  Matches these components with existing cloud service offerings Proposes deployment scenarios GA 731533 (c) DECIDE Consortium 59
  60. 60. OPTIMUS: theory GA 731533 (c) DECIDE Consortium 60 Eclipse Plug inWeb application + Integrated with ARCHITECT & OPTIMUS Simulation REST service DevOps Framework (redeployment) Integrated with the DevOps framework  OPTIMUS is distributed in two flavours:
  61. 61. OPTIMUS Theory  Classification of the microservices based on the requirements for each component : GA 731533 (c) DECIDE Consortium 61 Computing Needs for public access StorageDatabase Main microservices Detachable Resources
  62. 62. OPTIMUS Theory  Classification of the cloud offerings in services classes  Each service class has specific attributes to characterize it. GA 731533 (c) DECIDE Consortium 62 Virtual Machines DataBaseStorage
  63. 63. OPTIMUS in practice  In the tab ‘deployment simulation’ the functionalities provided by OPTIMUS are shown.  Microservice Edit • allows to add a detachable resource to the microservice • Allows to add more information about the requirements needed to deploy that microservice on a cloud service offering Simulate • Provides the user with the different deployment topologies GA 731533 (c) DECIDE Consortium 63
  64. 64. OPTIMUS in practice  Edit the microservices by selecting each microservice and including the infrastructure requirements GA 731533 (c) DECIDE Consortium 64
  65. 65. OPTIMUS in practice  You can add also detachable resources (databases) GA 731533 (c) DECIDE Consortium 65
  66. 66. OPTIMUS in practice  When going to the tab ‘Simulate’ the following screen is shown  A preferred cloud provider can be selected. If done so, then only cloud service offerings from that CSP will be selected. To do this, OPTIMUS calls ACSmI Discovery  When you click ‘Simulate’ the algorithm starts (see next) GA 731533 (c) DECIDE Consortium 66
  67. 67. OPTIMUS theory GA 731533 (c) DECIDE Consortium 67 DECIDE OPTIMUS: Algorithm Simulation REST service Uses NSGA-II Genetic Algorithm for generating optimized deployment schemas (*). The steps are: Initialize Initial population based on the “OPTIMUS problem” definition. Population is a set of solutions (deployment schemas) Evaluate Level of fulfillment of the FRs and NFRs established Select Comparisons between new Solutions and existing ones determine the solution to be kept Crossover & mutation Creation of new Solutions improving the existing ones Stop when the number of final solutions is less tan five (*) MOEA Framework, a free and open source Java library for developing and experimenting with multiobjective evolutionary algorithms
  68. 68. OPTIMUS in practice  The result is shown next. Select the configuration you wish GA 731533 (c) DECIDE Consortium 68 Select and save the schema
  69. 69. OPTIMUS in practice  A new section in the DECIDE.json has been generated. It’s under ‘schema’ GA 731533 (c) DECIDE Consortium 69
  70. 70. DECIDE OPTIMUS GA 731533 (c) DECIDE Consortium 70 Eclipse plugin Eclipse Plug in Integrated with OPTIMUS Clone Project Plugin editor/classification Plugin editor/simulation Plugin editor/schemas  The eclipse plug in has the same functionality
  71. 71. Multi-cloud SLA creation Support for Multi-cloud SLA creation
  72. 72. MCSLA GA 731533 (c) DECIDE Consortium 72 REST serviceWeb application + Stand alone + DECIDE DevOps Dashboard ISO/IEC 19086
  73. 73. MCSLA GA 731533 (c) DECIDE Consortium 73 MCSLA  Requests information from ACSmI for all CSLAs  It also allows to: Edit MCSLA View MCSLA  Definition of SLOs and SQOs  Aggregation of SLOs (*) (*) Performance and Availability
  74. 74. MCSLA GA 731533 (c) DECIDE Consortium 74  The MCSLA shown is the aggregation of the SLAs from the selected CSPs
  75. 75. MCSLA GA 731533 (c) DECIDE Consortium 75  The values suggested are the result from the aggregation but these can also be changed if wished
  76. 76. MCSLA  To save the changes: click commit. The application description has been updated in the section mcsla GA 731533 (c) DECIDE Consortium 76
  77. 77. Resources contracting Cloud resources contracting using ACSmI contracting
  78. 78. ACSmI contracting overview  ACSmI contracting offers the functionalities to contract the resources which are selected by OPTIMUS and noted in the DECIDE.json (schema section).  ACSmI contracting programatically connects to the available CSPs contracting functionalities and performs the different contracts automatically (without human intervention).  ACSmI contracting reads the schema in the DECIDE.json and automatically contracts the resources. These resources need to be: Registered in the ACSmI registry (transparent to the user) Available in the ACSmI contracting catalogue (transparent to the user) GA 731533 (c) DECIDE Consortium 78
  79. 79. ACSmI contracting overview  The user needs to have an account in ACSmI contracting to contract the resources.  The user can have an Amazon personal account and provide it to ACSmI contracting in case he/she wants to use this account.  Once the contracts are stablished, ACSmI contracting includes the information about the resources contracted in the DCIDE.json under the “virtual machines” section  With this information ADAPT DO can provision the contracted resources. GA 731533 (c) DECIDE Consortium 79
  80. 80. ACSmI contracting in practice  Once the MCSLA is created the DevOps team member can contract the resources to be used for the deployment. These resources are already selected and the ACSmI contracting tool guides the user through the contracting process: GA 731533 (c) DECIDE Consortium 80
  81. 81. ACSmI contracting in practice  The user needs to have an account. Only one account can be related to one DECIDE application. Cloud service contracting register form. GA 731533 (c) DECIDE Consortium 81
  82. 82. ACSmI contracting in practice  Cloud service contracting welcome page if the user is already logged GA 731533 (c) DECIDE Consortium 82
  83. 83. ACSmI contracting in practice  All the proposed resources and their characteristics are shown (in this case Amazon). If the user has an Amazon account, it can be used for the current contracting, otherwise ACSmI contracting offers the possibility to do it through the ACSmI platform directly, who has a specific framework contract with Amazon cloud provider. GA 731533 (c) DECIDE Consortium 83
  84. 84. ACSmI contracting in practice  Characteristics of the proposed contract GA 731533 (c) DECIDE Consortium 84
  85. 85. ACSmI contracting in practice  Once the contracts are stablished the user gets a success response and the contracts can be checked under the contracts tab. GA 731533 (c) DECIDE Consortium 85
  86. 86. ACSmI contracting in practice  Current active contracts GA 731533 (c) DECIDE Consortium 86
  87. 87. Components deployment Automatic Multi-cloud components deployment (microservices) in contracted cloud resources Using ADAPT Deployment Orchestrator (ADAPT-DO)
  88. 88. ADAPT DO overview  ADAPT DO provisions the machines contracted, install all the needed software to deploy the Docker images, downloads all the images from a Docker registry and deploys them in the corresponding cloud resource (see flow description in the following slides).  ADAPT DO sets up all the communications between the containers (with the information from the developer).  ADAPT DO reads the schema and the virtuals machines section in the DECIDE.json to automatically deploy the components. The user needs to provide information about: The components communication (ports, etc) The components source (images name, Docker registry information) GA 731533 (c) DECIDE Consortium 88
  89. 89. ADAPT DO overview  ADAPT DO installs the agents needed for the monitoring of the resources and monitors the application through ADAPT monitoring and the resources through ACSmI monitoring. GA 731533 (c) DECIDE Consortium 89
  90. 90. ADAPT DO flow GA 731533 (c) DECIDE Consortium 90 ADAPT Deployment Orchestrator DevOps Framework / State Machine Git repo Applicatio n descriptor ADAPT parses the Application Descriptor and detects how many VMs must be started and on which CSP
  91. 91. ADAPT DO flow GA 731533 (c) DECIDE Consortium 91 ADAPT Deployment Orchestrator DevOps Framework / State Machine VM VM VM VM VM VM ACSMI Git repo Applicatio n descriptor ADAPT invokes the ACSmI broker to ask for the provisioning of VMs on the target CSPs ACSmI applies the provisioning requests to the proper CSPs
  92. 92. ADAPT DO flow GA 731533 (c) DECIDE Consortium 92 ADAPT Deployment Orchestrator DevOps Framework / State Machine VM VM VM VM VM VM ACSMI runtim e security runtim e security runtim e security runtim e security runtim e security runtim e security Git repo Applicatio n descriptor When VMs are ready (have a public address) ADAPT accesses them and installs on each one the needed runtimes and configurations
  93. 93. ADAPT DO flow GA 731533 (c) DECIDE Consortium 93 ADAPT Deployment Orchestrator OPTIMUS VM VM VM VM VM VM ACSMI s runtim e security runtim e security runtim e security s s runtim e security runtim e security runtim e security s s s Git repo Applicatio n descriptor Microservice CSP VM microservice1 CSP1 VM1 microservice2 CSP1 VM2 microservice3 CSP1 VM3 microservice4 CSP2 VM1 microservice5 CSP2 VM2 microservice6 CSP2 VM3 Finally ADAPT can deploy the microservices according to the Application Description deployment schema
  94. 94. ADAPT DO in practice  STEP 1- ADAPT DO information: This information is automatically loaded from the DevOps framework and the DECIDE.json. The user only needs to continue to the next step. GA 731533 (c) DECIDE Consortium 94 Click to continue to the next step
  95. 95. ADAPT DO in practice  STEP 2 – Resources information: This information is automatically loaded from the DECIDE.json. If necessary, the user can change it (if the resources have been contracted manually). GA 731533 (c) DECIDE Consortium 95 Click to continue to the next step
  96. 96. ADAPT DO in practice  STEP 3 – Containers information: This information needs to be completed by the developer/operator of the application (in red mandatory fields). This information includes: Container name: To be decided by the developer Image name: As it is named in the registry loaded in the profile1. VM_Name: It is loaded from the DECIDE.json Tag: Image tag to be downloaded from the registry Docker registry: Select from the list the registry where the Docker images will be downloaded by ADAPT. This registry is configured in the DevOps user profile information1. GA 731533 (c) DECIDE Consortium 96 1See DevOps FW in practice profile creation
  97. 97. ADAPT DO in practice  Port mapping: This information needs to be provided by the developer/operator of the application. It corresponds to the actual mapping of the host (resource) ports and the containers ports.  Host mapping: This information needs to be provided by the developer/operator. It corresponds to the add-host command in Docker CLI, that configures an specific service in a container to reach external hosts.  Volume Mapping: This information needs to be provided by the developer/operator. It corresponds to the volumes to be added to the container for persistent storage of data.  Safe methods: This information needs to be provided by the developer/operator. It corresponds to the REST method against which ADAPT Monitoring will monitor the availability of the component (application level). This method needs to be safe (it should not have side effects in the application performance). GA 731533 (c) DECIDE Consortium 97
  98. 98. ADAPT DO in practice  Reverse Proxy Endpoints: This is a mechanisms which allows to set up a reverse proxy, so that microservices reference each other via name, while the actual adresses are stored in a remote location and always kept up to date during re-deployments. When a microservice refers to a another microservice, it requests the proxy for it by name and the proxy knows the target service that is responding at a specific moment.  Command parameters: These are parameters to be passed to a container for the EntryPoint command (see EntryPoint Docker command).  Entrypoint: This is the command to be run in the EntryPoint command (see EntryPoint Docker command).  Networks: This is the value for the network option when starting the container with Docker run.  Environment: This is the definition of environment variables when starting a container with Docker run.  Add consul service/Consul service port GA 731533 (c) DECIDE Consortium 98
  99. 99. ADAPT DO in practice GA 731533 (c) DECIDE Consortium 99  Generic container info
  100. 100. ADAPT DO in practice GA 731533 (c) DECIDE Consortium 100  Port mapping and host mapping information
  101. 101. ADAPT DO in practice GA 731533 (c) DECIDE Consortium 101  Volume mapping and safe methods.
  102. 102. ADAPT DO in practice GA 731533 (c) DECIDE Consortium 102  Reverse proxy, command paramenters, Entrypoint, Networks and Environment.
  103. 103. ADAPT DO in practice  Consul service port GA 731533 (c) DECIDE Consortium 103
  104. 104. ADAPT DO in practice  STEP 4 – This step submits the preparation information to ADAPT DO and launches the actual deployment process including (for more information see ADAPT DO Flow): Infrastructure initialization, planning and application (following the information of the Application description). Services initialization, planning and application (following the information of the Application description).  The user can perform the different steps one by one or all in one shot. GA 731533 (c) DECIDE Consortium 104
  105. 105. ADAPT DO in practice  STEP 4 – One by one option: Each of the phases are manually launched by the user. When one by one option is selected, first the preparation step needs to be submitted 1 2 3
  106. 106. ADAPT DO in practice  STEP 4 – One shot option:The whole process is launched at once.When one shot deployment, the whole process is launched with the corresponding option. The next phases are automatically launched when the previous one is finished 1
  107. 107. ADAPT DO in practice  STEP 4 – In both cases the “submit preparation step” or “one shot deployment (all in one deployment step)” will init the process by accepting the proposed data: The user needs to accept the proposed data submission
  108. 108. ADAPT DO in practice  STEP 4 – Once the actual deployment process has started the status of the different resources and microservices can be checked by the green ticks and the circles in the UI.
  109. 109. ADAPT DO in practice  STEP 4 – Once the actual deployment process has started the status of the different resources and microservices can be checked by the green ticks and the circles in the UI.
  110. 110. ADAPT DO in practice  Once the deployment process is finished the user can check the deployed application (in this case the sockshop) in the corresponding virtual machine (public IP in ADAPT UI or dockerHostPublicIp in the virtual machines section in the application description), accesing the desired component. In the Sockshop case , we need to access the front-end container. The front-end is deployed in the VM1_Amazon_Ubuntu16_04
  111. 111. ADAPT DO in practice  Once the deployment process is finished the user can check the deployed application (in this case the sockshop) in the corresponding virtual machine (public IP in ADAPT UI or dockerHostPublicIp in the virtual machines section in the application description), accessing the desired component. In the Sockshop case, we need to access the front-end container.
  112. 112. Components and underlying resources monitoring Proactive Multi-cloud components (microservices) and underlying resources (virtual machines) monitoring and continous SLA assessment
  113. 113. Generic monitoring overview in DECIDE  Monitoring approach in DECIDE GA 731533 (c) DECIDE Consortium 113 ADAPT monitoring ACSmI monitoring Out of scope (*) Source: Production Ready Microservices O’Reilly Layer 4: Microservices Layer 3: Application platform Layer 2: Communication Layer 1: Virtual Hardware Out of scope
  114. 114. Generic monitoring overview in DECIDE  Monitoring stack in DECIDE GA 731533 (c) DECIDE Consortium 114 Composed metrics visualization Composed metrics generation Raw metrics storage (TSDB) Agents for metrics collection ACSmI monitoring ADAPT monitoring ADAPT monitoring ACSmI monitoring ACSmI monitoring ADAPT monitoring Composed metrics assessment ACSmI monitoring ADAPT monitoring ADAPT monitoring MCSLA editor
  115. 115. ADAPT Mon overview  ADAPT Monitoring (ADAPT Mon) monitors the different components of the multi-cloud application through the safe methods introduced by the developer (see ADAPT DO) against the NFRs established in the initial phase (see DevOps framework): Availability and/or performance.  ADAPT Mon also monitors the fulfillment of the established NFRs (availability and/or performance) at application level if any (see DevOps framework).  NFR monitored metrics are generic and based on the ISO19086 standard.  Availability = MTBF/(MTBF+MTTR)  Performance = Response time  When any of the stablished NFRs is violated, ADAPT Mon triggers the Violation Handler which is the component in charge of managing violations.  Apart from the application layer monitoring, ADAPT Mon also launches the infrastructure monitoring, calling ACSmI mon. GA 731533 (c) DECIDE Consortium 115
  116. 116. ADAPT Mon in practice  Once the multi-cloud application is deployed the monitoring tabs are automatically activated, with the corresponding selected metrics (performance).
  117. 117. ADAPT Mon in practice  Once the multi-cloud application is deployed the monitoring tabs are automatically activated, with the corresponding selected metrics (availability).
  118. 118. ADAPT Mon in practice  When any of the established NFRs is violated, ADAPT Mon triggers the Violation Handler which is the component in charge of managing violations. GA 731533 (c) DECIDE Consortium 118 -> Alarms raised Visual feedback
  119. 119. ACSmI Mon overview  ACSmI Monitoring (ACSmI Mon) monitors the resources where the different components are being deployed.  ACSmI Mon monitors the compliance at run time of the following NFRs (stated in the CSPs SLAs). monitored metrics are generic and compliant with ISO19086  Location = Actual location of the VM  Availability = MTBF/(MTBF+MTTR)  Performance = Disk, Memory and CPU usage  Cost = Actual cost and usage records of each resource.  When any of the established SLOs is violated, ACSmi Mon triggers the Violation Handler which is the component in charge of managing violations.  When the violation is detected by ACSmI mon the ACSmI registry is updated so that the user (and other DECIDE tools like Optimus) are aware of this violation and can perform the required actions when looking for services in the next deployment. GA 731533 (c) DECIDE Consortium 119
  120. 120. ACSmI Mon in practice  ACSmI Mon does not include a graphical interface for metrics monitoring but the violations can be checked in the ACSmI Discovery user interface. GA 731533 (c) DECIDE Consortium 120 Violations detail Violations occurred
  121. 121. Multi-cloud application adaptation Semi-automatic Multi-cloud application adaptation if a violation occurs
  122. 122. Violation Handler overview  The Violation Handler (VH) is the component which manages the violations.  It receives the violations from ADAPT mon or ACSmI mon and performs the following steps:  Notifies the developer so he or she is aware of the violation.  If the technology risk of the application is low (see DevOps framework description) the VH triggers a new deployment process which includes the following automatic tasks: • Call OPTIMUS for a new deployment schema • Create a new MCSLA • Contract the new resources selected by OPTIMUS • Deploy the components in the new resources and stop the old ones. • Set up the monitoring components for the new deployed components/resources. GA 731533 (c) DECIDE Consortium 122
  123. 123. Violation handler in practice  The developer receives the email from the VH and the re- deployment process is launched (if the technological risk is low). GA 731533 (c) DECIDE Consortium 123 The re-deployment process is triggered with the call to OPTIMUS who writes the new schema in the application description
  124. 124. Violation handler in practice  The redeployment process is launched automatically if the technological risk is low. The different DECIDE components are subsequently automatically executed until the new redeployment takes place. Each component updates the required information in the application description. GA 731533 (c) DECIDE Consortium 124 OPTIMUS writes the new schema OPTIMUS updates the history New MCSLA is created New contracts are established The application is deployed in the new resources
  125. 125. Billing Proactive Multi-cloud components (microservices) and underlying resources (virtual machines) monitoring and continous SLA assessment
  126. 126. ACSmI Billing overview  ACSmI Billing keeps track of the usage of each cloud resource and the corresponding costs.  The user can access the ACSmI billing to check the active invoices and the usage records. GA 731533 (c) DECIDE Consortium 126

×