SALSA: A Framework for Dynamic Configuration of Cloud Services
•Download as PPTX, PDF•
1 like•1,518 views
This document summarizes a framework called SALSA that provides dynamic configuration of cloud services. It captures information about cloud service structures and deployment across multiple levels. It orchestrates configuration tasks across different levels including VMs, containers, and applications. The framework was prototyped and tested on various cloud platforms and services to demonstrate its ability to automate complex configuration tasks.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to SALSA: A Framework for Dynamic Configuration of Cloud Services
Similar to SALSA: A Framework for Dynamic Configuration of Cloud Services (20)
SALSA: A Framework for Dynamic Configuration of Cloud Services
- 1. SALSA: A Framework for Dynamic Configuration of Cloud Services Duc-Hung Le, Hong-Linh Truong, Georgiana Copil, Stefan Nastic and Schahram Dustdar Distributed Systems Group, Vienna University of Technology http://www.infosys.tuwien.ac.at/research/viecom/ 6th International Conference on Cloud Computing Technology and Science Singapore, 15-18 December 2014
- 2. Overview Scenario Research issues Framework for dynamic configuration Demo Conclusions and future work CloudCom 2014 2
- 3. Scenario - Complexity of configuration (1) Interact via web interface Service developer (3) Configure CloudCom 2014 3 (I) Interact via cloud API (2) Access to each VM by SSH/VNC > apt-get update > vim configure.conf > make clean install Error ! (II) Configure remotely via SSH/JSch > service X start > ./decommiss.sh > ./register.sh Warning ! Multiple IaaS cloud providers Elasticity controller
- 4. Scenario - Complexity of the cloud service configuration Load balancer Data node CloudCom 2014 4 M2MDaaS topology IoT topology Topology 1 Topology 2 Data controller Event Processing MOM Sensor Gateway Application level Legends: Connect via network
- 5. Scenario - Complexity of the cloud service configuration Java Load balancer Data node OS/ Java docker Linux Java Linux Java Linux CloudCom 2014 5 Tomcat docker Linux Java Linux Linux Container level Legends: Connect via network Placed on M2MDaaS topology IoT topology Topology 1 Topology 2 Data controller Event Processing MOM Sensor Gateway Application level
- 6. Scenario - Complexity of the cloud service configuration Java Load balancer Data node OS/ Java docker Linux Java Linux Java Linux m1.medium instance CloudCom 2014 6 Tomcat docker Linux Java Linux Linux Container level Virtual machine level m1.small instance m1.small instance c1.medium instance Cloud infrastructure 1 Public network c1.medium snapshot instance m1.large instance m1.medium instance Cloud infrastructure 2 Legends: Connect via network Placed on M2MDaaS topology IoT topology Topology 1 Topology 2 Data controller Event Processing MOM Sensor Gateway Application level
- 7. Scenario - Complexity of the cloud service configuration Java Load balancer Data node OS/ Java docker Linux Java Linux Java Linux m1.medium instance CloudCom 2014 7 Tomcat docker Linux Java Linux Linux Container level Virtual machine level m1.small instance m1.small instance c1.medium instance Public network c1.medium snapshot instance m1.large instance m1.medium instance Quick startup VMs High compute intensive VMs Low price VMs High data intensive VMs M2MDaaS topology IoT topology Topology 1 Topology 2 Data controller Event Processing MOM Sensor Gateway Application level
- 8. Requirements Elasticity controllers – Fine-grained actions. – Control all service levels. – Manage the whole service. Well-known tools - Chef, Puppet, Ansible… - Google CDM, Slipstream, Brooklyn, Juju … - OpenTOSCA Service developers Limitations - Support single configuration level. - Need human interaction. - Not support runtime configuration. Multi-cloud environment - Diversity - Dynamicity CloudCom 2014 8 – Automate complex tasks. – Manage only application level. – Semantic service management.
- 9. Research issues and Approach Research questions - Full automated configuration for complex cloud services. - Support multiple cloud infrastructures. - Configure multiple software stacks: VM, container, webapp. Approach – Capture information of multiple configuration levels: • Cloud service structure • Deployment information – Coordinate of the configuration tasks by orchestration. – Support many types of configuration on different levels. CloudCom 2014 9
- 10. Service configuration flow CloudCom 2014 10 Preprocess SALSA Framework Cloud systems Configuration plan VM Orchestrate configuration tasks Bootstrap VM Artifact deployment Configure cloud resources Lifecycle management Configure VM environment Setup configuration environment PUT [service_structure] Configured service Developer POST [unit_ID, scaleout] Controller DELETE [instance_ID] GET [serviceID]
- 11. SALSA Framework architecture Information services – Provide services information, e.g. artifact, relationships – Link to cloud providers and external repositories. – Enable third parties add configuration capabilities to the existing services. CloudCom 2014 11 Central service – Orchestrate the configuration tasks – Interface with multiple cloud providers. – Plug-in configuration algorithms Local configurator – Configure artifacts on VM. – Interface with multiple configuration tools.
- 12. Capturing configuration information Configuration capability categorized Primitive Add/remove Cloud service consist of consist of Service topology depend-on consist of Service unit consist of Service instance CloudCom 2014 12 associated at runtime Complex Start/stop Reconfigure Scale in/out Migrate Redeploy associate Deployment stack Web service App container Executable OS container Virtual machine … Cloud 1 Cloud N Multiple service levels Complex configuration capability Multiple deployment stacks [1] G. Copil, D. Moldovan, H.-L. Truong, and S. Dustdar, “Multi-level elasticity control of cloud services,”, ICSOC 2013
- 13. Generating deployment information Enrich missing information – Different service types. – Based on many criteria. Service unit eco-system – Connect to clouds – Service relationships Plugin-able enrichment algorithms – Enrich based on configuration capabilities. – Determine the relationship between service units. • Hosted-on relationship • Connect-to relationship CloudCom 2014 13
- 14. Orchestrate configuration tasks Service unit orchestrator Service unit orchestrator .. Service unit orchestrator actuate actions update states Cloud service orchestrator task Topology of service units task task Maintain the dependencies and the order of tasks. – Tasks can be executed in parallel. – Forward tasks to the right configurator module to execute. Plugin-able module for develop different algorithms – Distributed orchestration. – Workflow, plan-based. CloudCom 2014 14 Web app. configurator Container configurator System service configurator Virtual machine configurator Interface for applications configuration Interface for cloud systems configuration Environment configurator
- 15. Service placement at runtime Placement – Decide where to place the service unit. – Automatic manage the configuration of container and infrastructure. Placement algorithms: – Optimal for resource usage or performance. – Multi-tenant problem. Plugin-able module – Instance number based. – Resource threshold based. CloudCom 2014 15
- 16. Managing configuration states Configuration states information – Manage the dependencies between configuration tasks. – Represent service lifecycle – Feed-back to users. Collecting state information – States from many configuration levels – States of service structure Aggregating states on multiple levels and stacks – A configuration task exposed its state. – State of cloud service and topology are collected from service units. CloudCom 2014 16 CloudService Allocating FrontEnd topo Allocating Tomcat allocating Docker configuring VM deployed BackEnd topo Running Webapp 1 Allocating App 1 Running App 2 deployed Aggregated configuration states Actual configuration states
- 17. SALSA Prototype Interface with multiple cloud providers – OpenStack: by Jclouds – Flexiant: by Flexiant JADE API – StratusLab: by StratusLab client command line – Non-providers: configure existing machine Interface with multiple configuration tools – Script-based: bash – Unix like packages: apt-get, yum, gem – Chef enterprise and chef-solo Specific containers and artifact support – Tomcat, Docker – Web application in .war artifact CloudCom 2014 17 TOSCA service definition SALSA • Set of VMs running. • Cloud service runtime structure. • Unit configuration.
- 18. Tested services EventProcessing DataEnd DataNode Controller LB MOM EventProcessor MQTT CloudCom 2014 18 Market Place + database Iot.eclipse.org Data customer Sensor Sensors - ~3000 lightweight instances - On 3 clouds and normal servers - Dynamic add/remove M2MDaaS - Complexity: 8 units, 3 topologies - Running on 2 cloud systems DSG OpenStack - Private cloud - Vienna Flexiant - Public cloud - Commercial - London StratusLab - Public cloud - For research - Paris Marsa - Connect with external services - Distributed sites: Euro and Asia Normal servers - Hanoi and HCM city, Vietnam - Vienna Gateway GW Sensor Internet Sensor Iot.eclipse.org MQTT Gateway Scalable unit Static unit External service
- 19. Examples and demo Demo 1: Simplify complex configuration management – M2MDaaS: 3 topologies, 7 service units. – Multiple clouds: • OpenStack private cloud • Flexiant public cloud. – Connect sensors into the running M2MDaaS. – Maintain service units dependencies. Demo 2: Enable dynamic and multiple levels configuration – Focus on one topology of the M2MDaaS – Enable complex operations at runtime: scale in, scale out. – Enable configuration at infrastructure and application level. CloudCom 2014 19
- 20. Conclusions and future work Conclusion – Introduce a framework for developing and experimenting advanced configuration functionalities. – Provide initial features for full automatic and dynamic configuration of cloud services. – Support single cloud and multiple clouds. Future work: Focus more on the configuration of application level – Understand more configuration relationships of multiple levels and application structure. – Optimize the configuration at application level. CloudCom 2014 20
- 21. THANK YOU ! Duc-Hung Le d.le@dsg.tuwien.ac.at http://dsg.tuwien.ac.at/staff/dle/ Prototypes available http://tuwiendsg.github.io Distributed Systems Group Vienna University of Technology Austria
Editor's Notes
- In this presentation, we refer the cloud service as applications which are running on clouds and the service unit as application components.
- Let us consider a simple case with two stakeholders who want to configuring a service on several cloud infrastructure. By traditional way, firstly they need to connect with cloud providers to configure the infrastructure, either by using web interface or cloud API. After, they must connect to the VM and configure the application manually. These steps take many efforts if they only use services that clouds provided.
- Moreover, configuration of a cloud service is more complex with many service units that connect with each other via network, and the cloud service can be structured by several topology. Here we consider cloud service as an application running on the cloud and service units are application components. It becomes complex while the cloud service can also be structured by grouping units into topology.
- For running, each service unit also need a specific environments such as appropriate container or libraries.
- And all of them will be hosted by various types of infrastructure that are provided by clouds.
- User can also select appropriate configuration of the infrastructures that suitable for the services runtime.
- From the complexity of the configuration and cloud service, we observe the requirement for both stakeholders when configuring the cloud service. The elasticity controllers need to interact with all the level of the service structure and require the support in as much detail level as possible. Is there any mechanism to deal with all of them in a single endpoint. The service developer want to automate complex tasks and semantically managing the service. In this case, many additional information and automatic mechanisms are need to simplify their tasks. There are several well-known tools
- For these motivation, raise some research questions; - full automatic solution for configuring complex services: automate all the human involvement at runtime, manage cloud infrastructure which requires much of efforts, automatic connect and configure distributed units of the service. - also how to support multiple provider with: different API, capabilities, performance. our approach is firstly to capture the information at multiple levels, then design a central orchestration of the configuration tasks. Last by not least, in order to support configuring different types of service unit, we reuse as much as possible the existing configuration tools.
- So, going to more detail of the approach, firstly we investigate a general flow of configuration a cloud service. Starting from the user viewpoint, they will provide a document that describe the cloud service and the request for a configuration at initiation time or runtime. Then, they want to receive the feedback about the configuration progress. Internally, there are two phases. Firstly we pre process the user input to generate a configuration plan, and then a set of configuration tasks will be orchestrated to execute in right order and at right place.
- Following the configuration flow, we design the SALSA framework and defined functionalities which allow us to develop and plug-in more features into the framework. There are three main blocks in the architecture that follow traditional client-server pattern. We have one public central service and multiple local configurator to manage the distributed configuration tasks. The needed information will be provided by the information services
- For the service developers that need to automate most of the complex configurations, they only want to specify as less information as possible, which then their specification about the service would miss some details, and need to be enriched at the preprocess. When we aim to provide
- After this step, we have a full cloud service specification or need to reconfigure at runtime, many the configuration tasks need to be executed at right orders We start with the topology of service unit, which can be running or starting to be deployed. Some request for execute configuration tasks are called and need to be orchestrate by mean of execution these task in an proper orders to fulfill the dependencies between these service unit. To do that, we cooperate one center orch. and a set of SU orch. Unit orchestrator: Runs independently Handle multiple deployment stacks. Cloud service orchestrator: Maintains configuration states Maintains shared information
- At runtime, when user want to configure the application level without knowing about the container or infrastructure, we need to decide where to put the new service either on an existing container or on a new one. This feature enable the transparent between deployment stacks and simplify the service management while hiding the lower levels from the user. We plug into the framework two simple mechanisms, to place a service based on the instance number and based on resource threshold, which can be define by user.
- Lastly, we need to manage the configuration state in order to support the orchestration process and feedback to stakeholders.
- Above features was implemented in the SALSA prototype, which now can interact with multiple cloud providers and reuse common configuration tools to maximize the capability to execute complex configuration. We use TOSCA for describing the cloud service.
- For testing the feature to manage complex cloud service, we tested with the M2MDaaS cloud service with many service unit and relationship
- Demo 1 – service developer. Demo 2 – elasticity controller.