A Scalable Approach to Deploying and Managing Appliances


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • a layer of abstraction that makes procuring resources simple and intuitive Can we provide middleware that will do it for us
  • Instead of annotating images, annotate partitions
  • A Scalable Approach to Deploying and Managing Appliances

    1. 1. A Scalable Approach to Deploying and Managing Appliances Kate Keahey Rick Bradshaw, Narayan Desai, Tim Freeman Argonne National Lab, University of Chicago
    2. 2. Environmental Problem <ul><li>Complexity </li></ul><ul><li>Consistency </li></ul>?
    3. 3. Providers and Consumers Resource provider Resource consumers Has a limited number of resources Want the resources when they need them & as much as they need Has to balance the software needs of multiple users Want to use specific software packages Has to provide a limited execution environment for security reasons Wants as much control as possible over resources
    4. 4. Changing the Question Let’s see what’s available and adapt my problem to use it Can we provide the middleware that will enable this change of approach? Here is the environment I need to solve my problem -- deploy it on the Grid
    5. 5. Virtual Workspaces <ul><li>Dynamically overlay a required environment over resources in the Grid </li></ul><ul><ul><li>Configuration and Information </li></ul></ul><ul><ul><ul><li>Configuration management, e.g., Bcfg2, Pacman </li></ul></ul></ul><ul><ul><ul><li>Issues: How do I express what I want? How long will it take to install? How reliably will it provide the required environment? </li></ul></ul></ul><ul><ul><li>Virtual Machines </li></ul></ul><ul><ul><ul><li>Xen, VMware, etc. </li></ul></ul></ul><ul><ul><ul><li>Develop and test locally, deploy globally </li></ul></ul></ul><ul><ul><ul><li>Short deployment times </li></ul></ul></ul><ul><ul><li>Paper: “Quality of Service and Quality of Life in the Grid” </li></ul></ul>
    6. 6. Virtual Machines: The Good News <ul><li>Quality of Life </li></ul><ul><ul><li>A user can provision a pre-configured customized and consistent environment across the Grid </li></ul></ul><ul><ul><li>The site does not need to understand user’s environment needs in detail </li></ul></ul><ul><ul><li>The site can provision environments in a cost-effective manner </li></ul></ul><ul><li>And many other fine properties… </li></ul><ul><ul><li>Quality of Service </li></ul></ul><ul><ul><ul><li>Fine-grained enforcement </li></ul></ul></ul><ul><ul><ul><li>Performance isolation </li></ul></ul></ul><ul><ul><li>Convenient serialization </li></ul></ul><ul><ul><li>Suspend/resume </li></ul></ul><ul><ul><li>For more see http://workspace.globus.org </li></ul></ul>
    7. 7. Virtual Machines: The Bad News <ul><li>In order to run in the Grid a user now has to provide an image… </li></ul><ul><li>A site administrator now has to maintain potentially many more platforms than before… </li></ul><ul><li>How will the management of all these images scale? </li></ul><ul><li>If a user-provided image were to be deployed, how can it be integrated with its deployment context? </li></ul>
    8. 8. To Have a Cake and Eat It Too… Someone I could trust… Hopefully they can also manage it for me… Assuming I find such a person, how can I adapt this image to actually work with my site? provider Get someone else to configure the image for me…
    9. 9. To Have a Cake and Eat It Too… Assuming I find such a person, will I be able to log in to those image? client Get someone else to configure the image for me… Someone I could trust… Hopefully they can also manage it for me…
    10. 10. Overview <ul><li>Virtual Appliances </li></ul><ul><li>Configuring for contextualization </li></ul><ul><ul><li>Example </li></ul></ul><ul><li>Appliance Configuration and Management </li></ul><ul><li>Appliance Deployment </li></ul><ul><li>Conclusions </li></ul>
    11. 11. Virtual Appliances <ul><li>Environment to support a specific set of applications </li></ul><ul><li>Can be automatically adapted to many different deployment contexts </li></ul><ul><li>Examples of contextualization: </li></ul><ul><ul><li>IP address </li></ul></ul><ul><ul><li>IP adresses of critical services </li></ul></ul><ul><ul><li>Ssh keys </li></ul></ul><ul><ul><li>Security certificates </li></ul></ul>Context IP address SSH keys etc. VM Image
    12. 12. Overall Approach Appliance Producer Appliance Deployment Appliance Management build an appliance update an appliance manage appliance deployment A A’
    13. 13. Applicance Contextualization (Preparation) contextualization agent Contextualization template IP address signed by provider to have properties XYZ certificate
    14. 14. Appliance Contextualization (Deployment) delivery method Contextualization template IP address: etc. Validate signature: do we have properties XYZ ?
    15. 15. Example: Virtual Cluster <ul><li>Torque cluster </li></ul><ul><ul><li>Assign IP addresses </li></ul></ul><ul><ul><li>Create accounts </li></ul></ul><ul><ul><li>Name resolution </li></ul></ul><ul><ul><li>ssh/scp keys for the nodes </li></ul></ul><ul><ul><li>Torque configuration files </li></ul></ul><ul><li>The configuration template is consumed by self-contained Bcfg2 agent inside the VM </li></ul><Parameters> <Param name='DNSServer'> <List> <Item value=''/> <Item value=''/> </List> </Param> <Param name='nodenames'> <List> <Item value=''/> <Item value=''/> <Item value=''/> <Item value=''/> </List> </Param> <Param name='users'> <List> <Item name='user1' value='sad8hgewjnb'/> <Item name='user2' value='saasd2sjnb'/> </List> </Param> </Parameters>
    16. 16. Appliance Provider Software <ul><li>Incremental construction </li></ul><ul><li>Versioning </li></ul><ul><li>Describe capabilities </li></ul><ul><ul><li>Xen? Vmware? </li></ul></ul><ul><li>Testing of appliances </li></ul><ul><li>Maintenance </li></ul><ul><ul><li>Security RSS feed </li></ul></ul><ul><ul><ul><li>Bugtraq, US-CERT Security Advisories </li></ul></ul></ul><ul><li>Attestation and signing </li></ul><ul><ul><li>Automation is important! </li></ul></ul>SL3 OSG TeraGrid STAR CCSM … … SL4 …
    17. 17. Appliance Provider Software <ul><li>Bcfg2 </li></ul><ul><ul><li>Incrementally constructed configuration profiles </li></ul></ul><ul><ul><ul><li>E.g., OS, security services, application </li></ul></ul></ul><ul><ul><li>Node analysis capabilities </li></ul></ul><ul><ul><li>Supplied with many Linux distributions </li></ul></ul><ul><ul><li>http://trac.mcs.anl.gov/projects/bcfg2 </li></ul></ul><ul><li>rPath </li></ul><ul><ul><li>Recipe-style configuration </li></ul></ul><ul><ul><ul><li>Create a project, choose packages, “cook”, build the software appliance </li></ul></ul></ul><ul><ul><li>Freely available online </li></ul></ul><ul><ul><li>Many appliances available, integrated with EC2 </li></ul></ul><ul><ul><li>http://www.rpath.com/rbuilder/ </li></ul></ul>
    18. 18. Appliance Deployment <ul><li>Matching appliances to resources </li></ul><ul><ul><li>What VMM? What kernels? Etc. </li></ul></ul><ul><li>Secure admission of appliances </li></ul><ul><ul><li>Validate signature </li></ul></ul><ul><ul><li>Admission policies and workspace assertions </li></ul></ul><ul><ul><ul><li>E.g., no root access, configuration and versioning assertions </li></ul></ul></ul><ul><ul><li>SC05 Poster: “ Making your workspace secure: establishing trust with VMs in the Grid” </li></ul></ul><ul><li>Contextualization </li></ul><ul><ul><li>Providing contextualization information </li></ul></ul><ul><ul><li>Secure delivery </li></ul></ul><ul><ul><li>Host certificates, virtual clusters, etc. </li></ul></ul>
    19. 19. The Workspace Service Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node (1) The workspace service allows users to dynamically deploy and manage VMs on a pool of nodes <ul><li>(2) A Workspace is deployed based on (a) image + meta-data and (b) resource allocation </li></ul><ul><li>(3) Access is determined based on attribute authorization, image validation work in progress </li></ul><ul><li>(4) Contextualization: </li></ul><ul><li>- Multiple methods of IP address assignment </li></ul><ul><li>Host certificates </li></ul><ul><li>- Personalization work in progress </li></ul>VWS Service
    20. 20. Contextualization and Delivery <ul><li>The deployment software will rely on a range of services </li></ul><ul><ul><li>Certificate authorities, IP management, etc. </li></ul></ul><ul><li>Existing contextualization agents </li></ul><ul><ul><li>DHCP </li></ul></ul><ul><ul><li>Workspace DHCP delivery method </li></ul></ul><ul><li>Ad hoc methods </li></ul><ul><ul><li>E.g., current workspace tools, configuring certificates, etc. </li></ul></ul><ul><li>Configuration tools </li></ul><ul><ul><li>Needed for application-specific tools </li></ul></ul><ul><li>Delivery methods </li></ul><ul><ul><li>Kernel parameters </li></ul></ul><ul><ul><li>Secure communication over the network </li></ul></ul><ul><ul><li>Files </li></ul></ul>
    21. 21. Appliance Layers <ul><li>Layered Appliance </li></ul><ul><ul><li>A set of interdependent layers </li></ul></ul><ul><li>Appliance layers </li></ul><ul><ul><li>Less data needs to travel </li></ul></ul><ul><ul><li>More flexible </li></ul></ul><ul><ul><li>Faster deployment </li></ul></ul><ul><ul><li>Trust management </li></ul></ul><ul><li>Collaborative aspects of configuration </li></ul>System Layer Customization Layer Application Layer VO Layer
    22. 22. New Roles Appliance Producer (Virtual Organization) Appliance Deployer (Resource Providers) build an appliance update an appliance manage appliance deployment
    23. 23. Conclusions <ul><li>Virtualization has the potential to allow providers to reach more users </li></ul><ul><ul><li>Flexibility, fast turnaround, etc. </li></ul></ul><ul><ul><li>Examples: EC2 and others </li></ul></ul><ul><li>Configuration management is increasing in importance important </li></ul><ul><ul><li>Configuration for the masses… </li></ul></ul><ul><ul><li>We have the methods, but they need to be adapted </li></ul></ul><ul><li>The role of VOs will grow </li></ul><ul><ul><li>VO administrators trusted by the sites </li></ul></ul><ul><ul><li>VO security procedures </li></ul></ul>
    24. 24. Credits <ul><li>Workspace team </li></ul><ul><ul><li>Tim Freeman, Borja Sotomayor </li></ul></ul><ul><li>Bcfg2 </li></ul><ul><ul><li>Rick Bradshaw, Narayan Desai </li></ul></ul><ul><li>Thanks to </li></ul><ul><ul><li>Brett Adam, Ian Foster, Frank Siebenlist, Ravi Subramaniam, Marty Wesley </li></ul></ul>