Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Performance is the main concern that comes up as we talk about VMs. The results above are amazing and come from the paper as referenced. Also: “Xen and the Art of repeated research”: they did independent test of these results and confirmed them. They also showed that the software runs just as well on old and cheap workstations (i.e., the resuts show the same relationship). Benchmarks: 1) SPED INT2000: a CPU intensive benchmark 2) Linux bulid: about 7% in CPU, mainly file i/o, scheduling and memory management 3) Database access PostgreSQL database Open Source Database Benchmark suite on-line transaction processing: considerable load on the OS, many synchronous disk operations resulting in many protection domain transitions 4) SPEC WEB99: web servers, dynamic content, http post, disk workload (not just read-only), exercises the whole system, PCP traffic disk read-write activity, scheduling between many different httpd processes.
  • disk is easy new IO isn’t as fast as not all ring 0 ; win big time
  • Distribution/installation: using Xen requires patching the kernel in an installation but everything else stays the same. This means that there will be NO changes from the perspective of an application. So for example if we take fedora distribution and patch the kernel, everything will run exactly as before. Quite apart from that major Linux distributions are making efforts to package Xen and I tend to think it should make it to the Linux kernel within the next six months. Another question is: do I need root to run VMs? The answer is that yes, it does require an administrator to set it up (to the admins great relief I should imagine ;-). And with Xen it would require for an admin to also install Grid software that stands up client VMs --- not that very different from what we do today.
  • Provenance of VM images: an image may be configured by several different “actors” that we may or may not trust. Instance equality: if two instances have same image and everything else, but different names they are not equal. Signing: signing the image, digest is part of meta-data, signing the instance.
  • Three scenarios: Preconfigured job means we arrange for job startup as part of the boot process (so as soon as the VM boots, the job starts executing).All it takes is for the VM Manager to load up the VM. We prepared a VM, started the GT4 environment, and then paused it. All it takes is for the VM Manager to load up the VM, then the client makes a GRAM request. Regular job startup using GRAM With Xen the actual “unpause” operation takes about .1 second --- way less than processing the request on the server side. It actually takes more time to process a GRAM request because GRAM has a generic interface whereas starting a VM just takes an image and runs with it --- all the processing is embedded in the image itself. Unless the boot is needed for some site-specific configuration (assignment of IP addresses) it can also be “pre-computed” in which case starting up a preconfigured job in a VM would take about as much time as starting a job with GRAM
  • The previous slide shows some preliminary numbers for starting VMs. BUT: They don’t include encryption (yet) Moving images is moving a lot more bytes than your average input data I think, several strategies for dealing with that: You can cache some “frequently used images” on-site or close and only ship the “diff” of an image from a standard configuration (proposed by Rosenblum) If you operate on cold images then you can essentially use the proximity of a desirable image as scheduling criterion You can mount partitions (you can shlep the whole environment with you if you want but you don’t HAVE TO!) Scalabilty: interesting summer project
  • Basic message: we have done very well in terms of scalability in latency, bandwidth, data, etc. Time to scale in numbers of users, installations, Grids, projects, etc. For that effortless execution environments will be critical. Lots of emergent stuff.
  • ppt

    1. 1. Working Spaces: Virtual Machines in the Grid Kate Keahey [email_address] Argonne National Laboratory Tim Freeman, Frank Siebenlist {tfreeman, franks}@mcs.anl.gov
    2. 2. Towards Realizing the Grid Vision <ul><li>Quality of Service </li></ul><ul><ul><li>Dynamically controlled enforcement of various qualities </li></ul></ul><ul><ul><li>Not just per-process enforcement </li></ul></ul><ul><li>Quality of Life </li></ul><ul><ul><li>Being able to find the right configuration on the Grid </li></ul></ul>
    3. 3. We Need a Workspace! <ul><li>A configurable execution environment, container </li></ul><ul><ul><li>Good isolation properties </li></ul></ul><ul><ul><li>Good enforcement potential </li></ul></ul><ul><ul><li>Customizable software configuration </li></ul></ul><ul><ul><ul><li>Library signature, OS, maybe even 64/32-bit architectures </li></ul></ul></ul><ul><li>We need to be able to create, manage and deploy it </li></ul><ul><ul><li>We need to be able to negotiate/renegotiate an environment shape with a VO </li></ul></ul><ul><ul><li>A broker would then be able to negotiate resource allocations and map these workspaces onto </li></ul></ul>
    4. 4. Virtual Machines (VMs) <ul><li>VMs happen to have: </li></ul><ul><ul><li>Good isolation properties </li></ul></ul><ul><ul><ul><li>Generally enhanced security, audit forensics </li></ul></ul></ul><ul><ul><li>Good enforcement potential </li></ul></ul><ul><ul><li>Customizable software configuration </li></ul></ul><ul><ul><ul><li>Library signature, OS, maybe even 64/32-bit architectures </li></ul></ul></ul><ul><ul><li>Serialization property </li></ul></ul><ul><ul><ul><li>VM images (include RAM), can be copied </li></ul></ul></ul><ul><ul><li>The ability to pause and resume computations </li></ul></ul><ul><ul><ul><li>Allow migration </li></ul></ul></ul><ul><li>Common concern: </li></ul><ul><ul><li>Overhead: application, startup, resource usage </li></ul></ul>
    5. 5. Virtual Machines Primer <ul><li>Different types of virtual machines </li></ul><ul><ul><li>Full virtualization (VMware) </li></ul></ul><ul><ul><ul><li>Run multiple unmodified guest OSs </li></ul></ul></ul><ul><ul><li>Para-virtualization (Xen, UML, Denali) </li></ul></ul><ul><ul><ul><li>Run multiple guest OSs ported to a special architecture </li></ul></ul></ul><ul><ul><li>Single OS image (Vserver) </li></ul></ul><ul><li>Paper: “ From Sandbox to Playground: Dynamic Virtual Environments in the Grid”, Grid 2004 </li></ul>Hardware Virtual Machine Monitor (VMM) Guest OS (Linux) Guest OS (NetBSD) Guest OS (Windows)
    6. 6. The Need for Speed Paper: “Xen and the Art of Virtualization”, SOSP 2003 L X V U SPEC INT2000 (score) L X V U Linux build time (s) L X V U OSDB-OLTP (tup/s) L X V U SPEC WEB99 (score) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)
    7. 7. TCP results L X V U Tx, MTU 1500 (Mbps) L X V U Rx, MTU 1500 (Mbps) L X V U Tx, MTU 500 (Mbps) L X V U Rx, MTU 500 (Mbps) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 TCP bandwidth on Linux (L), Xen (X), VMWare Workstation (V), and UML (U)
    8. 8. Scalability L X 2 L X 4 L X 8 L X 16 0 200 400 600 800 1000 Simultaneous SPEC WEB99 Instances on Linux (L) and Xen(X)
    9. 9. Other Concerns <ul><li>License </li></ul><ul><ul><li>Open source (Xen, UML) </li></ul></ul><ul><ul><ul><li>Visible effects of open source community at work </li></ul></ul></ul><ul><ul><li>Commercial (VMware) </li></ul></ul><ul><ul><ul><li>Also, XenSource </li></ul></ul></ul><ul><li>Distribution/Installation </li></ul><ul><ul><li>Para-virtualization requires kernel modifications </li></ul></ul><ul><ul><ul><li>Yes, but … everything else stays the same </li></ul></ul></ul><ul><ul><ul><li>Xen is soon to be part of the Linux kernel, Fedora Core 4 (May ’05), is in Debian unstable, unofficial: Gentoo, Mandrake and SUSE distributions </li></ul></ul></ul><ul><ul><li>Privilege </li></ul></ul><ul><ul><ul><li>Xen (root, patch kernel, domain 0 privileges setup) </li></ul></ul></ul><ul><ul><ul><li>VMware Workstation (root, installation only) </li></ul></ul></ul>
    10. 10. New Technology, New Challenges <ul><li>How can we leverage the benefits of VMs in Grid technology? </li></ul><ul><li>How efficient will be this new technology? </li></ul><ul><li>How can we ensure a secure environment under these new assumptions? </li></ul><ul><li>How well will it all work in practice? </li></ul><ul><li>What new scenarios will this enable? </li></ul><ul><li>How will it change Grid computing? </li></ul><ul><li>What new problems will it create? </li></ul>
    11. 11. Integrating VMs into the Grid Architecture Client VM EPR inspect and manage Create VM image VM Factory VM Repository VM Manager create new VM image Resource VM start program request deploy & suspend use existing VM image
    12. 12. Supporting Services <ul><li>Factory </li></ul><ul><ul><li>Creates VM images </li></ul></ul><ul><ul><ul><li>Eventually it may have to support negotiation </li></ul></ul></ul><ul><ul><li>Images may be created based on an already existing image </li></ul></ul><ul><li>VM Repository </li></ul><ul><ul><li>Access to state describing a VM </li></ul></ul><ul><ul><li>Allows inspection, management, termination, potentially renegotiation, etc. </li></ul></ul><ul><li>VM Manager </li></ul><ul><ul><li>Service deploying VMs on nodes </li></ul></ul><ul><ul><li>Operations: stage, start, pause, stop, checkin </li></ul></ul><ul><li>Once deployed, jobs may be executed in the virtual machines in a variety of ways </li></ul>
    13. 13. Workspace Structure <ul><li>Elements of a workspace </li></ul><ul><ul><li>Workspace description (meta-data) </li></ul></ul><ul><ul><ul><li>EPR/name, category, state, etc., also hardware, network, software configuration, etc. </li></ul></ul></ul><ul><ul><li>Workspace implementation (VM image) </li></ul></ul><ul><li>Workspace “types” </li></ul><ul><ul><li>Workspaces conforming to set configurations </li></ul></ul><ul><ul><li>Provenance of VM images </li></ul></ul><ul><li>Workspace instances </li></ul><ul><ul><li>Workspace meta-data contains a name </li></ul></ul><ul><ul><li>Instance equality </li></ul></ul><ul><ul><li>Copying operation: copy image, meta-data, create a new name </li></ul></ul><ul><ul><li>Signing instances </li></ul></ul>
    14. 14. Security: New Opportunities, New Problems <ul><li>VMs introduce a new layer of trust </li></ul><ul><ul><li>VM monitor needs to be trusted as a technology </li></ul></ul><ul><ul><li>Trusted computing </li></ul></ul><ul><li>VMs can be serialized and transferred as data </li></ul><ul><ul><li>The integrity of a VM image needs to be protected (signing) </li></ul></ul><ul><ul><li>Private information on a VM image need to be protected (encryption) </li></ul></ul><ul><ul><ul><li>VM private key: should a VM be able to assert its identity? </li></ul></ul></ul><ul><ul><ul><li>Application private keys, security context </li></ul></ul></ul><ul><li>VMs can be migrated (source --- > target) </li></ul><ul><ul><li>Trust management: a VM image may be moved between parties that don’t trust each other </li></ul></ul><ul><ul><ul><li>A popular problem </li></ul></ul></ul><ul><ul><li>Key management: target VM needs to verify the integrity and identity of a VM image in ways acceptable to the client: key management </li></ul></ul><ul><ul><li>Security context has to be preserved or renegotiated </li></ul></ul>
    15. 15. Migrating Securely target VM image in transfer: Signed and encrypted VM vouched for by source VM vouched for by target reestablish security context
    16. 16. Security: New Twist on Old Problems <ul><li>Deployment problem </li></ul><ul><ul><li>Protecting the VM from the world </li></ul></ul><ul><ul><ul><li>VMs are only as secure as the software they run </li></ul></ul></ul><ul><ul><ul><li>Who maintains all those VMs? Local administrators would have to maintain too many images yet need to protect against vulnerabilities </li></ul></ul></ul><ul><ul><li>Protecting the world from the VM </li></ul></ul><ul><ul><ul><li>One could use one’s privileges as root on a VM (for example to generate harmful network traffic) </li></ul></ul></ul><ul><ul><ul><li>Although audit works great by the time the damage is done it is too late! </li></ul></ul></ul><ul><li>Deployment solutions </li></ul><ul><ul><li>VO certification and authorization </li></ul></ul><ul><ul><ul><li>Certification Authority to certify VM image </li></ul></ul></ul><ul><ul><ul><li>Site policies may require VMs to conform to site policies </li></ul></ul></ul><ul><ul><li>Detection: Intrusion Detection Systems </li></ul></ul><ul><ul><li>Actions </li></ul></ul><ul><ul><ul><li>Restricting network traffic: putting the good guys in jail </li></ul></ul></ul><ul><ul><ul><li>Right to take action </li></ul></ul></ul>
    17. 17. Job Startup <ul><li>c) job startup using a pre-configured job </li></ul><ul><li>b) job startup using an unpaused VM </li></ul><ul><li>a) job startup using GRAM </li></ul>
    18. 18. Things We Didn’t Talk About <ul><li>Security </li></ul><ul><ul><li>Processing of encryption and signing </li></ul></ul><ul><li>Moving images </li></ul><ul><ul><li>Image size: starting at 1MB, more typically 200 MB and upwards </li></ul></ul><ul><ul><li>Image diff (Rosenblum) </li></ul></ul><ul><ul><li>Proximity of image as matching criterion (VMPlant) </li></ul></ul><ul><ul><li>Mounting partitions (general on-site assembly) </li></ul></ul><ul><li>Scalability </li></ul><ul><ul><li>Distribute processes among existing VMs </li></ul></ul><ul><ul><li>Lightweight VMs: Denali </li></ul></ul><ul><li>Clusters </li></ul><ul><ul><li>Currently in progress: work on virtual cluster, collaboration with the COD team at Duke </li></ul></ul>
    19. 19. How does it work in practice? <ul><li>Recent project: combining VMs and Grids to create a platform for bioinformatics applications </li></ul><ul><li>Some of the conclusions: </li></ul><ul><ul><li>Use of virtual machines can significantly broaden the resource base </li></ul></ul><ul><ul><li>Saves installation time </li></ul></ul><ul><ul><ul><li>EMBOSS installation: ~45 minutes </li></ul></ul></ul><ul><ul><ul><li>Deploying a 2GB VM image: ~6.5 minutes </li></ul></ul></ul><ul><ul><ul><li>Peace of mind: priceless! </li></ul></ul></ul><ul><ul><li>Enforcement capabilities </li></ul></ul><ul><ul><ul><li>Depend on the implementation but are generally better than what we have now </li></ul></ul></ul><ul><li>SC04 poster: </li></ul><ul><ul><li>“ Quality of Life in the Grids: VMs meet Bioinformatics Applications”, D. Galron et al </li></ul></ul>
    20. 20. Virtual Playgrounds <ul><li>Define a virtual Grid in terms of requirements </li></ul><ul><ul><li>Virtual workspaces </li></ul></ul><ul><ul><li>Networking requirements, virtual network </li></ul></ul><ul><ul><li>Storage and other requirements </li></ul></ul><ul><li>Provide mechanisms to create a Grid </li></ul><ul><li>Provide services for the deployment of such “virtual playgrounds” on real resources </li></ul><ul><li>Ephemeral Grids built for a special purpose: </li></ul><ul><ul><li>Scientists getting up a Grid for the purposes of a specific experiment run </li></ul></ul><ul><ul><li>A scientific simulation that gets discarded or interrupted but can potentially be restored later </li></ul></ul>
    21. 21. Conclusions <ul><li>For Grids to scale we need a way to create and manage remote environments in the dynamically and effortlessly </li></ul><ul><li>Virtual is the new Real! </li></ul><ul><ul><li>VMs present a very compelling solution </li></ul></ul><ul><ul><ul><li>Efficiency, flexibility, migration, etc. </li></ul></ul></ul><ul><ul><li>… and introduce some new challenges </li></ul></ul><ul><ul><ul><li>New services, different models of sharing, security, etc. </li></ul></ul></ul><ul><li>Watch out for emergent properties and behaviors! </li></ul><ul><li>We need to work with the VM community to fine-tune requirements and features </li></ul><ul><ul><li>Open Source helps! </li></ul></ul><ul><li>A growing role for Virtual Organizations </li></ul><ul><li>Policy, Policy, Policy… </li></ul><ul><ul><li>Policy of resource owners, VOs, users… </li></ul></ul><ul><li>Closer to the dream on seamlessly negotiating, provisioning, renegotiating… </li></ul><ul><li>Status: a GT4 prototype that we keep evolving! </li></ul>
    22. 22. Related Efforts <ul><li>Condor </li></ul><ul><ul><li>ClassAds, glide-ins </li></ul></ul><ul><li>In-Vigo (VMPlant) </li></ul><ul><li>Virtuoso (VNET) </li></ul><ul><li>VIOLIN (UML + private networks) </li></ul><ul><li>Cluster on Demand </li></ul><ul><li>Workspaces Project </li></ul><ul><ul><li>www.mcs.anl.gov/workspace </li></ul></ul>
    23. 23. Credits <ul><li>Actively working: </li></ul><ul><ul><li>Tim Freeman </li></ul></ul><ul><ul><li>Frank Siebenlist </li></ul></ul><ul><ul><li>Xuehai Zhang </li></ul></ul><ul><li>Past contributions: </li></ul><ul><ul><li>Karl Doering (UCSD) </li></ul></ul><ul><ul><li>Daniel Galron (OSU) </li></ul></ul>