Your SlideShare is downloading. ×
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Maybe add agreements to this picture: workspaces are agreements, an agreement for resource usage is formed, a binding agreement is formed, the workspace is deployed Very important: the binding agreement can be renegotiated
  • Performance is the main concern that comes up as we talk about VMs. The results above are amazing and come from the paper as referenced. Also: “Xen and the Art of repeated research”: they did independent test of these results and confirmed them. They also showed that the software runs just as well on old and cheap workstations (i.e., the resuts show the same relationship). Benchmarks: 1) SPED INT2000: a CPU intensive benchmark 2) Linux bulid: about 7% in CPU, mainly file i/o, scheduling and memory management 3) Database access PostgreSQL database Open Source Database Benchmark suite on-line transaction processing: considerable load on the OS, many synchronous disk operations resulting in many protection domain transitions 4) SPEC WEB99: web servers, dynamic content, http post, disk workload (not just read-only), exercises the whole system, PCP traffic disk read-write activity, scheduling between many different httpd processes.
  • Distribution/installation: using Xen requires patching the kernel in an installation but everything else stays the same. This means that there will be NO changes from the perspective of an application. So for example if we take fedora distribution and patch the kernel, everything will run exactly as before. Quite apart from that major Linux distributions are making efforts to package Xen and I tend to think it should make it to the Linux kernel within the next six months. Another question is: do I need root to run VMs? The answer is that yes, it does require an administrator to set it up (to the admins great relief I should imagine ;-). And with Xen it would require for an admin to also install Grid software that stands up client VMs --- not that very different from what we do today.
  • Transcript

    • 1. Workspaces for CE Management Kate Keahey [email_address] Argonne National Laboratory
    • 2. Why Workspaces?
      • We need to be able to dynamically create an execution environment on remote resources
      • The aspects of workspaces:
        • Quality of Service: isolation and enforcement
        • Quality of Life: providing the right configuration at the right time
      Dream up an ideal environment run jobs Ideal environment is deployed in just the right place magic happens
    • 3. What are Virtual Workspaces?
      • A description of an execution environment
        • Software configuration requirements
          • OSG worker node, submit node for a Grid3 cluster
        • Resource allocation requirements
          • Use exactly X memory, at least Y disk space, Z bandwidth…
        • Sharing and isolation properties
          • Unix account, sandbox, various kinds of virtual machines …
        • And others…
      • Basic workspace example : a Unix account on a remote machine
      • Workspace can be managed and refined
        • In terms of lifetime, meta-data, access policies…
      • A workspace can be deployed on a resource
      • Jobs can be deployed in a workspace
      • A workspace can have various implementations
        • Dynamic accounts & configuration tools
          • Pacman, SoftEnv, Softricity
        • Virtual Machines
    • 4. Binding Wrokspaces to Resources Client CPU = Pentium Memory = 1GB Disk = 20GB Find a resource advertising the required capability capability description CPU >= 60% Pentium Memory = 256 MB Reserve capability vw1 capability description CPU >= 30% Pentium Memory = 512 MB Reserve capability for vw2 workspace description CPU >= 60% Pentium Memory = 256 MB Software. = OSG SE v1 workspace description CPU >= 30% Pentium Memory = 512MB Software = exp Bind to reservation Bind to reservation
    • 5. VW Implemenations: Virtual Machines
      • Advantages
        • Customizable software configuration
          • Library signature, OS, 64/32-bit architectures
        • Excellent enforcement potential
          • Enforcement on a sandbox rather than process level
        • Excellent isolation
          • Generally enhanced security, audit forensics
        • Pausing, serialization, and migration
          • VM images (include RAM), can be copied
      • Available implementations
        • Commercial (VMware)
        • Open source (Xen, UML)
          • Also support for Xen from XenSource and many Linux distributors
      • Xen is rapidly emerging as the most popular implementation
        • The fastest, freeest, the most open source, the most backed…
    • 6. The Need for Speed L X V U SPEC INT2000 (score) L X V U Linux build time (s) L X V U OSDB-OLTP (tup/s) L X V U SPEC WEB99 (score) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)
    • 7. DRAG Benchmark Results
      • DRAG suite: FFT-based benchmark
      • Comparison (by Xuehai Zhang, UC):
        • Linux: machine runs native 2.6 Linux.
        • Dom0: machine runs Xen and domain 0.
        • DomU: machine runs Xen, domain 0 and a user domain.
      • Similar performance as native Linux
        • <3% degradation, but sometimes actually better than native Linux
      • More details at
    • 8. Deployment Concerns
      • Distribution/Installation
        • Para-virtualization (Xen) requires kernel modifications
          • Yes, but … everything else stays the same
          • Work in progress on making Xen part of Linux kernel
            • False information of its conclusion seen recently!
          • Support from many Linux distributors: Fedora, Debian, SUSE, Gentoo, Mandrake, etc.
      • Privilege level(Xen)
        • Domain0 is a privileged domain, not a good environment for sharing.
        • If Xen configuration is going to be permanent using DomainU is recommended
          • Performance impact needs to be considered
    • 9. The Xen of Enforcement
      • CPU
        • Schedulers: BVT, FBVT, Round Robin, Atropos/SEDF
        • May be selected at boot time; BVT is default
        • Borrowed Virtual Time (BVT)
          • Fair share of CPU based on weights assigned to the domains
          • Work-conserving
        • Simple Earliest Deadline First (SEDF)
          • Reserves absolute shares of CPU for domains
      • Memory
        • Memory size specified in a configuration file
        • Can be readjusted from domain0
      • Disk
        • Export partitions
        • Logical Volume Manager (LVM) allows to grow and shrink the disk size
      • Networking
        • Standard Linux deployment tools: Domain0 can do traffic shaping for user domains.
    • 10. Workspace as a CE Environment CE WSS LCAS LCMAPS GRAM CE WSS LCAS LCMAPS GRAM headnode VW Manager CE WSS LCAS LCMAPS GRAM VW Manager CE broker
    • 11. Pros and Cons
      • Problems that VMs solve for us
        • Environment management
          • Configuration management
            • Running two different versions of CE software side-by-side
          • Enforcement and isolation
        • Graceful load management
          • Renegotiating the resource allocation
          • Live migration across nodes
      • Problems that VMs don’t solve for us
        • Job management: jobs within an environment still need to be managed
          • Job throttling
          • Job persistence, restart, etc.
        • The cost of perfect enforcement
          • Each CE will run a copy of similar services leading to potential inefficiencies
          • There has been some work in sharing e.g. shared libraries between VMs, but is inconclusive right now
    • 12. Other things VMs are not helpful for…
      • Digestion
      • Walking your dog
      • Laundry
      • Most sports
      • Painting walls
      • Knitting
      • Grocery shopping (unless you shop via peapod)
    • 13. Meanwhile, in a galaxy far, far away…
      • Similar ideas in US projects
        • Edge Services
          • effort led by Frank Wuertherwein in the context of OSG/CMS
        • Management of submit nodes
          • work with Rob Gardner and Mike Wilde in the context of Grid3/Atlas
      • Requirements:
        • Install and manage a complex configuration
          • Easy upgrades based on pre-configured images, consistent configuration across sites, version management, etc.
        • Control of resources
          • Guaranteed dedicated use of resources
        • Flexible load balancing
          • Widening the interface to a cluster based on need
    • 14. Edge Services
      • Edge Services: Services executing on the edge of a private/public network boundary
        • Typical configuration of today’s resources
          • Resources within a site are available only on a private network
          • Site can be accessed through a limited number of public addresses
        • Examples: CE, SE, GK, and others
      • Edge Services will be deployed in VM-based workspaces
        • Role-based deployment
        • Initially no advance reservations, no load balancing
        • A proof-of-concept activity
      • Draft document available:
        • =167
    • 15. Submit Node Management
      • Similar to the Edge Service activity with particular emphasis on:
        • Configuration management
          • Configure once, copy and deploy many times
        • Load balancing: widening the submit bottleneck to clusters based on need
    • 16. Conclusions
      • Workspaces solve environment management problems
        • Configuration management
          • Configure once, copy and deploy many times
          • Upgrading service versions
          • Running conflicting or incompatible services side-by-side
        • Enforcement
          • Guaranteed resources with respect to other users
      • There seems to be a confluence of ideas
        • Similar ideas in three different contexts
        • Coincidence?
      • Workspaces do not solve job management problems
        • Which leads into subjects we’ll talk about later…