• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
~ Slides Available
 

~ Slides Available

on

  • 314 views

 

Statistics

Views

Total Views
314
Views on SlideShare
314
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Introduced in the 60s, revived in the 90s Paper: “Survey of Virtual Machine Research”, 1976
  • Performance is the main concern that comes up as we talk about VMs. The results above are amazing and come from the paper as referenced. Also: “Xen and the Art of repeated research”: they did independent test of these results and confirmed them. They also showed that the software runs just as well on old and cheap workstations (i.e., the resuts show the same relationship). Benchmarks: 1) SPED INT2000: a CPU intensive benchmark 2) Linux bulid: about 7% in CPU, mainly file i/o, scheduling and memory management 3) Database access PostgreSQL database Open Source Database Benchmark suite on-line transaction processing: considerable load on the OS, many synchronous disk operations resulting in many protection domain transitions 4) SPEC WEB99: web servers, dynamic content, http post, disk workload (not just read-only), exercises the whole system, PCP traffic disk read-write activity, scheduling between many different httpd processes.
  • How do we make all those goodies available remotely?
  • Add security to this slide!

~ Slides Available ~ Slides Available Presentation Transcript

  • Virtual Infrastructure in the Grid Kate Keahey [email_address] Argonne National Laboratory
  • The Grid Metaphor
  • The Grid Metaphor
  • The Grid Metaphor How do we store energy? How do we charge for energy? How do we reliably deliver energy? What happens if a power station fails? How do we ensure quality of service? What elements make for a safe and efficient power Grid? How do we make sure that supply meets demand?
  • Providers and Consumers
    • Providers
      • Own, operate, and contribute physical resources
      • Require incentives to participate
      • Low participation costs
      • Protection from activities of the consumer
      • Ability to control and monitor resource usage
    • Consumers
      • Want on-demand access to computational resources at modest cost
      • The ability to configure them to meet their needs
      • Reasonable guarantees of resource availability
    • Scalability
      • Provider and consumer roles have to be decoupled
  • Requirements for a Grid Execution Environment
    • Environment and configuration
      • A VO should be able to provide the configuration it needs independently of the resource provider
    • Isolation
      • The provider needs to be able to delegate resource usage to the VO so that the VO can’t impact the resource provider -- and thus does not need to be under its control
    • Resource usage and accounting
      • The provider needs to be able to grant, enforce and account for VO resource usage in a way that is independent of how the resource is consumed
    • All of this must be available on-demand!
  • Virtual Workspaces
    • A virtual workspace is an abstraction of an execution environment that can be made dynamically available to authorized clients by using well-defined protocols.
    • Two dimensions:
      • Software configuration
      • Resource quota (CPU, memory, etc.)
    • Examples of Workspaces:
      • A physical machine configured to meet TeraGrid requirements
      • A cluster of virtual machines configured to meet OSG requirements
      • A cluster of physical machines running a hypervisor
  • Virtual Workspace Implementations
    • Physical resources
      • Allocate and configure a physical resource
        • Cluster on Demand (COD), Duke University
        • Bcfg project at ANL
      • As a method they are inflexible and coarse-grained
    • Virtual resources
      • Allocate resources for and deploy configured virtual machines
        • Existing efforts: In-Vigo, Virtuoso, VIOLIN, the workspace project…
      • Much more flexible, allowing migration and fine-grain enforcement.
  • Virtual Machine Basics
    • A VM can serialize all of its state (including RAM)
      • A VM image is simply a collection of files
        • Disk partitions, RAM, configuration file
      • Such image can be easily moved (migrated) between hypervisors of the same type
      • Such image can also be saved and used for rollbacks
    Hardware Virtual Machine Monitor (VMM) / Hypervisor Guest OS (Linux) Guest OS (NetBSD) Guest OS (Windows) App App App App App VM VM VM
  • The Need for Speed Paper: “Xen and the Art of Virtualization”, SOSP 2003 L X V U SPEC INT2000 (score) L X V U Linux build time (s) L X V U OSDB-OLTP (tup/s) L X V U SPEC WEB99 (score) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)
  • Summary: What Makes VMs a Great Workspace Implementation
    • VM properties:
      • Excellent isolation
        • Generally enhanced security, audit forensics
      • Fine-grain enforcement potential
        • Details depend on implementation
      • Customizable software configuration
        • Library signature, OS, maybe even 64/32-bit architectures
      • Serialization property
        • VM images (include RAM), can be copied
      • The ability to pause and resume computations
        • Allow migration
    • How do we make VMs available over the network and manage them so as to leverage this potential?
      • Challenges: security, enforcement, protocols
  • Deploying Workspaces in the Grid Workspace Wizard (VW Factory) Workspace Management Service (VW Repository) Workspace Service (VW Manager) manage workspace environment workspace metadata terminate workspace deployment manage activities within the workspace Workspace request a workspace workspace meta-data negotiate workspace deployment manage/monitor/renegotiate workspace deployment
  • Workspace Implementation
    • Protocols : Web Service Resource Framework (WSRF)
      • An extension of Web Services
      • Standard mechanisms for creation, inspection, notification, lifetime management
      • Globus Toolkit 4 implementation
        • Provides secure authentication, authorization as well as tools for fast transfer, replica management, monitoring, and others.
    • Creating a workspace
      • workspace meta-data (workspace image)
      • deployment descriptor (resource allocation)
    • Managing a workspace
      • renegotiate resource allocation
      • Standard WSRF management functions
    • Challenges: resource assignment, negotiation, etc.
    • To download visit http://workspace .globus.org
  • Putting it All Together R R R R R R R R R A C B VM1 VM2 VM3 VM4 resources deployment capabilities virtual machines jobs B D B E deployment capabilities
    • Deploying a workspace requires and creates a deployment capability
      • Required capability is described in workspace pre-requisites
    • Workspaces can be layered
  • Applications: Edge Services (1)
    • Edge Service: service executing on the edge of private and public network
    • ESF Requirements
      • Diverse configurations, easy to upgrade
      • Good potential for managing resource allocation
    • Status:
      • Testbed: SDSC, FNAL, UC
      • Multiple base images have been developed
      • One Edge Service deployed
      • Workspace Service developed
    • Timeframe: ~few months
  • Applications: Edge Services (2)
  • Applications: Virtual Clusters
    • Extends the abstraction of a workspace to a virtual cluster
    • Deploys a cluster on a site
      • SLURM, PBS/Torque implementations
      • Configures networking, shared storage
      • Image propagation, main deployment cost
    • Tech report available
  • Virtual Cluster: OSG Applications GADU fMRI Montage FOAM
  • Conclusions
    • In order to grow, we need to scale
    • In order to scale, we need to provide a reliable tool for separating producers and consumers
    • Virtualization provides a useful, scalable tool to decouple providers and consumers
      • Workspaces as physical resources
      • Workspaces as virtual machines
    • Looking forward
      • Grid economies