• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Scaling Xen within Rackspace Cloud Servers
 

Scaling Xen within Rackspace Cloud Servers

on

  • 127,371 views

Rackspace has years of experience with running Xen at scale, starting with Xen and migrating to XenServer. We will share why we use Xen/XenServer along with some of the issues that we've experienced. ...

Rackspace has years of experience with running Xen at scale, starting with Xen and migrating to XenServer. We will share why we use Xen/XenServer along with some of the issues that we've experienced. We will touch on our experience with migrating from Xen to XenServer and the challenges there. We will share information about Rackspace Cloud Servers architecture, and touch briefly on OpenStack when doing so. We will explain how we use Xen to quickly deploy new Openstack services with what we call Nova on Nova. And finally, we will discuss what additional features and improvements are needed and why.

Statistics

Views

Total Views
127,371
Views on SlideShare
4,557
Embed Views
122,814

Actions

Likes
4
Downloads
107
Comments
0

36 Embeds 122,814

http://www.xen.org 97366
http://xen.org 21552
http://www.xenproject.org 2826
http://xen.xensource.com 307
http://translate.googleusercontent.com 295
http://xenproject.org 250
http://staging.xen.org 111
http://xenorg.cloudaccess.net 28
http://www-archive.xenproject.org 17
http://131.253.14.66 8
http://www.xen.org. 6
http://blog.xen.org 6
http://webcache.googleusercontent.com 6
http://50.57.168.107 4
http://irq.tumblr.com 4
http://xen.org. 3
http://xen.org.sixxs.org 3
http://xen 2
http://safe.tumblr.com 2
http://mail.xen.org 2
http://xen.xensource.com&_=1349886236015 HTTP 1
https://www.google.it 1
http://www.xenproject.org. 1
http://131.253.14.98 1
http://xen.org&_=1348016971135 HTTP 1
http://74.6.238.254 1
http://www.xen.org&_=1352783455005 HTTP 1
http://www.xen.org&_=1352783478143 HTTP 1
https://www.google.com&_=1352570042024 HTTP 1
http://www.xen.org&_=1352285117656 HTTP 1
http://www.xen.org&_=1352285259296 HTTP 1
https://translate.googleusercontent.com 1
http://www.xen.org.sixxs.org 1
http://www.xen.org&_=1348163584571 HTTP 1
http://www.google.com&_=1349622827067 HTTP 1
https://twitter.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Scaling Xen within Rackspace Cloud Servers Scaling Xen within Rackspace Cloud Servers Presentation Transcript

    • Scaling Xen withinRackspace Cloud Servers Chris Behrens chris.behrens@rackspace.comXenSummit 2012, San Diego, CA USAAugust 28, 2012 1
    • Agenda•  Intro to Rackspace Cloud Servers•  Why Xen?•  Xen to XenServer•  OpenStack Basics•  OpenStack at Rackspace•  Scaling OpenStack Deployment•  Future RACKSPACE® HOSTING | WWW.RACKSPACE.COM 2
    • Intro to Cloud Servers RACKSPACE® HOSTING | WWW.RACKSPACE.COM 3
    • Intro to Cloud ServersOverview•  Public Cloud product under Rackspace’s Open Cloud portfolio•  First Gen –  Code originated from Slicehost acquisition –  Initially Xen 3.x, but migrated to XenServer•  Next Gen –  Public Launch 8/1/2012 –  Powered by OpenStack•  XenServer 6•  Linux, Windows, and FreeBSD VMs RACKSPACE® HOSTING | WWW.RACKSPACE.COM 4
    • Intro to Cloud ServersStatistics•  180,000+ total Rackspace customers (not just Cloud Servers)•  Tens of thousands of hosts•  Hundreds of thousands of VMs•  Millions of snapshots RACKSPACE® HOSTING | WWW.RACKSPACE.COM 5
    • Why Xen? RACKSPACE® HOSTING | WWW.RACKSPACE.COM 6
    • Why Xen?•  Thin hypervisor layer•  Open source•  Excellent driver support due to Linux dom0•  Good API•  Good performance•  Along with Linux, Windows VMs are a must –  Same virtualization technology desired for Windows –  Microsoft will support –  Good PV driver support RACKSPACE® HOSTING | WWW.RACKSPACE.COM 7
    • Xen to XenServer RACKSPACE® HOSTING | WWW.RACKSPACE.COM 8
    • Xen to XenServerCloud Servers use of Xen•  LVM –  Logical volumes per VM•  Base images stored as tar files•  Kernel and ramdisk outside of images•  Shell out to run ‘xm’ commands•  Snapshots w/ LVM –  Mounted and contents rsynced to another host•  No agent –  Modifications inside images done via mounting powered off VM RACKSPACE® HOSTING | WWW.RACKSPACE.COM 9
    • Xen to XenServerXen to XenServer changes•  VHDs for virtual disk images –  Tarred and gzipped•  Kernel and ramdisk inside images•  Snapshots via XenAPI –  Whole VDI chain wrapped in OVF container for upload to glance•  Agent inside VM images –  Communication via Xenstore –  File injection, password resets, networking setup RACKSPACE® HOSTING | WWW.RACKSPACE.COM 10
    • Xen to XenServerMigrating•  Pick a XenServer host•  Create VDI•  Attach VDI to dom0•  Partition, mkfs, and mount the new virtual disk•  LVM Snapshot original Xen VM and mount it•  rsync snapshot contents to mounted VDI on new host•  Power off original Xen VM and mount its virtual disk•  rsync again to new host to catch changes since snapshotting•  Extract a tar file into mounted VDI containing kernel, ramdisk, agent•  Unmount and detach VDI from dom0•  Create new VM record and attach VDI•  Start the new VM RACKSPACE® HOSTING | WWW.RACKSPACE.COM 11
    • OpenStack The Basics RACKSPACE® HOSTING | WWW.RACKSPACE.COM 12
    • OpenStackComponents•  Nova –  Compute controller with various hypervisor drivers•  Glance –  VM Image registry•  Swift –  Object storage•  Quantum/Melange –  Networking•  Cinder –  Volumes/Block storage•  Keystone –  Authentication –  Authorization –  API Endpoint discovery RACKSPACE® HOSTING | WWW.RACKSPACE.COM 13
    • OpenStackTypical OpenStack Deployment keystone nova glance nova-api glance-api glance-registry RabbitMQ MySQL nova-scheduler MySQL nova-network nova-compute swift RACKSPACE® HOSTING | WWW.RACKSPACE.COM 14
    • OpenStacknova-compute w/ XenAPI•  Runs in a utility domU on each host•  Images downloaded in dom0 Physical host running XenServer" –  glance XenAPI plugin•  Images attached to nova-compute dom0" nova utility domU" domU –  Partitioning changes glance plugin nova-compute –  file-system resizing agent plugin•  Monitors VMs’ power_state•  Snapshots/backups xenstore plugin nova instance•  VM resize/migration domUs –  rsync to new host•  Communicates w/ agent inside VMs –  Via xenstore RACKSPACE® HOSTING | WWW.RACKSPACE.COM 15
    • OpenStackXenAPI VM Calls•  VM.start / VM.start_on •  VM.add_to_VCPUs_params•  VM.destroy •  VM.get_VBDs•  VM.clean_reboot •  VM.get_record•  VM.hard_reboot •  VM.get_by_uuid•  VM.clean_shutdown •  VM.get_by_name_label•  VM.hard_shutdown •  VM.set_name_label•  VM.pause •  VM.add_to_xenstore_data•  VM.unpause •  VM.remove_from_xenstore_data•  VM.suspend •  VM.assert_can_migrate•  VM.resume •  VM.migrate_send•  VM.snapshot •  VM.pool_migrate RACKSPACE® HOSTING | WWW.RACKSPACE.COM 16
    • OpenStackRackspace Specifics RACKSPACE® HOSTING | WWW.RACKSPACE.COM 17
    • OpenStack Rackspace SpecificsSource Code•  Rackspace tracks trunk –  Deployed code generally less than 2 weeks behind trunk•  Some custom patches on top of trunk –  Features specific to Rackspace –  Custom scheduling to meet Rackspace needs –  Scaling with OpenStack Compute Cells RACKSPACE® HOSTING | WWW.RACKSPACE.COM 18
    • OpenStack Rackspace SpecificsOpenStack Compute Cells•  Scaling beyond small nova deployments –  Effectively ties multiple nova deployments together•  Developed by Rackspace•  Targeted for Grizzly OpenStack release•  Hierarchical tree –  Multiple parents supported•  API cell(s) at top of the tree•  Compute cell(s) underneath•  DB and RabbitMQ per cell•  Scalability –  Splits connections to DB and RabbitMQ –  Less work to schedule new VM placement RACKSPACE® HOSTING | WWW.RACKSPACE.COM 19
    • OpenStack Rackspace Specifics OpenStack Nova Multi-cell Diagram API Cell RabbitMQ MySQL nova-api nova-cellsCompute Cell 1" Compute Cell 2" nova-cells nova-cells RabbitMQ RabbitMQ MySQL MySQL nova-scheduler nova-scheduler nova-network nova-network nova-compute nova-compute RACKSPACE® HOSTING | WWW.RACKSPACE.COM 20
    • Deploying OpenStack RACKSPACE® HOSTING | WWW.RACKSPACE.COM 21
    • Deploying OpenStack‘Inception’•  Private internal cloud –  Based on Openstack, called iNova –  OpenStack services for public cloud run virtualized under iNova•  Easy to spin up additional services for load balancing –  Enables quick reaction to load spikes•  Easier deployments of new compute cells•  Automated reaction to downed hosts•  Enables new strategies for deployments –  Replace vs Upgrade RACKSPACE® HOSTING | WWW.RACKSPACE.COM 22
    • Deploying OpenStackiNova Diagram iNova" OpenStack Control Public Cloud Infrastructure for XenServer Public Cloud" hosts XenServer hosts" domUs XenServer host w/ Public Cloud OpenStack Control Infrastructure for iNova" XenServer hosts RACKSPACE® HOSTING | WWW.RACKSPACE.COM 23
    • Deploying OpenStackContinuous Deployment•  Trunk and custom branches merged multiple times daily –  Unit Tested and Packaged•  Configuration managed with puppet –  Follows same QE and Continuous Deployment rules as code•  QE Environment –  OpenStack control infrastructure deployed on iNova –  nova-compute upgraded on 200 hosts –  Smoke tests•  Staging Deploy –  Uses the same packages and puppet manifests RACKSPACE® HOSTING | WWW.RACKSPACE.COM 24
    • Future RACKSPACE® HOSTING | WWW.RACKSPACE.COM 25
    • FutureImprovements desired from Xen•  Better VM resource isolation –  I/O scheduling weights per virtual disk•  Built in live migration without shared storage•  More information about VM resource usage –  Disk usage –  Help us improve OpenStack code –  Help customers react to bad situations•  XenAPI call to write to live xenstore•  Improve XenServer upgrade experience RACKSPACE® HOSTING | WWW.RACKSPACE.COM 26
    • FutureRackspace’s Other Interests•  Better VDI introduction for new VMs –  Look at VDI.introduce vs mv’ing VDIs and doing an sr-scan•  Incremental backups –  Delta VHDs instead of all VHDs•  Updated plugins using python >= 2.6 –  Cleans up workarounds for no built-in ‘uuid’ module –  Allows use of newer python features like context managers•  Better continuous deployment and testing –  Virtualize all of OpenStack •  Requires spinning XenServer on XenServer •  Wipe out compute nodes for QE environment •  Enable virtual labs for OpenStack developers –  Puppet used to configure XenServer RACKSPACE® HOSTING | WWW.RACKSPACE.COM 27
    • FutureiNova with Dev/QE Environments OpenStack QE iNova" Environment" OpenStack XenServer Control ‘hosts’ Infrastructure XenServer hosts" OpenStack Dev Environment" domUs OpenStack XenServer Control ‘hosts’ Infrastructure XenServer host w/ OpenStack Control Infrastructure for iNova" RACKSPACE® HOSTING | WWW.RACKSPACE.COM 28
    • " Chris Behrens" chris.behrens@rackspace.com" RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COMRACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM 29