Your SlideShare is downloading. ×
0
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Scaling Xen within Rackspace Cloud Servers
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Scaling Xen within Rackspace Cloud Servers

129,462

Published on

Rackspace has years of experience with running Xen at scale, starting with Xen and migrating to XenServer. We will share why we use Xen/XenServer along with some of the issues that we've experienced. …

Rackspace has years of experience with running Xen at scale, starting with Xen and migrating to XenServer. We will share why we use Xen/XenServer along with some of the issues that we've experienced. We will touch on our experience with migrating from Xen to XenServer and the challenges there. We will share information about Rackspace Cloud Servers architecture, and touch briefly on OpenStack when doing so. We will explain how we use Xen to quickly deploy new Openstack services with what we call Nova on Nova. And finally, we will discuss what additional features and improvements are needed and why.

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
129,462
On Slideshare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
127
Comments
0
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Scaling Xen withinRackspace Cloud Servers Chris Behrens chris.behrens@rackspace.comXenSummit 2012, San Diego, CA USAAugust 28, 2012 1
  • 2. Agenda•  Intro to Rackspace Cloud Servers•  Why Xen?•  Xen to XenServer•  OpenStack Basics•  OpenStack at Rackspace•  Scaling OpenStack Deployment•  Future RACKSPACE® HOSTING | WWW.RACKSPACE.COM 2
  • 3. Intro to Cloud Servers RACKSPACE® HOSTING | WWW.RACKSPACE.COM 3
  • 4. Intro to Cloud ServersOverview•  Public Cloud product under Rackspace’s Open Cloud portfolio•  First Gen –  Code originated from Slicehost acquisition –  Initially Xen 3.x, but migrated to XenServer•  Next Gen –  Public Launch 8/1/2012 –  Powered by OpenStack•  XenServer 6•  Linux, Windows, and FreeBSD VMs RACKSPACE® HOSTING | WWW.RACKSPACE.COM 4
  • 5. Intro to Cloud ServersStatistics•  180,000+ total Rackspace customers (not just Cloud Servers)•  Tens of thousands of hosts•  Hundreds of thousands of VMs•  Millions of snapshots RACKSPACE® HOSTING | WWW.RACKSPACE.COM 5
  • 6. Why Xen? RACKSPACE® HOSTING | WWW.RACKSPACE.COM 6
  • 7. Why Xen?•  Thin hypervisor layer•  Open source•  Excellent driver support due to Linux dom0•  Good API•  Good performance•  Along with Linux, Windows VMs are a must –  Same virtualization technology desired for Windows –  Microsoft will support –  Good PV driver support RACKSPACE® HOSTING | WWW.RACKSPACE.COM 7
  • 8. Xen to XenServer RACKSPACE® HOSTING | WWW.RACKSPACE.COM 8
  • 9. Xen to XenServerCloud Servers use of Xen•  LVM –  Logical volumes per VM•  Base images stored as tar files•  Kernel and ramdisk outside of images•  Shell out to run ‘xm’ commands•  Snapshots w/ LVM –  Mounted and contents rsynced to another host•  No agent –  Modifications inside images done via mounting powered off VM RACKSPACE® HOSTING | WWW.RACKSPACE.COM 9
  • 10. Xen to XenServerXen to XenServer changes•  VHDs for virtual disk images –  Tarred and gzipped•  Kernel and ramdisk inside images•  Snapshots via XenAPI –  Whole VDI chain wrapped in OVF container for upload to glance•  Agent inside VM images –  Communication via Xenstore –  File injection, password resets, networking setup RACKSPACE® HOSTING | WWW.RACKSPACE.COM 10
  • 11. Xen to XenServerMigrating•  Pick a XenServer host•  Create VDI•  Attach VDI to dom0•  Partition, mkfs, and mount the new virtual disk•  LVM Snapshot original Xen VM and mount it•  rsync snapshot contents to mounted VDI on new host•  Power off original Xen VM and mount its virtual disk•  rsync again to new host to catch changes since snapshotting•  Extract a tar file into mounted VDI containing kernel, ramdisk, agent•  Unmount and detach VDI from dom0•  Create new VM record and attach VDI•  Start the new VM RACKSPACE® HOSTING | WWW.RACKSPACE.COM 11
  • 12. OpenStack The Basics RACKSPACE® HOSTING | WWW.RACKSPACE.COM 12
  • 13. OpenStackComponents•  Nova –  Compute controller with various hypervisor drivers•  Glance –  VM Image registry•  Swift –  Object storage•  Quantum/Melange –  Networking•  Cinder –  Volumes/Block storage•  Keystone –  Authentication –  Authorization –  API Endpoint discovery RACKSPACE® HOSTING | WWW.RACKSPACE.COM 13
  • 14. OpenStackTypical OpenStack Deployment keystone nova glance nova-api glance-api glance-registry RabbitMQ MySQL nova-scheduler MySQL nova-network nova-compute swift RACKSPACE® HOSTING | WWW.RACKSPACE.COM 14
  • 15. OpenStacknova-compute w/ XenAPI•  Runs in a utility domU on each host•  Images downloaded in dom0 Physical host running XenServer" –  glance XenAPI plugin•  Images attached to nova-compute dom0" nova utility domU" domU –  Partitioning changes glance plugin nova-compute –  file-system resizing agent plugin•  Monitors VMs’ power_state•  Snapshots/backups xenstore plugin nova instance•  VM resize/migration domUs –  rsync to new host•  Communicates w/ agent inside VMs –  Via xenstore RACKSPACE® HOSTING | WWW.RACKSPACE.COM 15
  • 16. OpenStackXenAPI VM Calls•  VM.start / VM.start_on •  VM.add_to_VCPUs_params•  VM.destroy •  VM.get_VBDs•  VM.clean_reboot •  VM.get_record•  VM.hard_reboot •  VM.get_by_uuid•  VM.clean_shutdown •  VM.get_by_name_label•  VM.hard_shutdown •  VM.set_name_label•  VM.pause •  VM.add_to_xenstore_data•  VM.unpause •  VM.remove_from_xenstore_data•  VM.suspend •  VM.assert_can_migrate•  VM.resume •  VM.migrate_send•  VM.snapshot •  VM.pool_migrate RACKSPACE® HOSTING | WWW.RACKSPACE.COM 16
  • 17. OpenStackRackspace Specifics RACKSPACE® HOSTING | WWW.RACKSPACE.COM 17
  • 18. OpenStack Rackspace SpecificsSource Code•  Rackspace tracks trunk –  Deployed code generally less than 2 weeks behind trunk•  Some custom patches on top of trunk –  Features specific to Rackspace –  Custom scheduling to meet Rackspace needs –  Scaling with OpenStack Compute Cells RACKSPACE® HOSTING | WWW.RACKSPACE.COM 18
  • 19. OpenStack Rackspace SpecificsOpenStack Compute Cells•  Scaling beyond small nova deployments –  Effectively ties multiple nova deployments together•  Developed by Rackspace•  Targeted for Grizzly OpenStack release•  Hierarchical tree –  Multiple parents supported•  API cell(s) at top of the tree•  Compute cell(s) underneath•  DB and RabbitMQ per cell•  Scalability –  Splits connections to DB and RabbitMQ –  Less work to schedule new VM placement RACKSPACE® HOSTING | WWW.RACKSPACE.COM 19
  • 20. OpenStack Rackspace Specifics OpenStack Nova Multi-cell Diagram API Cell RabbitMQ MySQL nova-api nova-cellsCompute Cell 1" Compute Cell 2" nova-cells nova-cells RabbitMQ RabbitMQ MySQL MySQL nova-scheduler nova-scheduler nova-network nova-network nova-compute nova-compute RACKSPACE® HOSTING | WWW.RACKSPACE.COM 20
  • 21. Deploying OpenStack RACKSPACE® HOSTING | WWW.RACKSPACE.COM 21
  • 22. Deploying OpenStack‘Inception’•  Private internal cloud –  Based on Openstack, called iNova –  OpenStack services for public cloud run virtualized under iNova•  Easy to spin up additional services for load balancing –  Enables quick reaction to load spikes•  Easier deployments of new compute cells•  Automated reaction to downed hosts•  Enables new strategies for deployments –  Replace vs Upgrade RACKSPACE® HOSTING | WWW.RACKSPACE.COM 22
  • 23. Deploying OpenStackiNova Diagram iNova" OpenStack Control Public Cloud Infrastructure for XenServer Public Cloud" hosts XenServer hosts" domUs XenServer host w/ Public Cloud OpenStack Control Infrastructure for iNova" XenServer hosts RACKSPACE® HOSTING | WWW.RACKSPACE.COM 23
  • 24. Deploying OpenStackContinuous Deployment•  Trunk and custom branches merged multiple times daily –  Unit Tested and Packaged•  Configuration managed with puppet –  Follows same QE and Continuous Deployment rules as code•  QE Environment –  OpenStack control infrastructure deployed on iNova –  nova-compute upgraded on 200 hosts –  Smoke tests•  Staging Deploy –  Uses the same packages and puppet manifests RACKSPACE® HOSTING | WWW.RACKSPACE.COM 24
  • 25. Future RACKSPACE® HOSTING | WWW.RACKSPACE.COM 25
  • 26. FutureImprovements desired from Xen•  Better VM resource isolation –  I/O scheduling weights per virtual disk•  Built in live migration without shared storage•  More information about VM resource usage –  Disk usage –  Help us improve OpenStack code –  Help customers react to bad situations•  XenAPI call to write to live xenstore•  Improve XenServer upgrade experience RACKSPACE® HOSTING | WWW.RACKSPACE.COM 26
  • 27. FutureRackspace’s Other Interests•  Better VDI introduction for new VMs –  Look at VDI.introduce vs mv’ing VDIs and doing an sr-scan•  Incremental backups –  Delta VHDs instead of all VHDs•  Updated plugins using python >= 2.6 –  Cleans up workarounds for no built-in ‘uuid’ module –  Allows use of newer python features like context managers•  Better continuous deployment and testing –  Virtualize all of OpenStack •  Requires spinning XenServer on XenServer •  Wipe out compute nodes for QE environment •  Enable virtual labs for OpenStack developers –  Puppet used to configure XenServer RACKSPACE® HOSTING | WWW.RACKSPACE.COM 27
  • 28. FutureiNova with Dev/QE Environments OpenStack QE iNova" Environment" OpenStack XenServer Control ‘hosts’ Infrastructure XenServer hosts" OpenStack Dev Environment" domUs OpenStack XenServer Control ‘hosts’ Infrastructure XenServer host w/ OpenStack Control Infrastructure for iNova" RACKSPACE® HOSTING | WWW.RACKSPACE.COM 28
  • 29. " Chris Behrens" chris.behrens@rackspace.com" RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COMRACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM 29

×