Xenserver Highload Frank Kohler


Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Xenserver Highload Frank Kohler

  1. 1. Virtualization 2008 Citrix XenServer 5.0 Frank Kohler, October 2008 Manager System Engineers Server Virtualization
  2. 2. Agenda • Introduction • Big Picture • XenServer • Competitive – Vs. Other Xen Virtualization – Vs. VMware – vs. Microsoft
  3. 3. Virtualization Aspects Partitioning: Presenting physical resources to individual users --> XEN, VMware, VirtualIron Aggregation: Combining multiple resources to a single unit --> PVM, OpenSSI, ScaleMP
  4. 4. Virtualization Big Picture physical partition into virtual instances. each partition w/resources
  5. 5. Virtualization Big Picture Software Virtualization CSS / OSS z/VM -- KVM bringing virtualization to x86, IA64, ... platforms coexistence of Linux and Windows
  6. 6. Server Virtualization
  7. 7. Virtualization Aspects [left] type II VMM: Application w/virtual extension [center] OS w/virtual extension [right] type I VMM: hardware w/virtual extension: Vanderpool (IVT/VT- x), Pacifica (AMD-V) x86, Power, Sparc, Itanium
  8. 8. Virtualization Challenges/Limitations Virtualization technology limited by hardware platform Do you want to get locked?
  9. 9. Step Back for the Full Picture
  10. 10. Case for Server Virtualization • Servers are costly to maintain – Costs encompass provisioning, housing, power, cooling, management, etc. – Complex, expensive management • Servers are poorly utilized – Yet, they proliferate as apps are added • Physical servers are static – Locked to one workload per server
  11. 11. What problems are we solving Data Centers are Data Centers static should be dynamic
  12. 12. Citrix Virtualization Division - History • XenSource founded in 2005 • Creators of Xen and leaders of Xen project • Acquired by Citrix Systems –Oct 2007 • Home of the Xen Hypervisor • Open source, next generation architecture • High performance bare metal virtualization engine • Deliver Commercial Virtualization Platform • Citrix XenServer Express, Standard, Enterprise Editions
  13. 13. Xen - the Industry Standard Hypervisor
  14. 14. Small Performance Overhead • Native 64 Bit Hypervisor • Code base ~50.000 lines of code • Approx 3180 C functions • Small Overhead • Guests Linux/Win: ½ – 8% • Supports large workloads • Up to 8 virtual CPUs per guest • Up to 128 GB RAM • Up to 32 GB per VM
  15. 15. • Supports large machines
  16. 16. Industry develops Xen Hypervisor -- xen.org
  17. 17. Architecture XenServer ‘Orlando’
  18. 18. Xen Architecture Open Source Storage Control Interface Drivers Xen Control Interface Virtualized Hardware VT/AMD-V Hardware
  19. 19. Paravirtualization • Relies on “modified” XenServer operating systems User User Apps Apps • Kernel and I/O paths know they are being virtualized • Cooperation provides best performance HALT Paravirtualized guests HALT makes high-speed calls HYPERCALL directly to the hypervisor VT/AMD-V Hardware
  20. 20. Hardware-Assisted Virtualization XenServer User User Apps Apps • Hardware-assist allows high performance without emulation HALT HALT Other guests benefit HYPERCALL from hardware- VT/AMD-V accelerated call Hardware translation
  21. 21. From Xen to XenSource to Citrix Xenserver
  22. 22. From Open Source to Enterprise Product xm  xe
  23. 23. >xe help -all log-set-output, message-create, message-list, message-param-get host-backup, host-bugreport-upload, host-call-plugin message-param-list, network-create, network-destroy, network-list host-compute-free-memory, host-cpu-list, host-cpu-param-get network-param-add, network-param-clear, network-param-get host-cpu-param-list, host-crashdump-destroy, host-crashdump-list network-param-list, network-param-remove, network-param-set, patch-apply host-crashdump-param-get, host-crashdump-param-list patch-clean, patch-destroy, patch-list, patch-param-clear host-crashdump-upload, host-data-source-forget, host-data-source-list patch-param-get, patch-param-list, patch-pool-apply, patch-precheck host-data-source-query, host-data-source-record, host-disable patch-upload, pbd-create, pbd-destroy, pbd-list, pbd-param-add host-dmesg, host-emergency-ha-disable pbd-param-clear, pbd-param-get, pbd-param-list, pbd-param-remove host-emergency-management-reconfigure, host-enable, host-evacuate host-forget, host-get-system-status, host-get-system-status-capabilities pbd-param-set, pbd-plug, pbd-unplug, pif-forget, pif-introduce, pif-list host-get-vms-which-prevent-evacuation, host-is-in-emergency-mode pif-param-add, pif-param-clear, pif-param-get, pif-param-list host-license-add, host-license-view, host-list, host-logs-download pif-param-remove, pif-param-set, pif-plug, pif-reconfigure-ip, pif-scan host-management-disable, host-management-reconfigure, host-param-add pif-unplug, pool-designate-new-master, pool-dump-database, pool-eject host-param-clear, host-param-get, host-param-list, host-param-remove pool-emergency-reset-master, pool-emergency-transition-to-master host-param-set, host-power-on, host-reboot, host-restore pool-ha-compute-hypothetical-max-host-failures-to-tolerate host-send-debug-keys, host-set-hostname-live, host-shutdown pool-ha-compute-max-host-failures-to-tolerate, pool-ha-disable host-shutdown-agent, host-sync-data, host-syslog-reconfigure pool-ha-compute-hypothetical-max-host-failures-to-tolerate pool-ha-compute-max-host-failures-to-tolerate, pool-ha-disable vm-cd-add, vm-cd-eject, vm-cd-insert, vm-cd-list, vm-cd-remove, vm-clone pool-ha-enable, pool-join, pool-list, pool-param-add, pool-param-clear vm-compute-maximum-memory, vm-copy, vm-crashdump-list pool-param-get, pool-param-list, pool-param-remove, pool-param-set vm-data-source-forget, vm-data-source-list, vm-data-source-query pool-recover-slaves, pool-restore-database, pool-sync-database vm-data-source-record, vm-destroy, vm-disk-add, vm-disk-list pool-vlan-create, sm-list, sm-param-get, sm-param-list, sr-create vm-disk-remove, vm-export, vm-import, vm-install, vm-list sr-destroy, sr-forget, sr-introduce, sr-list, sr-param-add vm-memory-shadow-multiplier-set, vm-migrate, vm-param-add sr-param-clear, sr-param-get, sr-param-list, sr-param-remove vm-param-clear, vm-param-get, vm-param-list, vm-param-remove sr-param-set, sr-probe, sr-scan, sr-update, task-cancel, task-list vm-param-set, vm-pause, vm-reboot, vm-reset-powerstate, vm-resume task-param-get, task-param-list, template-export, template-list vm-shutdown, vm-snapshot, vm-snapshot-with-quiesce, vm-start, vm-suspend template-param-add, template-param-clear, template-param-get vm-uninstall, vm-unpause, vm-vcpu-hotplug, vm-vif-list template-param-list, template-param-remove, template-param-set update-upload, user-password-change, vbd-create, vbd-destroy, vbd-eject blob-create, blob-get, blob-list, blob-param-clear, blob-param-get vbd-insert, vbd-list, vbd-param-add, vbd-param-clear, vbd-param-get blob-param-list, blob-param-set, blob-put, bond-create, bond-destroy vbd-param-list, vbd-param-remove, vbd-param-set, vbd-plug, vbd-unplug bond-list, bond-param-get, bond-param-list, cd-list, console-list vdi-clone, vdi-copy, vdi-create, vdi-destroy, vdi-forget, vdi-import console-param-add, console-param-clear, console-param-get vdi-introduce, vdi-list, vdi-param-add, vdi-param-clear, vdi-param-get console-param-list, console-param-remove, console-param-set vdi-param-list, vdi-param-remove, vdi-param-set, vdi-resize diagnostic-compact, diagnostic-db-log, diagnostic-db-stats vdi-snapshot, vdi-unlock, vdi-update, vif-create, vif-destroy, vif-list diagnostic-gc-stats, diagnostic-timing-stats, diagnostic-vdi-status vif-param-add, vif-param-clear, vif-param-get, vif-param-list diagnostic-vm-status, event-wait, log-get, log-get-keys, log-reopen vif-param-remove, vif-param-set, vif-plug, vif-unplug, vlan-create vlan-destroy, vlan-list, vlan-param-get, vlan-param-list
  24. 24. From Open Source to Enterprise Product XenServer Quality Assurance Enterprise Open Source XEN and Testing on Virtualization Enterprise Level Software Manual Regression Automated Regression Optimization BETA releases stable, tested stable, tested Platinum not tested Edition Proprietary unstable software Fixes Fixes
  25. 25. Other Architectures
  26. 26. Architectures
  27. 27. Monolithic & Microkernelized • Monolithic hypervisor • Microkernelized hypervisor – Simple partitioning functionality – Simpler than a modern kernel, – Increase reliability and minimize TCB but still complex – No third-party code – Contains its own drivers model – Drivers run within guests “Our view is that virtualization VM 1 is something that1 should be VM 2 VM 3 VM (“Admin”) built into the operating 2system.” (“Parent”) Virtual- VM VM 3 ization (“Child”) Steve Ballmer (“Child”) Stack Hypervisor Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Hypervisor Hardware Hardware Windows Server Virtualization VMware ESX Approach Approach
  28. 28. KVM Arch
  29. 29. XenServer Concepts
  30. 30. Xen and XenServer Dom0: stripped Centos5, smaller footprint, linux drivers XenCenter GUI Windows / C# Management Stack HP ProLiant Console XAPI Python Control Domain Storage Hardware, management VHD, iSCSI, Fibre Channel Xen OS Support Windows PV, Linux Kernels Interrupts, CPU, memory
  31. 31. Resource Pools Anchored VM Agile VM Agile VM XenServer XenServer XenServer Local Storage Shared Storage
  32. 32. service api {start | stop | restart} ProLiant Virtual Console Citrix Citrix Windows Egenera CIM XenCenter Workflow Studio Powershell PAN Manager bridge All these clients are using the Citrix XenServer same API pools
  33. 33. Storage Management
  34. 34. Storage Management The Old Way The XenServer way Storage operations controlled Leave storage to the storage by virtualization software experts. Open up an API for command and control. Storage API File System Intelligent storage reduced to Leverage the capabilities of commodity disk enterprise storage systems
  35. 35. XenServer APIs ProLiant Virtual Console Citrix Citrix Windows Egenera CIM XenCenter Workflow Studio Powershell PAN Manager bridge All storage is accessed through Citrix XenServer the same API pools • fdisk –l • fdisk /dev/sdb • mkfs -t ext3 /dev/sdb • xe sr-create – content-type= device- config: host-uuid= name-label= physical- size= shared= sm- config: type=lvm • vgdisplay • pvcreate /dev/sdb • vgextend VG_Name /dev/sdb • vgdisplay
  36. 36. NetApp: ONTAP API: snapshotting und recovery
  37. 37. XenServer Resource Pools Ability to join multiple physical servers into one logical pool of resources. Allows VM agility • Automatic Initial Placement • Live Migration Shared configurations • Networking xe pool-join master-address=a • Remote Storage master-username=root master Resiliency to single system failures password=password • Clustered management layer Example 1: To import multiple metadata files into a pool: • No backend management server # importscript.sh # Imports multiple numbered VM Metadata files • Built-in replicated database on each node ## Usage: ./importscript.sh filename echo Importing $1 xe vm-import filename=$1 metadata=true i=2 while [ -f $1$i ] do echo Importing $1$i xe vm-import filename=$1$i metadata=true i=`expr $i + 1` done
  38. 38. Active-active NIC Bonding Architecture Control Domain VM • New NIC • lspci Virtual eth 0 Bond 0 • ifconfig -a Switch Virtual • xe pif-scan host-uuid=UUID NIC • Announce new hardware: eth 1 Virtual xe pif-introduce device=eth0 Interface mac= 00:16:D3:C4:B9:BB host- uuid= Linux Device Drivers Xen Hypervisor Physical Physical NIC 1 NIC 2
  39. 39. Snapshotting VMs (CLI) New commands: • Perform a regular snapshot xe vm-snapshot vm=vm_name new-name-label=vm_snapshot_name • Perform a quiesced snapshot xe vm-snapshot-with-quiesce vm=vm_name new-name-label=vm_snapshot_name
  40. 40. High Availability
  41. 41. Management Architectures The Other Guys XenServer Traditional Management Next Generation Architecture Management Architecture • Single backend management • Clustered management layer server
  42. 42. Dynamic Failure Planning Adaptive Heuristics E-mail Alerting High Availability Features “Invisible”
  43. 43. High Availability XenApp XenApp XenApp XenApp XenApp XenApp Workload 1 Workload 2 Workload 3 Workload 1 Workload 2 Workload 3 XenServer 1 XenServer 2 Shared Storage
  44. 44. Marathon – HA Level Standard HA everRun VM everRun VM + Lockstep Option (Q4 ‘08) Zero downtime – Any failure Level 3 – System-Level Maintains application state Fault Tolerance Maintains memory state Zero downtime – I/O failures Automated setup & configuration Level 2 – Automated fault management Component-Level Fault Tolerance Guaranteed recovery Geographic protection Dynamic upgrades Best-effort failover Linux & Windows guest support Level 1 – Basic Failover No secondary resource allocation Shared-LUN support
  45. 45. enterprise features… Workflow Studio-enabled Auto backup of VM metadata Oracle DB certified hypervisor NetApp SR driver Workflow templates XenConvert P2V 10 x64 support Automated High Availability to protect against host failures SLES Role-based admin for streaming Increasednumberofsupportedhosts per pool Oracle Apps certified hypervisor Latest Xen 3.x hypervisor Host networking config via XenCenter Windows Server 2008 support 32 & 64-bit (incl. streaming) NIC bonding SLES 9 SP4 support Bulletproof Reliability New Guest /HV Support Sorting, searching, grouping, tagging in XenCenter Snapshots for advanced storage arrays XenConvert Intuitive XenCenter UI P2V Near bare-metal performance Search and Tag Virtual Machines Persistent performance data / trending Increasednumberofsupportedhosts per pool FT upgradable Alerting for key runtime events Business Continuity Automated High Availability to protect against host failures Windows PV drivers for max performance MSFT certified hypervisor multi-path support I/O IncreasednumberofsupportedhostphysicalNICs installation 10 to Xen Dell EqualLogic SR driver iSCSI FC + SAN Support via XenCenter FC SR creation Multi-site Disaster Recovery enablement QLogic / Emulex 8Gb FC HBA support Xen SDK XenAPI Auto VM placement and load balancing 32 and 64-bit RHEL support Full
  46. 46. Performance Comparisons
  47. 47. Performance comparisons
  48. 48. Moore’s Law: doubling transistors 2yrs 3.7 TFlops 25 racks 512 Servers 1000 sq ft 128 kW 3.7 TFlops 1 rack 53 Blades 40 sq ft 21 kW
  49. 49. XEN: build W2K3 DDK build
  50. 50. XEN & Application SAP scalability on XEN
  51. 51. Reducing XenApp Server Count Issue: 32-bit Windows can only address 4 GB RAM Underutilizes modern multi-core processors 4 GB RAM exhausted Plenty of processor cycles still available
  52. 52. Test Results – Performance Comparison Physical Server RAM exhausted at <90 users
  53. 53. Test Results – Performance Comparison
  54. 54. Optimized for XenApp XenServer 73% more users Other Guys
  55. 55. Pool Scalability
  56. 56. 1400 VMs, 28 hosts, 1 pool
  57. 57. Citrix XenServer OEM Edition
  58. 58. Strong Open Partner Ecosystem
  59. 59. Enterprise Ready!
  60. 60. Windows Certification
  61. 61. Disaster Recovery
  62. 62. Improved Hardware Support
  63. 63. User Experience
  64. 64. Beyond Citrix XenServer 5
  65. 65. Future of Xen, e.g. Client Initiative • Power Management • Security • S3 suspend to RAM, Rapid • TPM, emulated TPM Boot • OVF specs • GPU passthru w/IOMMU • USB passthru • WiFi / WiMax • Native UEFI support
  66. 66. Decision points for XenServer? • Stability, Performance • Ease of use: „10 minutes to Xen“; real :-) • High performance workloads • Best price/performance ratio • Provisioning Server • Close Partnership w/Microsoft
  67. 67. “Citrix is a great partner! They know how to partner best with Microsoft. They are the best example for ISV partnership.” —Steve Ballmer (Photo from Microsoft CIO Summit, Mar 1, 2007)
  68. 68. New Landing Pages http://www.xenserver5.com
  69. 69. Backup Slides
  70. 70. That’s all for Orlando folks! Engineering will return in 2009 in “Midnight Ride” Happy Selling!