Your SlideShare is downloading. ×

Highload Frank Kohler


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Virtualization 2008 Citrix XenServer 5.0 Frank Kohler, October 2008 Manager System Engineers Server Virtualization
  • 2. Agenda • Introduction • Big Picture • XenServer • Competitive – Vs. Other Xen Virtualization – Vs. VMware – vs. Microsoft
  • 3. Virtualization Aspects Partitioning: Presenting physical resources to individual users --> XEN, VMware, VirtualIron Aggregation: Combining multiple resources to a single unit --> PVM, OpenSSI, ScaleMP
  • 4. Virtualization Big Picture physical partition into virtual instances. each partition w/resources
  • 5. Virtualization Big Picture Software Virtualization CSS / OSS z/VM -- KVM bringing virtualization to x86, IA64, ... platforms coexistence of Linux and Windows
  • 6. Server Virtualization
  • 7. Virtualization Aspects [left] type II VMM: Application w/virtual extension [center] OS w/virtual extension [right] type I VMM: hardware w/virtual extension: Vanderpool (IVT/VT- x), Pacifica (AMD-V) x86, Power, Sparc, Itanium
  • 8. Virtualization Challenges/Limitations Virtualization technology limited by hardware platform Do you want to get locked?
  • 9. Step Back for the Full Picture
  • 10. Case for Server Virtualization • Servers are costly to maintain – Costs encompass provisioning, housing, power, cooling, management, etc. – Complex, expensive management • Servers are poorly utilized – Yet, they proliferate as apps are added • Physical servers are static – Locked to one workload per server
  • 11. What problems are we solving Data Centers are Data Centers static should be dynamic
  • 12. Citrix Virtualization Division - History • XenSource founded in 2005 • Creators of Xen and leaders of Xen project • Acquired by Citrix Systems –Oct 2007 • Home of the Xen Hypervisor • Open source, next generation architecture • High performance bare metal virtualization engine • Deliver Commercial Virtualization Platform • Citrix XenServer Express, Standard, Enterprise Editions
  • 13. Xen - the Industry Standard Hypervisor
  • 14. Small Performance Overhead • Native 64 Bit Hypervisor • Code base ~50.000 lines of code • Approx 3180 C functions • Small Overhead • Guests Linux/Win: ½ – 8% • Supports large workloads • Up to 8 virtual CPUs per guest • Up to 128 GB RAM • Up to 32 GB per VM
  • 15. • Supports large machines
  • 16. Industry develops Xen Hypervisor --
  • 17. Architecture XenServer ‘Orlando’
  • 18. Xen Architecture Open Source Storage Control Interface Drivers Xen Control Interface Virtualized Hardware VT/AMD-V Hardware
  • 19. Paravirtualization • Relies on “modified” XenServer operating systems User User Apps Apps • Kernel and I/O paths know they are being virtualized • Cooperation provides best performance HALT Paravirtualized guests HALT makes high-speed calls HYPERCALL directly to the hypervisor VT/AMD-V Hardware
  • 20. Hardware-Assisted Virtualization XenServer User User Apps Apps • Hardware-assist allows high performance without emulation HALT HALT Other guests benefit HYPERCALL from hardware- VT/AMD-V accelerated call Hardware translation
  • 21. From Xen to XenSource to Citrix Xenserver
  • 22. From Open Source to Enterprise Product xm  xe
  • 23. >xe help -all log-set-output, message-create, message-list, message-param-get host-backup, host-bugreport-upload, host-call-plugin message-param-list, network-create, network-destroy, network-list host-compute-free-memory, host-cpu-list, host-cpu-param-get network-param-add, network-param-clear, network-param-get host-cpu-param-list, host-crashdump-destroy, host-crashdump-list network-param-list, network-param-remove, network-param-set, patch-apply host-crashdump-param-get, host-crashdump-param-list patch-clean, patch-destroy, patch-list, patch-param-clear host-crashdump-upload, host-data-source-forget, host-data-source-list patch-param-get, patch-param-list, patch-pool-apply, patch-precheck host-data-source-query, host-data-source-record, host-disable patch-upload, pbd-create, pbd-destroy, pbd-list, pbd-param-add host-dmesg, host-emergency-ha-disable pbd-param-clear, pbd-param-get, pbd-param-list, pbd-param-remove host-emergency-management-reconfigure, host-enable, host-evacuate host-forget, host-get-system-status, host-get-system-status-capabilities pbd-param-set, pbd-plug, pbd-unplug, pif-forget, pif-introduce, pif-list host-get-vms-which-prevent-evacuation, host-is-in-emergency-mode pif-param-add, pif-param-clear, pif-param-get, pif-param-list host-license-add, host-license-view, host-list, host-logs-download pif-param-remove, pif-param-set, pif-plug, pif-reconfigure-ip, pif-scan host-management-disable, host-management-reconfigure, host-param-add pif-unplug, pool-designate-new-master, pool-dump-database, pool-eject host-param-clear, host-param-get, host-param-list, host-param-remove pool-emergency-reset-master, pool-emergency-transition-to-master host-param-set, host-power-on, host-reboot, host-restore pool-ha-compute-hypothetical-max-host-failures-to-tolerate host-send-debug-keys, host-set-hostname-live, host-shutdown pool-ha-compute-max-host-failures-to-tolerate, pool-ha-disable host-shutdown-agent, host-sync-data, host-syslog-reconfigure pool-ha-compute-hypothetical-max-host-failures-to-tolerate pool-ha-compute-max-host-failures-to-tolerate, pool-ha-disable vm-cd-add, vm-cd-eject, vm-cd-insert, vm-cd-list, vm-cd-remove, vm-clone pool-ha-enable, pool-join, pool-list, pool-param-add, pool-param-clear vm-compute-maximum-memory, vm-copy, vm-crashdump-list pool-param-get, pool-param-list, pool-param-remove, pool-param-set vm-data-source-forget, vm-data-source-list, vm-data-source-query pool-recover-slaves, pool-restore-database, pool-sync-database vm-data-source-record, vm-destroy, vm-disk-add, vm-disk-list pool-vlan-create, sm-list, sm-param-get, sm-param-list, sr-create vm-disk-remove, vm-export, vm-import, vm-install, vm-list sr-destroy, sr-forget, sr-introduce, sr-list, sr-param-add vm-memory-shadow-multiplier-set, vm-migrate, vm-param-add sr-param-clear, sr-param-get, sr-param-list, sr-param-remove vm-param-clear, vm-param-get, vm-param-list, vm-param-remove sr-param-set, sr-probe, sr-scan, sr-update, task-cancel, task-list vm-param-set, vm-pause, vm-reboot, vm-reset-powerstate, vm-resume task-param-get, task-param-list, template-export, template-list vm-shutdown, vm-snapshot, vm-snapshot-with-quiesce, vm-start, vm-suspend template-param-add, template-param-clear, template-param-get vm-uninstall, vm-unpause, vm-vcpu-hotplug, vm-vif-list template-param-list, template-param-remove, template-param-set update-upload, user-password-change, vbd-create, vbd-destroy, vbd-eject blob-create, blob-get, blob-list, blob-param-clear, blob-param-get vbd-insert, vbd-list, vbd-param-add, vbd-param-clear, vbd-param-get blob-param-list, blob-param-set, blob-put, bond-create, bond-destroy vbd-param-list, vbd-param-remove, vbd-param-set, vbd-plug, vbd-unplug bond-list, bond-param-get, bond-param-list, cd-list, console-list vdi-clone, vdi-copy, vdi-create, vdi-destroy, vdi-forget, vdi-import console-param-add, console-param-clear, console-param-get vdi-introduce, vdi-list, vdi-param-add, vdi-param-clear, vdi-param-get console-param-list, console-param-remove, console-param-set vdi-param-list, vdi-param-remove, vdi-param-set, vdi-resize diagnostic-compact, diagnostic-db-log, diagnostic-db-stats vdi-snapshot, vdi-unlock, vdi-update, vif-create, vif-destroy, vif-list diagnostic-gc-stats, diagnostic-timing-stats, diagnostic-vdi-status vif-param-add, vif-param-clear, vif-param-get, vif-param-list diagnostic-vm-status, event-wait, log-get, log-get-keys, log-reopen vif-param-remove, vif-param-set, vif-plug, vif-unplug, vlan-create vlan-destroy, vlan-list, vlan-param-get, vlan-param-list
  • 24. From Open Source to Enterprise Product XenServer Quality Assurance Enterprise Open Source XEN and Testing on Virtualization Enterprise Level Software Manual Regression Automated Regression Optimization BETA releases stable, tested stable, tested Platinum not tested Edition Proprietary unstable software Fixes Fixes
  • 25. Other Architectures
  • 26. Architectures
  • 27. Monolithic & Microkernelized • Monolithic hypervisor • Microkernelized hypervisor – Simple partitioning functionality – Simpler than a modern kernel, – Increase reliability and minimize TCB but still complex – No third-party code – Contains its own drivers model – Drivers run within guests “Our view is that virtualization VM 1 is something that1 should be VM 2 VM 3 VM (“Admin”) built into the operating 2system.” (“Parent”) Virtual- VM VM 3 ization (“Child”) Steve Ballmer (“Child”) Stack Hypervisor Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Drivers Hypervisor Hardware Hardware Windows Server Virtualization VMware ESX Approach Approach
  • 28. KVM Arch
  • 29. XenServer Concepts
  • 30. Xen and XenServer Dom0: stripped Centos5, smaller footprint, linux drivers XenCenter GUI Windows / C# Management Stack HP ProLiant Console XAPI Python Control Domain Storage Hardware, management VHD, iSCSI, Fibre Channel Xen OS Support Windows PV, Linux Kernels Interrupts, CPU, memory
  • 31. Resource Pools Anchored VM Agile VM Agile VM XenServer XenServer XenServer Local Storage Shared Storage
  • 32. service api {start | stop | restart} ProLiant Virtual Console Citrix Citrix Windows Egenera CIM XenCenter Workflow Studio Powershell PAN Manager bridge All these clients are using the Citrix XenServer same API pools
  • 33. Storage Management
  • 34. Storage Management The Old Way The XenServer way Storage operations controlled Leave storage to the storage by virtualization software experts. Open up an API for command and control. Storage API File System Intelligent storage reduced to Leverage the capabilities of commodity disk enterprise storage systems
  • 35. XenServer APIs ProLiant Virtual Console Citrix Citrix Windows Egenera CIM XenCenter Workflow Studio Powershell PAN Manager bridge All storage is accessed through Citrix XenServer the same API pools • fdisk –l • fdisk /dev/sdb • mkfs -t ext3 /dev/sdb • xe sr-create – content-type= device- config: host-uuid= name-label= physical- size= shared= sm- config: type=lvm • vgdisplay • pvcreate /dev/sdb • vgextend VG_Name /dev/sdb • vgdisplay
  • 36. NetApp: ONTAP API: snapshotting und recovery
  • 37. XenServer Resource Pools Ability to join multiple physical servers into one logical pool of resources. Allows VM agility • Automatic Initial Placement • Live Migration Shared configurations • Networking xe pool-join master-address=a • Remote Storage master-username=root master Resiliency to single system failures password=password • Clustered management layer Example 1: To import multiple metadata files into a pool: • No backend management server # # Imports multiple numbered VM Metadata files • Built-in replicated database on each node ## Usage: ./ filename echo Importing $1 xe vm-import filename=$1 metadata=true i=2 while [ -f $1$i ] do echo Importing $1$i xe vm-import filename=$1$i metadata=true i=`expr $i + 1` done
  • 38. Active-active NIC Bonding Architecture Control Domain VM • New NIC • lspci Virtual eth 0 Bond 0 • ifconfig -a Switch Virtual • xe pif-scan host-uuid=UUID NIC • Announce new hardware: eth 1 Virtual xe pif-introduce device=eth0 Interface mac= 00:16:D3:C4:B9:BB host- uuid= Linux Device Drivers Xen Hypervisor Physical Physical NIC 1 NIC 2
  • 39. Snapshotting VMs (CLI) New commands: • Perform a regular snapshot xe vm-snapshot vm=vm_name new-name-label=vm_snapshot_name • Perform a quiesced snapshot xe vm-snapshot-with-quiesce vm=vm_name new-name-label=vm_snapshot_name
  • 40. High Availability
  • 41. Management Architectures The Other Guys XenServer Traditional Management Next Generation Architecture Management Architecture • Single backend management • Clustered management layer server
  • 42. Dynamic Failure Planning Adaptive Heuristics E-mail Alerting High Availability Features “Invisible”
  • 43. High Availability XenApp XenApp XenApp XenApp XenApp XenApp Workload 1 Workload 2 Workload 3 Workload 1 Workload 2 Workload 3 XenServer 1 XenServer 2 Shared Storage
  • 44. Marathon – HA Level Standard HA everRun VM everRun VM + Lockstep Option (Q4 ‘08) Zero downtime – Any failure Level 3 – System-Level Maintains application state Fault Tolerance Maintains memory state Zero downtime – I/O failures Automated setup & configuration Level 2 – Automated fault management Component-Level Fault Tolerance Guaranteed recovery Geographic protection Dynamic upgrades Best-effort failover Linux & Windows guest support Level 1 – Basic Failover No secondary resource allocation Shared-LUN support
  • 45. enterprise features… Workflow Studio-enabled Auto backup of VM metadata Oracle DB certified hypervisor NetApp SR driver Workflow templates XenConvert P2V 10 x64 support Automated High Availability to protect against host failures SLES Role-based admin for streaming Increasednumberofsupportedhosts per pool Oracle Apps certified hypervisor Latest Xen 3.x hypervisor Host networking config via XenCenter Windows Server 2008 support 32 & 64-bit (incl. streaming) NIC bonding SLES 9 SP4 support Bulletproof Reliability New Guest /HV Support Sorting, searching, grouping, tagging in XenCenter Snapshots for advanced storage arrays XenConvert Intuitive XenCenter UI P2V Near bare-metal performance Search and Tag Virtual Machines Persistent performance data / trending Increasednumberofsupportedhosts per pool FT upgradable Alerting for key runtime events Business Continuity Automated High Availability to protect against host failures Windows PV drivers for max performance MSFT certified hypervisor multi-path support I/O IncreasednumberofsupportedhostphysicalNICs installation 10 to Xen Dell EqualLogic SR driver iSCSI FC + SAN Support via XenCenter FC SR creation Multi-site Disaster Recovery enablement QLogic / Emulex 8Gb FC HBA support Xen SDK XenAPI Auto VM placement and load balancing 32 and 64-bit RHEL support Full
  • 46. Performance Comparisons
  • 47. Performance comparisons
  • 48. Moore’s Law: doubling transistors 2yrs 3.7 TFlops 25 racks 512 Servers 1000 sq ft 128 kW 3.7 TFlops 1 rack 53 Blades 40 sq ft 21 kW
  • 49. XEN: build W2K3 DDK build
  • 50. XEN & Application SAP scalability on XEN
  • 51. Reducing XenApp Server Count Issue: 32-bit Windows can only address 4 GB RAM Underutilizes modern multi-core processors 4 GB RAM exhausted Plenty of processor cycles still available
  • 52. Test Results – Performance Comparison Physical Server RAM exhausted at <90 users
  • 53. Test Results – Performance Comparison
  • 54. Optimized for XenApp XenServer 73% more users Other Guys
  • 55. Pool Scalability
  • 56. 1400 VMs, 28 hosts, 1 pool
  • 57. Citrix XenServer OEM Edition
  • 58. Strong Open Partner Ecosystem
  • 59. Enterprise Ready!
  • 60. Windows Certification
  • 61. Disaster Recovery
  • 62. Improved Hardware Support
  • 63. User Experience
  • 64. Beyond Citrix XenServer 5
  • 65. Future of Xen, e.g. Client Initiative • Power Management • Security • S3 suspend to RAM, Rapid • TPM, emulated TPM Boot • OVF specs • GPU passthru w/IOMMU • USB passthru • WiFi / WiMax • Native UEFI support
  • 66. Decision points for XenServer? • Stability, Performance • Ease of use: „10 minutes to Xen“; real :-) • High performance workloads • Best price/performance ratio • Provisioning Server • Close Partnership w/Microsoft
  • 67. “Citrix is a great partner! They know how to partner best with Microsoft. They are the best example for ISV partnership.” —Steve Ballmer (Photo from Microsoft CIO Summit, Mar 1, 2007)
  • 68. New Landing Pages
  • 69. Backup Slides
  • 70. That’s all for Orlando folks! Engineering will return in 2009 in “Midnight Ride” Happy Selling!