Dan Downing Principal Consultant Dion Johnson
Agenda <ul><li>Introduction </li></ul><ul><ul><li>State of adoption of server virtualization </li></ul></ul><ul><ul><li>Op...
State of Adoption of Server Virtualization
Operational Challenges / Barriers
Server Virtualization Technologies <ul><li>Five key technologies </li></ul>The goals:  Enable more VMs on a single chip, m...
Six Critical Factors for Testers <ul><li>Anatomy of a Virtual System </li></ul><ul><li>Mapping workloads  from dedicated t...
1 - Anatomy of a Virtual System <ul><li>“ The abstraction of computer resources” (Wikipedia) </li></ul><ul><li>Type 1:  Na...
2 - Virtual Workload Modeling <ul><li>How quantify the dedicated server workloads? </li></ul><ul><li>Host of what configur...
2a - VMware Capacity Planner <ul><li>Monitor resources on each tier for a week (cpu, memory, disk & network IO) </li></ul>...
3 - VM Bottlenecks & Vulnerabilities <ul><li>#1 limiting resource:  Disk IO </li></ul><ul><ul><li>Thus high IO servers (DB...
4 - Performance Testing Techniques <ul><li>Test in parallel on dedicated and VM environments </li></ul><ul><ul><li>Elimina...
5 - System Resources to Measure <ul><li>VM resources </li></ul><ul><ul><li>Cpu, memory, disk IO, network IO (Unix:  Load A...
5a - Monitoring Host Resources VCenter monitoring host resources kry, in addition to monitoring resources on individual VM...
6 – Presenting Comparative Results Business process end-end response over load on same graph Overall response profiles btw...
6a – Comparative Page Times Differences btwn dedicated and Virtual page response times over load Average difference: Virtu...
6 b – Comparative CPU VM utilizations higher than on dedicated servers, but still low
6c – Comparative Bandwidth Bandwidth usage similar in magnitude, profile
Case Study: SAMPLE <ul><li>Consolidated 18 dedicated servers into 3 virtual hosts </li></ul><ul><li>Reduced rack space, po...
Environment Migration 18 dedicated, underutilized servers… … migrated to 3 Virtual Hosts… … with proven performance & reli...
Customer Benefits <ul><li>Test results enabled him to gain management confidence that the new solution was sound </li></ul...
In Conclusion <ul><ul><li>As testers, we need to stay ahead of the technologies curve to continue adding value to our orga...
Upcoming SlideShare
Loading in …5
×

Performance testing virtualized systems v5

340 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Performance testing virtualized systems v5

  1. 1. Dan Downing Principal Consultant Dion Johnson
  2. 2. Agenda <ul><li>Introduction </li></ul><ul><ul><li>State of adoption of server virtualization </li></ul></ul><ul><ul><li>Operational barriers and concerns </li></ul></ul><ul><ul><li>Virtualization technologies </li></ul></ul><ul><li>The six critical success factors a performance tester needs to know </li></ul><ul><ul><li>Anatomy if a virtual system </li></ul></ul><ul><ul><li>Principles of virtual workload modeling </li></ul></ul><ul><ul><li>Shortcomings and key bottlenecks of virtual systems </li></ul></ul><ul><ul><li>System resources: what to measure and how </li></ul></ul><ul><ul><li>Analyzing and presenting compelling results </li></ul></ul><ul><li>Case Study </li></ul><ul><ul><li>How a provider of insurance market intelligence went virtual, and how they proved performance </li></ul></ul>
  3. 3. State of Adoption of Server Virtualization
  4. 4. Operational Challenges / Barriers
  5. 5. Server Virtualization Technologies <ul><li>Five key technologies </li></ul>The goals: Enable more VMs on a single chip, minimize cpu overhead for IO, and enable monitoring & management Technology Example 1 Example 2 Differentiators Virtualization Software WMware’s vSphere/ESXi Microsoft’s Hyper-V (W2008 Server) <ul><li>Type 1 Native OS hypervisor vs. Type 2 “guest OS” </li></ul><ul><li>Broad OS support </li></ul>Processor architecture Intel Xeon 7500 8-core AMD Opteron 6000 12-core <ul><li>Core density </li></ul><ul><li>Threads, DIMMs per socket </li></ul>Disk IO Interface iSCSI FCoE <ul><li>Protocol to minimize cpu overhead, making network-attached storage perform like attached storage </li></ul><ul><li>Standard Ethernet cabling vs. Fibre </li></ul>Network Interface Broadcom iSoE Intel I/OAT <ul><li>TCP headers processed on NIC </li></ul><ul><li>Multiple NICs with “Teaming” </li></ul>Management Utilities VMware’s vCenter MS Virtual Machine Manager <ul><li>Model virtual workload </li></ul><ul><li>Monitor host and VM resources </li></ul>
  6. 6. Six Critical Factors for Testers <ul><li>Anatomy of a Virtual System </li></ul><ul><li>Mapping workloads from dedicated to virtual </li></ul><ul><li>Key bottlenecks of virtual systems </li></ul><ul><li>Effective comparative testing techniques </li></ul><ul><li>System resources to measure </li></ul><ul><li>Results analysis and presentation </li></ul>
  7. 7. 1 - Anatomy of a Virtual System <ul><li>“ The abstraction of computer resources” (Wikipedia) </li></ul><ul><li>Type 1: Native OS, runs as hypervisor on the host, provides peripheral drivers </li></ul><ul><li>Type 2: Guest OS that runs under main host OS and relies on host OS drivers </li></ul><ul><li>VMs with HW resources allocated by VM software </li></ul><ul><li>VM software running as native OS (“ Hypervisor”) </li></ul><ul><li>Hardware (multi-cpu, multi-core, multi-threaded, lots of memory, multi-NIC) </li></ul><ul><li>Network Attached Storage (NAS) or Storage Area Network (NAS) accessed over standard cabling & cpu-optimized protocols </li></ul>Type 1 Virtualized Server
  8. 8. 2 - Virtual Workload Modeling <ul><li>How quantify the dedicated server workloads? </li></ul><ul><li>Host of what configuration required to handle the total workload? </li></ul><ul><li>How many VMs are needed to handle the workload? </li></ul><ul><li>Which workloads mapped to which VMs? </li></ul>4 Key Questions: Web App Files DB Hardware
  9. 9. 2a - VMware Capacity Planner <ul><li>Monitor resources on each tier for a week (cpu, memory, disk & network IO) </li></ul><ul><li>Enter configuration and role of each server </li></ul><ul><li>Run the modeling tool, analyze the proposed host & VM mapping </li></ul><ul><li>Build prototype and test </li></ul>
  10. 10. 3 - VM Bottlenecks & Vulnerabilities <ul><li>#1 limiting resource: Disk IO </li></ul><ul><ul><li>Thus high IO servers (DB) not good candidates for virtualization </li></ul></ul><ul><li># 1 vulnerability: Redundancy / load balancing </li></ul><ul><ul><li>Like services should be distributed on more than one VM to ensure that system can continue to operate if a VM fails </li></ul></ul>App servers on different hosts
  11. 11. 4 - Performance Testing Techniques <ul><li>Test in parallel on dedicated and VM environments </li></ul><ul><ul><li>Eliminates time-of-day differences due to loads on shared components </li></ul></ul><ul><li>Test a light workload first </li></ul><ul><ul><li>Establish a baseline for both systems at low/no contention </li></ul></ul><ul><li>Test a 50% workload </li></ul><ul><ul><li>Compare performance scalability under reasonable load, without potentially overwhelming the DB </li></ul></ul><ul><li>Test fail-over of VMs </li></ul><ul><ul><li>Validate system recovery / availability /lost transactions if a virtual web / app / file server fails </li></ul></ul>
  12. 12. 5 - System Resources to Measure <ul><li>VM resources </li></ul><ul><ul><li>Cpu, memory, disk IO, network IO (Unix: Load Average) </li></ul></ul><ul><ul><li>Web server: queued requests </li></ul></ul><ul><ul><li>JVM: Heap utilization </li></ul></ul><ul><li>Host resources </li></ul><ul><ul><li>Cpu, memory, disk IO (% disk busy and IOPS), network IO </li></ul></ul><ul><ul><li>DB server: lock waits, deadlocks </li></ul></ul><ul><li>Monitoring utilities </li></ul><ul><ul><li>Windows: perfmon </li></ul></ul><ul><ul><li>Unix/Linux: nmon, sar </li></ul></ul>
  13. 13. 5a - Monitoring Host Resources VCenter monitoring host resources kry, in addition to monitoring resources on individual VMs, to id IO path bottlenecks
  14. 14. 6 – Presenting Comparative Results Business process end-end response over load on same graph Overall response profiles btwn dedicated and virtual platforms similar
  15. 15. 6a – Comparative Page Times Differences btwn dedicated and Virtual page response times over load Average difference: Virtual 7% faster on average Max difference: “a wash” (Virtual 1% faster on average)
  16. 16. 6 b – Comparative CPU VM utilizations higher than on dedicated servers, but still low
  17. 17. 6c – Comparative Bandwidth Bandwidth usage similar in magnitude, profile
  18. 18. Case Study: SAMPLE <ul><li>Consolidated 18 dedicated servers into 3 virtual hosts </li></ul><ul><li>Reduced rack space, power & cooling footprint and maintenance costs </li></ul><ul><li>Performance-tested and showed improved performance and increased efficiency </li></ul><ul><li>Preeminent provider of market intelligence to the commercial insurance industry </li></ul><ul><ul><li>Corporate, legal, financial and regulatory data for hundreds of thousands of public and private companies </li></ul></ul><ul><ul><li>Competitive information for underwriters, brokers and risk managers </li></ul></ul><ul><ul><li>Analytic tools for benchmarking and comparative analysis </li></ul></ul><ul><li>HQ in NYC </li></ul>Solution Business Challenge <ul><li>Needed to contain operating costs while assuring their customers 24/7 availability and performance </li></ul><ul><li>Have 18 dedicated, underutilized servers in a managed hosting Tier 4 data center </li></ul>
  19. 19. Environment Migration 18 dedicated, underutilized servers… … migrated to 3 Virtual Hosts… … with proven performance & reliability
  20. 20. Customer Benefits <ul><li>Test results enabled him to gain management confidence that the new solution was sound </li></ul><ul><li>Saved several $K/month in environment facility and hardware maintenance costs </li></ul><ul><li>Maintained performance and reliability </li></ul>
  21. 21. In Conclusion <ul><ul><li>As testers, we need to stay ahead of the technologies curve to continue adding value to our organizations </li></ul></ul><ul><ul><li>Virtualized systems are becoming the norm </li></ul></ul><ul><ul><li>While the fundamentals are the same, performance testing them require new understanding and know-how </li></ul></ul>

×