Managing Performance in a Virtual Environment


Published on

For more information on Virtualization Manager, visit:

Watch this webcast:

Watch this webcast with Eric Siebert, VMware® vExpert, blogger and published author, along with SolarWinds’ virtualization management expert, Jon Reeve. We’ll be discussing, “Managing Performance in a Virtual Environment.”

In a virtual world, the most important metrics all boil down to one thing: Performance. Virtual infrastructure performance really determines the end user experience, and it’s the one metric that aggregates the entire infrastructure including CPU, memory, storage and network.
So, the key for every virtualization administrator is to maximize performance while providing appropriate levels of availability. This includes key tasks like:

• Right-sizing your virtual machines by allocating the appropriate resources
• Isolating and troubleshooting performance bottlenecks
• Freeing up wasted resources (like zombie VMs, idle/stale VMs, etc.) so you have plenty of resources to allocate.

In this webinar, we’ll talk about how to perform these tasks and more.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Virtualization Manager - Server and Application Monitor
  • Aggreaget and format data
  • Managing Performance in a Virtual Environment

    1. 1. Performance Management in a Virtual Environment 2/22/12 IT Management Simplified
    2. 2. About the Speakers <ul><li>Eric Siebert </li></ul><ul><li>25+ years experience in the IT industry </li></ul><ul><li>3 time VMware® vExpert </li></ul><ul><li>Author of 2 books on virtualization </li></ul><ul><li>Top virtualization blogger </li></ul>Follow me on Twitter @ericsiebert Visit my website: <ul><li>Jonathan Reeve </li></ul><ul><li>SolarWinds, Senior Director of Product Management </li></ul><ul><li>Previously ran product management at Hyper9™ </li></ul><ul><li>Multiple successful start-ups in the IT space </li></ul>
    3. 3. Agenda <ul><li>Know where to start when it comes to monitoring performance </li></ul><ul><li>Know what to look for with performance </li></ul><ul><li>Learn the key resource counters you want to focus on </li></ul><ul><li>Know where to look when monitoring performance </li></ul><ul><li>Understanding the domino effect with virtualization </li></ul><ul><li>Learn how to interpreting the data to know what all those numbers mean </li></ul><ul><li>Learn how SolarWinds can provide you the tools you need to properly manage your virtual environment </li></ul>
    4. 4. Why is monitoring performance important? <ul><li>Nothing can kill a virtual environment faster than not having adequate performance for all VMs </li></ul><ul><li>VMs are all fighting for limited host resources </li></ul><ul><li>Virtual environments are like small children, they require constant monitoring and attention </li></ul><ul><li>If you don’t monitor, you could end up with a mess on your hands </li></ul><ul><li>Performance monitoring with virtualization is complicated – more to monitor and interpreting the statistics can be difficult </li></ul>© iStockphoto
    5. 5. Monitoring Performance – Where to Start <ul><li>Traditional non-virtual environments, performance monitored through guest OS using agents or WMI </li></ul><ul><li>Not effective in virtualized environments; guest OS is no longer seeing physical hardware </li></ul><ul><li>Virtualization layer is transparent to the guest OS </li></ul><ul><li>Virtual hardware is emulated by the hypervisor, the guest OS doesn’t know the big picture </li></ul><ul><li>Statistics measured inside guest OS are not an accurate reflection of the physical hardware of the host usage </li></ul><ul><li>VMware resource controls impact guest OS reporting </li></ul>
    6. 6. Know Where to Look <ul><li>Performance should be monitored at virtualization layer </li></ul><ul><li>Many statistics that are unique to virtualization </li></ul>
    7. 7. Monitor the Entire Stack <ul><li>Don’t completely ignore the guest OS; some stats are relevant </li></ul><ul><li>A good monitoring tool will look at both the virtualization and guest OS layer as well as the app layer </li></ul><ul><li>vCenter Server is focused on the virtualization layer </li></ul><ul><li>SolarWinds provides a complete performance picture of all layers </li></ul>Guest OS Stats Virtualization Stats Memory usage by app/process Active Memory Used CPU usage by app/process CPU % Ready Memory Pages/Sec Memory Swapped (vswp) Disk I/O by app/process Kernel Latency
    8. 8. Know What to Look for <ul><li>Virtual environments can generate hundreds of different kinds of performance stats </li></ul><ul><li>Statistic Levels determine which stats are collected </li></ul><ul><li>Not all stats useful, some are useful for monitoring health and usage, other are more useful for troubleshooting </li></ul><ul><li>There are different object levels where stats are collected: </li></ul><ul><ul><li>Datacenter </li></ul></ul><ul><ul><li>Cluster </li></ul></ul><ul><ul><li>Resource Pool </li></ul></ul><ul><ul><li>Host </li></ul></ul><ul><ul><li>Virtual Machines </li></ul></ul><ul><li>Sometimes need to look at different levels to see aggregated stats or those specific to an object </li></ul>
    9. 9. Why Not Use vCenter Server <ul><li>vCenter Server already collects performance statistics, why shouldn’t I just use it for monitoring? </li></ul><ul><li>Statistics are all dumped into one database, which can get very large with millions of rows </li></ul><ul><li>Database is also used for other vCenter Server data </li></ul><ul><li>90% of the database consists of performance statistics </li></ul><ul><li>The larger the database gets, more difficult it is to manage & vCenter Server slows down </li></ul><ul><li>Problems with database can cause vCenter Server to become unavailable </li></ul>
    10. 10. Why Not Use vCenter Server <ul><li>vCenter Server is a multi-function management tool </li></ul><ul><li>Responsible for many functions such as: </li></ul><ul><ul><li>Provisioning VMs and resources </li></ul></ul><ul><ul><li>VM & Host management </li></ul></ul><ul><ul><li>DRS, </li></ul></ul><ul><ul><li>vMotion®, </li></ul></ul><ul><ul><li>Distributed vSwitches </li></ul></ul><ul><li>Critical to the proper function of the whole environment </li></ul><ul><li>vCenter Server does many things OK but not everything great, offload the monitoring </li></ul><ul><li>SolarWinds can offload this and do it better </li></ul>
    11. 11. Key Statistics – CPU/Memory Statistic What It Means Why It’s Important CPU - Ready VM stat – amount of time in ms spent waiting for CPU High value indicates bottleneck or too many vSMP VMs as VMs wait for CPU time CPU - Used VM/Host stat – amount of CPU time used in ms High value indicates VMs are saturating vCPUs and may benefit from additional vCPUs if CPU Ready is low CPU - Usage VM/Host stat – total amount of CPU usage measured as a % For VMs usage measured by host (avg. of all vCPUs), for hosts actively used percentage of total available CPU Mem - Swapped VM/Host stat – amount of mem in KB swapped to virtual swap file Large number indicate lack of physical memory or memory limits, optimally this should be zero Mem - Active VM/Host stat – amount of true host mem used by VM in KB For VMs amount of memory actively used, for hosts sum of all VM active memory plus host overhead Mem - Ballooned VM/Host stat – amount of memory in KB that is used by balloon driver For VMs this is the amount of physical memory reclaimed by balloon driver, for hosts it’s the sum plus overhead
    12. 12. Key Statistics – Disk/Network Statistic What It Means Why It’s Important Disk – G/AVG Host stat – amount of time in ms it takes to process SCSI command High latency has a big impact on VM performance, once it exceeds 30ms VMs will slow, above 50ms is severe Disk - Qued Host stat – amount of time in ms that commands spend in VMkernel queue before sent to device queue Above 5ms indicates SCSI commands spending too much time in VMkernel®, may need to increase queue depth Disk – Usage VM/Host stat – aggregated disk I/O rate measured in KBps Good general stat to show disk activity, for hosts its combined total of all VMs Disk – Commands VM/Host stat – number of SCSI commands that have been issued Another good general stat that shows disk activity in IOPS instead, for hosts its combined total of all commands sent to disk targets Net – Dropped TX/RX Host stat – number of transmit & receive packets dropped If greater than zero you may need additional pNICs in a vSwitch or receive buffers increased Net - Usage VM/Host stat – combined transmit & receive rates measured in KBps Good general stat for net activity, for VMs sum of traffic across all vNICs, for hosts sum of all traffic across all pNICs
    13. 13. Understanding Disk Latency <ul><li>Total Guest Latency (GAVG) is measured where I/O enters VMkernel to the point it arrives at storage device </li></ul><ul><li>Kernel Latency (KAVG) is time I/O spends in VMkernel </li></ul><ul><li>Queue Latency (QAVG) is part of VMkernel but measured independently </li></ul><ul><li>Device Latency (DAVG) is time I/O spends in driver code and storage device </li></ul><ul><li>GAVG = DAVG + KAVG </li></ul><ul><li>GAVG should be < 20ms </li></ul><ul><li>High DAVG indicates problem with storage array being too busy or improperly architected </li></ul><ul><li>High KAVG can be result of queue depth too small </li></ul>© SolarWinds Whitepaper: Storage I/O Bottlenecks in a Virtual Environment
    14. 14. Knowing Where to Look <ul><li>Virtual environments are complicated with many moving parts, shared resources and dependencies </li></ul><ul><li>Pinpointing cause of performance problems can be complicated and frustrating </li></ul><ul><li>Problems can be very disruptive, must be quickly resolved </li></ul><ul><li>First question asked when problems seemingly occur out of nowhere is: what changed? </li></ul><ul><li>Something as innocent as changing a setting can have a big impact </li></ul><ul><li>SolarWinds can track changes using host & VM DNA so events can be correlated to problems </li></ul>
    15. 15. Domino Effect in Virtual Environments <ul><li>Because of shared resources, small things can have big ripple effects </li></ul><ul><li>Performance problems can effect VMs within a host and within a cluster </li></ul><ul><li>Once one VM puts intense pressure on host resources, it can cause a denial of service type attack </li></ul><ul><li>Chain reaction can cause all VMs on multiple hosts to become unresponsive </li></ul><ul><li>Must continually monitor and know normal resource usage patterns so you can spot irregularities </li></ul><ul><li>SolarWinds can show you resource trends and alert you when contention begins to occur </li></ul>
    16. 16. Interpreting the Data <ul><li>Many performance statistics that are unique to virtual environments like: CPU Ready, Kernel Latency, Memory Consumed, Queue Depth, Memory Ballooned </li></ul><ul><li>If you don’t know what these stats are, how can you understand what the values mean </li></ul><ul><li>Trying to interpret performance stats and understanding relationships to other stats can be very difficult </li></ul><ul><li>SolarWinds can be your interpreter and help translate the numbers into more meaningful reports and dashboards </li></ul><ul><li>Help you understand relationships between resources & applications </li></ul>
    17. 17. SolarWinds Virtualization Manager DEMO
    18. 18. Summary <ul><li>You can’t afford to be reactive when it comes to performance </li></ul><ul><li>Being proactive can help you prevent big problems from occurring </li></ul><ul><li>If you don’t monitor constantly how do you know if you have a problem today or if its been there all along </li></ul><ul><li>To be successful at virtualization, you must maintain good performance </li></ul><ul><li>Effects of poor performance can be far-reaching </li></ul><ul><li>Having proper tool to monitor performance is critical </li></ul><ul><li>Use the right tool for the job: SolarWinds Virtualization Manager </li></ul>