This document discusses common performance testing mistakes and provides recommendations to avoid them. The five main "wrecking balls" that can ruin a performance testing project are: 1) lacking knowledge of the application under test, 2) not seeing the big picture and getting lost in details, 3) disregarding monitoring, 4) ignoring workload specification, and 5) overlooking software bottlenecks. The document emphasizes the importance of understanding the application, building a mental model to identify potential bottlenecks, and using monitoring to measure queues and resource utilization rather than just time-based metrics.

1. 1 | P a g e
Performance testing wreaking balls
Leonid Grinshpan, PhD
There are many ways to ruin a performance testing project, there is just a handful of ways to do it right. This publication analyses the most widespread performance testing blunders. It is impossible in one article to expose all the varieties of testing wrongdoings; as such, this publication is definitely an open- ended. It is based on my posts on the Practical Performance Analyst web site (http://www1.practicalperformanceanalyst.com/); I’m thankful to the site owner and manager Trevor Warren for giving me an opportunity and incentive to expose to engineering community my opinion on the topic.
Contents
Wreaking ball 1. Lacking a knowledge of application under test ................................................................. 2
Wreaking ball 2. Not seeing the forest for the trees .................................................................................... 5
Wreaking ball 3. Disregarding monitoring .................................................................................................... 8
Wreaking ball 4. Ignoring workload specification ...................................................................................... 12
Wreaking ball 5. Overlooking software bottlenecks ................................................................................... 15

2. 2 | P a g e
Wreaking ball 1. Lacking a knowledge of application under test
When we have to fix a car engine, the first thing we do is learning how the engine works. But when we plan to load test and tune an application, knowledge of its functions and internals sometimes is considered an unnecessary burden. If an acquaintance with the application is not included into “must to do” list of performance project activities, than with 100% certainty the project will be ruined by application unawareness wreaking ball. To avoid a project fiasco, here’s what we have to know about application before we start deploying a performance testing framework.
Application functionality
What application is intended to do? What tasks does it help to automate? What are the user’s categories (for example, planners, analysts, reviewers, C-level executives, etc.)? How the users of each category are working with the application? Does application process the batch jobs? Does it do it concurrently with the users’ interactive requests?
Knowledge of application functionality and usage enables to specify realistic workload to be generated by load testing tools.
Application components and tuning parameters
Today’s enterprise applications are not monolith software masses; they are comprised of a number of software components hosted on different hardware servers. The most prevalent are Web servers, Application servers, and Databases. Depending on functionality, an application can include reporting, consolidation, printing, data transformation, and many other components processing users’ requests. The components have tuning parameters that makes them adaptable to varying service demands. The examples of tuning parameters are: Java Virtual Machine minimum and maximum heap sizes, number of software threads supporting concurrent processing by a server, number of database connections, etc.
If we know functionality of application’s component as well as each component tuning parameter, we can fix a bottleneck caused by particular component by changing the values of its tuning variables.
Transactions
The users communicate with an application by initiating the transactions that require allocation of the system resources to be processed. The intensity of the user transactions at any given time depends on the number of users actively interacting with a system. It also is a derivative of the pace each user submits one transaction after another.
In order to generate test workload that closely emulates real business workload, we have to identify the following transaction characteristics:

3. 3 | P a g e
- List of transactions generated by the users.
- Per each transaction its intensity expressed in the average number of times a transaction was initiated per one hour by single user.
- Per each transaction, the number of users requesting it.
Transaction profiles
Transaction profile is a measure of single transaction demand for system resources. A transaction triggers a multitude of processing activities by application components hosted on different hardware servers. Each component allocates its resources for transaction processing for a particular time interval. In general, each component has the following assets to be allocated:
Active resources:
- CPU time (data processing)
- I/O time (data transfer)
- Network time (data transfer)
Passive resources:
- Software connections to the servers and services (for example, Web server connections, database connections)
- Software threads
- Storage space
- Memory space
- Software locks
Active resources implement transaction processing and data transfer. Passive resources provide access to active resources. A consumption of an active resource is measured in the time interval it was serving a transaction. A metric for a passive resource usage depends on passive resource type: for software connections and threads it is a number of connections and threads; for memory and storage it is a size of allocated memory. A transaction profile is a set of numbers (vector) specifying quantity of each resource consumed by transaction during its processing by hardware components.
Knowledge of transaction profiles is important for correct deployment of performance monitors to ensure that they record meaningful performance counters capable to identify shortage of system resources under load.
Application processes
For operating system (OS) an application represents a set of processes working under OS control and receiving from OS resources needed to satisfy the demands from user transactions. We have to know the processes of our application in order to monitor system resources allocated to each process. Windows Task Manager provides basic information on each process behavior:

4. 4 | P a g e
Windows Performance Monitor delivers broad range of performance counters for each process. This is an example of performance counter readings for FrameworkService process:

5. 5 | P a g e
In UNIX universe, depending on the UNIX flavor, the process performance counters might look like below:
Monitoring server level counters (like total CPU utilization) might show shortage of a particular hardware resource, but won’t identify which process experiences a resource deficit. To identify that process a monitoring has to be set up per each application process.
Application logs and error files
Application logs and error files are data gold mines - they keep valuable information on executed transactions including, but not limited to, the times transaction was processed by different application components. On some occasions such information let quickly identify a component that is causing a bottleneck.
Information in log and error files helps to check if a workload generated by load testing tools is in sync with the test requirements.
Wreaking ball 2. Not seeing the forest for the trees
In the course of performance testing implementation we usually realize that an application, we have tasked to find out the performance malfunction causes, features tremendous complexity. Today’s distributed application is comprised of a tanglewood of physical and logical objects interacting in a very intricate manner using multiple rules, algorithms, and protocols while serving communities of concurrent users generating fluctuating workloads. That makes application performance testing and troubleshooting extremely cumbersome and time consuming. Identification of a few tuning parameters capable to eliminate a bottleneck can be compared to locating a needle in a large and messy haystack because application features hundreds of tuning parameters on system and application levels.

6. 6 | P a g e
System level tuning parameters define the management policies of the operating systems. Microsoft offers 112 pages document “Performance Tuning Guidelines for Windows Server 2008 R2” (http://tinyurl.com/qx4v4gy). AIX operating system tuning guide by IBM is even fattier – it has 744 pages (http://tinyurl.com/o3b66o8).
Application tuning parameters control application demand for system resources as well as configurations of application internal logical objects like software threads, connection pools, etc. Application vendors publish comprehensive tuning documentations to help optimize their products. Here are a few examples of Oracle all-inclusive performance tuning publications: “Oracle® Fusion Middleware Performance and Tuning Guide” (http://tinyurl.com/kurmd9p), “Oracle® JRockit Performance Tuning Guide” (http://tinyurl.com/mggv55j), “Oracle® Fusion Middleware Performance and Tuning for Oracle WebLogic Server” (http://tinyurl.com/panje7g).
Is it possible to perceive an application in a way that scales down its complexity (similar to taking an aerial view of the Earth)? In other words, can we conceptualize an application to abstract from numerous details and concentrate only on its objects that have a potential to create the bottlenecks? Shortly, can less be more?
The answer is “yes”. An application complexity reduction process encompasses building its mental model. Wikipedia defines a mental model as an explanation of someone's thought process about how something works in the real world (http://en.wikipedia.org/wiki/Mental_model).
In what follows we show how to build a mental model of application that exposes the relations between demand for application services and supply of application resources. By devising a mental model we conceptually leap into different perspective on application; that perspective highlights application components, their interconnections, as well as transaction processing inside application.
Mental model constructs
We need three constructs to build a mental model that serves our purpose:
1. Nodes – represent hardware components processing user requests. The nodes symbolize servers, appliances, and networks.
2. Interconnections among nodes – they stand for connections among hardware components. Interconnections and nodes define application topology.
3. Transactions – characterize user requests for application services. If we visualize a transaction as a physical object (for example, a car), we can create in our mind an image of a car-transaction visiting different nodes and spending some time in each one while receiving a service.
The model’s constructs associate with application objects as shown in a table:

7. 7 | P a g e
Component of application
Matching element in a model
Users and their computers
Node “users”
Network
Node “network”
Server
Node “server”
Transactions initiated by users
Cars
An application can be represented by a mental model as shown on the picture below:
A transaction starts its journey when a user clicks on a menu item or a link that implicitly initiates a transaction. In a model it means that a transaction leaves a node “users”. After that it gets processed in the nodes “network” and “server”. At the end of its journey a transaction comes back to a node “users “. Total time a transaction has spent in the nodes “network” and “server” is a transaction response time.
How mental models help to identify the bottlenecks
Let’s consider what can cause a delay in a processing of a car-transaction in a node. One obvious reason – a node does not have enough capacity when a number of car-transactions concurrently requested a service. In such a case some car-transactions will receive a service, but the others will wait. Another delay reason is not quite obvious. For example, in order to process a transaction in a CPU, an application has to request and to receive a particular memory space. But what will happen if a memory is not available? Obviously, a transaction will wait. In general, this fact means that transaction processing can be delayed as a result of a limited access to a node even if node is not fully utilized.
We came to important conclusion: transaction delay can be cause by two circumstances – a shortage of a resource capacity and a limited access to a resource.

8. 8 | P a g e
That means that in order to indentify where the bottlenecks can potentially take place, we have to monitor all hardware resources that are processing transactions, as well as all objects providing access to resources. Among such objects there are the physical ones (like memory and disk space), as well as the logical programmatic constructs (like software threads, connection pools, locks, semaphores, etc).
The model on above picture suggests that the bottlenecks in our application might happen when there are insufficient CPU and I/O resources in hardware server, as well as when the server has limited memory, application spawns insufficient software threads or features poorly tuned connection pool. Indeed, low network throughput also can cause the bottlenecks.
The model consolidates our bottleneck troubleshooting efforts into right directions; we will deploy monitors to collect server’s CPU and I/O meaningful performance counters as well as we will monitor server memory availability and the behavior of the connection pools and software threads for our application processes. We also will use network monitors to assess network latency and throughput.
The bottom line – the mental models expose application fundamentals distilled of innumerable application particulars that conceal the roots of performance issues.
From mental models to queuing models
The application mental models are indispensable instruments streamlining our performance testing and troubleshooting activities. The mental models point to the facts that a bottleneck happens when a node does not have sufficient capacity or access to the node is limited. In both cases processing of a transaction will be delayed because transaction will be placed into a waiting queue.
Queuing is a major phenomenon defining application performance, but mental models cannot quantitatively asses its impact on transaction times, as well as on application architecture. If we want to find out an application architecture delivering application performance according to a service level agreement, we have to transition from the application mental models to their queuing models. The book [1] can be a guide in that journey.
Wreaking ball 3. Disregarding monitoring
In order to find out what causes unacceptable production system performance we have to monitor its hardware and software components during normal operation or while running a load test. A system monitoring framework can be compared to medical doctor diagnostic equipment – without it a doctor has in his disposal only patient’s complains but not the data objectively describing vital body functions. Running performance tests without monitoring performance counters or monitoring the wrong ones is a useless undertaking as it does not deliver any information on application performance under load.

9. 9 | P a g e
Today’s systems expose hundreds of counters available for monitoring. The commonly utilized counter’s categories are:
- The counters reporting utilization of system resources during some time interval (for example, percent of total CPU utilizations, percent of CPU utilization by a particular process, percent of physical disk utilization)
- Resource throughput measured in a number of operations executed by a resource during particular time interval (for example, network throughput measured in bytes/second, number of I/O Reads/second)
The common characteristic of both categories is that all their counters are time dependent. That means an accuracy of reported by a counter parameter value depends on accuracy of the time measurement. Unfortunately, in dominant today virtualized and cloud computing environments a timekeeping is flawed. A detailed discussion on that can be found in [2,3]; here we just point at its fundamental reason: in a virtual environment hypervisor treats guest operating system (OS) as any other process that can be stopped and resumed at any time. When guest OS is stopped, it cannot accept time interrupts from hardware clock. That means guest OS misses time intervals as it cannot measure time when it is not running; that makes making time-dependent metric not representative. Taking into consideration this fact, what are the right objects to monitor in virtualized environments?
To find it out let’s invoke a system’s conceptual queuing model. We described a representation of the systems by the models in a previous paragraph and we stated that a queuing model is an abstract representation of a system that includes a depiction of the system resources as well as the demands for resources generated by the users (more on queuing models of the systems can be found in a book [1]).
Queuing models create systematic framework for system performance analysis and capacity planning. The queues are the major phenomenon defining system performance, because waiting time in the queues adds up to the time a transaction is processing by system resources. A queue is an indicator of an imbalance between demand generated by fluctuating user’s workload and availability of system’s resources to satisfy the demand. As such, while troubleshooting a performance bottleneck, it is necessary to find out where in a system the queues are building up and exceeding the acceptable thresholds. That can be done by monitoring internal system queues. Because instantaneous counts of the queue lengths do not depend on implementation of a system timekeeping mechanism, this approach delivers representative performance metrics for any environment.
We already noted that application requests two kinds of resources to process the user transactions - active and passive. In order to be processed by any active resource, a transaction has to request and get allocated the passive resources. If any resource needed for transaction processing is not available because all supply is taken by other transactions, a transaction will wait in a queue until resource is released. Indeed, wait time will increase transaction response time and application performance will suffer.

10. 10 | P a g e
Observing queues is not an exceptional task – it can be done using built into OS performance monitors and counter-reporting commands. We compiled in the table below information on the Windows counters that deliver instantaneous queue lengths for different system objects. The table is far from all- inclusive, but it is sufficient enough to demonstrate queue-based performance monitoring tactic.
Counter
Description
Active resources queues
Queue Length (object Server Work Queues)
Queue Length is the current length of the server work queue for particular CPU. A sustained queue length greater than four might indicate processor congestion.
Processor Queue Length (object System)
Processor Queue Length is the number of threads in the processor queue. Unlike the disk counters, this counter shows ready threads only, not threads that are running. There is a single queue for processor time even on computers with multiple processors. Therefore, if a computer has multiple processors, you need to divide this value by the number of processors servicing the workload. A sustained processor queue of less than 10 threads per processor is normally acceptable, dependent of the workload.
Current Disk Queue Length (object Physical Disk)
Current Disk Queue Length is the number of requests outstanding on the disk at the time the performance data is collected. This counter might reflect a transitory high or low queue length, but if there is a sustained load on the disk drive, it is likely that this will be consistently high. This counter shows not only waiting requests, but also the ones that are processing. That is why requests delays are proportional to the length of this queue minus the number of spindles on the disks. For good performance, this difference should average less than two.
Output Queue Length (object Network Interface)
Output Queue Length is the length of the output packet queue on Network Interface Card. If this is longer than two, there are delays and the bottleneck should be eliminated. Microsoft informs that Output Queue Length counter value is not reliable on computers with symmetric multiprocessing (SMP) architecture; the details are here: http://support.microsoft.com/kb/822226.
Jobs (object Print Queue)
The counter shows current number of jobs in a print queue.
Passive resources queues
Current Queue Size (object HTTP Service Request Queues)
The counter shows number of requests in the Web server queue. Requests begin to accumulate when IIS falls behind in dequeueing requests. The limit is set by the application pool’s queueLength attribute, and defaults to 1000. When limit is reached, HTTP.SYS returns 503 Service Unavailable [http://blog.leansentry.com/2013/07/all- about-iis-asp-net-request-queues/] .
Requests Queued (object ASP.NET)
The number of requests waiting to be processed. When this number starts to increment linearly with respect to client load, the Web server computer has reached the limit of concurrent requests that it can process [http://www.techiesbytes.com/2010/05/using-perfmon-for-web-application- based.html].

11. 11 | P a g e
Requests in Application Queue (object ASP.NET Applications)
In Classic mode, ASP.NET will queue all incoming requests to the per-application queue when there are not enough threads. The threads available for request processing are determined by available threads in the CLR thread pool, minus the reserved threads set by the httpRuntime/minFreeThreads andhttpRuntime/minFreeLocalThreads attributes.
This queue indicates poor performance, and does not guarantee FIFO policy in application pools with multiple applications. Because threads are shared between multiple apps, so a single app can starve the other applications of available threads [http://blog.leansentry.com/2013/07/all-about-iis-asp-net-request-queues/].
Requests Queued (object ASP.NET v4.0.30319)
In Integrated mode, ASP.NET will queue all incoming requests after the configured concurrency limit is reached. Concurrency limit is set by the MaxConcurrentRequestsPerCPU registry key or applicationPool/maxConcurrentRequestsPerCPU attribute (Defaults to 12 on .NET 2.0/3.5, and 5000 on .NET 4.0+) and MaxConcurrentThreadsPerCPU registry key or the applicationPool/MaxConcurrentThreadsPerCPU attribute (defaults to 0, disabled) [http://blog.leansentry.com/2013/07/all-about-iis-asp-net-request-queues/].
Current Queue Length (object .NET CLR LocksAndThreads)
This counter (pertaining to Common Language Runtime Microsoft’s execution environment) displays the last recorded number of threads currently waiting to acquire a managed lock in an application. You may want to run dedicated tests for a particular piece of code to identify the average queue length for the particular code path. This helps identify inefficient synchronization mechanisms [http://blog.monitis.com/2012/09/14/improving-net-performance-part-17- measuring-net-application-performance-ii/].
Objects .NET Data Provider for SqlServer and .NET Data Provider for Oracle
Both objects have ten counters each providing the counts (not rates!) characterizing size and status of connection pools and connections. Information on monitoring and tuning connection pools can be found here:
http://betav.com/blogadmin/mt- search.cgi?search=Managing+and+Monitoring+.NET+Connection+Pools&IncludeBlogs=3&limit=20
Thread State and
Thread Wait Reason(object Thread)
A software thread is a logical vehicle that moves execution forward. Each process initiates one or a few threads. Monitoring its thread states provides information on what causes delays in process execution.
Thread State counter reports the current state of the thread. It is 0 for Initialized, 1 for Ready, 2 for Running, 3 for Standby, 4 for Terminated, 5 for Wait, 6 for Transition, 7 for Unknown. A Running thread is using a processor; a Standby thread is about to use one. A Ready thread wants to use a processor, but is waiting for a processor because none are free. A thread in Transition is waiting for a resource in order to execute, such as waiting for its execution stack to be paged in from disk. A Waiting thread has no use for the processor because it is waiting for a peripheral operation to complete or a resource to become free.
Thread Wait Reason counter is only applicable when the thread is in the Wait state. There are multiple wait reasons, but each one indicates that execution of the thread is suspended as it is waiting for a particular event. The examples of thread wait reasons:

12. 12 | P a g e
waiting for a memory page to be freed waiting for a component of the Windows NT Executive waiting for a page to be written to disk
For enumeration of wait reasons visit http://msdn.microsoft.com/en- us/library/ms804615.aspx
This list of queue-reporting counters is extracted from Windows Performance Monitor; its goal is to demonstrate the primary tenets of queue monitoring. Today’s application landscape is tremendously diversified and requires monitoring of the queue-reporting counters for multiple UNIX flavors, the counters pertaining to different technologies, as well as application-specific counters exposed by built-in instrumentation. The common denominator is the same – if a counter shows that a queue exceeds an acceptable threshold than system performance is degraded.
In addition to timekeeping issue in VMs, the technological advances like hyperthreading, power management, CPU entitlement, and the others also distort time-dependent performance counters [4]. That makes queue monitoring a trusted and preferred methodology for a wide range of the systems that are built upon sophisticated technologies.
Wreaking ball 4. Ignoring workload specification
The essential task of any application is processing of a generated by the users production workload according to the requirements stated in a service level agreement. To ensure that application is capable to successfully execute such a task, the performance engineers carry out load testing. Correct specification of production workload is a cornerstone of the load testing projects; it enables to come up with proper application sizing and tuning recommendations.
This paragraph analyses the components of application workload; it demonstrates that ignoring or skewing production workload specifications while generating workload by load testing tools makes tuning recommendations meaningless, or ever worse, misleading.
1. The users communicate with application by issuing the transactions that represent the requests to perform particular application functions. Each request triggers a transaction execution process that consumes various hardware and software resources: CPUs, memory, network bandwidth, software threads, database connections, etc. The transactions examples are: login into system with user name and password; calculate revenue of plasma TV sales in New England stores in August 2012; estimate the expenses to launch new product line in China, create company profit and loss statement. Each transaction ends up by delivering the results to the users. As an example, Figure 1 shows a financial report generated as a reply to a reporting transaction (http://www.oracle.com/technetwork/middleware/bi-foundation/financial- reporting-large-125270.gif).

13. 13 | P a g e
Figure1 Financial report generated as a reply to a reporting transaction
A rich functionality supported by an application for each line of business is reflected in a broad assortment of transactions available to the users. Some applications have hundreds transactions accessible over user interfaces; such a richness of application’s front-ends sometimes collides with a need for simplicity of a user experience. Nevertheless, the business applications must be in tune with a complexity of the business processes and the way to implement it is to let users to initiate the transactions delivering needed information.
The flow of transactions constitutes application’s transactional workload. The intensity of the transactions at any given time depends on a number of users actively interacting with a system. It also is a derivative of a pace each user submits one transaction after another. The interval between consecutive transactions from the same user can be substantial as a user needs time to assess a reply from an application to a previous request, and to prepare the next one.
A workload specification includes a list of transactions, a number of concurrent users initiating each transaction, and, per each transaction, its rate measured in a number of times it is set off during one hour by one user. This is an example of characterization of a user-generated workload:
List of transactions
Number of
users initiating
transaction
Transaction rate
(average number of transactions
initiated by one user per one hour)
Retrieve financial report
100
5

14. 14 | P a g e
Consolidate sale data
25
2
Enter quantity of sold on-line products
400
2
The table indicates that business users request every hour 100*5 = 500 financial reports, launch consolidation of sales data 25*2 = 50 times, and enter a quantity of sold on-line products 400*2 = 800 times. Let’s assume that the table specifies a real production workload; appropriately sized and tuned for this workload application will be able to process it according to a service level agreement.
But what will happen if we are using for application tuning a load testing framework that generates a workload with downward or upward deviations from the numerical values in the table? In this case our load test does not generate production workload, and under such a skewed workload application will have either idle or insufficient capacity. Tuning application for a skewed workload is technically doable, but because such workload never happens in real life, our tuning efforts are not just useless, they are misleading, they degrade quality of service and might lead to unnecessary expenses if we recommend to purchase additional hardware.
While executing performance tuning and capacity planning projects, we have to be well aware of how representative are our manufactured workloads of the real production workloads. Let’s consider presented on Figure 2 results of a load test conducted using LoadRunner, and find out how a workload, generated by this test, associates with a production workload.
Figure 2 Load test results as reported by LoadRunner

15. 15 | P a g e
The test includes the following transactions executed in such an order:
1. User logs in
2. Open Planning application
3. Open Web Form
4. Save Form
5. If test time not expired, go to 1, otherwise stop the test
As column “Pass” indicates, each transaction was executed 372 times during test which lasted 7 minute and 37 seconds. That is equal to 372 / 7 min 37 sec = ~ 50 executions of each transaction every minute. Because the number of concurrent users is 36, every user initiated one transaction (50/36)*4 = ~5.6 times per minute. Definitely, a workload generated by this test is not production one but stressful. This workload can be used for testing stability of the system servers, but not for sizing and tuning of production environment.
Detailed discussion of application transactional workloads and importance of their specifications for proper sizing and tuning can be found in a Chapter 3 of the book [1].
Wreaking ball 5. Overlooking software bottlenecks
Software bottlenecks are often overlooked causes of poor application performance because, when present, they limit usage of hardware resources making IT departments satisfied with infrastructure capacity but system users unhappy with long transaction times.
The effect of software bottlenecks can be demonstrated by the following analogy (see picture below). Let’s mentally model hardware server by a highway toll station, each server’s CPU by one toll booth, incoming transactions by the cars, and waiting queue by the cars congregating on a toll plaza.

16. 16 | P a g e
The cars arrive at tool plaza from different directions and they are jammed in toll booth approaching lanes because plaza has too few of them. As soon as a car gets through the jam, it immediately finds an idle toll booth. Increasing a number of tool booths cannot improve plaza’s throughput; what will do the trick is adding approaching lanes. More lanes dissolve traffic congestion and keep toll booths busy.
What constitutes application’s “approaching lanes”? They are software tuning parameters regulating access to hardware resources. The most common parameters are:
 Memory space an application can use
 Number of threads providing access to CPUs/cores
 Number of connection to database
 Number of Web sessions with interactive users
Insufficient settings of any of these parameters limit access to available hardware resources keeping some of them idle and at the same time causing unacceptable increase of transaction time. Performance engineers have to find optimal values of software tuning parameters to ensure application performance in sync with business requirements.
1. Leonid Grinshpan. Solving Enterprise Applications Performance Puzzles: Queuing Models to the Rescue, Wiley-IEEE Press; 1 edition, 2012 (http://tinyurl.com/7hbalv5)
2. VMware document: “Timekeeping in VMware Virtual Machines” http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf
3. Bernd Harzog. “White Paper: Application Performance Management for Virtualized and Cloud based Environments” http://www.virtualizationpractice.com/blog/wp- content/plugins/downloads- manager/upload/APM_for_Virtualized_and_Cloud_Hosted_Applications.pdf
4. Adrian Cockcroft “Utilization is Virtually Useless as a Metric!” http://www.hpts.ws/papers/2007/Cockcroft_CMG06-utilization.pdf