Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Getting start with Performance Testing


Published on

i hope this would help you to get start with performance test basic and jargon's.

Published in: Education, Technology
  • very nice
    Are you sure you want to  Yes  No
    Your message goes here

Getting start with Performance Testing

  2. 2. ALL RIGHTS RESERVED 2013 Agenda: Introduction to Performance Testing Why Performance Testing? Performance Testing jargons Performance Test Metrics and Process Limitations of Manual Performance Testing Benefits of Performance Test Automation  Tools used for Performance Testing
  3. 3. ALL RIGHTS RESERVED 2013 Performance Testing Wikipedia says: “In the computer industry, software performance testing is used to determine the speed or effectiveness of a computer, network, software program or device. This process can involve quantitative tests done in a lab, such as measuring the response time or the number of MIPS (millions of instructions per second) at which a system functions. Qualitative attributes such as reliability, scalability and interoperability may also be evaluated. Performance testing is often done in conjunction with stress testing.” Our Simplified Definitions Performance Testing = how fast & stable is the system?
  4. 4. ALL RIGHTS RESERVED 2013 Why Performance Testing?
  5. 5. ALL RIGHTS RESERVED 2013 Oh it’s 10 o’clock Got to book my Rail Ticket..
  7. 7. ALL RIGHTS RESERVED 2013 Poor Performance Affects: • Loss of revenue  Amazon states that for every 100ms of latency, they lose 1% of their sales.  Google's experiments prove statistically that slowing its search results by 400ms costs millions of dollars per year due to lost advertisement revenue.  Industry experts estimate that revenue of $3 billion has lost in 2012 because of slow site performance. • Loss of customers  Google says they could lose 8 million searches daily if their search results are 1/2 second slower.  Flipkart says “57% of shoppers will abandon a site after waiting 3 seconds for a page to load, out of them 80% don’t return again on same site.”
  8. 8. ALL RIGHTS RESERVED 2013 Continued… • Loss of productivity  Reddit, states that their performance focus is on page load speed. “If we can get 10% more performance, we immediately see 10% more traffic.”  Better performing website (speed improvements) increased productivity by 7-12%. • Backlog of Work  On an average 1.2 million people try to book tickets in the first ten minutes of the Tatkal timeframe, Of that only 50000 people come out of the website with tickets, at the success rate of 4.16% only due to service backlog during peak. • Media attention and damage to your brand  Times Of India Ranked IRCTC “1st” out of 8 ‘worst’ Indian government websites in 2012.
  9. 9. ALL RIGHTS RESERVED 2013 Why Performance Testing? • Identifies problems early on before they become costly to resolve. • Produces better quality, more scalable code. • Prevents revenue and credibility loss due to poor Web site performance. • Enables intelligent planning for future expansion. • To ensure that the system meets performance expectations such as response time, throughput etc.
  10. 10. ALL RIGHTS RESERVED 2013 Performance Testing Jargons  Before going into the details, we should understand the factors that governs Performance testing:  Business Transaction  Think Time  Concurrent User load  Simultaneous User load  Throughput  Response Time  Tuning  Baseline Test  Benchmarking  User Abandonment  Performance Bottleneck  Load Testing  Stress Testing  Spike Testing  Volume Testing
  11. 11. ALL RIGHTS RESERVED 2013 Business Transaction • It refers to a sequence of request-response and the related information exchange between the client and the server in order to accomplish a business requirement. • For example, an online banking customer transferring money from one account to other is a business transaction.
  12. 12. ALL RIGHTS RESERVED 2013 Think Time • It refers to the time taken by the user for thinking or clicking on any of the web page links, buttons, etc while navigating through the web site. • Think time is a very important parameter which needs to be set while scripting the scenarios using the performance testing tools. • The business analysts of the application or web site management team or sometimes even the end user survey might give a realistic picture about the think time requirements of a transaction.
  13. 13. ALL RIGHTS RESERVED 2013 User Abandonment • It refers to the situation wherein the end users exit from the web site because of the slow performance of the site. • The user abandonment rate varies from web site to web site. A low priority web site might experience a high user abandonment rate compared to the payroll web site. Analysis on the abandonment rate of the site needs to be done before concluding on the load on the site.
  14. 14. ALL RIGHTS RESERVED 2013 Simultaneous User load • The simultaneous users have the active session on the server at any point of time wherein each user will be executing different transactions. • For example, if we say 100 simultaneous users load, then there will be 100 active sessions opened up in the server, wherein each user will be performing different set of transactions – one logging in, another viewing reports, another navigating to the next page, etc. • The simultaneous user load of a web site would be always greater than the concurrent user load of a web site.
  15. 15. ALL RIGHTS RESERVED 2013 Concurrent User load • The concurrent users connect to the server and perform the same operation at any point of time. • For example, if we say 100 concurrent user load, all 100 users would be logging in at the same point of time, view the reports at the same point of time, etc. For example, an online banking web site might have 10,000 – 20,000 simultaneous user load, but 1000 to 1500 concurrent user load.
  16. 16. ALL RIGHTS RESERVED 2013 Baseline Test  Make you familiar with the operational behavior of each app/server.  Tells you all about the performance of system under normal conditions.  It refers to the test conducted to measure the application performance for limited virtual user load.  The baseline test is often conducted to collect the metrics about the system performance for 1,5,50…. user load .
  17. 17. ALL RIGHTS RESERVED 2013 Benchmarking  Benchmarks are baselines at known and defined levels of load.  The objective of the benchmark test is to validate the correctness of the test scripts and to check the readiness of the system before subjecting it to a high load.  Normally for a benchmark test, 15 -20% of the target load can be considered.
  18. 18. It is equally important to find out how much time each of the transactions took to complete. Response time is defined as the delay between the point of request and the first response from the product. The response time increases proportionally to the user load. ALL RIGHTS RESERVED 2013 Response Time • It is equally important to find out how much time each of the transactions took to complete. • Response time is defined as the delay between the point of request and the first response from the product. • The response time increases proportionally to the user load.
  19. 19. ALL RIGHTS RESERVED 2013 Throughput • Capability of a product to handle multiple transactions in a give period. • Throughput represents the number of requests/business transactions processed by the product in a specified time duration. • As the number of concurrent users increase, the throughput increases almost linearly with the number of requests. As there is very little congestion within the Application Server system queues.
  20. 20. ALL RIGHTS RESERVED 2013 Tuning • Tuning is the procedure by which product performance is enhanced by setting different values to the parameters of the product, operating system and other components. • Tuning improves the product performance without having to touch the source code of the product.
  21. 21. ALL RIGHTS RESERVED 2013 Performance Bottleneck • It refers to the slow spot, the effects of which are widely felt. It refers to the situation/areas which do not allow the application to perform as per its ideal specifications. • For example, the response time increase for the load of 100 virtual users because of improper setting of HTTP connections parameter in the IIS server, CPU utilization reaching 95% during 100 users load are typical performance bottlenecks. A bottleneck might lead to a failure if mitigation actions are not taken.
  22. 22. ALL RIGHTS RESERVED 2013 Load Testing • Process of exercising the system under test by feeding it the largest tasks it can operate with. • Constantly increasing the load on the system via automated tools to simulate real time scenario with virtual users.
  23. 23. ALL RIGHTS RESERVED 2013 Stress Testing • Trying to break the system under test by overwhelming its resources or by taking resources away from it. • Purpose is to make sure that the system fails and recovers gracefully.
  24. 24. ALL RIGHTS RESERVED 2013 Spike Testing • It refers to test conducted by subjecting the system to a short burst of concurrent load to the system. This test might be essential while conducting performance tests on an auction site wherein a sudden load is expected. • The goal is to determine whether performance will suffer, the system will fail, or it will be able to handle dramatic changes in load.
  25. 25. ALL RIGHTS RESERVED 2013 Volume Testing • It refers to the tests designed to measure the throughput of the system more in a batch processing, messaging kind of environment. The objective of this test is to identify the processing capacity. • For example, if you want to volume test your application with a specific database size, you will expand your database to that size and then test the application's performance on it.
  26. 26. ALL RIGHTS RESERVED 2013 Transaction Based Metrics Server Based Metrics Metrics to be Captured Comments Throughput (per Sec) Transactions per sec Response Times / Elapsed Times Time taken to process a transaction Types of Errors Totals for different types of errors for a particular test Count of Errors Total errors for a particular test. Transaction Count Total transactions processed for the time period Metrics to be Captured Comments CPU Usage System% Idle% Memory Usage Used% Used in GB Memory available Disk I/O Disk Read KB/sec Disk Write KB/sec IO/sec Network Activity MB/sec Packets/sec Size of packets Bandwidth used
  27. 27. ALL RIGHTS RESERVED 2013 Performance Testing Process
  28. 28. ALL RIGHTS RESERVED 2013 Performance Testing Manual OR Automation…
  29. 29. ALL RIGHTS RESERVED 2013 Testers Load Generation System Under Test Do you have the testing resources? • Testing personnel • Client machines How do you coordinate and synchronize users? How do you collect and analyze results? How do you achieve test repeatability? Analysis? 123.20 All of you, click the GO button again Manual Performance Testing Limitations Web server Database server Coordinator
  30. 30. ALL RIGHTS RESERVED 2013 Continued… Manual Testing Limitations  Expensive, requiring large amounts of both personnel and machinery.  Complicated, especially co-ordinating and synchronising multiple testers  Involves a high degree of organization, especially to record and analyse results meaningfully  Repeatability of the manual tests is limited
  31. 31. ALL RIGHTS RESERVED 2013 Load Generation System Under Test Benefits of Automation Web server Database server Vuser host Analysis Controller Solves the resource limitations •Runs many Vusers on a few machines •Controller manages the virtual users •Analyze results with graphs and reports Solves the resource limitations •Runs many Vusers on a few machines •Controller manages the virtual users •Analyze results with graphs and reports
  32. 32. ALL RIGHTS RESERVED 2013 Continued… Using Automated Tools  Reduces personnel requirements by replacing human users with virtual users or Vusers. These Vusers emulate the behaviour of real users.  Because numerous Vusers can run on a single computer, the tool reduces the amount of hardware required for testing.  Monitors the application performance online, enabling you to fine-tune your system during test execution.  It automatically records the performance of the application during a test. You can choose from a wide variety of graphs and reports to view the performance data.  Because the tests are fully automated, you can easily repeat them as often as you need.
  33. 33. ALL RIGHTS RESERVED 2013 Some common Tools used for Performance Testing Open Source • OpenSTA • Diesel Test • TestMaker • Grinder • LoadSim • Jmeter • Rubis Commercial • LoadRunner • Silk Performer • Qengine • Empirix e-Load
  34. 34. ALL RIGHTS RESERVED 2013