Why average response time is not a right measure of your webapplication's performance  nishant_verma
Upcoming SlideShare
Loading in...5

Like this? Share it with your network

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. + People don’t know what they want until you show it to them. Why “Average” Response Time is not a right measure of your web application's performance?
  • 2. +For the next 15 minutes n  Web Application Performance Testing n  Average Response Time n  Solution to Average Response Time n  Beyond Average Response Time n  Questions
  • 3. +Web Application Performance Testing
  • 4. + Web Application Performance Testing How to do Design Test/ Scenarios Establish Performance Acceptance Criteria Identify Environment Execute Test Analyze Reports / Tune/ Retests What we collect Response Time Throughput CPU Utilization Memory
  • 5. +Average Response Time n  Response Time is defined as the time it takes for each Web page to load. n  How we capture? n  Hit the server with a specified number of user (could be in pattern) and then capture the response time of the server. n  Refer the plot of ART on next page.
  • 6. +Average Response Time 0 100 200 300 400 500 600 Page 1 (in ms) Page 2 (in ms) Page 3 (in ms)
  • 7. +Let’s take a real life scenario… n  Consider a typical web application n  Load Scenario (1): n  60% of requests | 300ms n  20% of requests |10ms n  went through real quick. n  mostly served from cache. n  20% of requests | 10 sec n  Requests stuck up due to external api’s or n  let’s say some DB locks. n  Average Response Time: 2.182 sec
  • 8. +Fallacy - #1 What’s wrong with 2.182 sec? n  Bad Indicator of user “satisfaction” given that 20% of your users are unhappy with your site. n  80% of users who are facing fast response time can’t compensate the others who are leaving the website. n  20% of users are very critical depending on which part of application they are on. n  E.g. Sign Up, Log In, Check out Learning: It hides important outliers.
  • 9. +Fallacy - #2 Increasing complexity of website Huh ! n  Websites are little complicated these days. Different pages have different normal response time. n  Login thru LinkedIn API taking 5 sec is ok. n  A kernel cached image taking 5 sec to serve is not ok. n  Averaging these two together wouldn’t really let me know anything. n  It’s unfair to compare the Average Response Time of a page over time. n  Every time the component of the traffic changes, it will have impact on ART. Learning:Your Average Response Time is not a fixed number.
  • 10. +Fallacy - #3 Averages are meant more for averaging the total quantity being measured, not the number of samples ! n  Load scenario (2): n  10 request | ~300ms n  1 request | 5 minute n  hit triggered a cache miss, which got stuck in some almost never hit DB query. n  Average Response Time spiked to 27.3 sec ! n  First impression “Something is seriously wrong!” Learning:Your average response time never tells you the number of users affected by the problem.
  • 11. What we need?
  • 12. + Apdex n  Created by Peter Sevcik of Apdex Alliance (apdex.org). n  How is it calculated? n  ApdexT = (# Satisfied requests + # Tolerating requests / 2) (# Total requests) APDEX: Application Performance Index
  • 13. + How it scores over Average Response Time n  Load Scenario(1). 60% of requests take 300ms, 20% take 10ms, and 20 percent are getting unacceptable latencies at 10 seconds. n  Average Response Time = 2.182 sec; Wrong Confidence n  Apdex2s is 80%; Pin points the problem n  Load Scenario(2). 10 requests takes 300ms and 1 request takes 5 minute. n  Average Response Time = 27.3 seconds; False Alarm n  Apdex2s is 90.9 % ; Good health ü  It does a good job. ü  Indicates correct health of application. ü  Highlights even when the small percentage of user face performance issue (so called outliers).
  • 14. + One last thing… Load Testing Stress Testing Soak Testing Performance Tuning
  • 15. +