0
+
People don’t know what they want until you show
it to them.
Why “Average” Response Time is not a right
measure of your w...
+For the next 15 minutes
n  Web Application Performance Testing
n  Average Response Time
n  Solution to Average Respons...
+Web Application Performance Testing
+
Web Application Performance Testing
How to do
Design Test/
Scenarios
Establish
Performance
Acceptance
Criteria
Identify
...
+Average Response Time
n  Response Time is defined as the time it takes for each Web page
to load.
n  How we capture?
n...
+Average Response Time
0
100
200
300
400
500
600
Page 1 (in ms)
Page 2 (in ms)
Page 3 (in ms)
+Let’s take a real life scenario…
n  Consider a typical web application
n  Load Scenario (1):
n  60% of requests | 300m...
+Fallacy - #1
What’s wrong with 2.182 sec?
n  Bad Indicator of user “satisfaction” given that 20% of your
users are unhap...
+Fallacy - #2
Increasing complexity of website Huh !
n  Websites are little complicated these days. Different pages have
...
+Fallacy - #3
Averages are meant more for averaging the total quantity
being measured, not the number of samples !
n  Loa...
What we need?
+ Apdex
n  Created by Peter Sevcik of Apdex Alliance (apdex.org).
n  How is it calculated?
n  ApdexT = (# Satisfied req...
+
How it scores over Average Response
Time
n  Load Scenario(1). 60% of requests take 300ms, 20% take 10ms, and
20 percent...
+ One last thing…
Load Testing
Stress
Testing
Soak Testing
Performance
Tuning
+
Upcoming SlideShare
Loading in...5
×

Why average response time is not a right measure of your webapplication's performance nishant_verma

281

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
281
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Why average response time is not a right measure of your webapplication's performance nishant_verma"

  1. 1. + People don’t know what they want until you show it to them. Why “Average” Response Time is not a right measure of your web application's performance?
  2. 2. +For the next 15 minutes n  Web Application Performance Testing n  Average Response Time n  Solution to Average Response Time n  Beyond Average Response Time n  Questions
  3. 3. +Web Application Performance Testing
  4. 4. + Web Application Performance Testing How to do Design Test/ Scenarios Establish Performance Acceptance Criteria Identify Environment Execute Test Analyze Reports / Tune/ Retests What we collect Response Time Throughput CPU Utilization Memory
  5. 5. +Average Response Time n  Response Time is defined as the time it takes for each Web page to load. n  How we capture? n  Hit the server with a specified number of user (could be in pattern) and then capture the response time of the server. n  Refer the plot of ART on next page.
  6. 6. +Average Response Time 0 100 200 300 400 500 600 Page 1 (in ms) Page 2 (in ms) Page 3 (in ms)
  7. 7. +Let’s take a real life scenario… n  Consider a typical web application n  Load Scenario (1): n  60% of requests | 300ms n  20% of requests |10ms n  went through real quick. n  mostly served from cache. n  20% of requests | 10 sec n  Requests stuck up due to external api’s or n  let’s say some DB locks. n  Average Response Time: 2.182 sec
  8. 8. +Fallacy - #1 What’s wrong with 2.182 sec? n  Bad Indicator of user “satisfaction” given that 20% of your users are unhappy with your site. n  80% of users who are facing fast response time can’t compensate the others who are leaving the website. n  20% of users are very critical depending on which part of application they are on. n  E.g. Sign Up, Log In, Check out Learning: It hides important outliers.
  9. 9. +Fallacy - #2 Increasing complexity of website Huh ! n  Websites are little complicated these days. Different pages have different normal response time. n  Login thru LinkedIn API taking 5 sec is ok. n  A kernel cached image taking 5 sec to serve is not ok. n  Averaging these two together wouldn’t really let me know anything. n  It’s unfair to compare the Average Response Time of a page over time. n  Every time the component of the traffic changes, it will have impact on ART. Learning:Your Average Response Time is not a fixed number.
  10. 10. +Fallacy - #3 Averages are meant more for averaging the total quantity being measured, not the number of samples ! n  Load scenario (2): n  10 request | ~300ms n  1 request | 5 minute n  hit triggered a cache miss, which got stuck in some almost never hit DB query. n  Average Response Time spiked to 27.3 sec ! n  First impression “Something is seriously wrong!” Learning:Your average response time never tells you the number of users affected by the problem.
  11. 11. What we need?
  12. 12. + Apdex n  Created by Peter Sevcik of Apdex Alliance (apdex.org). n  How is it calculated? n  ApdexT = (# Satisfied requests + # Tolerating requests / 2) (# Total requests) APDEX: Application Performance Index
  13. 13. + How it scores over Average Response Time n  Load Scenario(1). 60% of requests take 300ms, 20% take 10ms, and 20 percent are getting unacceptable latencies at 10 seconds. n  Average Response Time = 2.182 sec; Wrong Confidence n  Apdex2s is 80%; Pin points the problem n  Load Scenario(2). 10 requests takes 300ms and 1 request takes 5 minute. n  Average Response Time = 27.3 seconds; False Alarm n  Apdex2s is 90.9 % ; Good health ü  It does a good job. ü  Indicates correct health of application. ü  Highlights even when the small percentage of user face performance issue (so called outliers).
  14. 14. + One last thing… Load Testing Stress Testing Soak Testing Performance Tuning
  15. 15. +
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×