• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Why Average Response Time is not a right measure of your web application's performance  Nishant Verma
 

Why Average Response Time is not a right measure of your web application's performance Nishant Verma

on

  • 1,955 views

Presentation from vodQA Bangalore Get Set Test

Presentation from vodQA Bangalore Get Set Test

Statistics

Views

Total Views
1,955
Views on SlideShare
1,955
Embed Views
0

Actions

Likes
0
Downloads
13
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Hi I am Nishant Verma and today I am here to present a lightening talk for you folks for the next 15 minutes. So let’s start: (Slide 1)Not sure how many of you have read this quote before. But I would expect you folks to relate to this quote at the end of the 15 minutes.[ENTER]How many of you have previously worked on Performance Testing? (Show of hand would be fine!) How many of you are aware of Average Response Time? (Show of hands!)All right Cool!
  • So at the high level agenda looks like: What it is? How we do it? What we collect?What is it? And What’s wrong with Average Response Time?Solution - What is it? How to use it? - - - - - -  And then in the end we will try to take some questions if time permits.
  • (Slide 3)So what is Performance Testing? [Click]It’s a type of testing with focus on non-functional aspects of the application. It’s done and owned by a tester. [Click] What we typically measure is [Click] Responsiveness of the application [Click] Scalability [Click] Throughput Why we do it? [Click] To establish a baseline of application performance [Click] To determine compliance.Response time, or Responsiveness, is simply how quickly an interactive system responds to user input.Throughput is the amount of transactions produced over time during a test. It’s also expressed as the amount of capacity that a website or application can handle.
  • How to do:So we start with designing the test scenarios which could be a business case .Then we establish the acceptance criteria (basically defining exit criteria)Identifying a test environment, sometimes we even create a separate Performance Environment with scale down version of actual production environment.Put a tested build preferably a release candidate build. Execution- How we do it ? Typically what we do is we make a load profile which could be just bombarding the app server with What we collect: Responsiveness:Response time, or Responsiveness, is simply how quickly an interactive system responds to user input. Basically, “Throughput” is the amount of transactions produced over time during a test. It’s also expressed as the amount of capacity that a website or application can handle.
  • 1. Response Time is defined as the time it takes for each Web page to load.2. There are bunch of tools in market like LR which suggests to go by 90% percentile.3. What we generally do is hit the server with a specified number of user (could be in pattern) and then capture the response time of the server.
  • Let’s say an E-commerce application like Flipkart
  • We need a performance Indicator Help take the go-no-go decision. Without looking at all possible URLs. Without parsing the request log to find any outliers.Indicator of the number of users affected by the performance problem.Indicator of over all health of the website, without missing outliers.
  • Let’s re look at the load scenario once again. And this time comparing it with Apdex Index.
  • This is just a tip of iceberg.Average Response Time or Apdex Index is just the tip of the so called Performance Testing ice berg….Load Testing – To verify application behavior under normal and peak load conditions.Stress Testing-To determine or validate an application’s behavior when it is pushed beyond normal or peak load conditions.Soak Testing provides a measure of a system's stability over an extended period of time.

Why Average Response Time is not a right measure of your web application's performance  Nishant Verma Why Average Response Time is not a right measure of your web application's performance Nishant Verma Presentation Transcript

  • +Why “Average” Response Time is not a right measure of your webapplications performance?By- Nishant Verma
  • +For the next 15 minutesn  Web Application Performance Testingn  Average Response Timen  Solution to Average Response Timen  Beyond Average Response Timen  Questions
  • +Web Application Performance Testing
  • +Web Application Performance TestingHow to doDesign Test/ScenariosEstablishPerformanceAcceptanceCriteriaIdentifyEnvironmentExecute TestAnalyzeReports / Tune/RetestsWhat wecollectResponse TimeThroughputCPU UtilizationMemory
  • +Average Response Timen  Response Time is defined as the time it takes for each Web pageto load.n  How we capture?n  Hit the server with a specified number of user (could be in pattern)and then capture the response time of the server.n  Refer the plot of ART on next page.
  • +Average Response Time0100200300400500600Page 1 (in ms)Page 2 (in ms)Page 3 (in ms)
  • +Problems with Average ResponseTimen  It’s a bad way of tracking website performance.n  It doesn’t indicate or hint about the outliers.n  We tend to average the number of samples than the quantityitself.n  It represents a distorted picture in case of exceptionalscenario with extremely high response time.
  • What we need?
  • + Apdexn  Created by Peter Sevcik of Apdex Alliance (apdex.org).n  How is it calculated?n  ApdexT = (# Satisfied requests + # Tolerating requests / 2)(# Total requests)APDEX: Application Performance Index
  • +How it scores over Average ResponseTimen  It help takes us Go-No-Go decision.n  It’s doesn’t hide outliers. It yells the number of users affected by theproblem.n  It doesn’t get distorted when there are extremely high responsetimes.n  It’s a measure towards Satisfaction score and can be derived basedon threshold. It’s not a fix number.
  • + One last thing…Load TestingStressTestingSoak TestingPerformanceTuning
  • +Some Useful Readingsn  http://apdex.org/n  http://en.wikipedia.org/wiki/Apdexn  http://www.apdex.org/overview.htmln  https://newrelic.com/docs/site/apdexn  https://newrelic.com/docs/site/apdex-dissatisfaction