Your SlideShare is downloading. ×
  • Like
  • Save
Web Load Testing for Dummies
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Web Load Testing for Dummies


Web applications that perform well can strengthen a company's brand, reputation, and create customer loyalty. Web applications that perform poorly put all of that at risk. Web load testing is a …

Web applications that perform well can strengthen a company's brand, reputation, and create customer loyalty. Web applications that perform poorly put all of that at risk. Web load testing is a critical component to any risk management plan for web applications.

Join Scott Barber, President of PerfTestPlus and Colin Mason, Product Manager, Compuware to learn the whys and wherefores of Web Load Testing as they present Web Load Testing for Dummies.

You will learn:

* The ins and outs of web load testing — know what to expect from web load testing
* The importance of outside-in load testing — determine what the performance feels like to an actual user
* Why and when to test — set goals, gather your team, and implement
* How to manage ongoing analysis— monitor how your testing is going
* How diagnostics tools combined with web load testing dramatically reduces time to problem resolution

Scott Barber, Chief Technologist, President and CEO of PerfTestPlus is a writer, consultant, and speaker. He is a prominent thought-leader in the area of software system performance testing and served as the Executive Director of the Association for Software Testing, and is the co-founder of the Workshop on Performance and Reliability.
He has contributed to several books including the upcoming How to Reduce the Cost of Testing, Taylor & Francis.

Colin Mason, Product Manager for Gomez Web Load Testing, at Compuware, has over a decade of performance testing experience and has conducted over hundreds of load tests for companies world wide. He has presented at numerous industry conferences, including STAR, and has authored industry-acclaimed papers on performance testing web applications.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Scott BarberColin Mason
  • 2. Agenda• What is Web Load Testing & Why does it matter?• Let’s get started- How to approach• Nuts and Bolts- Executing tests• Analyzing and Reporting Results• Gomez Approach
  • 3. What’s Web Load Testing? Web Load Testing is How You Avoid This “It’s an unforgivable mistake that cost them thousands of dollars,” she said. “After all the money spent with the hype of the collection, no one thought it would translate to web traffic?”
  • 4. The Stakes of Positive User Experience • Internet population has increased from 360 Million in 2000 to 1.8 billion in 2009. • Around Twenty Five per cent of the world’s population now use the Internet. North America (74%) has the highest penetration.
  • 5. Many merchants suffer site slowdowns & problems under load US holiday shopping season starts around Thanksgiving for Online RetailersFrom completely offline,to intermittent "servicedisruptions“, scheduledmaintenance messages orslowdowns
  • 6. Online Gaming –So Successful in Driving Traffic it Crashed
  • 7. Why Web Performance Matters Average impact of 1 second delay in response time for Web users Customer Page Views Conversions Satisfaction 0% -2% -4% -6% -8% -7% -10% -12% -11% -14% • Lost revenues* -16% • Brand damage • More support calls -16% -18% • Increase costs • LOB dissatisfaction with IT *Online business doing $100K/day = $2.5M/year in lost revenue
  • 8. The Browser Is Becoming THE Integration PlatformNumber of hosts accessed directly by the browser,per user transaction, averaged across 3,000 companies 8+
  • 9. The Challenge Of Delivering Quality Web Experiences Systems …user ismanagement NOT happy tools: “OK” The Web Application Delivery Chain • Inconsistent geo performance Cloud Customers • Poorly • Bad performance under load Private Public • Network peering performing Browsers • Blocking content delivery problems Local JavaScript Data Center • Bandwidth ISP • Inconsistent • Incorrect geo-targeted content 3rd Party/ Virtual/Physical Environment throttling Cloud Services CSS rendering • Inconsistent • Browser/device DB• Configuration Web App Load Network peering • connectivity incompatibilityMainframe Servers Servers Servers errors Balancers problems Major • Page size too • Application ISP big • Outages design issues • Network resource • Conflicting • Code defects shortage HTML tag Storage • Insufficient • Faulty content support Network infrastructure transcoding Content • Too many •Delivery SMS routing / objects latency issues Networks • Content not Web • Configuration issues Mobile WAN optimized for Services • Oversubscribed POP Components Optimization Mobile device • Poor routing optimization Carrier • Low cache hit • Low cache hit rate Devices rate Employees Employees
  • 10. Testing from Different PerspectivesInside out- – Assess code/componentsMiddle in- – Assess Components but not user experienceOutside in-Assess – Web application entirely including user experience
  • 11. Evolution of Load Testing Load Test 2.0 (Outside In) Load Test 1.5 ( Middle in) Load Test 1.0 ( Inside Out) Cloud Customers Private Public Browsers Local Data Center ISP 3rd Party/ Virtual/Physical Environment Cloud Services DB App Web LoadMainframe Servers Servers Servers Balancers Major ISP Storage Network Content Delivery Networks Web Mobile WAN Services Components Optimization Mobile Carrier Devices Employees Employees
  • 12. Web Load Testing is the only way to Assess whether actual users will be frustrated by performance without actually frustrating actual users.
  • 13. Let’s get started• Gather the Team – Determining goals and establishing measures and metrics are tasks for which more minds are better than fewer – especially when gathering initial information• Establish goals – Start with questions – Convert to goals• Measurements and Metrics – Determine what data you need to collect, and how you will process that data to determine if goals have been achieved
  • 14. Gathering Your TeamEveryone’s perspective and concerns matter, so gather thewhole team including:• All the developers• All the testers• All the analysts• All the managers (project, development, test, and so on)• The project’s executive sponsor• At least one representative from each of – Product management – Marketing – IT/Support Operations
  • 15. Establishing Goals• Solicit Multiple Goals- Business, Technical, etc• Get the Right Questions from Everyone• Establish Goals that Address those Questions
  • 16. Measurements and Metrics • For each goal, determine what information will answer: – Has this goal been achieved? – To what degree? – What needs to be done to achieve this goal? • Decide what data must be collected to provide that information • Figure out how to collect that data
  • 17. Nuts and Bolts• When to Test – Early and often is best, but let’s be reasonable. ☺• How Much Load – Until it breaks is fun, but what does that mean?!?• Which Transactions – You could do them all… but probably not in your lifetime.• Diagnostic tools – A great idea, but how can I convince the others?• Selecting a load testing tool – Oh, they’re all the same. Aren’t they?
  • 18. When to Test (and Retest)Launching New Apps/Features Deploying New InfrastructureMajor Marketing Campaigns Penetrating New Markets
  • 19. How Much Load?• More load can provide more data, but that data doesn’t mean much unless you have a target load.• Establishing a Target Load – Historic data – Competitive analysis – Sales projections & fancy math• During your research, look for – Peak number of user sessions – Peak number of page views – Average length of time an individual session – The average number of pages the typical user views• Plan to test above and below the target… just to make sure
  • 20. Which Transactions to Test Common activities (get from logs) e.g. Resource hogs (get from developers/admins) Even if these activities are both rare and not risky SLA’s, Contracts and other stuff that will get you sued What the users will see and are mostly likely to complain about. What is likely to earn you bad press New technologies, old technologies, places where it’s failed before, previously under-tested areas Don’t argue with the boss (too much)
  • 21. Which Transactions to Test
  • 22. Diagnostic toolsDiagnostic tools are a great addition to a web load tester’stoolbox:• Easier Data Collection• Speed up data analysis• Simplifies root cause analysis• Shortens the test->tune->re-test cycle
  • 23. Selecting a Tool (or Tools)Select the right tool for the Job!
  • 24. Details to watch for before testing• Test environment should Mirror Production• Validate your test data• Establish a test data restore point before you begin testing – for repeat tests• Simulate ramp-up and cool-down periods appropriately.
  • 25. Running the Tests• Use the system manually during test execution so you can compare your observations with the results data at a later time.• Maintain a test execution log that captures notes and observations for each run.• Run test tasks in one- to two-day batches. – See the tasks through to completion – Be willing to take important detours• Analyze results immediately so that you can modify your test plan accordingly.• Communicate test results frequently and openly across the team.
  • 26. Analyzing and Reporting• Analyzing Data – Data analysis is a team sport• Looking for Problems – You will encounter many types of problems, a classification scheme helps• Dealing with issues – Ultimately, you want to make your users like using your site• Reporting Sensibly – Stakeholders want answers, not data and jargon
  • 27. Analyzing Data• Gather the Team Again. Additionally include: – At least one non-technical manager or executive – One or more person(s) with the following skills: • Spreadsheet • Statistics • Data visualization• Go back to your goals and metrics – If you can’t assess goal achievement, you are either: • Not done yet • Not collecting the correct data
  • 28. Looking for Problems• Here are the things you are looking for:• Bottlenecks: Systemic issues that cause user slowdown• Slow spots: Isolated pages, objects, or activities that take a long time but don’t impact other users’ activities• Limits: Points at which performance changes noticeably• Defects/errors: Things that are broken or don’t work
  • 29. Gomez Solution
  • 30. Industry First 360o Web Load Testing Solution • Find and fix performance & scalability problems from the First Mile to the Last Mile • Accelerate time to market by identifying user experience problems and isolating the root cause down to the line of code • Reduce testing iterations and cycle time with proactive problem resolution and prevention • Validate architecture for maximum scalability Business Development “Competitive “Agility!” advantage!” Gomez 360o Common metrics Common tools,Web Load Testing Application data, Performance diagnostics
  • 31. Why 360o Load Testing Matters • Hours/days to solve problems • Weeks/months to360o Load Testing solve problem • Session is recorded, eliminating need to • Multiple test reproduce errors cycles • Drill down to • Poor offending line of communication code between QA and developers • Eliminates testing iterations
  • 32. Reducing Load Test Time Developing Test Run Reproduction Refine Capturing Multiple test iterations Re Run Tests Reproduction Refine Capturing needed to analyze root-cause and fix the problem Re Run Tests Reproduction Problem Analysis Problem Solving time Developing Test Run Reproduction Refine Capturing • Eliminates test Iterations Re Run Tests Reproduction Refine Capturing • Go directly to problem analysis • Frees up resources Re Run Tests Reproduction Problem Analysis for other projects Problem Solving time
  • 33. Questions?