Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Perform Realistic Web
Performance Testing To
Achieve Peak User
Experiences
Mike Gualtieri
Senior Analyst
           y
Forr...
Realistic web
       ea st c eb
performance testing is
 c t ca
 critical to ac e g
             achieving
     flawless us...
Agenda

     • Why your customers won’t wait
     • Wh t is realistic performance testing
       What i     li ti     f   ...
Agenda

     • Why your customers won’t wait
     • Wh t is realistic performance testing
       What i     li ti     f   ...
Performance means:
                                                                           Gomez Media Home
     • Raw ...
Performance means:
                                                                           Gomez Media Article
     • C...
Realistic Web Performance Testing Is A
    Forrester Best P ti
    F     t B t Practice
        • Best Practices: Attainin...
Blazing Fast
Web Sites = Customers
Traditional web performance testing
     doesn’t cut it
     d      ’t t
         •          Traditional testing is expens...
Most Developers Are Building Web Apps.
                    “What types of new custom applications are your firm’s develope...
New trends make optimal user
     experiences even more important
          i                i    tt
         •          T...
Agenda

      • Why your customers won’t wait?
      • Wh t i realistic performance testing
        What is   li ti     f ...
“Realistic testing” means testing from your
     customer s
     customer’s point of view
         • Where they do it
    ...
The user’s distance from the content
     matters
       tt




16   Entire contents © 2008 Forrester Research, Inc. All r...
Browsers render differently




         Source: Actual Experience XF Real-User Monitoring Data; Gomez, Inc.


17   Entire...
What features should a realistic web
     performance testing platform have? It should:
       • Test from the end user’s ...
Agenda

      • What do customers want?
      • Wh t i realistic performance t ti
        What is   li ti     f        tes...
Recommendations (1 of 2)
       • Design a test plan based on how real users interact
         with your web application
 ...
Recommendations (2 of 2)
       • Testing early to nip problems in the bud
       • Test often to maintain a consistently ...
In this fiercely competitive
environment, realistic web
performance testing is the
        right answer.
Thank you

      Mike Gualtieri
      +1 617 613 6145
       1 617.613.6145
      mgualtieri@forrester.com
      www.forre...
Gomez Profile


                   ▪ First SaaS Web Application
▪ Headquarters       Experience Management Suite
         ...
Today’s Testing Challenges

Never Enough Time                                  Small QA Teams




                        ...
When To Load Test

Launching New Apps/Features    Deploying New Infrastructure




                                       ...
Web Application Delivery
    In A Web 2.0 World


       ▪ Web applications
               pp
         increasingly
      ...
Load 2.0 Testing Solutions

Overcome Limitations of 1.0 Testing Approaches

▪ Find and fix problems across the
  entire we...
Ensure Web Experiences Scale
                               Under Load With Reality Load XF



▪ Deploy web applications a...
Critical To Test Across Entire
      Web Application Delivery Chain




Key Load Testing Metrics
 • Success rate of end-us...
Load Testing 2.0
                Performance Testing For A Web 2.0 World



                             ▪ Self-service te...
Critical To Test Across Entire
    Web Application Delivery Chain
                      3
       2


1
Ensuring Performance Of
                          Login Process




1   Company
     • Online presence for a p p
         ...
Application Bottleneck Causes
Immediate Response Time Issue



           • As users were added the
                      ...
Ensuring Performance Of
              E-Commerce Transaction

         2




Company
 • Online Retailer
Situation
 • Typic...
Response Times Rise Due To
                                 Third-Party Object Error

                                    ...
Ensuring Performance In
                                              Target Markets
                                     ...
No Performance Issues Detected
                                  From Data-Center

Increase and hold load and not exceed
r...
Performance Issues Detected
                            From Real User Desktops

Key geographies for this customer are
   ...
Performance Issues Detected For
                             Target Market End Users

                                 • L...
Gomez Offers Full Lifecycle Solutions
                            For Web Experience Management




“Gomez has made specta...
Find Out How Gomez Can Help You

Ensuring great experiences in a Web 2.0                  Some of Our Customers
world requ...
Achieving Peak User Experiences & Optimizing Web Performance - Load Testing, Stress Testing & Performance Testing in an Ag...
Achieving Peak User Experiences & Optimizing Web Performance - Load Testing, Stress Testing & Performance Testing in an Ag...
Upcoming SlideShare
Loading in …5
×

Achieving Peak User Experiences & Optimizing Web Performance - Load Testing, Stress Testing & Performance Testing in an Agile Web 2.0 World

2,135 views

Published on

Ensuring great experiences for today’s rich and complex web applications requires a new approach to performance testing. Application owners, e-business and marketing managers, QA & IT operations all need to ensure mission & business critical transactions perform flawlessly when:
* Releasing new web applications and features
* Upgrading or virtualizing IT infrastructure
* Adding third-party or cloud-based services such as CDNs, shopping carts and news feeds
* Penetrating new markets

Join us as Forrester’s Senior Analyst, Mike Gualtieri, and Gomez’s CTO, Imad Mouline, discuss:
* Best practices for launching new applications and infrastructure with confidence
* Finding and fixing problems prior to launch across the entire web application delivery chain
* Maximizing the effectiveness of major marketing campaigns
* Why the new “load testing 2.0” approach finds problems legacy tools couldn’t detect, and makes solutions accessible to a broader range of departments and organizations due to their ease of use and on-demand delivery

Published in: Technology
  • Be the first to comment

Achieving Peak User Experiences & Optimizing Web Performance - Load Testing, Stress Testing & Performance Testing in an Agile Web 2.0 World

  1. 1. Perform Realistic Web Performance Testing To Achieve Peak User Experiences Mike Gualtieri Senior Analyst y Forrester Research May 13, 2009
  2. 2. Realistic web ea st c eb performance testing is c t ca critical to ac e g achieving flawless user experiences. p
  3. 3. Agenda • Why your customers won’t wait • Wh t is realistic performance testing What i li ti f t ti • Recommendations 4 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  4. 4. Agenda • Why your customers won’t wait • Wh t is realistic performance testing What i li ti f t ti • Recommendations 5 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  5. 5. Performance means: Gomez Media Home • Raw page load speed Page Benchmark April 10 ’09 – May 10 ’09 0:00 - 24:00 EST 6 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  6. 6. Performance means: Gomez Media Article • Customer workflow speed Search Transaction Benchmark 1 April 10 ’09 – May 10 ’09 0:00 - 24:00 EST 2 3 7 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  7. 7. Realistic Web Performance Testing Is A Forrester Best P ti F t B t Practice • Best Practices: Attaining And Maintaining Blazing Fast Web Sites • Best Practices: Blazing Fast Web Site Infrastructure • Case Study: How Orbitz.com Takes You Places Faster • Case Study: How Msnbc.com Is First In News 8 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  8. 8. Blazing Fast Web Sites = Customers
  9. 9. Traditional web performance testing doesn’t cut it d ’t t • Traditional testing is expensive • Cost of licenses, hardware and training • Time consuming to implement • Idiosyncrasies are often difficult to master • Traditional testing is not realistic • Scope is limited to internal clean-room testing • Not focused on end-user point of view p • Misses too many pieces in the web application delivery chain such as CDN, 3rd-party components, geography, etc… 11 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  10. 10. Most Developers Are Building Web Apps. “What types of new custom applications are your firm’s developers building” (multiple responses accepted) 2008 Base: 977 platform software decision-makers at North American and European enterprises and SMBs 12 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  11. 11. New trends make optimal user experiences even more important i i tt • The Millennial generation expects more – 18 to 27 year olds – The tech-savvy generation • Web site redesigns – RIA More Web sites are composed of 3rd-party • components • Virtualization f infrastructure including cloud Vi t li ti of i f t t i l di ld • Mobile 13 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  12. 12. Agenda • Why your customers won’t wait? • Wh t i realistic performance testing What is li ti f t ti • Recommendations 14 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  13. 13. “Realistic testing” means testing from your customer s customer’s point of view • Where they do it – LA, Edmonton, London, Bangkok, Des Moines, Miami, Cannes, Dubrovnik, Osaka, Pusan • Wh they d What h do – Key tasks, transactions, and business processes • How they do it – Browsers galore, mobile devices • When they do it (and how frequently) – Peak times, spikes p 15 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  14. 14. The user’s distance from the content matters tt 16 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  15. 15. Browsers render differently Source: Actual Experience XF Real-User Monitoring Data; Gomez, Inc. 17 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  16. 16. What features should a realistic web performance testing platform have? It should: • Test from the end user’s point of view – Different locations – Different browsers – Different loads – Different tasks • Be easy for developers, testers and business people to use. • Identify and resolve problems along the entire web application delivery chain. 18 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  17. 17. Agenda • What do customers want? • Wh t i realistic performance t ti What is li ti f testing • Recommendations 19 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  18. 18. Recommendations (1 of 2) • Design a test plan based on how real users interact with your web application • Use testing tools that allow you to test from your user’s point of view: – Wh Where, Wh t How, Wh What, H When • Test the entire Web application • Democratize testing: – Build a test team that includes anyone who has a stake in the outcome 20 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  19. 19. Recommendations (2 of 2) • Testing early to nip problems in the bud • Test often to maintain a consistently supreme user experience • Follow Blazing Fast Web Performance Best Practices to keep your Web site humming 21 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  20. 20. In this fiercely competitive environment, realistic web performance testing is the right answer.
  21. 21. Thank you Mike Gualtieri +1 617 613 6145 1 617.613.6145 mgualtieri@forrester.com www.forrester.com 23 Entire contents © 2008 Forrester Research, Inc. All rights reserved.
  22. 22. Gomez Profile ▪ First SaaS Web Application ▪ Headquarters Experience Management Suite pe e ce a age e t Su te ▪ Lexington, MA ▪ Web Performance and Load Testing ▪ Global Offices ▪ Cross-Browser Testing Cross Browser ▪ UK ▪ Web Performance Monitoring ▪ Germany Ge a y ▪ 80 000+ internet experience 80,000+ measurement points in 162 ▪ Switzerland countries ▪ China ▪ Over 2,500 Customers Worldwide ▪ 12 out of 20 most visited US Websites utilize G tili Gomez ▪ 300 Employees 24
  23. 23. Today’s Testing Challenges Never Enough Time Small QA Teams 25 Catching All the Problems Testing the End-User Perspective
  24. 24. When To Load Test Launching New Apps/Features Deploying New Infrastructure 26 Major Marketing Campaigns Penetrating New Markets
  25. 25. Web Application Delivery In A Web 2.0 World ▪ Web applications pp increasingly assembled at browser level ▪ 3rd party performance from ads, videos, shopping cart vary widely under load ▪ Content Delivery Network configuration and effectiveness impact end-user experience d i 27
  26. 26. Load 2.0 Testing Solutions Overcome Limitations of 1.0 Testing Approaches ▪ Find and fix problems across the entire web application delivery chain ▪ Inside and outside the firewall ▪ Flexible and easy to use ▪ On-demand – no installation required ▪ Highly scalable to accommodate peak loads ▪ Supports Agile development ▪ Test from the end-user perspective 28
  27. 27. Ensure Web Experiences Scale Under Load With Reality Load XF ▪ Deploy web applications and infrastructure with confidence ▪ Combine high volume web application load testing with broad geographic experience testing ▪ Ensure on-demand testing g flexibility ▪ Detect failures in user experience not caught by traditional load testing ▪ Testing across the entire web application d li delivery chain li ti hi 29
  28. 28. Critical To Test Across Entire Web Application Delivery Chain Key Load Testing Metrics • Success rate of end-user workflow • Response time of end-user workflow end user • Quality of end-user workflow delivery 30
  29. 29. Load Testing 2.0 Performance Testing For A Web 2.0 World ▪ Self-service testing gives you control to test as needed ▪ Test internal & external web application components to ensure transactions perform under load ▪ Test cloud-based applications & services ▪ SaaS based Load Testing 2.0 solutions enable on-demand, scalable testing without g Combine high volume C bi hi h l associated hardware investment application load testing & maintenance costs with broad geographic ▪ Test the full web application pp experience testing delivery chain all the way to your end users’ desktops 31
  30. 30. Critical To Test Across Entire Web Application Delivery Chain 3 2 1
  31. 31. Ensuring Performance Of Login Process 1 Company • Online presence for a p p p popular television show Situation • Following episodes of the television show the web site sees high traffic spikes • Goal was to achieve 1500 logins per minute • Load tested database optimizations aimed to improve performance in anticipation of another large traffic spike 33
  32. 32. Application Bottleneck Causes Immediate Response Time Issue • As users were added the added, response time of step 3 (the login) climbed immediately • The test bottlenecked at 160 logins per minute (Goal 1500) • But quickly dropped off as users received server errors • New login query was not optimized and was bottlenecking the database servers’ CPU ’ CPUs 34
  33. 33. Ensuring Performance Of E-Commerce Transaction 2 Company • Online Retailer Situation • Typical e-Commerce type application •SSeveral 3 d P ti now i l 3rd Parties involved i l d in serving up key content • Goal was to validate performance of entire application
  34. 34. Response Times Rise Due To Third-Party Object Error The transaction rate The load increases increases and then throughout the test falls off as response times climb Errors are seen, all on a 3rd party object • 3 d party h d 3rd t hardware was i insufficient f overall d ffi i t for ll demands on application d li ti • Based on SLAs 3rd party had to improve performance to get paid
  35. 35. Ensuring Performance In Target Markets 3 Company • Regional Online News Source Situation • Began testing for the 2008 election season • Goal was to validate performance of overall application, and specifically for users in 2 key geographies • Testing started with the homepage
  36. 36. No Performance Issues Detected From Data-Center Increase and hold load and not exceed response times of 4 seconds and Success There was only Rate of 99% 1 page error and 11 errors total out of 60000+ Page response transactions times stayed under 4 seconds, outside of one brief blip p By traditional test standards the test passed
  37. 37. Performance Issues Detected From Real User Desktops Key geographies for this customer are New York and Pennsylvania Last Mile data showing substantial number of measurements greater than 4 seconds
  38. 38. Performance Issues Detected For Target Market End Users • Last Mile Users exceeding the 4- second goal – 29% • Overall availability was 96.3% • Availability for Last Mile Users exceeding 4 seconds – 87% • Last Mile Users reporting 10+ g second response times – 8% By these standards the test f il d B th t d d th t t failed • Cutting 1 5 to 2 sec off the response time for the page would move 1.5 sec. another 16% of the Last Mile users into the 4 sec. response time range
  39. 39. Gomez Offers Full Lifecycle Solutions For Web Experience Management “Gomez has made spectacular p g p progress in the last 18 months…to become the leader in web experience management.” Jean-Pierre Garbani, Vice President, Principal Analyst, Forrester Research 41 Tech Horizons: Evaluating Gomez’s Web Experience Management Services, August 2008
  40. 40. Find Out How Gomez Can Help You Ensuring great experiences in a Web 2.0 Some of Our Customers world requires a new approach to web application load testing - Reality Load XF Reality Load XF • Self-service or full turnkey solution with Gomez Professional S i Pf i l Services offerings ff i • Tests outside-in, with drill down to web application components • SaaS with no investment or maintenance costs and rapid payback • Full desktop browser testing across globally distributed geographies Learn More: Reality Load XF Solution Brief http://www.gomez.com/pdfs/SB_RealityLoad.pdf Speak To A Gomez Solutions Expert 42 Toll Free: 877.372.6732

×