Achieving Peak User Experiences & Optimizing Web Performance - Load Testing, Stress Testing & Performance Testing in an Agile Web 2.0 World

Perform Realistic Web Performance Testing To Achieve Peak User Experiences Mike Gualtieri Senior Analyst y Forrester Research May 13, 2009

Realistic web ea st c eb performance testing is c t ca critical to ac e g achieving flawless user experiences. p

Performance means: Gomez Media Article • Customer workflow speed Search Transaction Benchmark 1 April 10 ’09 – May 10 ’09 0:00 - 24:00 EST 2 3 7 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Realistic Web Performance Testing Is A Forrester Best P ti F t B t Practice • Best Practices: Attaining And Maintaining Blazing Fast Web Sites • Best Practices: Blazing Fast Web Site Infrastructure • Case Study: How Orbitz.com Takes You Places Faster • Case Study: How Msnbc.com Is First In News 8 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Blazing Fast Web Sites = Customers

Traditional web performance testing doesn’t cut it d ’t t • Traditional testing is expensive • Cost of licenses, hardware and training • Time consuming to implement • Idiosyncrasies are often difficult to master • Traditional testing is not realistic • Scope is limited to internal clean-room testing • Not focused on end-user point of view p • Misses too many pieces in the web application delivery chain such as CDN, 3rd-party components, geography, etc… 11 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Most Developers Are Building Web Apps. “What types of new custom applications are your firm’s developers building” (multiple responses accepted) 2008 Base: 977 platform software decision-makers at North American and European enterprises and SMBs 12 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

New trends make optimal user experiences even more important i i tt • The Millennial generation expects more – 18 to 27 year olds – The tech-savvy generation • Web site redesigns – RIA More Web sites are composed of 3rd-party • components • Virtualization f infrastructure including cloud Vi t li ti of i f t t i l di ld • Mobile 13 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

“Realistic testing” means testing from your customer s customer’s point of view • Where they do it – LA, Edmonton, London, Bangkok, Des Moines, Miami, Cannes, Dubrovnik, Osaka, Pusan • Wh they d What h do – Key tasks, transactions, and business processes • How they do it – Browsers galore, mobile devices • When they do it (and how frequently) – Peak times, spikes p 15 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

What features should a realistic web performance testing platform have? It should: • Test from the end user’s point of view – Different locations – Different browsers – Different loads – Different tasks • Be easy for developers, testers and business people to use. • Identify and resolve problems along the entire web application delivery chain. 18 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Recommendations (1 of 2) • Design a test plan based on how real users interact with your web application • Use testing tools that allow you to test from your user’s point of view: – Wh Where, Wh t How, Wh What, H When • Test the entire Web application • Democratize testing: – Build a test team that includes anyone who has a stake in the outcome 20 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Recommendations (2 of 2) • Testing early to nip problems in the bud • Test often to maintain a consistently supreme user experience • Follow Blazing Fast Web Performance Best Practices to keep your Web site humming 21 Entire contents © 2008 Forrester Research, Inc. All rights reserved.

In this fiercely competitive environment, realistic web performance testing is the right answer.

Gomez Profile ▪ First SaaS Web Application ▪ Headquarters Experience Management Suite pe e ce a age e t Su te ▪ Lexington, MA ▪ Web Performance and Load Testing ▪ Global Offices ▪ Cross-Browser Testing Cross Browser ▪ UK ▪ Web Performance Monitoring ▪ Germany Ge a y ▪ 80 000+ internet experience 80,000+ measurement points in 162 ▪ Switzerland countries ▪ China ▪ Over 2,500 Customers Worldwide ▪ 12 out of 20 most visited US Websites utilize G tili Gomez ▪ 300 Employees 24

Today’s Testing Challenges Never Enough Time Small QA Teams 25 Catching All the Problems Testing the End-User Perspective

When To Load Test Launching New Apps/Features Deploying New Infrastructure 26 Major Marketing Campaigns Penetrating New Markets

Web Application Delivery In A Web 2.0 World ▪ Web applications pp increasingly assembled at browser level ▪ 3rd party performance from ads, videos, shopping cart vary widely under load ▪ Content Delivery Network configuration and effectiveness impact end-user experience d i 27

Load 2.0 Testing Solutions Overcome Limitations of 1.0 Testing Approaches ▪ Find and fix problems across the entire web application delivery chain ▪ Inside and outside the firewall ▪ Flexible and easy to use ▪ On-demand – no installation required ▪ Highly scalable to accommodate peak loads ▪ Supports Agile development ▪ Test from the end-user perspective 28

Ensure Web Experiences Scale Under Load With Reality Load XF ▪ Deploy web applications and infrastructure with confidence ▪ Combine high volume web application load testing with broad geographic experience testing ▪ Ensure on-demand testing g flexibility ▪ Detect failures in user experience not caught by traditional load testing ▪ Testing across the entire web application d li delivery chain li ti hi 29

Critical To Test Across Entire Web Application Delivery Chain Key Load Testing Metrics • Success rate of end-user workflow • Response time of end-user workflow end user • Quality of end-user workflow delivery 30

Load Testing 2.0 Performance Testing For A Web 2.0 World ▪ Self-service testing gives you control to test as needed ▪ Test internal & external web application components to ensure transactions perform under load ▪ Test cloud-based applications & services ▪ SaaS based Load Testing 2.0 solutions enable on-demand, scalable testing without g Combine high volume C bi hi h l associated hardware investment application load testing & maintenance costs with broad geographic ▪ Test the full web application pp experience testing delivery chain all the way to your end users’ desktops 31

Critical To Test Across Entire Web Application Delivery Chain 3 2 1

Ensuring Performance Of Login Process 1 Company • Online presence for a p p p popular television show Situation • Following episodes of the television show the web site sees high traffic spikes • Goal was to achieve 1500 logins per minute • Load tested database optimizations aimed to improve performance in anticipation of another large traffic spike 33

Application Bottleneck Causes Immediate Response Time Issue • As users were added the added, response time of step 3 (the login) climbed immediately • The test bottlenecked at 160 logins per minute (Goal 1500) • But quickly dropped off as users received server errors • New login query was not optimized and was bottlenecking the database servers’ CPU ’ CPUs 34

Ensuring Performance Of E-Commerce Transaction 2 Company • Online Retailer Situation • Typical e-Commerce type application •SSeveral 3 d P ti now i l 3rd Parties involved i l d in serving up key content • Goal was to validate performance of entire application

Response Times Rise Due To Third-Party Object Error The transaction rate The load increases increases and then throughout the test falls off as response times climb Errors are seen, all on a 3rd party object • 3 d party h d 3rd t hardware was i insufficient f overall d ffi i t for ll demands on application d li ti • Based on SLAs 3rd party had to improve performance to get paid

Ensuring Performance In Target Markets 3 Company • Regional Online News Source Situation • Began testing for the 2008 election season • Goal was to validate performance of overall application, and specifically for users in 2 key geographies • Testing started with the homepage

No Performance Issues Detected From Data-Center Increase and hold load and not exceed response times of 4 seconds and Success There was only Rate of 99% 1 page error and 11 errors total out of 60000+ Page response transactions times stayed under 4 seconds, outside of one brief blip p By traditional test standards the test passed

Performance Issues Detected From Real User Desktops Key geographies for this customer are New York and Pennsylvania Last Mile data showing substantial number of measurements greater than 4 seconds

Performance Issues Detected For Target Market End Users • Last Mile Users exceeding the 4- second goal – 29% • Overall availability was 96.3% • Availability for Last Mile Users exceeding 4 seconds – 87% • Last Mile Users reporting 10+ g second response times – 8% By these standards the test f il d B th t d d th t t failed • Cutting 1 5 to 2 sec off the response time for the page would move 1.5 sec. another 16% of the Last Mile users into the 4 sec. response time range

Gomez Offers Full Lifecycle Solutions For Web Experience Management “Gomez has made spectacular p g p progress in the last 18 months…to become the leader in web experience management.” Jean-Pierre Garbani, Vice President, Principal Analyst, Forrester Research 41 Tech Horizons: Evaluating Gomez’s Web Experience Management Services, August 2008

Find Out How Gomez Can Help You Ensuring great experiences in a Web 2.0 Some of Our Customers world requires a new approach to web application load testing - Reality Load XF Reality Load XF • Self-service or full turnkey solution with Gomez Professional S i Pf i l Services offerings ff i • Tests outside-in, with drill down to web application components • SaaS with no investment or maintenance costs and rapid payback • Full desktop browser testing across globally distributed geographies Learn More: Reality Load XF Solution Brief http://www.gomez.com/pdfs/SB_RealityLoad.pdf Speak To A Gomez Solutions Expert 42 Toll Free: 877.372.6732

Achieving Peak User Experiences & Optimizing Web Performance - Load Testing, Stress Testing & Performance Testing in an Agile Web 2.0 World

0 comments

Notes on slide 1

Favorites, Groups & Events