Perform Realistic Web
Testing To Ensure Blazing
Fast Web Site Performance
Mike Gualtieri
Senior Analyst
Forrester Research
James Staten
Principal Analyst
Forrester Research
June 16, 2009

Realistic Web
performance testing is
critical to achieving
flawless user
experiences.

Agenda
• Why your customers won’t wait
• What is realistic performance testing?
• Recommendations
4
Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Agenda
• Why your customers won’t wait
• What is realistic performance testing?
• Recommendations
5
Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Performance means:
• Raw page load speed Gomez Media Home
Page Benchmark
April 10 ’09 – May 10 ’09
0:00 - 24:00 EST
6
Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Performance means:
Gomez Media Article
• Customer workflow speed Search Transaction
1 Benchmark
April 10 ’09 – May 10 ’09
0:00 - 24:00 EST
2
3
7
Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Blazing fast
Web sites = customers

Traditional Web performance testing
doesn’t cut it
• Traditional testing is expensive.
• Cost of licenses, hardware, and training
• Time-consuming to implement
• Idiosyncrasies of the software are often difficult to
master.
• Traditional testing is not realistic.
• Scope is limited to internal, clean-room testing.
• Not focused on end-user point of view
• Misses too many pieces in the Web application
delivery chain such as content delivery network
(CDN), third-party components, geography, etc.
9
Entire contents © 2008 Forrester Research, Inc. All rights reserved.

Most developers are building Web apps
“What types of new custom applications are your firm’s developers building?”
(Multiple responses accepted)
Base: 977 platform software decision-makers at North American and European enterprises and SMBs
Source: Enterprise And SMB Software Survey, North America And Europe, Q4 2008
10 contents © 2008 Forrester Research, Inc. All rights reserved.
Entire

New trends make optimal user
experiences even more important
• The Millennial generation expects more.
– 18- to 27-year-olds
– The tech-savvy generation
• Web site redesigns
– Rich internet applications (RIA)
• More Web sites are composed of third-party
components.
• Virtualization of infrastructure, including cloud
• Mobile apps availability is exploding.
11 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Agenda
• Why your customers won’t wait?
• What is realistic performance testing?
• Recommendations
12 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Realistic testing means taking a
walk in your customers’ shoes.

“Realistic testing” means testing from
your customers’ point of view
• Where do they access your site from?
– Bangkok, Cannes, Des Moines, Dubrovnik,
Edmonton, LA, London, Miami, Osaka, Pusan
• What do they do?
– Key tasks, transactions, and business processes
• How do they do it?
– Browsers galore, mobile devices
• When do they do it (and how frequently)?
– Peak times, spikes
14 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

The user’s distance from the content
matters
Source: Tom Leighton, “Improving Performance on the Internet,” Queue, October 2008
15 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Boston users of forrester.com have the
fastest response time
Source: Gomez (www.gomez.com)
16 contents © 2008 Forrester Research, Inc. All rights reserved.
Entire

Browsers render differently
Source: Gomez (http://www.gomez.com)
17 contents © 2008 Forrester Research, Inc. All rights reserved.
Entire

What features should a realistic Web
performance testing platform have?
It should:
• Test from the end user’s point of view.
– Different locations
– Different browsers
– Different loads
– Different tasks
• Be easy for developers, testers, and businesspeople
to use.
• Identify and resolve problems along the entire Web
application delivery chain.
18 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Web sites are increasingly held hostage
by third-party apps
Source: Gomez (www.gomez.com)
19 contents © 2008 Forrester Research, Inc. All rights reserved.
Entire

Vendors offering performance
measurement solutions
• Gomez
• Keynote Systems
• Cordiant Technologies
20 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Agenda
• What do customers want?
• What is realistic performance testing?
• Recommendations
21 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Recommendations
• Design a test plan based on how real users interact
with your Web application.
• Use testing tools that allow you to test from your
user’s point of view:
– Where, what, how, and when
• Test the entire Web application.
• Democratize testing:
– Build a test team that includes anyone who has a
stake in the outcome.
22 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Recommendations (cont.)
• Test early to nip problems in the bud.
• Test often to maintain a consistently supreme user
experience.
• Follow the “Blazing Fast Web Site Performance” best
practices to keep your Web site humming.
23 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Don’t leave user experience to
chance.

Perform realistic Web testing
to achieve blazing fast Web
site performance.
25 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Thank you
Mike Gualtieri
+1 617.613.6145
mgualtieri@forrester.com
James Staten
+1 650.380.3733
jstaten@forrester.com
www.forrester.com
26 contents © 2008
Entire Forrester Research, Inc. All rights reserved.

Selected Forrester research
• February 4, 2009, “Best Practices: Attaining And
Maintaining Blazing Fast Web Site Performance”
• February 4, 2009, “Case Study: How Orbitz Takes
You Places Faster”
• August 15, 2008, “Best Practices: Blazing Fast Web
Site Infrastructure”
• August 15, 2008, “Case Study: How Msnbc.com Is
First In News”
27 contents © 2008
Entire Forrester Research, Inc. All rights reserved.