Did you know that 85% of users complain about slow response time? Poor web application performance can directly impact your bottom line
The success of your critical eBusiness initiatives depends on your ability to deliver quality web experiences. Unfortunately, 65% of applications are not properly load tested prior to launch, resulting in lost revenue, increased support costs and brand damage. So how can you ensure success when launching new applications, adding features, deploying new infrastructure, rolling out marketing campaigns, or preparing for seasonal spikes like the holiday shopping season?
Join us as our guest speaker, Melinda Ballou, IDC’s Program Director for Application Life-Cycle Management research discusses challenges, drivers and best practices for effective web performance testing and quality life-cycle management for today’s rich and complex applications. Additional topics that Imad Mouline, Gomez’s CTO will cover in this session are:
Best practices for ensuring the success of critical eBusiness initiatives
The end-user experience and business impact of emerging web technologies like Rich Internet Applications, virtualization, cloud computing and Web 2.0
A new approach for web performance and load testing that’s easy to use, delivered on-demand, and enables you to find and fix problems before they impact customers
Who Should Watch: Line of Business and eCommerce Managers, Interactive Marketing, Brand Managers, Project Managers and IT Operations Executives.
IDC & Gomez Webinar --Best Practices: Protect Your Online Revenue Through Web Performance Testing
1. Best Practices
Protect your Online Revenue through Web
Performance Testing
Melinda Carol-Ballou - Application Life-Cycle Management
pp y g
& IT Executive Strategies, IDC
Imad Mouline - CTO Gomez
CTO,
2. Optimizing Application Quality and
Performance to Drive the Business
Melinda Ballou
Application Life-Cycle Management
IT Executive Strategies
E ti St t i
September 2009
Copyright 2009 IDC. Reproduction is forbidden unless authorized. All rights reserved.
21. Best Practices: Protecting your Online Revenue
BP #1: Establish a Common Language
between IT and the business
BP #2: Make Customer Experience
Management part of the culture
BP #3: Implement testing &
management for end‐to‐end business
processes from the customer/end
processes – from the customer/end‐
user perspective
22. BP #1: Establish a Common Language Between IT & the Business:
Customer Satisfaction Index
Apdex (Application Quickly isolate and prioritize
Performance Index)
P f I d ) opportunities for performance
ii f f
Single measurement of improvement based on real
user satisfaction with customer satisfaction
Web performance
W b f Why is customer satisfaction
h f
Based on response time decreasing?
thresholds What do users have in
common? ?
How do I compare against
Satisfied Tolerating Frustrated my competitors?
http://www.apdex.org 2007
27. BP #3: Implement Testing/Management for End‐to‐end
Business Process from the Customer/End‐user Perspective
Behind‐the‐ …user is
firewall testing NOT happy
tools: “OK”
Users
Load
Balancing
Web
Servers
App
Servers The traditional view of Web Application Delivery
The traditional view of Web Application Delivery
DB
Servers
Storage
Mobile
Components
Traditional zone
of control
28. Critical to Test Across Entire Web Application Delivery Chain
Behind‐the‐ …user is
firewall testing NOT happy
tools: “OK”
The Web Application Delivery Chain
3rd Party/ Browsers
Cloud Services Local ISP and Devices Users
Load • Inconsistent geo performance • Network peering • Poorly
Balancing • Bad performance under load
Bad performance under load p
problems p
performing g
• Blocking content delivery • Bandwidth throttling JavaScript
Web • Inconsistent • Browser/device
Servers • Incorrect geo‐targeted content
connectivity incompatibility
• Configuration • Page size
App • Network peering too big
Servers errors problems
• Application •TToo many
DB design issues • Outages Internet objects
• Network resource • Low cache
Servers • Code defects Major shortage hit rate
• Insufficient ISP • Configuration issues • Faulty content
Storage infrastructure transcoding
• Oversubscribed POP
Mobile • Poor routing optimization SMS routing /
• SMS i /
Components • Low cache hit rate latency issues
Content Delivery Mobile
Networks Carrier
Traditional zone Zone of customer expectation
of control
29. Load Testing 2.0
Performance Testing For A Web 2.0 World
Self‐service testing gives you
control to test as needed
Test internal & external web
T ti t l& t l b
application components to ensure
transactions perform under load
Test cloud‐based applications &
Test cloud‐based applications &
services
SaaS based Load Testing 2.0
solutions enable on‐demand, ,
scalable testing without associated
hardware investment &
maintenance costs
Combine hi h volume
C bi high l Test the full web application
delivery chain all the way to your
application load testing end users’ desktops
with broad geographic
experience testing from
the outside-in
30. Gomez Reality Load XF:
On‐Demand Realistic Load Testing from Browser to Data Center
Backbone Last Mile
Last Mile
High volume load (HTTP, Browser) Real‐world load
Find infrastructure breaking points Find user experience breaking points
Define capacity headroom Accurately measure response time
3rd Party/ Browsers
Cloud Services Local ISP and Devices Users
Load 100,000+
Balancing consumer‐
grade
d
Web desktops
Servers
168+
App 100+ countries
Servers commercial‐
grade
DB nodes & Internet 2,500+
data ISPs
Servers Major centers
ISP
Storage Major
mobile
Mobile carriers
around
Components the globe
Content Delivery Mobile
Networks Carrier
31. Ensuring Performance of All 3rd Party Components
Company: Online Retailer
• Several 3rd Parties now involved in serving up key content
• G l was to validate performance of entire application
Goal t lid t f f ti li ti
3rd Party/
y/ Browsers
Cloud Services Local ISP and devices Users
Load *
Balancing
Web
Servers
App
Servers
DB Internet
Servers Major
ISP
Storage
Mobile
Components
Content Delivery Mobile
Networks Carrier
32. Response Times Rise Due To 3rd Party Object Error
The load increases The transaction rate
throughout the test increases and then
falls off as response
times climb
Errors are seen, all
on a 3rd party object
• 3 d party h d
3rd t hardware was i
insufficient f overall d
ffi i t for ll demands on application
d li ti
• Based on SLAs 3rd party had to improve performance to get paid
33. Ensuring Performance in Key Markets
Company: Regional Online News Source
• Began testing for the 2008 election season
• G l was to validate overall performance f
Goal t lid t ll f focusing i 2 k regions
i in key i
3rd Party/
y/ Browsers
Cloud Services Local ISP and devices Users
Load
Balancing
*
Web
Servers
App
Servers
DB Internet
Servers Major
ISP
Storage
Mobile
Components
Content Delivery Mobile
Networks Carrier
34. No Performance Issues Detected From Data-Center
Increase and hold load and not exceed
response times of 4 seconds and Success There was only
Rate of 99% 1 page error and
11 errors total
out of 60000+
Page response transactions
times stayed under
4 seconds, outside
of one brief blip
By traditional test standards the test passed
35. Performance Issues Detected From Real User
Desktops
Key geographies for this customer are
New York and Pennsylvania
Last Mile data showing substantial number of
measurements greater than 4 seconds
36. The Gomez Platform: Web Application Experience Management
Self‐
Self‐Service SaaS Portal
Web Web Load Web Web
Cross‐Browser and Performance Performance Performance
Testing Testing Management Business Analysis
• Compatibility High volume
• High‐volume 20
Real user monitoring
• Real‐user monitoring Web performance
• Web performance
• Functionality backbone load • Last Mile monitoring analytics
• Performance • Real‐world • Backbone monitoring • Benchmarks
Last Mile load • Business dashboard
Web 2.0, RIA, Streaming, Mobile
Web 2 0 RIA Streaming Mobile
Dashboards, Recording & Alerting & APIs & Education &
Metrics & Analytics Provisioning Diagnostics Data Feeds Best Practices
World’s Most Comprehensive Testing Network
168+ countries 2,500+ ISPs
500+ 100+ 5,000+ 100,000+
combos of commercial nodes supported consumer‐grade
browsers & O/S and data centers mobile devices desktops
39. Ensuring Performance Of Login Process
Company: Online presence for a popular TV show
• Following episodes of the TV show the web site sees high traffic spikes
• Goal was to achieve 1500 logins per minute
• Load tested DB to improve performance in anticipation of another traffic spike
3rd Party/
y/ Browsers
Cloud Services Local ISP and devices Users
Load
Balancing 1
Web
Servers
App
Servers
DB Internet
Servers Major
ISP
Storage
Mobile
Components
Content Delivery Mobile
Networks Carrier
40. Application Bottleneck Causes Immediate Response
Time Issue
• As users were added the
added,
response time of step 3 (the
login) climbed immediately
• The test bottlenecked at 160
logins per minute (Goal 1500)
• But quickly dropped off as
users received server errors
• New login query was not
optimized and was
bottlenecking the database
servers’ CPU
’ CPUs