Your SlideShare is downloading. ×
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The Evolution of the Architecture of Web Applications Requires a Load Testing Revolution

1,554

Published on

Long-held best practices of n-tier application architecture are being challenged. Old design patterns are new again. This session discusses the architecture evolution of Web applications, load testing …

Long-held best practices of n-tier application architecture are being challenged. Old design patterns are new again. This session discusses the architecture evolution of Web applications, load testing approaches, and the future.

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,554
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
45
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The Evolution of the Architecture of WebApplications Requires a Load Testing Revolution Imad Mouline – CTO – Compuware APM @imadmouline
  • 2. Where The Data Comes From Synthetic monitoring and load testing scripts from 3,000 enterprises, across broad number of verticals  Number of distinct scripts ranges from 45K to 68K, depending on data mining exercise  150+ backbone / data center / cloud testing locations, and thousands of Last Mile / desktop testing locations Real user monitoring measurements from 200+ sites  Users from around the world  Measurement samples range from 117 million to 526 million pages / user interactions, depending on data mining exercise
  • 3. Observation 1 Web applications are becoming more composite
  • 4. The Client Is Becoming THE Integration Platform * Source: Gomez 2010
  • 5. By The Numbers – February 2010 Number of hosts accessed directly by the browser, per user transaction, averaged across 3,000 companies Measurement Number of hosts per city user transaction Hong Kong 5.51 Beijing 6.91 London 7.46 New York 8.17 Frankfurt 8.66
  • 6. By The Numbers – June 2010 Number of hosts accessed directly by the browser, per user transaction, averaged across 3,000 companies Measurement Number of hosts per city user transaction Hong Kong 7.56 Beijing 8.57 London 8.59 New York 8.85 Frankfurt 8.87
  • 7. By The Numbers – September 2010 Number of hosts accessed directly by the browser, per user transaction, averaged across 3,000 companies Measurement Number of hosts per city user transaction Hong Kong 6.82 Beijing 8.87 London 7.95 New York 9.82 Frankfurt 8.71 Paris 10.12 Stockholm 10.48 Helsinki 12.71
  • 8. By The Numbers – November 2010 Number of hosts accessed directly by the browser, per user transaction, averaged across 3,000 companies Measurement city Number of hosts per user transaction Hong Kong 5.50 Paris 6.27 Amsterdam 6.90 London 7.25 Frankfurt 7.45 Beijing 9.10 Stockholm 9.61 New York 10.50 Helsinki 11.57
  • 9. Observation 1a Enterprises Are Adopting the Cloud (with or without their knowledge)
  • 10. Web Applications Are Moving To The Cloud – June 2010 Percentage of web app transactions that include at least one object hosted on Amazon EC2Amazon EC2 Region PercentageEC2 Asia Pacific - Singapore 0.002EC2 US West - Northern California 0.659EC2 EU - Ireland 2.733EC2 US East - Northern Virginia 16.194 TOTAL 19.588
  • 11. Enterprises ARE Adopting Cloud Computing – Nov 2010 Percentage of web app transactions that include at least one object hosted on Amazon EC2Amazon EC2 Region PercentageEC2 Asia Pacific - Singapore 0.151EC2 EU - Ireland 1.578EC2 US West - Northern California 2.066EC2 US East - Northern Virginia 24.144 TOTAL 27.938
  • 12. Observation 2 Content is becoming increasingly dynamic and distributed
  • 13. Geographic Distribution Of Content Sources How many cities does content come from to form the average transaction? Distribution of host cities by test Distribution of host cities by test (measured from single location) (measured from multiple locations) >30 21-30 >30 21-30 8% 4% 2% 7% 1 1 11-20 27% 1 11% 2-5 34% 11-20 6-10 12% 6-10 13% 11-20 6-10 21-30 15% 2-5 >30 31% 2-5 36%Source: Gomez Active Backbone Monitoring Sample of 12,000 production monitoring scripts Multiple runs over 24 hours
  • 14. Observation 3 Content is becoming increasingly inter-dependent
  • 15. Browser Impact on Response TimeResponse times differences across Firefox and IE for a 6-steptransaction Internet Explorer 7 Firefox 3.5
  • 16. Major Differences In IE And Firefox Waterfall ChartsIE 7 Waterfall Chart Firefox 3.5 Waterfall Chart Connection 3 Connection 1 Connection 2 Connection 1 Connection 6 Connection 5 Connection 4 Connection 2 Connection 7 Connection 8 Connection 9 Connection 3 Connection 10 Connection 5 Connection 4 Connection 12 Connection 6 Connection 11 Connection 7
  • 17. Performance & Availability Issues Can Be Browser Specific Issue is 3rd party content blocking on IE onlyPerformance issue impacting Internet Explorer 7 Internet Explorer 7 Firefox 3.5
  • 18. Load Testing with HTTP Playback: Testing from NYC vs. Atlanta Some of the measurements are different. The load order is the same using HTTP- From NY or Atlanta.
  • 19. Load Testing with IE Playback: Testing from NYC vs. Atlanta The load order is different between Atlanta and NY
  • 20. Observation 4 Processing is increasingly being pushed to the client
  • 21. Significant Performance Differences Across Browsers Load Time Perceived Render 7 6 5 4 3 2 1 0Source: Gomez Real-User Monitoring (October 2010) Real users around the world 526 million page measurements Broadband connections only 200+ sites
  • 22. Browsers Are Evolving To Support Heavier Client-Side CodeHTML5 support  Application cache canvas, audio, video, local storage, geo-location, web workers etc.CSS3 Support  Webfonts, animations, gradients, shadows, etc.Performance improvements  Faster JavaScript processing  Parallel download of JS scripts  More parallel connections  Resource pre-fetching  Multi-threading in JS Key Trend - more and more client-side processing
  • 23. RIA Frameworks Adoption 25.18 % of transactions surveyed depend on at least one of these frameworks Percentage of transactions that leverage framework 14.00% 12.00% 10.00% 8.00% 6.00% 4.00% 2.00% 0.00%Source: Gomez Active Backbone Monitoring ~ 3,000 enterprises 48K+ distinct transactions active at least once during a 1 hr time period
  • 24. Observation 5 Mobile Users Are Becoming Less Patient
  • 25. Web & Mobile Site Performance Impacts User Behavior Abandonment Rate Across 200+ Sites / 177+ Million Page Views over 1 week / All Browsers vs. iPhone Safari 30 25Abandonment Rate (%) 20 Abandonment Rate - All Browsers 15 Abandonment Rate - iPhone Safari 10 5 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Page Load Time Band (sec.)Source: Gomez real user monitoring
  • 26. Significant Performance Differences Across Browsers & Devices Load Time Perceived Render 12 10 8 6 4 2 0Source: Gomez Real-User Monitoring (October 2010) Real users around the world 526 million page measurements Broadband & wireless connections only 200+ sites
  • 27. Recap of Observations1. Web applications are getting more composite  The browser is becoming the integration platform of choice  Cloud-hosted app components are mainstream2. Content is becoming increasingly dynamic3. Content is becoming increasingly inter-dependent4. Processing is moving to the client5. Mobile users are becoming less patient
  • 28. The Traditional View of Web Application Delivery Systems …user ismanagement happy tools: “OK”Load UsersBalancersWebServersMobileComponentsAppServersDB Web applicationServersStorageMainframeNetwork Traditional zone of control
  • 29. The Reality of Web Application Delivery Traditional Approach Slow response time …user is NOT happy DB Network Server Geographic disparitiesLoad UsersBalancers Transactions fail 4 sec’sWebServers 22 sec’sMobileComponentsAppServers Faulty display or operationDBServersStorageMainframeNetwork Traditional zone Traditional zone of control of control
  • 30. The Challenge of Managing Application Performance The Application Delivery Chain Customers Local Browsers Data Center ISP 3rd Party/ Virtual/physical environment Cloud Services DB App Web LoadMainframe Servers Servers Servers Balancers Major Storage Network ISP Content Delivery Networks Web Mobile WAN VirtualServices Components Optimization Desktops Mobile Carrier Devices Employees Employees (via WAN)
  • 31. The Challenge of Managing Application Performance Traditional Approach …user is The Application Delivery Chain NOT happy DB Network Server Customers Local Browsers Data Center • Inconsistent geo performance ISP • Bad performance under load • Network peering 3rd Party/ • Poorly Virtual/physical environment delivery • Blocking content problems Cloud Services performing DB • Incorrect geo-targeted Load App Web content • Bandwidth JavaScriptMainframe Servers Servers Servers Balancers throttling • Browser/device • Inconsistent incompatibility • Configuration • Network peering connectivity • Page size errors problems too big • Application • Outages Major • Too many Storage Network design issues ISP • Network resource objects • Code defects shortage Content • Low cache • Insufficient • Delivery Faulty content hit rate infrastructureWAN Networks transcoding Web Mobile VirtualServices Components Optimization Desktops • SMS routing / Mobile latency issues • Configuration issues Carrier • Oversubscribed POP Devices Employees • Poor routing optimization Employees (via WAN) • Low cache hit rate
  • 32. Simplification of the Problem is Key The Application Delivery Chain Customers Local Browsers Data Center ISP 3rd Party/ Virtual/physical environment Cloud Services DB App Web LoadMainframe Servers Servers Servers Balancers Major Storage Network ISP Content Delivery Networks Web Mobile WAN VirtualServices Components Optimization Desktops Mobile Carrier Devices Employees Employees (via WAN)
  • 33. Test Across the Entire Web Application Delivery Chain The Application Delivery Chain Load Testing 2.0 Load Testing 1.5 Load Testing 1.0 Customers Local Browsers Data Center ISP 3rd Party/ Virtual/physical environment Cloud Services DB App Web LoadMainframe Servers Servers Servers Balancers Major Storage Network ISP Content Delivery Networks Web Mobile WAN VirtualServices Components Optimization Desktops Mobile Carrier Devices Employees Employees (via WAN)
  • 34. Load Testing 1.0 Works For Specific Situations Company: Online presence for a popular TV show • Following episodes of the TV show the web site sees high traffic spikes • Goal was to achieve 1500 logins per minute • Load tested DB to improve performance in anticipation of another traffic spike 3rd Party/ BrowsersLoad Cloud Services Local ISP and devices UsersBalancersWebServersMobileComponentsAppServers InternetDB MajorServers ISPStorageMainframeNetwork Content Delivery Mobile Networks Carrier
  • 35. Application Bottleneck Causes Response Time Issue • As users were added, the response time of step 3 (the login) climbed immediately • The test bottlenecked at 160 logins per minute (Goal 1500) • But quickly dropped off as users received server errors • New login query was not optimized and was bottlenecking the database servers’ CPUs
  • 36. Application Bottleneck – Re-test •After tuning- application performance improved. •New Bottleneck occurred 1300 logins per minute. •Bandwidth limit reached at 90 Mbps Summary: •Problem found inside firewall •Fixes made for application issue •Retest shows second issue-bandwidth 1.0 1.5 2.0 •First test      •Second test 1.0 1.5 2.0    
  • 37. Load Testing from the Cloud misses end-user perspective Company: Online Gaming Site Testing a new rollout in support of a new sports season • Support anticipated traffic increases • Load tested system using cloud and Last Mile to validate performance for real users in new geographies. 3rd Party/ Browsers Load Cloud Services Local ISP and devices Users Balancers Web Servers Mobile Components App Servers Internet DB Major Servers ISP Storage Mainframe Network Content Delivery Mobile Networks Carrier
  • 38. View from the Cloud • First 20 minutes Cloud testing shows acceptable performance • After 2500 users, Response time climbs, Availability drops, Error rate climbs
  • 39. View from the Last Mile • Last Mile shows different story • Availability is terrible even at minimal load for real users Summary: Cloud-only testing may give misleading availability data Cloud starts with 100% availability Less than 25% for the Last Mile 1.0 1.5 2.0 
  • 40. Load testing 1.0 and 1.5 miss regional issues Company: Regional Online News Source • Began testing for the election season • Goal was to validate overall performance focusing in 2 key regions 3rd Party/ Browsers and Load Cloud Services Local ISP devices Users Balancers Web Servers Mobile Components App Internet Servers Major DB ISP Servers Storage Mainframe Content Delivery Mobile Network Networks Carrier
  • 41. No Performance Issues Detected From Data Center Increase and hold load and not exceed response times of 4 seconds and Success Rate of 99% There was only 1 page error and 11 errors total out of 60000+ transactions Page response times stayed under 4 seconds, outside of one brief blip 1.0 or 1.5 load testing shows tests passed
  • 42. Last Mile Case Study: Primary Geographies Key geographies for this customer are New York and Pennsylvania. The response time never met the 4 second average goal Summary: Availability was Less Last Mile shows goal not than 99% reached Cloud can’t detect the end user issue 1.0 1.5 2.0 
  • 43. The Internet is global – where your customers are mattersCompany: International Hotel chain • New reservations system rollout • New global server load balancing rolled out across multiple data centers • Validate that system works globally 3rd Party/ Browsers and Load Cloud Services Local ISP devices Users Balancers Web Servers Mobile Components App Internet Servers Major DB ISP Servers Storage Mainframe Content Delivery Mobile Network Networks Carrier
  • 44. Major Hotel Reservation System unavailable in 4 countries 0% availability in UK, Germany, Japan 99%+ availability in US, Canada, France Summary: Internal U.S. test looked good Distributed testing fails in key locations. 1.0 1.5 2.0 ? 
  • 45. Load Testing 2.0 shows you what your customer will see Company: eRetailer fashion • 100% virtual store • Daily sales spike driving 90% of revenue stream 3rd Party/ Browsers and Load Cloud Services Local ISP devices Users Balancers Web Servers Mobile Components App Internet Servers Major DB ISP Servers Storage Mainframe Content Delivery Mobile Network Networks Carrier
  • 46. Load Testing with multiple browsers shows discrepanciesAvailability vastly different between browsers
  • 47. Comparison of Performance across the country - FirefoxUsing Firefox browser – shows 100% availability for websiteWide variations in response time based on geography
  • 48. Comparison of Performance across the country – IE IE Browser : shows under 12 percent availability Availability and performance tied to geography
  • 49. Page Element Downloads: IE Versus Firefox- Order VariesRoot Cause: Summary:•Third party ad provider Only full browsers in targetmodifying the DOM locations can show what really happens.•Depending on the load orderof the third party the JavaScript 1.0 1.5 2.0in the ad would overwrite theDOM but only on IE 
  • 50. Load Testing Approaches : Which one is best for you? Load Test 1.0 Load Test 1.5 Load Test 2.0 HTTP : Behind the HTTP : Data Centers Browser : Data Real World Desktops Firewall Centers Last Mile Traditional Client/ Datacenter Testing Server TestAccuracy of End-User Incomplete Incomplete Indicative Most AccurateResponse TimeAccuracy of Invalid Indicative Indicative Most AccurateApplication AvailabilityAbility to drive large Yes-requires Best Better Goodload volume substantial hardwareUnderstand CDN No Misleading Misleading MostImpact AccurateUnderstand 3rd Party No Minimal Some Most(ads, feeds, etc…) AccurateRealistic object No No Yes Yesdownload Static OnlyVisibility behind the Best Good Good Goodfirewall
  • 51. So what? Now what? Broaden the definition of your web application Test early, test often Know your end-users, and test from their perspective Pay attention to all 4 buckets:  Data Center, Internet, 3rd Parties, Client Look for the breaking point of the end-user experience, not just the breaking point of the application infrastructure
  • 52. See you at CMG11Dec 5th - 9th, 2011 Washington, DC

×