Web performance testing


Published on

Presentation from the October 2011 meeting of the Northern Virginia Test Automation Interest Group on testing web performance.

Published in: Technology

Web performance testing

  1. 1. Web Performance Testing NoVa Test Automation Interest Group Patrick Meenan Twitter: @PatMeenan [email_address] blog.patrickmeenan.com
  2. 2. Front-End vs. Back-End
  3. 4. Back-End This is the HTML that largely contains the content of the page and defines the structure.
  4. 5. Front-End CSS : Layout and style Images Graphical Elements
  5. 6. Waterfall Diagram Base Page CSS Images and Graphical Elements Javascript Start Render Document Complete (onLoad)
  6. 7. Front-End vs Back-End Base Page Back-End Processing
  7. 8. Front-End: 3.2s* Back-End: 0.010s* *WebPagetest performance as measured by New Relic
  8. 9. Measuring Performance
  9. 10. velocityconf.com/velocity2011/public/schedule/detail/18039
  10. 11. Active Testing <ul><li>Consistent results </li></ul><ul><ul><li>Good for detecting and diagnosing issues that are 100% repeatable and on pages you are monitoring </li></ul></ul><ul><li>Lots of detail </li></ul><ul><li>Availability monitoring </li></ul><ul><li>Not representative of actual end-user performance </li></ul><ul><ul><li>Usually tested from data centers </li></ul></ul><ul><ul><li>Sometimes co-located with CDN provider equipment </li></ul></ul>
  11. 12. US Broadband Bandwidth www.fcc.gov/document/measuring-broadband-america-report-consumer-broadband-performance-us
  12. 13. US Broadband Latency www.fcc.gov/document/measuring-broadband-america-report-consumer-broadband-performance-us
  13. 14. Bandwidth/Latency Impact www.belshe.com/2010/05/24/more-bandwidth-doesnt-matter-much/ (Fixed 60ms RTT) (Fixed 5Mbps Bandwidth)
  14. 15. Backbone Testing Actual Users
  15. 16. DSL – Back-End: 17% Backbone – Back-End: 52%
  16. 17. When is “Done”? <ul><li>Easy answer is onLoad </li></ul><ul><li>But… </li></ul>
  17. 18. When is “Done”? Twitter onLoad (1.9s) Twitter end of activity (6.8s)
  18. 19. Synthetic Browsers <ul><li>Know HOW your active testing is done </li></ul><ul><ul><li>Some only test the base page </li></ul></ul><ul><ul><li>Synthetic browsers usually not reflective of real browser </li></ul></ul><ul><ul><li>Real browser testing </li></ul></ul><ul><ul><li>Connectivity (backbone, last-mile) </li></ul></ul><ul><li>www.engadget.com </li></ul><ul><ul><li>Synthetic Browser: 447 requests </li></ul></ul><ul><ul><li>IE 8: 241 requests </li></ul></ul>
  19. 20. Real User Monitoring (RUM) <ul><li>Full coverage on all pages visited by users </li></ul><ul><li>Real performance information for end-user experience </li></ul><ul><li>No availability monitoring </li></ul><ul><li>Limited detail </li></ul><ul><ul><li>… but getting better </li></ul></ul>
  20. 21. Navigation Timing – IE9, Chrome, Firefox 7 <ul><li>window.performance.timing </li></ul>dvcs.w3.org/hg/webperf/raw-file/tip/specs/NavigationTiming/Overview.html Front-End Back-End
  21. 22. Resource Timing (draft) w3c-test.org/webperf/specs/ResourceTiming/ Timing for every network-loaded resource
  22. 23. _gaq.push(['_trackPageLoadTime']);
  23. 26. Back-End Only
  24. 27. Frontend Single Point Of Failure
  25. 28. Asynchronous Javascript <ul><li>It’s not just good for web performance… </li></ul><ul><li><script type=&quot;text/javascript&quot; src=&quot;http://platform.twitter.com/widgets.js&quot;></script> </li></ul><ul><li>vs. </li></ul><ul><li><script type=&quot;text/javascript&quot;> (function() { var tw = document.createElement('script'); tw.type = 'text/javascript'; tw.async = true; tw.src = 'http://platform.twitter.com/widgets.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(tw, s); })(); </script> </li></ul>
  26. 29. It prevents Frontend SPOF User stares at a blank window for 21 seconds (usually < 2 seconds)
  27. 30. What it feels like
  28. 31. Testing for Frontend SPOF <ul><li>blackhole.webpagetest.org (aka </li></ul><ul><li>Hosts file: ajax.googleapis.com apis.google.com www.google-analytics.com connect.facebook.net platform.twitter.com </li></ul><ul><li>WebPagetest.org: setDnsName platform.twitter.com blackhole.webpagetest.org navigate your.url.com </li></ul>
  29. 32. WebPagetest
  30. 33. The basics – testing “pages” <ul><ul><li>www.webpagetest.org </li></ul></ul><ul><ul><li>Submit an URL…. </li></ul></ul>
  31. 34. From multiple locations globally…
  32. 35. The basics – get “results” <ul><li>Performance and optimization information </li></ul>
  33. 36. What it looks like
  34. 37. What it looks like
  35. 38. What if…
  36. 39. Comparing arbitrary tests
  37. 40. The comparison
  38. 41. More advanced capabilities <ul><li>Automating Rich Applications </li></ul><ul><ul><li>Authenticating into form-protected applications </li></ul></ul><ul><ul><li>Interacting with Web Apps (Mail, Calendar, etc) </li></ul></ul><ul><ul><li>Testing a navigation flow from one page to another </li></ul></ul><ul><ul><li>Execute arbitrary Javascript </li></ul></ul><ul><li>REALLY Advanced Options </li></ul><ul><ul><li>Re-writing DNS lookups (directing traffic to a test/staging box) </li></ul></ul><ul><ul><li>Custom User-Agent strings & HTTP Headers </li></ul></ul><ul><ul><li>Pre-populating cookies </li></ul></ul><ul><ul><li>Custom test end conditions </li></ul></ul><ul><ul><li>Capture Dynatrace Ajax Edition sessions </li></ul></ul><ul><ul><li>Tcpdump capture </li></ul></ul>
  39. 42. Private Instances <ul><li>http://sites.google.com/a/webpagetest.org/docs/private-instances </li></ul><ul><li>Bulk Testing </li></ul><ul><li>EC2 Agent Public AMI’s </li></ul><ul><ul><li>US (East/West) </li></ul></ul><ul><ul><li>Europe </li></ul></ul><ul><ul><li>Asia (Singapore/Tokyo) </li></ul></ul><ul><li>Public Projects </li></ul><ul><ul><li>HTTP Archive </li></ul></ul><ul><ul><li>Blaze Mobitest </li></ul></ul>
  40. 43. REST API <ul><li>Submit tests (API key required for public instance) </li></ul><ul><ul><li>Check status </li></ul></ul><ul><ul><li>Fetch Result </li></ul></ul><ul><li>XML </li></ul><ul><li>HAR </li></ul><ul><li>http://sites.google.com/a/webpagetest.org/docs/advanced-features/webpagetest-restful-apis </li></ul>
  41. 44. WPT Monitor <ul><li>Tony Perkins (Sabre) </li></ul><ul><li>http://www.webpagetest.org/forums/forumdisplay.php?fid=21 </li></ul>
  42. 45. HTTP Archive <ul><li>http://httparchive.org/ </li></ul>