Things to be aware of when you are measuring the performance of your web sites (both synthetic testing and Real User Monitoring).

1. Measuring Web Performance
Patrick Meenan
pmeenan@webpagetest.org
@patmeenan

2. Types of Measurement
velocityconf.com/velocity2011/public/schedule/detail/18039

3. Measuring Web Performance
SYNTHETIC TESTING

4. Active Testing
• Consistent results
– Good for detecting and diagnosing issues that are 100% repeatable
and on pages you are monitoring
• Lots of detail
• Availability monitoring
• Not representative of actual end-user
performance
– Usually tested from data centers
– Sometimes co-located with CDN provider
equipment

5. Bandwidth/Latency Impact
(Fixed 60ms RTT) (Fixed 5Mbps Bandwidth)
5
www.belshe.com/2010/05/24/more-bandwidth-doesnt-matter-much/

6. US Broadband Latency
WebPagetest DSL
WebPagetest Cable
WebPagetest FIOS Backbone
6
www.fcc.gov/document/measuring-broadband-america-report-consumer-broadband-performance-us

7. Actual
Users
Backbone
Testing
7

8. DSL – Back-End: 17%
Backbone – Back-End: 52%
8

9. Mac Traffic Shaping
OS X Lion + Xcode 4.1
Cable, DSL, 3G, Edge, WiFi and Custom
GUI for dummynet (which has been available for a while)
osxdaily.com/2011/08/10/simulate-internet-connectivity-bandwidth-speeds-network-link-conditioner/

10. Synthetic Browsers
• Know HOW your active testing is done
– Some only test the base page
– Synthetic browsers usually not reflective of real
browser
– Real browser testing
– Connectivity (backbone, last-mile)
• www.engadget.com
– Synthetic Browser: 447 requests
– IE 8: 241 requests

11. Measuring Web Performance
REAL USER MONITORING (RUM)

12. Real User Monitoring (RUM)
• Full coverage on all pages visited by users
• Real performance information for end-user
experience
• No availability monitoring
• No competitive reference
• Limited detail
– … but getting better

13. Navigation Timing
window.performance.timing
IE 9+
Chrome
Firefox 9+
Android 4+
Front-End
Back-End
13
dvcs.w3.org/hg/webperf/raw-file/tip/specs/NavigationTiming/Overview.html

14. Resource Timing (draft)
Timing for every network-loaded resource
w3c-test.org/webperf/specs/ResourceTiming/

18. Back-End Only

19. Access Logs
• Overall Response Times - %D (Apache)
• Custom variables (app-specific timings)
• Logster + graphite = graphs
• Lots of data-mining opportunities
– Broken links/content (404’s with useful referer information)
• Mike Brittain (Etsy) Presentation
– http://www.slideshare.net/mikebrittain/web-performance-culture-
and-tools-at-etsy-11159635

20. Measuring Web Performance
THE METRICS

21. When is “Done”?
Easy answer is onLoad
But…

22. When is “Done”?
Twitter onLoad (2.4s) Twitter end of activity (8.2s)

23. 2 Pages, Identical Performance
…

24. http://youtu.be/Ipk1k_IobYg

25. Start Render
Identical

26. Load Time
Identical

27. Last Visual Change
Identical

28. Visual “Completeness”

30. Measuring Web Performance
INTRODUCING…

31. The Speed Index
The user experience of A was 7.4 times faster than B
sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index

33. Amazon.com vs. Twitter
…

35. Amazon vs. Twitter
Amazon Twitter
Speed Index 1501 6014 (+300%)
onload 2.4 2.4
Fully Loaded 6.4 8.2 (+28%)
www.webpagetest.org/video/view.php?id=120414_22f2206da973b41621c499e2be01d555cdac6ab0&bare=1

37. Measuring Web Performance
WHAT’S NEW IN WEBPAGETEST

38. New Waterfall UI
Nav Timing – domContentLoaded
(Firefox/Chrome)
Nav Timing – load event
(Firefox/Chrome)
Pop-up request details

39. Dependencies (Chrome)
Line of code that
Triggered request

40. Response Bodies

41. Dev Tools Timeline (Chrome)

42. Dev Tools Timeline (Chrome)

43. Creating Videos

44. Creating Videos (cont)
Customize:
- End Point/Times
- Specific Run
- First/Repeat View
- Labels
www.webpagetest.org/video/compare.php?tests=120416_QX_3ZKFP%2C120416_DR_3ZKFJ&bare=1

45. SPOF Demonstrating
www.webpagetest.org/video/view.php?id=120416_d635d742a17d7eecf57df6282bcea87f5de2322c&bare=1

47. Custom viewport size
Chrome/Firefox

48. EC2 AMIs
IE 7
m1.small or
IE 8
c1.medium
IE 9, Chrome, Firefox (single image)

49. News/Updates
• Twitter:
@patmeenan
• Google+
PatMeenan@gmail.com
WebPagetest
• WebPagetest Forums
http://www.webpagetest.org/forums/