Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Metrics, Metrics Everywhere (but where the heck do you start?)

3,939 views

Published on

Not surprisingly, there’s no one-size-fits-all performance metric (though life would be simpler if there were). Different metrics will give you different critical insights into whether or not your pages are delivering the results you want — both from your end user’s perspective and ultimately from your organization’s perspective. Join Tammy Everts, and walk through various metrics that answer performance questions from multiple perspectives. You’ll walk away with a better understanding of your options, as well as a clear understanding of how to choose the right metric for the right audience.

Published in: Technology
  • Be the first to comment

Metrics, Metrics Everywhere (but where the heck do you start?)

  1. 1. Metrics, metrics everywhere (but where the heck do you start?)
  2. 2. @tameverts @cliffcrocker #velocityconf
  3. 3. Who cares about performance today? How do I measure performance? How fast am I? How fast should I be? How do I get there?
  4. 4. The myth of a single metric
  5. 5. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  6. 6. Who cares about performance?
  7. 7. “47% of consumers expect a web page to load in 2 seconds or less.” Akamai, 2009
  8. 8. 1s = $27M DNS 144ms Start render 1.59s Hero image render 2.01s
  9. 9. How do I measure performance?
  10. 10. Androiddevicefragmentation OpenSignal,August2014
  11. 11. RUM versus plus synthetic
  12. 12. RUM 101
  13. 13. Technology for collecting performance metrics directly from the end user’s browser Involves instrumenting your site via JavaScript Measurements are fired across the network to a collection point through a small request object (beacon)
  14. 14. What makes RUM great  Always on  Every user, every browser, everywhere  Able to capture human behavior/events  Only getting better
  15. 15. Questions your RUM data can answer
  16. 16. What are my users’ environments?
  17. 17. How do visitors move through my site?
  18. 18. How are my third-party scripts performing in real time?
  19. 19. What impact does performance have on my business?
  20. 20. Synthetic Monitoring 101
  21. 21. Uses automated agents (bots) to measure your website from different physical locations A set “path” or URL is defined Tests are run either ad hoc or scheduled, and data is collected
  22. 22. What makes synthetic monitoring great  Rich data collected (waterfall, video, filmstrip, HTTP headers)  Consistent “clean room” baseline  Nothing to install  Doesn’t require users / ability to measure pre-production and competition
  23. 23. Questions your synthetic data can answer
  24. 24. How do I compare to the competition?
  25. 25. How does the design of my pages affect performance?
  26. 26. How does the newest version of my site compare to previous versions?
  27. 27. How well am I sticking to my performance budget?
  28. 28. What if my third parties fail? Original: 3.5s SPOF: 22.7s
  29. 29. 36© 2014 SOASTA CONFIDENTIAL - All rights reserved. Why are these numbers so different? I don’t trust your data. Your numbers are wrong. How are you calculating page load time? I can’t share two sets of numbers with the business?
  30. 30. “But it loads so much faster for me!?!” 2015 Macbook Pro Warm browser cache FIOS X86 – Windows 7 VM Completely cold cache/DNS Throttled bandwidth
  31. 31. boomerang.js Episodes
  32. 32. W3C Performance Working Group
  33. 33. How fast am I?
  34. 34. Navigation Timing API
  35. 35. Browser support for Navigation Timing
  36. 36. 45© 2014 SOASTA CONFIDENTIAL - All rights reserved. Network operations Front-end developer Marketing and site operations Designer C-level
  37. 37. Use case: Measure network performance
  38. 38. I need visibility into…  issues with authoritative DNS servers  impact of denial of service attacks on end users  efficiency of connection re-use  tier 1 connectivity issues (load balancer, web server)
  39. 39. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  40. 40. Measuring DNS and TCP function getPerf() { var timing = window.performance.timing; return { dns: timing.domainLookupEnd - timing.domainLookupStart}; connect: timing.connectEnd - timing.connectStart}; }
  41. 41. What’s with all those zeros!  Connection reuse  DNS caching  Prefetching
  42. 42. Focus on higher percentiles 85th percentile Median (50th)
  43. 43. Use case: Measure front-end browser events
  44. 44. How do I…  understand the impact of back-end versus front-end on page speed?  investigate how DOM complexity affects performance?  measure a “fully loaded” page?
  45. 45. Start render DNS TCP TTFB DOM load event DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  46. 46. Isolate front-end vs. back-end
  47. 47. Isolate front-end vs. back-end function getPerf() { var timing = window.performance.timing; return { ttfb: timing.responseStart - timing.connectEnd}; basePage: timing.responseEnd - timing.responseStart}; frontEnd: timing.loadEventStart - timing.responseEnd}; }
  48. 48. Front-end Back-end
  49. 49. Investigate DOM events function getPerf() { var timing = window.performance.timing; return { DLoading: timing.domLoading – timing.navigationStart}; DInt: timing.domInteractive – timing.navigationStart}; DContLoaded: timing.domContentLoadedEventEnd – timing.navigationStart; DContLoadTime: timing.domContentLoadedEventEnd – timing.domContentLoadedEventStart}; DComplete: timing.domComplete - timing.navigationStart}; PLoad: timing.loadEventStart -
  50. 50. 2618 DOM nodes 86 DOM nodes Visualizing DOM complexity
  51. 51. Use case: Measure object-level performance
  52. 52. I need to understand…  how third-party content affects my performance  how long a group of assets takes to download  how assets served by a CDN are performing
  53. 53. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  54. 54. Resource Timing interface
  55. 55. Browser support for Resource Timing
  56. 56. Cross-Origin Resource Sharing (CORS) Start/End time only unless Timing-Allow-Origin HTTP Response Header defined Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list- or-null | "*"
  57. 57. Resource Timing var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker- dualtone-150x150.png’; var me = performance.getEntriesByName(rUrl)[0]; var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart } Measuring a single resource
  58. 58. Other uses for Resource Timing  Slowest resources  Time to first image (download)  Response time by domain  Time a group of assets  Response time by initiator type (element type)  Cache-hit ratio for resources For examples see: https://github.com/lognormal/beyond-page-metrics
  59. 59. Using Resource Groups PLT impact not due resource groups PLT impact correlates with improvement from image domains
  60. 60. Use case: Measure the user experience
  61. 61. I just need to understand…  when users perceive the page to be ready  how long until my page begins to render  when content above the fold is visible
  62. 62. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  63. 63. The fallacy of “First Paint” in the wild  Support for First Paint is not standardized between browsers  Metric can be misleading (i.e. painting a white screen)
  64. 64. First Paint is not equal to Start Render! Chrome – “First Paint” True Start Render
  65. 65. Start Render and filmstrips
  66. 66. User Timing Interface  Allows developers to measure performance of their applications through high-precision timestamps  Consists of “marks” and “measures”  PerformanceMark: Timestamp  PerformanceMeasure: Duration between two given marks
  67. 67. Measure duration between two marks performance.mark(“startTask”); //Some stuff you want to measure happens here performance.mark(“stopTask”); //Measure the duration between the two marks performance.measure(“taskDuration”,“startTask”,“stopTask”);
  68. 68. How long does it take to display the main product image on my site?
  69. 69. Record when an image loads <img src=“image1.gif” onload=“performance.mark(‘image1’)”> For more interesting examples, see: Measure hero image delay http://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/ Measure aggregate times to get an “above fold time” http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user- experience.html
  70. 70. How do I measure performance when the onload event no longer matters? Use case: Measure performance of SPAs
  71. 71. onload event visible resources
  72. 72. Measuring SPAs • Accept the fact that onload no longer matters • Tie into routing events of SPA framework • Measure downloads during soft refreshes • (support in boomerang.js for Angular and other SPA frameworks) See: http://www.soasta.com/blog/angularjs-real-user- monitoring-single-page-applications/
  73. 73. How fast should I be?
  74. 74. Use case: Measure business impact
  75. 75. I need to understand…  how performance affects business KPIs  how our site compares to our competitors
  76. 76. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  77. 77. 2% increase in conversions for every 1 second of improvement
  78. 78. Cut load times in half Increased sales by 13%
  79. 79. So what? You must look at your own data.
  80. 80. Not all pages are created equal For a typical ecommerce site, conversion rate drops by up to 50% when “browse” pages increase from 1 to 6 seconds
  81. 81. Not all pages are created equal However, there is much less impact to conversion when “checkout” pages degrade
  82. 82. Conversion Impact Score
  83. 83. How do I get there?
  84. 84. Create a performance budget See: Setting a Performance Budget http://timkadlec.com/2013/01/setting-a-performance-budget/ Performance Budget Metrics http://timkadlec.com/2014/11/performance-budget-metrics/
  85. 85. Set meaningful, page-specific SLAs
  86. 86. “Response time measured using resource timing from Chrome browsers in the United States should not exceed a median (50th percentile) of 100ms or a 95th percentile of 500ms for a population of more than 500 users in a 24-hour period.”
  87. 87. “Vendor will make an effort to ensure average response times for content is within reasonable limits.”
  88. 88. Chapter 8: Changing Culture at Your Organization
  89. 89. performancebacon.com performancebeacon.com
  90. 90. Thanks!
  91. 91. Meet us at booth #801

×