Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How I learned to stop worrying and love UX metrics

661 views

Published on

This talk at the 2018 performance.now() conference (Amsterdam) walks through a brief history of UX and web performance research, highlighting landmark studies that helped connect the dots between performance and user experience. I also demystify the current state of performance metrics and help you understand what you need to focus on for your site and your users.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

How I learned to stop worrying and love UX metrics

  1. 1. @tameverts #PerfNow How I learned to stop worrying and love UX metrics
  2. 2. @tameverts WPOstats.com
  3. 3. What is the best UX metric? How fast should it be? How can we stay on track?
  4. 4. https://www.cnet.com/news/appliance-science-the-well-done-physics-chemistry-of-the-toaster/
  5. 5. TTFB DNS TCP Start render DOM loading DOM ready Page load Fully loaded TTI Resource timing Number of requests Bytes in Speed Index PageSpeed Navigation timing DOM elements DOM size Visually complete TTFP TTFMP
  6. 6. “The real thing we are after is to create a user experience that people love and they feel is fast… and so we might be front-end engineers, we might be dev, we might be ops, but what we really are is perception brokers.” Steve Souders
  7. 7. How do we measure perception… at scale?
  8. 8. Is it happening? Is it useful? Is it usable? Is it delightful? https://developers.google.com/web/fundamentals/ performance/user-centric-performance-metrics
  9. 9. TTFB DNS TCP TTI FCP FMP FID OMG WTF
  10. 10. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric… #PerfNow@tameverts
  11. 11. A brief history of performance metrics
  12. 12. Load Time the time from the start of the initial navigation until the beginning of the window load event
  13. 13. BBC loses 10% of users for every additional second it takes their site to load Ancestry.com saw a 7% conversion rate increase after improving load time by 64% FT.com reduced desktop load times to 1.5s and mobile to 2.1s and increased user engagement by 30% @tameverts #PerfNow
  14. 14. Start Render the time from the start of the initial navigation until the first non-white content is painted
  15. 15. start render repeat visits
  16. 16. wow!
  17. 17. @tameverts #PerfNow
  18. 18. Speed Index average time at which visible parts of the page are in the viewport
  19. 19. When good metrics go bad…
  20. 20. First Paint First Contentful Paint First Meaningful Paint First CPU Idle First Interactive
  21. 21. First Paint (FP) Pixels first start to render
  22. 22. First Contentful Paint (FCP) Text and graphics start to render… BUT often catches non-meaningful paints (e.g. headers, nav bars) https://tinyurl.com/paint-metrics
  23. 23. First Meaningful Paint (FMP) The paint after which the biggest ATF layout change has happened and web fonts have loaded https://tinyurl.com/paint-metrics
  24. 24. https://speedcurve.com/blog/an-analysis-of-chromiums-paint-timing-metrics/
  25. 25. “The rendering pipeline is very complex, and the timestamp should be the latest timestamp the browser is able to note in the pipeline (best effort). Typically the time at which the frame is submitted to the OS for display is recommended for this API.” https://w3c.github.io/paint-timing/#sec-terminology
  26. 26. Analysis of 40 top Alexa-ranked sites 95% of FP events occur before Start Render 85% of FCP events occur before Start Render 50% of FMP events occur before Start Render https://speedcurve.com/blog/ an-analysis-of-chromiums-paint-timing-metrics/
  27. 27. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric…
  28. 28. First CPU Idle (formerly known as First Interactive) Page is minimally interactive Most visible UI elements are interactive Responds to user input reasonably quickly
  29. 29. Time to Interactive (TTI) (formerly known as Time to Consistently Interactive) Displays useful content Event handlers are registered for most visible elements Page responds to user interaction within 50ms
  30. 30. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric…
  31. 31. Okay… so then what?
  32. 32. Hero Rendering Times H1 Largest Image Largest Background Image First Painted Hero Last Painted Hero https://speedcurve.com/blog/web-performance-monitoring-hero-times/
  33. 33. One-size-fits-most (not all) Dependent on how page is built Don’t work for image carousels and popups Hero times gotchas
  34. 34. So which metric is best?
  35. 35. http://lab.speedcurve.com/rendering/picker.php
  36. 36. And the winner is…
  37. 37. It depends! (You probably saw that coming.)
  38. 38. Deliver any content? Start Render Deliver significant amount of content? Speed Index, FMP Deliver critical content? Hero Rendering Timeshttps://speedcurve.com/blog/rendering-metrics/
  39. 39. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric…
  40. 40. Custom metrics Measure performance with high-precision timestamps Available in both synthetic and RUM (yay!) https://www.w3.org/TR/user-timing/ https://speedcurve.com/blog/user-timing-and-custom-metrics/
  41. 41. how long does it take to display the main product image on my site?
  42. 42. Time to First Tweet The time from clicking the link to viewing the first tweet on each page’s timeline Pinner Wait Time (PWT) The time from initiating an action (e.g., tapping a pin) until the action is complete (pin close-up view is loaded) Time to Interact (TTI) @tameverts #PerfNow
  43. 43. @tameverts #PerfNow
  44. 44. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric…
  45. 45. What else can we track?
  46. 46. First Click First Scroll First Key First Input Delay (only available with real user monitoring)
  47. 47. Great! We have metrics. Now… what to do with them?
  48. 48. Performance budgets FTW!
  49. 49. Thresholds YOU create for metrics that are meaningful for YOUR site https://addyosmani.com/blog/performance-budgets/ Milestone timings (e.g. start render) Quantity-based (e.g. image weight) Rules-based (e.g. Lighthouse scores)
  50. 50. A good performance budget should show you… What your budget is When you go out of bounds How long you’re out of bounds When you’re back within budget
  51. 51. Recommended UX perf budgets* Metric How to track it Budget Start render Synthetic & RUM 2s First Painted Hero Synthetic 2s Speed Index Synthetic 4s First Interactive Synthetic 4s First Input Delay RUM 10ms @tameverts *See super important stuff on the next slide
  52. 52. Super important! Look at your own data Monitor your competitors No sandbagging allowed Take a step-by-step approach if necessary Use synthetic and RUM (numbers may will vary)
  53. 53. Pro tips Create budgets for your popular and regularly changing pages Review violations early and always Compare before and after releases Update budgets accordingly https://www.zillow.com/engineering/bigger-faster-more-engaging-budget/
  54. 54. Takeaways
  55. 55. 1. There is no unicorn metric 2. Use your eyes 3. Don’t just look at averages and medians 4. Validate your metrics by correlating them with business and engagement KPIs 5. Set targets (e.g., performance budgets) 6. Make yourself accountable (e.g., alerts) 7. Don’t be afraid to investigate new metrics! @tameverts #PerfNow
  56. 56. thanks! @tameverts speedcurve.com/blog

×