Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
@tameverts
#PerfNow
How I learned
to stop worrying
and love UX metrics
@tameverts
WPOstats.com
What is the best UX metric?
How fast should it be?
How can we stay on track?
https://www.cnet.com/news/appliance-science-the-well-done-physics-chemistry-of-the-toaster/
TTFB DNS TCP Start render
DOM loading DOM ready Page load Fully loaded
TTI Resource timing Number of requests Bytes in
Spe...
“The real thing we are after is
to create a user experience that
people love and they feel is fast…
and so we might be fro...
How do we measure
perception… at scale?
Is it happening?
Is it useful?
Is it usable?
Is it delightful?
https://developers.google.com/web/fundamentals/
performance...
TTFB DNS TCP
TTI FCP FMP
FID OMG WTF
❑ Correlates to what users actually see in the browser
❑ Is easy to use and accessible right out of the box
❑ Recognizes t...
A brief history
of performance metrics
Load Time
the time from the start of the initial
navigation until the beginning of the
window load event
BBC loses 10% of users for every additional
second it takes their site to load
Ancestry.com saw a 7% conversion rate
incre...
Start Render
the time from the start of the initial
navigation until the first non-white
content is painted
start render repeat visits
wow!
@tameverts #PerfNow
Speed Index
average time at which visible parts of
the page are in the viewport
When good metrics go bad…
First Paint
First Contentful Paint
First Meaningful Paint
First CPU Idle
First Interactive
First Paint (FP)
Pixels first start to render
First Contentful Paint (FCP)
Text and graphics start to render…
BUT often catches non-meaningful paints
(e.g. headers, nav...
First Meaningful Paint (FMP)
The paint after which the biggest
ATF layout change has happened
and web fonts have loaded
ht...
https://speedcurve.com/blog/an-analysis-of-chromiums-paint-timing-metrics/
“The rendering pipeline is very complex, and
the timestamp should be the latest timestamp
the browser is able to note in t...
Analysis of 40 top Alexa-ranked sites
95% of FP events occur before Start Render
85% of FCP events occur before Start Rend...
❑ Correlates to what users actually see in the browser
❑ Is easy to use and accessible right out of the box
❑ Recognizes t...
First CPU Idle
(formerly known as First Interactive)
Page is minimally interactive
Most visible UI elements are interactiv...
Time to Interactive (TTI)
(formerly known as Time to Consistently Interactive)
Displays useful content
Event handlers are ...
❑ Correlates to what users actually see in the browser
❑ Is easy to use and accessible right out of the box
❑ Recognizes t...
Okay… so then what?
Hero Rendering Times
H1
Largest Image
Largest Background Image
First Painted Hero
Last Painted Hero
https://speedcurve.com...
One-size-fits-most (not all)
Dependent on how page is built
Don’t work for image carousels and
popups
Hero times gotchas
So which metric is best?
http://lab.speedcurve.com/rendering/picker.php
And the winner is…
It depends!
(You probably saw that coming.)
Deliver any content?
Start Render
Deliver significant
amount of content?
Speed Index, FMP
Deliver critical content?
Hero R...
❑ Correlates to what users actually see in the browser
❑ Is easy to use and accessible right out of the box
❑ Recognizes t...
Custom metrics
Measure performance with high-precision timestamps
Available in both synthetic and RUM (yay!)
https://www.w...
how long does it take
to display the main
product image on my
site?
Time to First Tweet
The time from clicking the link to viewing
the first tweet on each page’s timeline
Pinner Wait Time (P...
@tameverts #PerfNow
❑ Correlates to what users actually see in the browser
❑ Is easy to use and accessible right out of the box
❑ Recognizes t...
What else can we track?
First Click
First Scroll
First Key
First Input Delay
(only available with real user monitoring)
Great!
We have metrics.
Now… what to do with them?
Performance budgets FTW!
Thresholds YOU create for metrics
that are meaningful for YOUR site
https://addyosmani.com/blog/performance-budgets/
Miles...
A good performance budget
should show you…
What your budget is
When you go out of bounds
How long you’re out of bounds
Whe...
Recommended UX perf budgets*
Metric How to track it Budget
Start render Synthetic & RUM 2s
First Painted Hero Synthetic 2s...
Super important!
Look at your own data
Monitor your competitors
No sandbagging allowed
Take a step-by-step approach if nec...
Pro tips
Create budgets for your popular and regularly
changing pages
Review violations early and always
Compare before an...
Takeaways
1. There is no unicorn metric
2. Use your eyes
3. Don’t just look at averages and medians
4. Validate your metrics by corr...
thanks!
@tameverts
speedcurve.com/blog
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
How I learned to stop worrying and love UX metrics
Upcoming SlideShare
Loading in …5
×

How I learned to stop worrying and love UX metrics

909 views

Published on

This talk at the 2018 performance.now() conference (Amsterdam) walks through a brief history of UX and web performance research, highlighting landmark studies that helped connect the dots between performance and user experience. I also demystify the current state of performance metrics and help you understand what you need to focus on for your site and your users.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

How I learned to stop worrying and love UX metrics

  1. 1. @tameverts #PerfNow How I learned to stop worrying and love UX metrics
  2. 2. @tameverts WPOstats.com
  3. 3. What is the best UX metric? How fast should it be? How can we stay on track?
  4. 4. https://www.cnet.com/news/appliance-science-the-well-done-physics-chemistry-of-the-toaster/
  5. 5. TTFB DNS TCP Start render DOM loading DOM ready Page load Fully loaded TTI Resource timing Number of requests Bytes in Speed Index PageSpeed Navigation timing DOM elements DOM size Visually complete TTFP TTFMP
  6. 6. “The real thing we are after is to create a user experience that people love and they feel is fast… and so we might be front-end engineers, we might be dev, we might be ops, but what we really are is perception brokers.” Steve Souders
  7. 7. How do we measure perception… at scale?
  8. 8. Is it happening? Is it useful? Is it usable? Is it delightful? https://developers.google.com/web/fundamentals/ performance/user-centric-performance-metrics
  9. 9. TTFB DNS TCP TTI FCP FMP FID OMG WTF
  10. 10. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric… #PerfNow@tameverts
  11. 11. A brief history of performance metrics
  12. 12. Load Time the time from the start of the initial navigation until the beginning of the window load event
  13. 13. BBC loses 10% of users for every additional second it takes their site to load Ancestry.com saw a 7% conversion rate increase after improving load time by 64% FT.com reduced desktop load times to 1.5s and mobile to 2.1s and increased user engagement by 30% @tameverts #PerfNow
  14. 14. Start Render the time from the start of the initial navigation until the first non-white content is painted
  15. 15. start render repeat visits
  16. 16. wow!
  17. 17. @tameverts #PerfNow
  18. 18. Speed Index average time at which visible parts of the page are in the viewport
  19. 19. When good metrics go bad…
  20. 20. First Paint First Contentful Paint First Meaningful Paint First CPU Idle First Interactive
  21. 21. First Paint (FP) Pixels first start to render
  22. 22. First Contentful Paint (FCP) Text and graphics start to render… BUT often catches non-meaningful paints (e.g. headers, nav bars) https://tinyurl.com/paint-metrics
  23. 23. First Meaningful Paint (FMP) The paint after which the biggest ATF layout change has happened and web fonts have loaded https://tinyurl.com/paint-metrics
  24. 24. https://speedcurve.com/blog/an-analysis-of-chromiums-paint-timing-metrics/
  25. 25. “The rendering pipeline is very complex, and the timestamp should be the latest timestamp the browser is able to note in the pipeline (best effort). Typically the time at which the frame is submitted to the OS for display is recommended for this API.” https://w3c.github.io/paint-timing/#sec-terminology
  26. 26. Analysis of 40 top Alexa-ranked sites 95% of FP events occur before Start Render 85% of FCP events occur before Start Render 50% of FMP events occur before Start Render https://speedcurve.com/blog/ an-analysis-of-chromiums-paint-timing-metrics/
  27. 27. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric…
  28. 28. First CPU Idle (formerly known as First Interactive) Page is minimally interactive Most visible UI elements are interactive Responds to user input reasonably quickly
  29. 29. Time to Interactive (TTI) (formerly known as Time to Consistently Interactive) Displays useful content Event handlers are registered for most visible elements Page responds to user interaction within 50ms
  30. 30. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric…
  31. 31. Okay… so then what?
  32. 32. Hero Rendering Times H1 Largest Image Largest Background Image First Painted Hero Last Painted Hero https://speedcurve.com/blog/web-performance-monitoring-hero-times/
  33. 33. One-size-fits-most (not all) Dependent on how page is built Don’t work for image carousels and popups Hero times gotchas
  34. 34. So which metric is best?
  35. 35. http://lab.speedcurve.com/rendering/picker.php
  36. 36. And the winner is…
  37. 37. It depends! (You probably saw that coming.)
  38. 38. Deliver any content? Start Render Deliver significant amount of content? Speed Index, FMP Deliver critical content? Hero Rendering Timeshttps://speedcurve.com/blog/rendering-metrics/
  39. 39. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric…
  40. 40. Custom metrics Measure performance with high-precision timestamps Available in both synthetic and RUM (yay!) https://www.w3.org/TR/user-timing/ https://speedcurve.com/blog/user-timing-and-custom-metrics/
  41. 41. how long does it take to display the main product image on my site?
  42. 42. Time to First Tweet The time from clicking the link to viewing the first tweet on each page’s timeline Pinner Wait Time (PWT) The time from initiating an action (e.g., tapping a pin) until the action is complete (pin close-up view is loaded) Time to Interact (TTI) @tameverts #PerfNow
  43. 43. @tameverts #PerfNow
  44. 44. ❑ Correlates to what users actually see in the browser ❑ Is easy to use and accessible right out of the box ❑ Recognizes that not all pixels and page elements are equal ❑ Allows us to customize what we measure on specific pages The best UX metric…
  45. 45. What else can we track?
  46. 46. First Click First Scroll First Key First Input Delay (only available with real user monitoring)
  47. 47. Great! We have metrics. Now… what to do with them?
  48. 48. Performance budgets FTW!
  49. 49. Thresholds YOU create for metrics that are meaningful for YOUR site https://addyosmani.com/blog/performance-budgets/ Milestone timings (e.g. start render) Quantity-based (e.g. image weight) Rules-based (e.g. Lighthouse scores)
  50. 50. A good performance budget should show you… What your budget is When you go out of bounds How long you’re out of bounds When you’re back within budget
  51. 51. Recommended UX perf budgets* Metric How to track it Budget Start render Synthetic & RUM 2s First Painted Hero Synthetic 2s Speed Index Synthetic 4s First Interactive Synthetic 4s First Input Delay RUM 10ms @tameverts *See super important stuff on the next slide
  52. 52. Super important! Look at your own data Monitor your competitors No sandbagging allowed Take a step-by-step approach if necessary Use synthetic and RUM (numbers may will vary)
  53. 53. Pro tips Create budgets for your popular and regularly changing pages Review violations early and always Compare before and after releases Update budgets accordingly https://www.zillow.com/engineering/bigger-faster-more-engaging-budget/
  54. 54. Takeaways
  55. 55. 1. There is no unicorn metric 2. Use your eyes 3. Don’t just look at averages and medians 4. Validate your metrics by correlating them with business and engagement KPIs 5. Set targets (e.g., performance budgets) 6. Make yourself accountable (e.g., alerts) 7. Don’t be afraid to investigate new metrics! @tameverts #PerfNow
  56. 56. thanks! @tameverts speedcurve.com/blog

×