• Philip Tellis
• @bluesmoon
• philip@bluesmoon.info
• SOASTA
• boomerang
UX Fest 2013 / 2013-10-01 RUM Distillation 101 1
RUM Distillation 101
Philip Tellis / philip@bluesmoon.info
UX Fest 2013 / 2013-10-01
UX Fest 2013 / 2013-10-01 RUM Distill...
Did you know that most Rum comes from the
Caribbean?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 3
But the largest selling Dark Rum in the world is
Old Monk
UX Fest 2013 / 2013-10-01 RUM Distillation 101 4
Jean-Antoine Nollet – 1746
UX Fest 2013 / 2013-10-01 RUM Distillation 101 5
Today Measuring the experience of Real Users is not
quite as shocking.
UX Fest 2013 / 2013-10-01 RUM Distillation 101 6
Real Users
UX Fest 2013 / 2013-10-01 RUM Distillation 101 7
Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from ...
Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from ...
Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from ...
Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from ...
Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from ...
What can we measure?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 8
Anything that the user does!
UX Fest 2013 / 2013-10-01 RUM Distillation 101 9
Anything that the user does!
(in the browser)
UX Fest 2013 / 2013-10-01 RUM Distillation 101 9
But we really care about performance here...
UX Fest 2013 / 2013-10-01 RUM Distillation 101 10
So let’s talk about How
UX Fest 2013 / 2013-10-01 RUM Distillation 101 11
Modern Browsers – no problem
UX Fest 2013 / 2013-10-01 RUM Distillation 101 12
Modern Browsers – no problem
UX Fest 2013 / 2013-10-01 RUM Distillation 101 13
Real Users don’t always use the latest browsers.
UX Fest 2013 / 2013-10-01 RUM Distillation 101 14
So there’s a lot of magic to support the other
31.38%
UX Fest 2013 / 2013-10-01 RUM Distillation 101 15
The other 31.38%...
1 Get a timestamp before they leave a page
2 Save it in a cookie
3 Compare with timestamp when the pag...
The other 31.38%...
1 Get a timestamp before they leave a page
2 Save it in a cookie
3 Compare with timestamp when the pag...
...magic
• What if the user goes off-site between pages?
• What if there’s SSL involved?
• What about new tab?
• What abou...
But it doesn’t need to be...
We put all this magic into boomerang
http://lognormal.github.com/boomerang/doc/
UX Fest 2013 ...
boomerang
• BSD licensed JavaScript library
• Measure load time, bandwidth, latency
• DNS lookup time, NavTiming, Connecti...
And it beacons all of this back to a predefined URL
UX Fest 2013 / 2013-10-01 RUM Distillation 101 20
So what then?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 21
Are you Ops, F2E or Product?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 22
Spot the datacenter
UX Fest 2013 / 2013-10-01 RUM Distillation 101 23
It’s actually in Nederland
UX Fest 2013 / 2013-10-01 RUM Distillation 101 24
How does bandwidth change around the world?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 25
Did you know that IE7 seems to be faster than IE8?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 26
But that’s mostly IE9 users browsing in IE7
compatibility mode.
UX Fest 2013 / 2013-10-01 RUM Distillation 101 27
What does a website’s performance distribution look
like?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 28
That’s my Blog (please visit)
UX Fest 2013 / 2013-10-01 RUM Distillation 101 29
Log-normal distribution
UX Fest 2013 / 2013-10-01 RUM Distillation 101 30
Bimodal distribution
UX Fest 2013 / 2013-10-01 RUM Distillation 101 31
Distributions
• Sparse distribution suggests that you don’t have enough
data points
• Log-normal distribution is typical
•...
Distributions are interesting, but how does it affect
engagement?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 33
Did you know that user patience wrt performance is
a cultural thing?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 34
It’s tied both to where a user is from, and also the
type of service.
UX Fest 2013 / 2013-10-01 RUM Distillation 101 35
different types of sites
UX Fest 2013 / 2013-10-01 RUM Distillation 101 36
We borrowed a biological term called LD50 and
applied it to Load Time v/s Bounce Rate
UX Fest 2013 / 2013-10-01 RUM Distil...
LD50
UX Fest 2013 / 2013-10-01 RUM Distillation 101 38
This helps you fix only the stuff that’s hurting you.
UX Fest 2013 / 2013-10-01 RUM Distillation 101 39
Did you know that users bounce if a page is too
fast?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 40
But that’s mostly because error pages are the fastest.
UX Fest 2013 / 2013-10-01 RUM Distillation 101 41
Measure → Hypothesize → Test → Fix
← Repeat ←
UX Fest 2013 / 2013-10-01 RUM Distillation 101 42
What delighted users a few years ago is now an
expected baseline, the absence of which will frustrate
them.
UX Fest 2013 /...
Changing Expectations
UX Fest 2013 / 2013-10-01 RUM Distillation 101 44
Thank You!
Questions?
UX Fest 2013 / 2013-10-01 RUM Distillation 101 45
• Philip Tellis
• @bluesmoon
• philip@bluesmoon.info
• www.SOASTA.com
• boomerang
• LogNormal Blog
UX Fest 2013 / 2013-10-...
Image Credits
• Jean-Antoine Nollet
http://en.wikipedia.org/wiki/Jean-Antoine_Nollet
• New York Times Olympic Medal Winner...
Upcoming SlideShare
Loading in...5
×

RUM Distillation 101 -- Part I

677

Published on

Part I of RUM Distillation 101. Part II is by Jonathan Klein available here: http://www.slideshare.net/jnklein/uxfest-performance-a

Published in: Technology
1 Comment
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
677
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

RUM Distillation 101 -- Part I

  1. 1. • Philip Tellis • @bluesmoon • philip@bluesmoon.info • SOASTA • boomerang UX Fest 2013 / 2013-10-01 RUM Distillation 101 1
  2. 2. RUM Distillation 101 Philip Tellis / philip@bluesmoon.info UX Fest 2013 / 2013-10-01 UX Fest 2013 / 2013-10-01 RUM Distillation 101 2
  3. 3. Did you know that most Rum comes from the Caribbean? UX Fest 2013 / 2013-10-01 RUM Distillation 101 3
  4. 4. But the largest selling Dark Rum in the world is Old Monk UX Fest 2013 / 2013-10-01 RUM Distillation 101 4
  5. 5. Jean-Antoine Nollet – 1746 UX Fest 2013 / 2013-10-01 RUM Distillation 101 5
  6. 6. Today Measuring the experience of Real Users is not quite as shocking. UX Fest 2013 / 2013-10-01 RUM Distillation 101 6
  7. 7. Real Users UX Fest 2013 / 2013-10-01 RUM Distillation 101 7
  8. 8. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience UX Fest 2013 / 2013-10-01 RUM Distillation 101 7
  9. 9. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience UX Fest 2013 / 2013-10-01 RUM Distillation 101 7
  10. 10. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience UX Fest 2013 / 2013-10-01 RUM Distillation 101 7
  11. 11. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience UX Fest 2013 / 2013-10-01 RUM Distillation 101 7
  12. 12. Real Users • Get delighted • Get frustrated • Have credit cards • ... that they are attached to • Form an impression from experience UX Fest 2013 / 2013-10-01 RUM Distillation 101 7
  13. 13. What can we measure? UX Fest 2013 / 2013-10-01 RUM Distillation 101 8
  14. 14. Anything that the user does! UX Fest 2013 / 2013-10-01 RUM Distillation 101 9
  15. 15. Anything that the user does! (in the browser) UX Fest 2013 / 2013-10-01 RUM Distillation 101 9
  16. 16. But we really care about performance here... UX Fest 2013 / 2013-10-01 RUM Distillation 101 10
  17. 17. So let’s talk about How UX Fest 2013 / 2013-10-01 RUM Distillation 101 11
  18. 18. Modern Browsers – no problem UX Fest 2013 / 2013-10-01 RUM Distillation 101 12
  19. 19. Modern Browsers – no problem UX Fest 2013 / 2013-10-01 RUM Distillation 101 13
  20. 20. Real Users don’t always use the latest browsers. UX Fest 2013 / 2013-10-01 RUM Distillation 101 14
  21. 21. So there’s a lot of magic to support the other 31.38% UX Fest 2013 / 2013-10-01 RUM Distillation 101 15
  22. 22. The other 31.38%... 1 Get a timestamp before they leave a page 2 Save it in a cookie 3 Compare with timestamp when the page loads UX Fest 2013 / 2013-10-01 RUM Distillation 101 16
  23. 23. The other 31.38%... 1 Get a timestamp before they leave a page 2 Save it in a cookie 3 Compare with timestamp when the page loads Then there’s some magic to handle edge-cases UX Fest 2013 / 2013-10-01 RUM Distillation 101 16
  24. 24. ...magic • What if the user goes off-site between pages? • What if there’s SSL involved? • What about new tab? • What about those strange cases where they leave a page after they load a new one? • What about those common cases where they open multiple pages from a search engine but don’t actually look at them? UX Fest 2013 / 2013-10-01 RUM Distillation 101 17
  25. 25. But it doesn’t need to be... We put all this magic into boomerang http://lognormal.github.com/boomerang/doc/ UX Fest 2013 / 2013-10-01 RUM Distillation 101 18
  26. 26. boomerang • BSD licensed JavaScript library • Measure load time, bandwidth, latency • DNS lookup time, NavTiming, Connection Type • Page Parameters • And anything else you can write a plugin for UX Fest 2013 / 2013-10-01 RUM Distillation 101 19
  27. 27. And it beacons all of this back to a predefined URL UX Fest 2013 / 2013-10-01 RUM Distillation 101 20
  28. 28. So what then? UX Fest 2013 / 2013-10-01 RUM Distillation 101 21
  29. 29. Are you Ops, F2E or Product? UX Fest 2013 / 2013-10-01 RUM Distillation 101 22
  30. 30. Spot the datacenter UX Fest 2013 / 2013-10-01 RUM Distillation 101 23
  31. 31. It’s actually in Nederland UX Fest 2013 / 2013-10-01 RUM Distillation 101 24
  32. 32. How does bandwidth change around the world? UX Fest 2013 / 2013-10-01 RUM Distillation 101 25
  33. 33. Did you know that IE7 seems to be faster than IE8? UX Fest 2013 / 2013-10-01 RUM Distillation 101 26
  34. 34. But that’s mostly IE9 users browsing in IE7 compatibility mode. UX Fest 2013 / 2013-10-01 RUM Distillation 101 27
  35. 35. What does a website’s performance distribution look like? UX Fest 2013 / 2013-10-01 RUM Distillation 101 28
  36. 36. That’s my Blog (please visit) UX Fest 2013 / 2013-10-01 RUM Distillation 101 29
  37. 37. Log-normal distribution UX Fest 2013 / 2013-10-01 RUM Distillation 101 30
  38. 38. Bimodal distribution UX Fest 2013 / 2013-10-01 RUM Distillation 101 31
  39. 39. Distributions • Sparse distribution suggests that you don’t have enough data points • Log-normal distribution is typical • Bi-modal distribution suggests two (or more) distributions combined UX Fest 2013 / 2013-10-01 RUM Distillation 101 32
  40. 40. Distributions are interesting, but how does it affect engagement? UX Fest 2013 / 2013-10-01 RUM Distillation 101 33
  41. 41. Did you know that user patience wrt performance is a cultural thing? UX Fest 2013 / 2013-10-01 RUM Distillation 101 34
  42. 42. It’s tied both to where a user is from, and also the type of service. UX Fest 2013 / 2013-10-01 RUM Distillation 101 35
  43. 43. different types of sites UX Fest 2013 / 2013-10-01 RUM Distillation 101 36
  44. 44. We borrowed a biological term called LD50 and applied it to Load Time v/s Bounce Rate UX Fest 2013 / 2013-10-01 RUM Distillation 101 37
  45. 45. LD50 UX Fest 2013 / 2013-10-01 RUM Distillation 101 38
  46. 46. This helps you fix only the stuff that’s hurting you. UX Fest 2013 / 2013-10-01 RUM Distillation 101 39
  47. 47. Did you know that users bounce if a page is too fast? UX Fest 2013 / 2013-10-01 RUM Distillation 101 40
  48. 48. But that’s mostly because error pages are the fastest. UX Fest 2013 / 2013-10-01 RUM Distillation 101 41
  49. 49. Measure → Hypothesize → Test → Fix ← Repeat ← UX Fest 2013 / 2013-10-01 RUM Distillation 101 42
  50. 50. What delighted users a few years ago is now an expected baseline, the absence of which will frustrate them. UX Fest 2013 / 2013-10-01 RUM Distillation 101 43
  51. 51. Changing Expectations UX Fest 2013 / 2013-10-01 RUM Distillation 101 44
  52. 52. Thank You! Questions? UX Fest 2013 / 2013-10-01 RUM Distillation 101 45
  53. 53. • Philip Tellis • @bluesmoon • philip@bluesmoon.info • www.SOASTA.com • boomerang • LogNormal Blog UX Fest 2013 / 2013-10-01 RUM Distillation 101 46
  54. 54. Image Credits • Jean-Antoine Nollet http://en.wikipedia.org/wiki/Jean-Antoine_Nollet • New York Times Olympic Medal Winners http://www.nytimes.com/interactive/2012/08/05/sports/olympics/the- 100-meter-dash-one-race-every-medalist-ever.html UX Fest 2013 / 2013-10-01 RUM Distillation 101 47
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×