Your SlideShare is downloading. ×
Rum for Breakfast
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Rum for Breakfast

2,495
views

Published on

Real user monitoring is one of the best ways of learning “the truth” about what visitors experience on your web site, but it comes at a cost. The real world is messy and noisy making it hard to know …

Real user monitoring is one of the best ways of learning “the truth” about what visitors experience on your web site, but it comes at a cost. The real world is messy and noisy making it hard to know exactly what’s going on. Filtering your data, splitting it along multiple dimensions, and determining what to discard are important second steps on the path to insightful RUM analysis, and in this session, we’ll go into some of the details.

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,495
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Velocity 2012 / 2012-06-26 RUM for Breakfast 1
  • 2. RUM for BreakfastBuddy Brewer, Carlos Bueno, Philip Tellis Velocity 2012 / 2012-06-26Velocity 2012 / 2012-06-26 RUM for Breakfast 2
  • 3. Real Users Velocity 2012 / 2012-06-26 RUM for Breakfast 3
  • 4. Real Browsers Velocity 2012 / 2012-06-26 RUM for Breakfast 4
  • 5. https://github.com/lognormal/boomerang/ Velocity 2012 / 2012-06-26 RUM for Breakfast 5
  • 6. Carlos built the dns plugin Velocity 2012 / 2012-06-26 RUM for Breakfast 6
  • 7. Buddy built the navtiming plugin Velocity 2012 / 2012-06-26 RUM for Breakfast 7
  • 8. tl;dr 1 Measure a bunch of stuff in the browser 2 Use high school stats that we vaguely remember 3 Randomly invent insights Velocity 2012 / 2012-06-26 RUM for Breakfast 8
  • 9. 1 MeasureVelocity 2012 / 2012-06-26 RUM for Breakfast 9
  • 10. 2 AnalyzeVelocity 2012 / 2012-06-26 RUM for Breakfast 10
  • 11. Log-Normal Distribution Velocity 2012 / 2012-06-26 RUM for Breakfast 11
  • 12. Log-Normal Distribution The logarithm of the x-axis follows a Normal distribution Velocity 2012 / 2012-06-26 RUM for Breakfast 11
  • 13. Log-Normal Distribution Use the Geometric Mean for pure Log-Normal distributions Velocity 2012 / 2012-06-26 RUM for Breakfast 12
  • 14. Log-Normal Distribution Performance data does not always follow a "pure" Log-Normal distribution Velocity 2012 / 2012-06-26 RUM for Breakfast 13
  • 15. Look at the entire spread ...Velocity 2012 / 2012-06-26 RUM for Breakfast 14
  • 16. Look at the entire spreadwhich often approaches an infinite widthVelocity 2012 / 2012-06-26 RUM for Breakfast 14
  • 17. Distill Velocity 2012 / 2012-06-26 RUM for Breakfast 15
  • 18. • 0.8% of hits are fake/abusive• 0.2-0.5% of hits are from a stale cache• 0.1% of hits are absurd• Timestamps in the future (or past depending on how you interpret it)• Bots ignore robots.txt across domains• "Interesting" caches/copies Velocity 2012 / 2012-06-26 RUM for Breakfast 16
  • 19. Even with beacons, you need to sanitize your input Velocity 2012 / 2012-06-26 RUM for Breakfast 17
  • 20. Band-pass filtering Velocity 2012 / 2012-06-26 RUM for Breakfast 18
  • 21. Band-pass filtering • Strip everything outside a reasonable range • Bandwidth range: 4kbps - 4Gbps • Page load time: 0ms - 600s • You may need to relook at the ranges all the time Velocity 2012 / 2012-06-26 RUM for Breakfast 18
  • 22. IQR filtering Velocity 2012 / 2012-06-26 RUM for Breakfast 19
  • 23. IQR filtering Derive the range from the data Velocity 2012 / 2012-06-26 RUM for Breakfast 19
  • 24. Sampling Velocity 2012 / 2012-06-26 RUM for Breakfast 20
  • 25. Margin of Error σ ±1.96 √n Velocity 2012 / 2012-06-26 RUM for Breakfast 21
  • 26. MoE & Sample size There is an inverse square root correlation between sample size and margin of error Velocity 2012 / 2012-06-26 RUM for Breakfast 22
  • 27. How big a sample is representative? Select nsuch that σ 1.96 √n ≤ 5%µ Velocity 2012 / 2012-06-26 RUM for Breakfast 23
  • 28. This needs to be at your lowest drilldown level Velocity 2012 / 2012-06-26 RUM for Breakfast 24
  • 29. 3 InsightVelocity 2012 / 2012-06-26 RUM for Breakfast 25
  • 30. How does performanceimpact human behavior?
  • 31. 8 million pages1.5 million visits50 different dimensions
  • 32. very fast sessions had high bounce rates70.00%52.50%35.00%17.50% 0% 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
  • 33. bounce rate vs. load time70.00%52.50%35.00%17.50% 0% 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
  • 34. bounce rate vs. DOM interactive70.00%52.50%35.00%17.50% 0% 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.5 9 9.5 10 10.5 11 11.5 12 12.5
  • 35. bounce rate vs. front end time80.00%60.00%40.00%20.00% 0% 0.5 2 3.5 5 6.5 8 9.5 11 12.5 14 15.5 17 18.5 20 21.5 23 24.5 26 27.5 29
  • 36. is my web site performance toxic to my users?http://www.flickr.com/photos/21560098@N06/3796822070
  • 37. LD50 - when do half the users bounce?http://www.flickr.com/photos/thecosmopolitan/6117530924
  • 38. Bounce rate =50% Back end time 1.7 sec DOM Loading 1.8 secDOM Interactive 2.75 secFront end time 3.5 secDOM Complete 4.75 sec Load event 5.5 sec
  • 39. Future directionsWhat is the LD50 for your site?Other bounce rates? 40%? 30%?Other variables? (critical contentvisible, etc)Other behaviors? Conversions,revenue, pages per session, actions,when do people make tea?
  • 40. Thank youVelocity 2012 / 2012-06-26 RUM for Breakfast 26
  • 41. Questions?Buddy Brewer @bbrewer Carlos Bueno @archivdPhilip Tellis @bluesmoon
  • 42. Photo credits • Rum on Ice – wiserbailey on flickr • Laptop-top Cat – wabisabi2015 on flickr • About NCSA Mosaic – ncsa • Distilled – Lost Albatross on flickr • Anthon Berg Chocolates – ulterior epicure on flickr • KilroySchematic – on WikiPedia • Boxplot vs PDF – on WikiPedia Velocity 2012 / 2012-06-26 RUM for Breakfast 27