• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Rum for Breakfast
 

Rum for Breakfast

on

  • 2,914 views

Real user monitoring is one of the best ways of learning “the truth” about what visitors experience on your web site, but it comes at a cost. The real world is messy and noisy making it hard to ...

Real user monitoring is one of the best ways of learning “the truth” about what visitors experience on your web site, but it comes at a cost. The real world is messy and noisy making it hard to know exactly what’s going on. Filtering your data, splitting it along multiple dimensions, and determining what to discard are important second steps on the path to insightful RUM analysis, and in this session, we’ll go into some of the details.

Statistics

Views

Total Views
2,914
Views on SlideShare
2,852
Embed Views
62

Actions

Likes
0
Downloads
21
Comments
0

3 Embeds 62

http://lanyrd.com 45
http://eventifier.co 16
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Rum for Breakfast Rum for Breakfast Presentation Transcript

    • Velocity 2012 / 2012-06-26 RUM for Breakfast 1
    • RUM for BreakfastBuddy Brewer, Carlos Bueno, Philip Tellis Velocity 2012 / 2012-06-26Velocity 2012 / 2012-06-26 RUM for Breakfast 2
    • Real Users Velocity 2012 / 2012-06-26 RUM for Breakfast 3
    • Real Browsers Velocity 2012 / 2012-06-26 RUM for Breakfast 4
    • https://github.com/lognormal/boomerang/ Velocity 2012 / 2012-06-26 RUM for Breakfast 5
    • Carlos built the dns plugin Velocity 2012 / 2012-06-26 RUM for Breakfast 6
    • Buddy built the navtiming plugin Velocity 2012 / 2012-06-26 RUM for Breakfast 7
    • tl;dr 1 Measure a bunch of stuff in the browser 2 Use high school stats that we vaguely remember 3 Randomly invent insights Velocity 2012 / 2012-06-26 RUM for Breakfast 8
    • 1 MeasureVelocity 2012 / 2012-06-26 RUM for Breakfast 9
    • 2 AnalyzeVelocity 2012 / 2012-06-26 RUM for Breakfast 10
    • Log-Normal Distribution Velocity 2012 / 2012-06-26 RUM for Breakfast 11
    • Log-Normal Distribution The logarithm of the x-axis follows a Normal distribution Velocity 2012 / 2012-06-26 RUM for Breakfast 11
    • Log-Normal Distribution Use the Geometric Mean for pure Log-Normal distributions Velocity 2012 / 2012-06-26 RUM for Breakfast 12
    • Log-Normal Distribution Performance data does not always follow a "pure" Log-Normal distribution Velocity 2012 / 2012-06-26 RUM for Breakfast 13
    • Look at the entire spread ...Velocity 2012 / 2012-06-26 RUM for Breakfast 14
    • Look at the entire spreadwhich often approaches an infinite widthVelocity 2012 / 2012-06-26 RUM for Breakfast 14
    • Distill Velocity 2012 / 2012-06-26 RUM for Breakfast 15
    • • 0.8% of hits are fake/abusive• 0.2-0.5% of hits are from a stale cache• 0.1% of hits are absurd• Timestamps in the future (or past depending on how you interpret it)• Bots ignore robots.txt across domains• "Interesting" caches/copies Velocity 2012 / 2012-06-26 RUM for Breakfast 16
    • Even with beacons, you need to sanitize your input Velocity 2012 / 2012-06-26 RUM for Breakfast 17
    • Band-pass filtering Velocity 2012 / 2012-06-26 RUM for Breakfast 18
    • Band-pass filtering • Strip everything outside a reasonable range • Bandwidth range: 4kbps - 4Gbps • Page load time: 0ms - 600s • You may need to relook at the ranges all the time Velocity 2012 / 2012-06-26 RUM for Breakfast 18
    • IQR filtering Velocity 2012 / 2012-06-26 RUM for Breakfast 19
    • IQR filtering Derive the range from the data Velocity 2012 / 2012-06-26 RUM for Breakfast 19
    • Sampling Velocity 2012 / 2012-06-26 RUM for Breakfast 20
    • Margin of Error σ ±1.96 √n Velocity 2012 / 2012-06-26 RUM for Breakfast 21
    • MoE & Sample size There is an inverse square root correlation between sample size and margin of error Velocity 2012 / 2012-06-26 RUM for Breakfast 22
    • How big a sample is representative? ￿ Select n￿such that ￿ σ ￿ ￿1.96 √n ￿ ≤ 5%µ Velocity 2012 / 2012-06-26 RUM for Breakfast 23
    • This needs to be at your lowest drilldown level Velocity 2012 / 2012-06-26 RUM for Breakfast 24
    • 3 InsightVelocity 2012 / 2012-06-26 RUM for Breakfast 25
    • How does performanceimpact human behavior?
    • 8 million pages1.5 million visits50 different dimensions
    • very fast sessions had high bounce rates70.00%52.50%35.00%17.50% 0% 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
    • bounce rate vs. load time70.00%52.50%35.00%17.50% 0% 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
    • bounce rate vs. DOM interactive70.00%52.50%35.00%17.50% 0% 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.5 9 9.5 10 10.5 11 11.5 12 12.5
    • bounce rate vs. front end time80.00%60.00%40.00%20.00% 0% 0.5 2 3.5 5 6.5 8 9.5 11 12.5 14 15.5 17 18.5 20 21.5 23 24.5 26 27.5 29
    • is my web site performance toxic to my users?http://www.flickr.com/photos/21560098@N06/3796822070
    • LD50 - when do half the users bounce?http://www.flickr.com/photos/thecosmopolitan/6117530924
    • Bounce rate >=50% Back end time 1.7 sec DOM Loading 1.8 secDOM Interactive 2.75 secFront end time 3.5 secDOM Complete 4.75 sec Load event 5.5 sec
    • Future directionsWhat is the LD50 for your site?Other bounce rates? 40%? 30%?Other variables? (critical contentvisible, etc)Other behaviors? Conversions,revenue, pages per session, actions,when do people make tea?
    • Thank youVelocity 2012 / 2012-06-26 RUM for Breakfast 26
    • Questions?Buddy Brewer @bbrewer Carlos Bueno @archivdPhilip Tellis @bluesmoon
    • Photo credits • Rum on Ice – wiserbailey on flickr • Laptop-top Cat – wabisabi2015 on flickr • About NCSA Mosaic – ncsa • Distilled – Lost Albatross on flickr • Anthon Berg Chocolates – ulterior epicure on flickr • KilroySchematic – on WikiPedia • Boxplot vs PDF – on WikiPedia Velocity 2012 / 2012-06-26 RUM for Breakfast 27