Measuring the web with Boomerang (YUIConf 2010)

Introduction How does it work? Using boomerang Data Measuring the web with boomerang Philip Tellis / philip@bluesmoon.info YUIConf 2010 – 2010-11-08 YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction How does it work? Using boomerang Data $ ﬁnger philip Philip Tellis philip@bluesmoon.info @bluesmoon geek - paranoid - speedfreak yahoo http://bluesmoon.info/ YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Less than 20% of page load time is something we can measure and ﬁx during development YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang It’s what we can’t control that bites us YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations browsers YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations plugins YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations OSes YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations viruses YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations antiviruses YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations microwaves YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations baby monitors YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations naughty neighbours YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations ﬁle shares YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations governments YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations rodents YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Too many variations Try simulating all that in the lab! YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang We need to measure real end-user performance YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang We need to measure real end-user performance from the real end-user’s box YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang While this might work, it isn’t necessarily representative YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang What about JavaScript? YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang boomerang is... A piece of javascript that you add to your web page where it measures and beacons back to you, the end user’s perceived performance of your page YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang How? <script src="boomerang.js" type="text/javascript"> </script> <script type="text/javascript"> BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php" }); </script> YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang And soon... YUI.use(’gallery-boomr’, function(Y) { Y.BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php" }); }); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang What does it do? Measures user’s bandwidth and latency to your server Measures the current page’s load time Beacons these results back to your server YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Time How does it work? The adversary Using boomerang Measure twice Data boomerang Let’s fork this talk Do you want to: Lean how boomerang works internally? Learn how to use boomerang? YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy How does boomerang work? YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy Let’s take that one at a time YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy Measuring latency Download a 32 byte gif 10 times in sequence Measure the time to download each Discard the ﬁrst measurement because it’s overpriced Calculate the arithmetic mean, standard deviation and margin of error of the rest YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy Wait, did you say overpriced? The ﬁrst image might require a DNS lookup and TCP handshake Slow start is not an issue since 32 bytes ﬁts in 1 packet YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy Measuring bandwidth After the latency test is done, we download progressively larger images Stop at the ﬁrst image that times out Redownload that image a few more times Calculate the median, standard deviation and margin of error of the largest images YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy Measuring latency before bandwidth helps here Those 10 latency images do a lot to widen the TCP window size The bandwidth images make much better use of bandwidth The image we end with uses the most bandwidth YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy How do we measure page load time? In the onbeforeunload event, measure the time and store it in a cookie In the onload event, check the cookie, and measure the difference with the current time We also make sure that the page that set the cookie is the referrer of the current page YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy What? Two pages? Yes, this needs two pages and cookies. If those aren’t supported, we try to use the WebTiming API. YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction Latency How does it work? Bandwidth Using boomerang Load time Data Accuracy How accurate is it? Latency measurements are very accurate (±1%) Bandwidth is to an order of magnitude. For bad connections can be ±30% Page load time sometimes has outliers, you need post-ﬁltering The margin of error tells you how good your data is YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data Include it on your page <script src="boomerang.js" type="text/javascript"> </script> <script type="text/javascript"> BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php" }); </script> YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data For most sites, that’s about it YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data You probably want to do more YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data Handle subdomains Set the site_domain parameter: BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php", site_domain: "mysite.com" }); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data This lets you measure transitions across *.mysite.com YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data Measure more than just load time <html><head> <script>var t_pagestart=new Date().getTime();</scrip ... <script>var th=new Date().getTime();</script> </head> <body> ... <script>var tj=new Date().getTime();</script> ... YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data ... <script src="boomerang.js"></script> ... var te=new Date().getTime(); BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart). setTimer("t_body", te-th). setTimer("t_js", te-tj); </script></body></html> YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data This adds the t_head, t_body and t_js ﬁelds to the beacon YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data Loading dynamic content BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php", auto_run: false }); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data Loading dynamic content // Just before download starts BOOMR.plugins.RT.startTimer("t_done"); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data Loading dynamic content // Just after download finishes BOOMR.plugins.RT.done(); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Basic Introduction Multi-domain How does it work? In-page timers Using boomerang AJAX Data Much more http://yahoo.github.com/boomerang/doc/howtos/ YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data The beacon A simple GET request to the beacon All parameters passed in the query string Extra timers are passed in as a comma separated list in t_other before_beacon JavaScript event ﬁred just before the beacon is sent YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data What should we do with the data? Sanity checking to: Remove fake data Remove abusive data Maybe just rate limiting YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data What can we do with the data? Statistical analysis to: Remove outliers Aggregate based on bandwidth blocks Measure trends over time and correlate them with code changes YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data Bandwidth blocks Ref: Analysing Bandwidth & Latency – YUI Blog YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data Bandwidth blocks Data points from some countries may require narrower bands YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data Geographic data Looking at latency from different geographic locations can tell you where your next mirror should be YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data ISPs Grouping data by ISP can tell you who’s behaving badly YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data More data Write plugins to get more performance data We already have a DNS plugin I’m thinking of an IPv6 v/s IPv4 plugin What about a full WebTiming plugin? Can we measure connection setup time? YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction shoulda How does it work? coulda Using boomerang woulda Data You decide Once you have the data, you can do anything with it YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction How does it work? Using boomerang Data Thank you http://github.com/yahoo/boomerang http://yahoo.github.com/boomerang/doc/ YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction How does it work? Using boomerang Data Photo credits ﬂickr.com/photos/21233184@N02/4389412851 YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction How does it work? Using boomerang Data Contact me Philip Tellis philip@bluesmoon.info @bluesmoon geek - paranoid - speedfreak yahoo http://bluesmoon.info/ slideshare.net/bluesmoon YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction How does it work? Using boomerang Data References github.com/yahoo/boomerang More bandwidth doesn’t matter (much) – Mike Belshe Analysing Bandwidth & Latency – YUI Blog It’s the latency, stupid – Stuart Cheshire The statistics of web performance YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Measuring the web with Boomerang (YUIConf 2010)

by Philip Tellis

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 24

Statistics

Measuring the web with Boomerang (YUIConf 2010) Presentation Transcript