Measuring the web with Boomerang (YUIConf 2010)

Introduction
How does it work?
Using boomerang
Data
Measuring the web with boomerang
Philip Tellis / philip@bluesmoon.info
YUIConf 2010 – 2010-11-08
YUIConf 2010 – 2010-11-08 Measuring the web with boomerang

Introduction
How does it work?
Using boomerang
Data
$ ﬁnger philip
Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paranoid - speedfreak
yahoo
http://bluesmoon.info/

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Less than 20% of page load time is something we can measure
and ﬁx during development

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
It’s what we can’t control that bites us

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
browsers

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
plugins

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
OSes

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
viruses

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
antiviruses

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
microwaves

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
baby monitors

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
naughty neighbours

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
ﬁle shares

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
governments

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
rodents

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
Try simulating all that in the lab!

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
We need to measure real end-user performance

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
We need to measure real end-user performance from the real
end-user’s box

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
While this might work, it isn’t necessarily representative

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
What about JavaScript?

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
boomerang is...
A piece of javascript that you add to your web page where it
measures and beacons back to you, the end user’s perceived
performance of your page

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
How?
<script src="boomerang.js" type="text/javascript">
</script>
<script type="text/javascript">
BOOMR.init({
user_ip: "<user’s ip address>",
beacon_url: "http://mysite.com/beacon.php"
});
</script>

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
And soon...
YUI.use(’gallery-boomr’, function(Y) {
Y.BOOMR.init({
});
});

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
What does it do?
Measures user’s bandwidth and latency to your server
Measures the current page’s load time
Beacons these results back to your server

Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Let’s fork this talk
Do you want to:
Lean how boomerang works internally?
Learn how to use boomerang?

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
How does boomerang work?

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Let’s take that one at a time

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Measuring latency
Download a 32 byte gif 10 times in sequence
Measure the time to download each
Discard the ﬁrst measurement because it’s overpriced
Calculate the arithmetic mean, standard deviation and
margin of error of the rest

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Wait, did you say overpriced?
The ﬁrst image might require a DNS lookup and TCP
handshake
Slow start is not an issue since 32 bytes ﬁts in 1 packet

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Measuring bandwidth
After the latency test is done, we download progressively
larger images
Stop at the ﬁrst image that times out
Redownload that image a few more times
Calculate the median, standard deviation and margin of
error of the largest images

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Measuring latency before bandwidth helps here
Those 10 latency images do a lot to widen the TCP
window size
The bandwidth images make much better use of bandwidth
The image we end with uses the most bandwidth

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
How do we measure page load time?
In the onbeforeunload event, measure the time and
store it in a cookie
In the onload event, check the cookie, and measure the
difference with the current time
We also make sure that the page that set the cookie is the
referrer of the current page

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
What? Two pages?
Yes, this needs two pages and cookies. If those aren’t
supported, we try to use the WebTiming API.

Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
How accurate is it?
Latency measurements are very accurate (±1%)
Bandwidth is to an order of magnitude. For bad
connections can be ±30%
Page load time sometimes has outliers, you need
post-ﬁltering
The margin of error tells you how good your data is

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Include it on your page
<script src="boomerang.js" type="text/javascript">
</script>
<script type="text/javascript">
BOOMR.init({
});
</script>

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
For most sites, that’s about it

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
You probably want to do more

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Handle subdomains
Set the site_domain parameter:
BOOMR.init({
beacon_url: "http://mysite.com/beacon.php",
site_domain: "mysite.com"
});

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
This lets you measure transitions across *.mysite.com

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Measure more than just load time
<html><head>
<script>var t_pagestart=new Date().getTime();</scrip
...
<script>var th=new Date().getTime();</script>
</head>
<body>
...
<script>var tj=new Date().getTime();</script>
...

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
...
<script src="boomerang.js"></script>
...
var te=new Date().getTime();
BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart).
setTimer("t_body", te-th).
setTimer("t_js", te-tj);
</script></body></html>

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
This adds the t_head, t_body and t_js ﬁelds to the beacon

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Loading dynamic content
BOOMR.init({
beacon_url: "http://mysite.com/beacon.php",
auto_run: false
});

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
// Just before download starts
BOOMR.plugins.RT.startTimer("t_done");

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
// Just after download finishes
BOOMR.plugins.RT.done();

Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Much more
http://yahoo.github.com/boomerang/doc/howtos/

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
The beacon
A simple GET request to the beacon
All parameters passed in the query string
Extra timers are passed in as a comma separated list in
t_other
before_beacon JavaScript event ﬁred just before the
beacon is sent

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
What should we do with the data?
Sanity checking to:
Remove fake data
Remove abusive data
Maybe just rate limiting

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
What can we do with the data?
Statistical analysis to:
Remove outliers
Aggregate based on bandwidth blocks
Measure trends over time and correlate them with code
changes

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
Bandwidth blocks
Ref: Analysing Bandwidth & Latency – YUI Blog

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
Bandwidth blocks
Data points from some countries may require narrower bands

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
Geographic data
Looking at latency from different geographic locations can tell
you where your next mirror should be

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
ISPs
Grouping data by ISP can tell you who’s behaving badly

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
More data
Write plugins to get more performance data
We already have a DNS plugin
I’m thinking of an IPv6 v/s IPv4 plugin
What about a full WebTiming plugin?
Can we measure connection setup time?

Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
You decide
Once you have the data, you can do anything with it

Introduction
How does it work?
Using boomerang
Data
Thank you
http://github.com/yahoo/boomerang
http://yahoo.github.com/boomerang/doc/

Introduction
How does it work?
Using boomerang
Data
Photo credits
ﬂickr.com/photos/21233184@N02/4389412851

Introduction
How does it work?
Using boomerang
Data
Contact me
Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paranoid - speedfreak
yahoo
http://bluesmoon.info/
slideshare.net/bluesmoon

Introduction
How does it work?
Using boomerang
Data
References
github.com/yahoo/boomerang
More bandwidth doesn’t matter (much) – Mike Belshe
Analysing Bandwidth & Latency – YUI Blog
It’s the latency, stupid – Stuart Cheshire
The statistics of web performance

Measuring the web with Boomerang (YUIConf 2010)

Recommended

Recommended

More Related Content

Similar to Measuring the web with Boomerang (YUIConf 2010)

Similar to Measuring the web with Boomerang (YUIConf 2010) (20)

More from Philip Tellis

More from Philip Tellis (20)

Recently uploaded

Recently uploaded (20)

Measuring the web with Boomerang (YUIConf 2010)