Boomerang at FOSS.IN/2010

Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data

Measuring the web with boomerang

Philip Tellis / philip@bluesmoon.info

FOSS.IN/2010 – 2010.12.15

FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang

Introduction
How does it work?
Using boomerang
Data

$ ﬁnger philip

Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paranoid - speedfreak
yahoo
http://bluesmoon.info/


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Less than 20% of page load time is something we can measure
and ﬁx during development


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

It’s what we can’t control that bites us


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

browsers


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

plugins


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

OSes


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

viruses


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

antiviruses


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

microwaves


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

baby monitors


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

naughty neighbours


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

ﬁle shares


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

governments


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

rodents


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

Too many variations

Try simulating all that in the lab!


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

We need to measure real end-user performance


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

We need to measure real end-user performance from the real
end-user’s box


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

While this might work, it isn’t necessarily representative


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

What about JavaScript?


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

boomerang is...
A piece of javascript that you add to your web page where it
measures and beacons back to you, the end user’s perceived
performance of your page


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

How?

<script src="boomerang.js" type="text/javascript">
</script>
<script type="text/javascript">
BOOMR.init({
user_ip: "<user’s ip address>",
beacon_url: "http://mysite.com/beacon.php"
});
</script>


Introduction
Time
How does it work?
The adversary
Measure twice
Using boomerang
boomerang
Data

What does it do?

Measures user’s network throughput and latency to your
server
Measures the current page’s load time
Beacons these results back to your server


Introduction
Latency
How does it work?
Bandwidth/Throughput
Load time
Using boomerang
Accuracy
Data

How does boomerang work?


Introduction
Latency
How does it work?
Load time
Using boomerang
Accuracy
Data

Let’s take that one at a time


Introduction
Latency
How does it work?
Load time
Using boomerang
Accuracy
Data

Measuring latency

Download a 32 byte gif 10 times in sequence
Measure the time to download each
Discard the ﬁrst measurement because it’s overpriced
Calculate the arithmetic mean, standard deviation and
margin of error of the rest


Introduction
Latency
How does it work?
Load time
Using boomerang
Accuracy
Data

Wait, did you say overpriced?

The ﬁrst image might require a DNS lookup and TCP
handshake
Slow start is not an issue since 32 bytes ﬁt in 1 packet


Introduction
Latency
How does it work?
Load time
Using boomerang
Accuracy
Data

Measuring throughput

After the latency test is done, we download progressively
larger images
Stop at the ﬁrst image that times out
Redownload that image a few more times
Calculate the median, standard deviation and margin of
error of the largest images


Introduction
Latency
How does it work?
Load time
Using boomerang
Accuracy
Data

Measuring latency before throughput helps here

Those 10 latency images do a lot to widen the TCP
window size
The bandwidth images make much better use of available
bandwidth
The image we end with uses the most bandwidth


Introduction
Latency
How does it work?
Load time
Using boomerang
Accuracy
Data

How do we measure page load time?

In the onbeforeunload event, measure the time and
store it in a cookie
In the onload event, check the cookie, and measure the
difference with the current time
We also make sure that the page that set the cookie is the
referrer of the current page


Introduction
Latency
How does it work?
Load time
Using boomerang
Accuracy
Data

What? Two pages?

Yes, this needs two pages and cookies. If those aren’t
supported, we try to use the WebTiming API.


Introduction
Latency
How does it work?
Load time
Using boomerang
Accuracy
Data

How accurate is it?

Latency measurements are very accurate (±1%)
Bandwidth is to an order of magnitude. For bad
connections can be ±30%
Page load time sometimes has outliers, you need
post-ﬁltering
The margin of error tells you how good your data is


Introduction
How does it work?
Using boomerang
Data

Github

https://github.com/yahoo/boomerang/


Introduction
How does it work?
Using boomerang
Data

Get the code


Introduction
How does it work?
Using boomerang
Data

Get the code

git clone
git@github.com:yourname/boomerang.git


Introduction
How does it work?
Using boomerang
Data

Submit code


Introduction
How does it work?
Using boomerang
Data

boomerang.js


Introduction
How does it work?
Using boomerang
Data

plugins

cp plugin.js yourplugin.js


Introduction
How does it work?
Using boomerang
Data

The Makeﬁle

make PLUGINS="plugin list" MINIFIER="/minifier/path"

make usage


Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data

Include it on your page

<script src="boomerang.js" type="text/javascript">
</script>
<script type="text/javascript">
BOOMR.init({
beacon_url: "http://mysite.com/beacon.php"
});
</script>


Introduction Basic
Data

For most sites, that’s about it


Introduction Basic
Data

You probably want to do more


Introduction Basic
Data

Handle subdomains

Set the site_domain parameter:

BOOMR.init({
beacon_url: "http://mysite.com/beacon.php",
site_domain: "mysite.com"
});


Introduction Basic
Data

This lets you measure transitions across *.mysite.com


Introduction Basic
Data

Measure more than just load time

<html><head>
<script>var t_pagestart=new Date().getTime();</scrip
...
<script>var th=new Date().getTime();</script>
</head>
<body>
...
<script>var tj=new Date().getTime();</script>
...


Introduction Basic
Data

...
<script src="boomerang.js"></script>
...
var te=new Date().getTime();
BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart).
setTimer("t_body", te-th).
setTimer("t_js", te-tj);
</script></body></html>


Introduction Basic
Data

This adds the t_head, t_body and t_js ﬁelds to the beacon


Introduction Basic
Data

Loading dynamic content

BOOMR.init({
beacon_url: "http://mysite.com/beacon.php",
auto_run: false
});


Introduction Basic
Data


// Just before download starts
BOOMR.plugins.RT.startTimer("t_done");


Introduction Basic
Data


// Just after download finishes
BOOMR.plugins.RT.done();


Introduction Basic
Data

Much more

http://yahoo.github.com/boomerang/doc/howtos/


Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data

The beacon

GET request to the beacon URL (response ignored)
All parameters passed in the query string
Extra timers are passed in as a comma separated list in
t_other
before_beacon JavaScript event ﬁred just before the
beacon is sent


Introduction
Data

What should we do with the data?

Sanity checking to:
Remove fake data
Remove abusive data
Maybe just rate limiting


Introduction
Data

What can we do with the data?

Statistical analysis to:
Remove outliers
Aggregate based on bandwidth blocks
Measure trends over time and correlate them with code
changes


Introduction
Data

Bandwidth blocks

Ref: Analysing Bandwidth & Latency – YUI Blog


Introduction
Data

Bandwidth blocks

Data points from some countries may require narrower bands


Introduction
Data

Geographic data

Looking at latency from different geographic locations can tell
you where your next mirror should be


Introduction
Data

ISPs

Grouping data by ISP can tell you who’s behaving badly


Introduction
Data

More data

Write plugins to get more performance data
We already have a DNS plugin
I’m thinking of an IPv6 v/s IPv4 plugin
What about a full WebTiming plugin?
Can we measure connection setup time?


Introduction
Data

You decide

Once you have the data, you can do anything with it


Introduction
How does it work?
Using boomerang
Data

Thank you
http://github.com/yahoo/boomerang
http://yahoo.github.com/boomerang/doc/


Introduction
How does it work?
Using boomerang
Data

Photo credits

ﬂickr.com/photos/21233184@N02/4389412851


Introduction
How does it work?
Using boomerang
Data

Contact me

Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paranoid - speedfreak
yahoo
http://bluesmoon.info/
slideshare.net/bluesmoon


Introduction
How does it work?
Using boomerang
Data

References

github.com/yahoo/boomerang
More bandwidth doesn’t matter (much) – Mike Belshe
Analysing Bandwidth & Latency – YUI Blog
It’s the latency, stupid – Stuart Cheshire
The statistics of web performance


Boomerang at FOSS.IN/2010

Recommended

Recommended

More Related Content

More from Philip Tellis

More from Philip Tellis (20)

Recently uploaded

Recently uploaded (20)

Boomerang at FOSS.IN/2010