Your SlideShare is downloading. ×
0
Introduction
How does it work?
Using boomerang
Data
Measuring the web with boomerang
Philip Tellis / philip@bluesmoon.info...
Introduction
How does it work?
Using boomerang
Data
$ finger philip
Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - p...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
YUIConf 2010 – 2010-11-08 M...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
YUIConf 2010 – 2010-11-08 M...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
YUIConf 2010 – 2010-11-08 M...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Less than 20% of page load ...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
It’s what we can’t control ...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
browser...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
plugins...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
OSes
YU...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
viruses...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
antivir...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
microwa...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
baby mo...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
naughty...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
file sha...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
governm...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
rodents...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Too many variations
Try sim...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
We need to measure real end...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
We need to measure real end...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
YUIConf 2010 – 2010-11-08 M...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
While this might work, it i...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
What about JavaScript?
YUIC...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
boomerang is...
A piece of ...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
How?
<script src="boomerang...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
And soon...
YUI.use(’galler...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
What does it do?
Measures u...
Introduction
How does it work?
Using boomerang
Data
Time
The adversary
Measure twice
boomerang
Let’s fork this talk
Do you...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
How does boomerang work?
YUIConf ...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Let’s take that one at a time
YUI...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Measuring latency
Download a 32 b...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Wait, did you say overpriced?
The...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Measuring bandwidth
After the lat...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
Measuring latency before bandwidt...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
How do we measure page load time?...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
What? Two pages?
Yes, this needs ...
Introduction
How does it work?
Using boomerang
Data
Latency
Bandwidth
Load time
Accuracy
How accurate is it?
Latency measu...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Include it on your page
<script...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
For most sites, that’s about it...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
You probably want to do more
YU...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Handle subdomains
Set the site_...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
This lets you measure transitio...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Measure more than just load tim...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
...
<script src="boomerang.js">...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
This adds the t_head, t_body an...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Loading dynamic content
BOOMR.i...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Loading dynamic content
// Just...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Loading dynamic content
// Just...
Introduction
How does it work?
Using boomerang
Data
Basic
Multi-domain
In-page timers
AJAX
Much more
http://yahoo.github.c...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
The beacon
A simple GET request to the beacon
Al...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
What should we do with the data?
Sanity checking...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
What can we do with the data?
Statistical analys...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
Bandwidth blocks
Ref: Analysing Bandwidth & Late...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
Bandwidth blocks
Data points from some countries...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
Geographic data
Looking at latency from differen...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
ISPs
Grouping data by ISP can tell you who’s beh...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
More data
Write plugins to get more performance ...
Introduction
How does it work?
Using boomerang
Data
shoulda
coulda
woulda
You decide
Once you have the data, you can do an...
Introduction
How does it work?
Using boomerang
Data
Thank you
http://github.com/yahoo/boomerang
http://yahoo.github.com/bo...
Introduction
How does it work?
Using boomerang
Data
Photo credits
flickr.com/photos/21233184@N02/4389412851
YUIConf 2010 – ...
Introduction
How does it work?
Using boomerang
Data
Contact me
Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paran...
Introduction
How does it work?
Using boomerang
Data
References
github.com/yahoo/boomerang
More bandwidth doesn’t matter (m...
Upcoming SlideShare
Loading in...5
×

Measuring the web with Boomerang (YUIConf 2010)

3,050

Published on

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,050
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
52
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

Transcript of "Measuring the web with Boomerang (YUIConf 2010)"

  1. 1. Introduction How does it work? Using boomerang Data Measuring the web with boomerang Philip Tellis / philip@bluesmoon.info YUIConf 2010 – 2010-11-08 YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  2. 2. Introduction How does it work? Using boomerang Data $ finger philip Philip Tellis philip@bluesmoon.info @bluesmoon geek - paranoid - speedfreak yahoo http://bluesmoon.info/ YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  3. 3. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  4. 4. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  5. 5. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  6. 6. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Less than 20% of page load time is something we can measure and fix during development YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  7. 7. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang It’s what we can’t control that bites us YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  8. 8. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations browsers YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  9. 9. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations plugins YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  10. 10. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations OSes YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  11. 11. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations viruses YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  12. 12. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations antiviruses YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  13. 13. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations microwaves YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  14. 14. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations baby monitors YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  15. 15. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations naughty neighbours YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  16. 16. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations file shares YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  17. 17. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations governments YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  18. 18. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations rodents YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  19. 19. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Too many variations Try simulating all that in the lab! YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  20. 20. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang We need to measure real end-user performance YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  21. 21. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang We need to measure real end-user performance from the real end-user’s box YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  22. 22. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  23. 23. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang While this might work, it isn’t necessarily representative YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  24. 24. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang What about JavaScript? YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  25. 25. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang boomerang is... A piece of javascript that you add to your web page where it measures and beacons back to you, the end user’s perceived performance of your page YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  26. 26. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang How? <script src="boomerang.js" type="text/javascript"> </script> <script type="text/javascript"> BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php" }); </script> YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  27. 27. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang And soon... YUI.use(’gallery-boomr’, function(Y) { Y.BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php" }); }); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  28. 28. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang What does it do? Measures user’s bandwidth and latency to your server Measures the current page’s load time Beacons these results back to your server YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  29. 29. Introduction How does it work? Using boomerang Data Time The adversary Measure twice boomerang Let’s fork this talk Do you want to: Lean how boomerang works internally? Learn how to use boomerang? YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  30. 30. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy How does boomerang work? YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  31. 31. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy Let’s take that one at a time YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  32. 32. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy Measuring latency Download a 32 byte gif 10 times in sequence Measure the time to download each Discard the first measurement because it’s overpriced Calculate the arithmetic mean, standard deviation and margin of error of the rest YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  33. 33. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy Wait, did you say overpriced? The first image might require a DNS lookup and TCP handshake Slow start is not an issue since 32 bytes fits in 1 packet YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  34. 34. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy Measuring bandwidth After the latency test is done, we download progressively larger images Stop at the first image that times out Redownload that image a few more times Calculate the median, standard deviation and margin of error of the largest images YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  35. 35. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy Measuring latency before bandwidth helps here Those 10 latency images do a lot to widen the TCP window size The bandwidth images make much better use of bandwidth The image we end with uses the most bandwidth YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  36. 36. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy How do we measure page load time? In the onbeforeunload event, measure the time and store it in a cookie In the onload event, check the cookie, and measure the difference with the current time We also make sure that the page that set the cookie is the referrer of the current page YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  37. 37. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy What? Two pages? Yes, this needs two pages and cookies. If those aren’t supported, we try to use the WebTiming API. YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  38. 38. Introduction How does it work? Using boomerang Data Latency Bandwidth Load time Accuracy How accurate is it? Latency measurements are very accurate (±1%) Bandwidth is to an order of magnitude. For bad connections can be ±30% Page load time sometimes has outliers, you need post-filtering The margin of error tells you how good your data is YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  39. 39. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX Include it on your page <script src="boomerang.js" type="text/javascript"> </script> <script type="text/javascript"> BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php" }); </script> YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  40. 40. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX For most sites, that’s about it YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  41. 41. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX You probably want to do more YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  42. 42. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX Handle subdomains Set the site_domain parameter: BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php", site_domain: "mysite.com" }); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  43. 43. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX This lets you measure transitions across *.mysite.com YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  44. 44. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX Measure more than just load time <html><head> <script>var t_pagestart=new Date().getTime();</scrip ... <script>var th=new Date().getTime();</script> </head> <body> ... <script>var tj=new Date().getTime();</script> ... YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  45. 45. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX ... <script src="boomerang.js"></script> ... var te=new Date().getTime(); BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart). setTimer("t_body", te-th). setTimer("t_js", te-tj); </script></body></html> YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  46. 46. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX This adds the t_head, t_body and t_js fields to the beacon YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  47. 47. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX Loading dynamic content BOOMR.init({ user_ip: "<user’s ip address>", beacon_url: "http://mysite.com/beacon.php", auto_run: false }); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  48. 48. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX Loading dynamic content // Just before download starts BOOMR.plugins.RT.startTimer("t_done"); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  49. 49. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX Loading dynamic content // Just after download finishes BOOMR.plugins.RT.done(); YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  50. 50. Introduction How does it work? Using boomerang Data Basic Multi-domain In-page timers AJAX Much more http://yahoo.github.com/boomerang/doc/howtos/ YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  51. 51. Introduction How does it work? Using boomerang Data shoulda coulda woulda The beacon A simple GET request to the beacon All parameters passed in the query string Extra timers are passed in as a comma separated list in t_other before_beacon JavaScript event fired just before the beacon is sent YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  52. 52. Introduction How does it work? Using boomerang Data shoulda coulda woulda What should we do with the data? Sanity checking to: Remove fake data Remove abusive data Maybe just rate limiting YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  53. 53. Introduction How does it work? Using boomerang Data shoulda coulda woulda What can we do with the data? Statistical analysis to: Remove outliers Aggregate based on bandwidth blocks Measure trends over time and correlate them with code changes YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  54. 54. Introduction How does it work? Using boomerang Data shoulda coulda woulda Bandwidth blocks Ref: Analysing Bandwidth & Latency – YUI Blog YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  55. 55. Introduction How does it work? Using boomerang Data shoulda coulda woulda Bandwidth blocks Data points from some countries may require narrower bands YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  56. 56. Introduction How does it work? Using boomerang Data shoulda coulda woulda Geographic data Looking at latency from different geographic locations can tell you where your next mirror should be YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  57. 57. Introduction How does it work? Using boomerang Data shoulda coulda woulda ISPs Grouping data by ISP can tell you who’s behaving badly YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  58. 58. Introduction How does it work? Using boomerang Data shoulda coulda woulda More data Write plugins to get more performance data We already have a DNS plugin I’m thinking of an IPv6 v/s IPv4 plugin What about a full WebTiming plugin? Can we measure connection setup time? YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  59. 59. Introduction How does it work? Using boomerang Data shoulda coulda woulda You decide Once you have the data, you can do anything with it YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  60. 60. Introduction How does it work? Using boomerang Data Thank you http://github.com/yahoo/boomerang http://yahoo.github.com/boomerang/doc/ YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  61. 61. Introduction How does it work? Using boomerang Data Photo credits flickr.com/photos/21233184@N02/4389412851 YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  62. 62. Introduction How does it work? Using boomerang Data Contact me Philip Tellis philip@bluesmoon.info @bluesmoon geek - paranoid - speedfreak yahoo http://bluesmoon.info/ slideshare.net/bluesmoon YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  63. 63. Introduction How does it work? Using boomerang Data References github.com/yahoo/boomerang More bandwidth doesn’t matter (much) – Mike Belshe Analysing Bandwidth & Latency – YUI Blog It’s the latency, stupid – Stuart Cheshire The statistics of web performance YUIConf 2010 – 2010-11-08 Measuring the web with boomerang
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×