DIY Synthetic: Private WebPagetest Magic
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

DIY Synthetic: Private WebPagetest Magic

on

  • 1,636 views

I gave this talk at Velocity NY 2013, and it covers the installation and use of a private WebPagetest instance, as well as the open source tool wpt-script which Etsy built.

I gave this talk at Velocity NY 2013, and it covers the installation and use of a private WebPagetest instance, as well as the open source tool wpt-script which Etsy built.

Statistics

Views

Total Views
1,636
Views on SlideShare
1,630
Embed Views
6

Actions

Likes
3
Downloads
20
Comments
0

2 Embeds 6

http://jkle.in 4
https://www.elcurator.net 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

DIY Synthetic: Private WebPagetest Magic Presentation Transcript

  • 1. DIY Synthetic: Private WebPagetest Magic Velocity NY 2013 Jonathan Klein @jonathanklein Wednesday, October 16, 13
  • 2. Slides, Links jkle.in/wpt Wednesday, October 16, 13
  • 3. About Me Wednesday, October 16, 13
  • 4. About Me • Performance Engineer At Etsy Wednesday, October 16, 13
  • 5. About Me • Performance Engineer At Etsy • I write the Etsy Site Performance Reports Wednesday, October 16, 13
  • 6. About Me • Performance Engineer At Etsy • I write the Etsy Site Performance Reports • Organize Boston Web Perf Meetup Group Wednesday, October 16, 13
  • 7. Etsy Stats Wednesday, October 16, 13
  • 8. Etsy Stats • World’s largest handmade marketplace Wednesday, October 16, 13
  • 9. Etsy Stats • World’s largest handmade marketplace • 1.5 billion page views/month Wednesday, October 16, 13
  • 10. Etsy Stats • World’s largest handmade marketplace • 1.5 billion page views/month • Almost $1B in sales last year Wednesday, October 16, 13
  • 11. +160k to a page = +12% bounce rate on mobile Wednesday, October 16, 13
  • 12. Synthetic Monitoring Wednesday, October 16, 13
  • 13. Wednesday, October 16, 13
  • 14. Synthetic Monitoring Browser Instrumentation Navigation Timing API Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic Wednesday, October 16, 13 Real User Monitoring “Real User Monitoring”
  • 15. Synthetic Monitoring Browser Instrumentation Navigation Timing API Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic Wednesday, October 16, 13 Real User Monitoring “Real User Monitoring”
  • 16. Synthetic Monitoring Browser Instrumentation Navigation Timing API Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic Wednesday, October 16, 13 Real User Monitoring “Real User Monitoring”
  • 17. Synthetic Monitoring Browser Instrumentation Navigation Timing API Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic Wednesday, October 16, 13 Real User Monitoring “Real User Monitoring”
  • 18. Synthetic Monitoring Browser Instrumentation Navigation Timing API Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic Wednesday, October 16, 13 Real User Monitoring “Real User Monitoring”
  • 19. Synthetic Monitoring Browser Instrumentation Navigation Timing API Consistent trending over time Highly variable Largely in your control Last mile difficulties Great for identifying regressions Great for comparing across geographies/browsers Not super realistic Wednesday, October 16, 13 Real User Monitoring “Real User Monitoring”
  • 20. Wednesday, October 16, 13
  • 21. $$$$ Wednesday, October 16, 13
  • 22. Wednesday, October 16, 13
  • 23. Background Wednesday, October 16, 13
  • 24. Wednesday, October 16, 13
  • 25. Wednesday, October 16, 13
  • 26. Wednesday, October 16, 13
  • 27. EC2 Test Agents Wednesday, October 16, 13
  • 28. Wednesday, October 16, 13
  • 29. No Built-in Automation Wednesday, October 16, 13
  • 30. Wednesday, October 16, 13
  • 31. Wednesday, October 16, 13
  • 32. wpt-script Wednesday, October 16, 13
  • 33. Wednesday, October 16, 13
  • 34. Motivation Wednesday, October 16, 13
  • 35. Why We Use It Wednesday, October 16, 13
  • 36. Why We Use It • Wanted long term synthetic trending Wednesday, October 16, 13
  • 37. Why We Use It • Wanted long term synthetic trending • Automated tests Wednesday, October 16, 13
  • 38. Why We Use It • Wanted long term synthetic trending • Automated tests • More control Wednesday, October 16, 13
  • 39. Why We Use It • Wanted long term synthetic trending • Automated tests • More control • Low cost Wednesday, October 16, 13
  • 40. Easy Configuration Wednesday, October 16, 13
  • 41. {     "server": "http://www.yourwptinstall.com",     "pending_dir": "private",     "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"],     "urls": {         "label": "http://www.example.com",         "other_label": "http://www.anotherexample.com"     },     "graphite": "graphite.example.com",     "logging_ns": "webpagetest.private",     "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  • 42. {     "server": "http://www.yourwptinstall.com",     "pending_dir": "private",     "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"],     "urls": {         "label": "http://www.example.com",         "other_label": "http://www.anotherexample.com"     },     "graphite": "graphite.example.com",     "logging_ns": "webpagetest.private",     "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  • 43. {     "server": "http://www.yourwptinstall.com",     "pending_dir": "private",     "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"],     "urls": {         "label": "http://www.example.com",         "other_label": "http://www.anotherexample.com"     },     "graphite": "graphite.example.com",     "logging_ns": "webpagetest.private",     "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  • 44. {     "server": "http://www.yourwptinstall.com",     "pending_dir": "private",     "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"],     "urls": {         "label": "http://www.example.com",         "other_label": "http://www.anotherexample.com"     },     "graphite": "graphite.example.com",     "logging_ns": "webpagetest.private",     "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  • 45. {     "server": "http://www.yourwptinstall.com",     "pending_dir": "private",     "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"],     "urls": {         "label": "http://www.example.com",         "other_label": "http://www.anotherexample.com"     },     "graphite": "graphite.example.com",     "logging_ns": "webpagetest.private",     "splunkLog": "/var/log/webpagetest/results.log" } Wednesday, October 16, 13
  • 46. # Run Test */25 * * * * php run.php Wednesday, October 16, 13
  • 47. # Run Test */25 * * * * php run.php -c foo.conf Wednesday, October 16, 13
  • 48. # Run Test */25 * * * * php run.php -c foo.conf # Get Results * * * * * php get_results.php Wednesday, October 16, 13
  • 49. Advanced Features Wednesday, October 16, 13
  • 50. {     "server": "http://example.com",     "pending_dir": "private",     "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"],     "urls": {         "label": "http://www.example.com",         "other_label": "http://www.webpagetest.org"     },     "graphite": "graphite.example.com",     "logging_ns": "webpagetest.private",     "splunkLog": "/var/log/webpagetest/results.log",     "prepend": "Login",     "username": "someuser",     "password": "somepassword",     "run_options": {         "video": 0     } } Wednesday, October 16, 13
  • 51. {     "server": "http://example.com",     "pending_dir": "private",     "locations": ["US_East_wptdriver:Firefox", "US_East_wptdriver:Chrome"],     "urls": {         "label": "http://www.example.com",         "other_label": "http://www.webpagetest.org"     },     "graphite": "graphite.example.com",     "logging_ns": "webpagetest.private",     "splunkLog": "/var/log/webpagetest/results.log", "prepend": ["BlockThirdParty", "bypass_cdn.txt"],     "run_options": {         "video": 0     } } Wednesday, October 16, 13
  • 52. foreach ($prepends as $prepend) { if (file_exists($scripts_dir . $prepend)) { $script .= file_get_contents($scripts_dir . $prepend); } elseif (method_exists($this, $prepend)) { $script .= $this->$prepend(); } } Wednesday, October 16, 13
  • 53. bypass_cdn.txt setDns setDns setDns setDns setDns setDns setDns setDns setDns Wednesday, October 16, 13 site.etsystatic.com 38.106.64.123 img0.etsystatic.com 38.106.64.125 img1.etsystatic.com 38.106.64.125 img2.etsystatic.com 38.106.64.125 img3.etsystatic.com 38.106.64.125 ny-image0.etsy.com 38.106.64.125 ny-image1.etsy.com 38.106.64.125 ny-image2.etsy.com 38.106.64.125 ny-image3.etsy.com 38.106.64.125
  • 54. Gathering Results Wednesday, October 16, 13
  • 55. <?php // Splunk $splunkLogger = new SplunkLogger($config['splunkLog'], $logging_ns); foreach ($results as $result) { $splunkLogger->log($result); } // Graphite $grapher = new Grapher($config['graphite'], $logging_ns); $grapher->graphResults($results); Wednesday, October 16, 13
  • 56. Wednesday, October 16, 13
  • 57. Wednesday, October 16, 13
  • 58. Wednesday, October 16, 13
  • 59. https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics Wednesday, October 16, 13
  • 60. Wednesday, October 16, 13
  • 61. Wednesday, October 16, 13
  • 62. Wednesday, October 16, 13
  • 63. Disadvantages vs. Paid Tools Wednesday, October 16, 13
  • 64. Disadvantages vs. Paid Tools • Harder to set up/maintain Wednesday, October 16, 13
  • 65. Disadvantages vs. Paid Tools • Harder to set up/maintain • Functionality Wednesday, October 16, 13
  • 66. Disadvantages vs. Paid Tools • Harder to set up/maintain • Functionality • Regions Wednesday, October 16, 13
  • 67. Disadvantages vs. Paid Tools • Harder to set up/maintain • Functionality • Regions • Full page oriented Wednesday, October 16, 13
  • 68. Disadvantages vs. Paid Tools • Harder to set up/maintain • Functionality • Regions • Full page oriented • Less consistent Wednesday, October 16, 13
  • 69. Advantages vs. Paid Tools Wednesday, October 16, 13
  • 70. Advantages vs. Paid Tools • Cost Wednesday, October 16, 13
  • 71. Advantages vs. Paid Tools • Cost • Flexibility Wednesday, October 16, 13
  • 72. Advantages vs. Paid Tools • Cost • Flexibility • People know the tool Wednesday, October 16, 13
  • 73. Advantages vs. Paid Tools • Cost • Flexibility • People know the tool • Multi-page flows Wednesday, October 16, 13
  • 74. Advantages vs. Paid Tools • Cost • Flexibility • People know the tool • Multi-page flows • Scripting Wednesday, October 16, 13
  • 75. Our Use Cases Wednesday, October 16, 13
  • 76. WebPagetest Wednesday, October 16, 13 Catchpoint
  • 77. WebPagetest • Multi-page flows Wednesday, October 16, 13 Catchpoint
  • 78. WebPagetest • Multi-page flows • Scripted tests Wednesday, October 16, 13 Catchpoint
  • 79. WebPagetest • Multi-page flows • Scripted tests • High volume testing Wednesday, October 16, 13 Catchpoint
  • 80. WebPagetest • Multi-page flows • Scripted tests • High volume testing • Ad-hoc tests Wednesday, October 16, 13 Catchpoint
  • 81. WebPagetest Catchpoint • Multi-page flows • DNS • Scripted tests • High volume testing • Ad-hoc tests Wednesday, October 16, 13
  • 82. WebPagetest Catchpoint • Multi-page flows • DNS • Scripted tests • API • High volume testing • Ad-hoc tests Wednesday, October 16, 13
  • 83. WebPagetest Catchpoint • Multi-page flows • DNS • Scripted tests • API • High volume testing • Single Object Tests • Ad-hoc tests Wednesday, October 16, 13
  • 84. WebPagetest Catchpoint • Multi-page flows • DNS • Scripted tests • API • High volume testing • Single Object Tests • Ad-hoc tests • Multiple locations Wednesday, October 16, 13
  • 85. WebPagetest Catchpoint • Multi-page flows • DNS • Scripted tests • API • High volume testing • Single Object Tests • Ad-hoc tests • Multiple locations • Comparisons Wednesday, October 16, 13
  • 86. Future Work Wednesday, October 16, 13
  • 87. Improving the Process Wednesday, October 16, 13
  • 88. Improving the Process • Manage disk space Wednesday, October 16, 13
  • 89. Improving the Process • Manage disk space • Multiple regions Wednesday, October 16, 13
  • 90. Improving the Process • Manage disk space • Multiple regions • More canned scripts Wednesday, October 16, 13
  • 91. Improving the Process • Manage disk space • Multiple regions • More canned scripts • Automate upgrades Wednesday, October 16, 13
  • 92. Leveraging WebPagetest Wednesday, October 16, 13
  • 93. Leveraging WebPagetest • Front-end SPOF Wednesday, October 16, 13
  • 94. Leveraging WebPagetest • Front-end SPOF • User Timings (AFT) Wednesday, October 16, 13
  • 95. Leveraging WebPagetest • Front-end SPOF • User Timings (AFT) • Speed Index Wednesday, October 16, 13
  • 96. Wednesday, October 16, 13
  • 97. Improving WebPagetest Wednesday, October 16, 13
  • 98. WebPagetest + Vagrant Wednesday, October 16, 13
  • 99. WebPagetest + Vagrant • Install VirtualBox Wednesday, October 16, 13
  • 100. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant Wednesday, October 16, 13
  • 101. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant • https://gist.github.com/jklein/6992296 Wednesday, October 16, 13
  • 102. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant • https://gist.github.com/jklein/6992296 •vagrant up Wednesday, October 16, 13
  • 103. WebPagetest + Vagrant • Install VirtualBox • Install Vagrant • https://gist.github.com/jklein/6992296 •vagrant up • http://localhost:8080 Wednesday, October 16, 13
  • 104. Wednesday, October 16, 13
  • 105. 800MB Wednesday, October 16, 13
  • 106. webpagetest-api Wednesday, October 16, 13
  • 107. Wednesday, October 16, 13
  • 108. Both Are Good Options Wednesday, October 16, 13
  • 109. Results > Tools Wednesday, October 16, 13
  • 110. Thanks! Wednesday, October 16, 13
  • 111. Get in Touch @jonathanklein jonathan@etsy.com www.etsy.com/careers Wednesday, October 16, 13