WebPagetest Power Users - Velocity 2014

6,164 views
5,992 views

Published on

Slides for my tutorial from Velocity 2014 on some of the more advanced features in WebPagetest.

Video is available on Youtube:

Part 1: http://youtu.be/6UeRMMI_IzI
Part 2: http://youtu.be/euVYHee1f1M

Published in: Technology, Design

WebPagetest Power Users - Velocity 2014

  1. 1. WebPagetest Power Users Patrick Meenan pmeenan@webpagetest.org @patmeenan
  2. 2. The Basics ● Presentation is available on slideshare ○ http://www.slideshare.net/patrickmeenan ○ Mostly contains links to everything I’ll be showing ● Video will be uploaded to YouTube ○ https://www.youtube.com/user/patmeenan ● I’ll send links and announcements of both to Twitter and Google+ ○ https://twitter.com/patmeenan - @patmeenan ○ https://plus.google.com/u/0/+PatrickMeenan/posts
  3. 3. What we’re covering 1. Multiple Runs 2. medianMetric 3. Visual Comparisons 4. Filmstrip URL Manipulation 5. Video Comparison View 6. Better Screen Shots 7. Chrome timelines 8. Mobile device vs emulation (data reduction) 9. bodies/tcpdump/cloudshark 10. Chrome Tracing 11. Images 12. Keep UA String 13. Resource blocking 14. SPOF testing 15. Filmstrip comparison debugging/overlays 16. Scripting multi-step/authentication 17. Scripting request manipulation (headers, DNS, command-line) 18. User Timing 19. Custom metrics/success criteria 20. API 21. CI - node wrapper, Travis, Jenkins, Grunt budget 22. Bulk tests (url/host substitution) 23. Results logging (splunk, logster, etc) 24. Benchmarks Using WebPagetest The API Private Instances
  4. 4. Plot Full Results ● Test with > 1 run ● Click “Plot Full Results” link below data table
  5. 5. medianMetric ● Defaults to onload (“loadTime”) ● Can be any metric by adding &medianMetric=<blah> to URL ● Valid metric names are the identifiers returned in the API
  6. 6. Visual Comparisons ● Log in ● Run Individual tests with the options you want ○ For best results, label each test run ● Go to your Test History ● Select the tests you want to compare ● Click Compare
  7. 7. Filmstrip URL Manipulation http://www.webpagetest.org/video/compare.php?tests=140609_KC_P6Z,140609_Z5_NDE
  8. 8. Filmstrip URL Manipulation http://www.webpagetest.org/video/compare.php?tests=140609_KC_P6Z,140609_Z5_NDE ● Test ID’s for the tests that are being compared ● In the order to be displayed ● Comma-delimited ● Selects the First View run with the median onload time
  9. 9. Each test can be configured 140609_KC_P6Z-l:First+View-r:3-c:0-e:7.3 ● -l: (Label) ● -r: (Run) ● -c: (Cached? 0 = First View, 1 = Repeat View) ● -e: (end time in seconds or category) ○ doc (onload) ○ full (fully loaded) ○ visual (visually complete - first time it gets to 100%) ○ all (last visual change)
  10. 10. Video Comparison View ● Get the filmstrips configured the way you want ● Click “Create Video” ● Click “View as data comparison”
  11. 11. Better Screenshots ● PNG Screen Shots ● Control JPEG compression level ● Usually API-only, but also available through hidden parameters pngss= 0/1, enables full-resolution PNG screen shots (not video frames) iq= <jpeq quality level>, sets compression level for all JPEG images (screen shots and video frames) http://www.webpagetest.org/?pngss=1&iq=75 Regular Resolution vs Improved
  12. 12. Chrome Timelines ● Advanced Settings -> Chrome -> Timeline ○ Always enabled on Android with video capture ○ Call stacks are optional (but helpful for debugging) ● Processing Breakdown link ○ top of details page ○ left of waterfall thumbnail ● Timeline CPU Time below waterfall including in filmstrip ● Timeline links to the left of the waterfall (download or view)
  13. 13. CPU Impact - www.amazon.com Nexus 5 Desktop VM Moto G Moto E 200ms ticks Filmstrip Comparison
  14. 14. CPU Impact - www.google.com/search?q=flowers Nexus 5 Desktop VM Moto G Moto E 200ms ticks Nexus 7 (2013) Filmstrip Comparison
  15. 15. Mobile Device vs Emulation - http://www.theguardian. com/uk?view=mobile Nexus 5 Desktop VM Moto G Moto E Nexus 7 (2013) 500ms ticks Filmstrip Comparison
  16. 16. Data Reduction Proxy ● Android-specific ● Only available with actual devices ● Advanced Settings -> Chrome ● Sample Test and Comparison
  17. 17. Moar Data! ● Response Bodies ○ Desktop-only (or mobile emulation), all browsers ○ Stores text resources served to test browser (HTML, CSS, JS) ○ Available in “object” tab of waterfall dialog ○ Also as download from request detail ● TCP Dump ○ Desktop and Android Mobile ○ pcap download left of waterfall ○ Sharable through cloudshark (view link) ■ Filter “http” ■ Find request ■ Analysis tools->Follow TCP Stream ■ Show only this stream
  18. 18. Chrome Tracing ● Filing a Chrome Performance bug (crbug.com)? ● Debugging a performance issue and need more detail? ○ Miss the days of writing raw assembly? yeah, it’s like that ● Provides tracing of internal browser activity ● Downloadable or viewable directly in the browser ○ WASD to zoom/pan
  19. 19. Images ● View all images link below waterfall in detail view (main test) ● Images displayed in the order they were requested ● Look for: ○ visible images loading before slideshow/lower images ○ things that should be in sprites ○ Images with lots of savings ○ PNGs that should be JPEGs ○ Anything else “odd” ○ Decide if you really need to deliver all of those images while loading ● JPEG Analysis view
  20. 20. Preserve Original UA String ● Advanced Settings -> “Preserve Original User Agent String” ● Usually Appends PTST <version> to UA string ○ Identify/filter synthetic tests ● Sometimes sites do weird things (sometimes on purpose) ● AOL blocks ads for example
  21. 21. Resource Blocking - Desktop Only ● Good for what-if testing ○ How does the site load without my javascript? ○ How does it load without ads? ● Only works for external resources ● Substring match ● “Fails Fast” - returns an immediate error ● Combine with visual comparison for great effect
  22. 22. SPOF Testing ● Good for testing impact of 3rd-party outages ● Blackhole FQDN’s only (and entirely) ○ Coming soon, “SPOF all domains except…” ● Goes directly to a Filmstrip comparison ○ Comparing Original vs SPOF ● SPOF routes given domains to blackhole.webpagetest.org ○ blackhole.webpagetest.org is just a host with a firewall rule to drop all incoming packets ● Non-Responding behaves VERY differently from Fast-Fail ● Would your monitoring detect it?
  23. 23. Test Comparison ● Waterfall overlay/slider ● Comparison Graphs
  24. 24. Scripting - Multi-step ● Simplest case is multiple distinct pages ● logData 0/1 ○ logData 0 ○ navigate www.amazon.com ○ logData 1 ○ navigate http://www.amazon.com/High-Performance-Web-Sites-Essential-ebook/dp/B0028N4WHY/ref=sr_1_2?ie=UTF8&qid=1402339606&sr=8-2&keywords=even+faster+websites ● Test with shared site resources ● vs combineSteps ○ combineSteps ○ navigate www.amazon.com ○ navigate http://www.amazon.com/High-Performance-Web-Sites-Essential-ebook/dp/B0028N4WHY/ref=sr_1_2?ie=UTF8&qid=1402339606&sr=8-2&keywords=even+faster+websites
  25. 25. Scripting - Forms ● Common use is testing pages behind an auth curtain ○ Check the “sensitive data” box to have cookies stripped ● Multiple ways to automate: ○ Automate login form ○ Re-use existing session cookie ● Script Structure navigate <landing page> xxx (not measured) xxx (not measured) xxxAndWait (measured) ● WPT depends on network activity for end state
  26. 26. Scripting - Form Commands ● setValue ● setInnerText ● setInnerHtml ● selectValue ● click, sendClick ● sendKeyDown, sendKeyUp, sendKeyPress ● submitForm ● exec
  27. 27. exec(AndWait) ● Executes arbitrary javascript in the context of the page ● Best to keep it to plain javascript (don’t depend on in-page code) ● Debug scripts with local dev tools console before submitting
  28. 28. Script - Get the Weather in Santa Clara logData 0 navigate http://www.weather.com/ exec document.getElementById("typeaheadBox").value = 95054 logData 1 execAndWaitdocument.getElementById("headerSearchForm").submit() Result - Filmstrip
  29. 29. Scripting - Request Manipulation ● setDns, setDnsName, overrideHost ● setCookie ● setHeader, addHeader ● block
  30. 30. Scripting - Misc ● setViewportSize ● sleep Full scripting docs: https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting
  31. 31. Custom Browser Settings ● Chrome ○ Custom command-line options (advanced settings) ■ Specify PAC scripts (proxy) ■ Custom host override ■ Disable SPDY ■ Toggle features ● Firefox ○ firefoxPref script command ■ Sets any firefox pref value before launching browser
  32. 32. User Timing ● performance.mark(‘label’) ● Exposed as metrics in WebPagetest results ○ In raw data ○ As purple vertical lines in waterfall ○ Last mark exposed as a unique metric ● Trivial polyfill for older browsers ● Validate RUM markers using video capture/filmstrip
  33. 33. Start Render From JS Light green line in waterfall Chrome: var startTime = window.chrome.loadTimes()['requestTime'] ? window.chrome.loadTimes()['requestTime'] : window.chrome.loadTimes()['startLoadTime']; var render = window.chrome.loadTimes().firstPaintTime - startTime; IE: var render = window.performance.timing.msFirstPaint;
  34. 34. Custom Metrics ● Runs arbitrary javascript snippet(s) at the end of the test ● Defined on a per-test basis or instance-wide ○ New HTTP Archive metrics (DOM depth, iFrame count, etc) ● Can be simple one-liners or more complex multi-function operations ● Can override existing metrics ● Can return numeric or string values ● Lots of uses: ○ Validate expected page content ○ Collect RUM beacon results ○ Separate results based on A/B or type of ad served ○ Count widgets, etc
  35. 35. Custom Metrics - Simple [iframe-count] return document.getElementsByTagName("iframe").length;
  36. 36. Custom Metrics - Validate Content [result] var result = 499; try { var copyright = document.getElementsByClassName('copy-text')[0].innerText.slice(-8); result = copyright == "AOL Inc." ? 0 : 498; } catch(e) { } return result; Changes test result to: ● 499 if the copyright block doesn’t exist ● 498 if the copyright block exists but the values don’t match ● Leaves it as a success (0) if it exists and matches
  37. 37. API https://sites.google.com/a/webpagetest.org/docs/advanced-features/webpagetest-restful-apis 1. Submit Test 2. Poll for test status 3. Get test result
  38. 38. API - Submitting a test ● API key required for public instance ○ email me (pmeenan@webpagetest.org) for key ○ keys are quite limited (~200 “page loads” per day) ○ mostly for proof of concept, early dev before deploying private instance ● Anything you can do with the UI you can do with the API ○ If a field isn’t documented, just inspect the HTML on the UI to get the field name :-) ● Make sure to encode parameters ○ urlencode if GET ○ form encode if POST
  39. 39. API Response http://www.webpagetest.org/runtest.php?url=www.aol.com&f=json { "statusCode":200, "statusText":"Ok", "data":{ "testId":"140610_FY_N2F", "ownerKey":"04121a1e5b17a59ba9ddd666ae020cf52cee4abe", "jsonUrl":"http://www.webpagetest.org/jsonResult.php?test=140610_FY_N2F", "xmlUrl":"http://www.webpagetest.org/xmlResult/140610_FY_N2F/", "userUrl":"http://www.webpagetest.org/result/140610_FY_N2F/", "summaryCSV":"http://www.webpagetest.org/result/140610_FY_N2F/page_data.csv", "detailCSV":"http://www.webpagetest.org/result/140610_FY_N2F/requests.csv" } }
  40. 40. Polling status ● Periodically poll the JSON/XML URL ● statusCode = 200 : Test Done ○ Test itself may still have failed ● statusCode = 1xx : Test pending/waiting ● statusCode = 4xx : Error locating the test
  41. 41. API test result http://www.webpagetest.org/xmlResult.php?test=140610_FY_N2F&requests=1&breakdown=1
  42. 42. Bulk testing ● Submit a bunch of tests ● Periodically check status ● Do “something interesting” with the results
  43. 43. Bulk Testing ● PHP ○ https://github.com/WPO-Foundation/webpagetest/tree/master/bulktest ● Python ○ https://github.com/WPO-Foundation/webpagetest/tree/master/batchtool ● HTTP Archive ○ http://httparchive.org/ ○ https://github.com/HTTPArchive/httparchive
  44. 44. Node API Wrapper Created by Marcel Duran (@marcelduran) https://github.com/marcelduran/webpagetest-api $ npm install webpagetest -g webpagetest test -k <key> http://twitter.com/marcelduran webpagetest status 140610_F0_T5K webpagetest results 140610_F0_T5K webpagetest test http://twitter.com/marcelduran --poll 5 --timeout 60
  45. 45. CI Integration https://github.com/marcelduran/webpagetest-api/wiki/Test-Specs Match result to provided specs { "median": { "firstView": { "requests": 20, "render": 400, "loadTime": 3000, "score_gzip": { "min": 90 } } } }
  46. 46. CI Integration cont. Uses Mocha with reporters suitable for integration with: ● Jenkins ● Travis-CI ● Drone.io ● Just about anything else you can imagine
  47. 47. grunt-perfbudget Created by Tim Kadlec (@tkadlec) https://github.com/tkadlec/grunt-perfbudget
  48. 48. Bulk Testing in the UI (Private Instances) ● Submit list of URLs to test ● UI options apply to each test ● Scripts can be run against each URL ○ %URL% in script gets replaced with test URL ○ %HOST% in script gets replaced with host from test URL ○ %HOSTR% in script gets replaced with host from URL after redirects ● Aggregate results ○ Download as CSV (subset of data) ○ Compare tests against each other
  49. 49. Results logging (Private Instances) ● As each test comes in, full result data is written to log files ○ Page Data (load time, Speed index, requests, custom metrics, etc) ○ Request Data (details for every individual request) ● Each test run or request is on it’s own log file line ● JSON-formatted ● Config information is included ○ Test URL ○ Run # ○ First/Repeat View ○ Test label ○ Location ○ Browser ○ Connectivity ○ Test ID ○ URL to test result
  50. 50. Results logging - integration ● Splunk ● Logster ○ Logster -> StatsD = Arbitrary metrics trending automatically ● Track individual requests across all pages ○ ads JS performance ○ CDN response times ○ Effectiveness of image compression ○ so much more
  51. 51. Benchmarks (Private Instances) ● Still pretty experimental ● Recurring tests run automatically ○ Configuration is a bit ugly but flexible ● Tests are scheduled, run and aggregated automatically ○ Top-level trended aggregate view ○ All-metrics trended aggregate view ○ Per-page trended view ○ Scatter Plot for a given run ○ Filmstrip comparisons
  52. 52. bit.ly/usingwebpagetest
  53. 53. Google Booth Wednesday 6/25 ● Lightning Talks ○ 10:45 am - Meet4Speed, Pat Meenan ○ 2:40 pm - The PageSpeed Optimization Platform - Joshua Marantz ● Meet the Speaker ○ 2:40 pm - WebPagetest Q&A, Pat Meenan Thursday, 6/26 ● Lightning Talks ○ 10:45 am The Case for Tunneling Sockets Over HTTP, Wenbo Zhu ○ 2:40 pm - Optimize For Bandwidth - the what and why, Matt Atterbury ● Meet the Speaker ○ 2:40 pm - Making the Web POSH - Q&A, Jan Maessen
  54. 54. Feedback Please! http://velocityconf.com/velocity2014/public/content/mobile-app
  55. 55. Thank You!

×