Web Performance BootCamp 2013


Published on

These are the slides I used for my 1-day Performance Workshop at HTML5 DevConf this year.

Published in: Technology, Design
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • It’s impossible to teach you what you need to know in one day!
  • Not database or middleware or hardware!
  • Why Science?Experiment -> get numbers -> analyze -> see outcomesHypotheses, testing,…
  • There are few good books in this space.
  • I’ll stay around for questions afterwards.
  • Sources:Compuware, April 2011.“Google has started using web-site performance as a signal in its search engine rankings.” Source: Using site speed in web search ranking: http://googlewebmastercentral.blogspot.com/2010/04/using-site-speed-in-web-search-ranking.htmlSource: http://blog.yottaa.com/2010/11/secret-sauce-for-successful-web-site-web-performance-optimization-wpo
  • This diagram gives you an idea of the number of different use cases and actors across a large organization who are concerned with performance and the user experience. Every one of these people needs to know something about performance.It’s really complicated in more ways than one!
  • It does not matter how fast the system is in the data centerThere is no reason anything should be down. It could be slower, but not down.Globalization: product is ready to be used anywhere.
  • Not latency or bandwidth or anything else – response time.
  • Robust statistics, John Tukey
  • 0-2525-5050-7575-100
  • Number of Jupiter’s satellites: 63 or 67? Different sources show different number. Wikipedia: 63 or 67, depending on the article. Caltech web page was showing 63.
  • Pattern: Your project is part of the system – optimize for the overall system performance, not just what you can see, even if that means your part is less-than-perfectly optimized.
  • 10-15 minutes on this slide
  • A simple OSI mnemonic
  • Observe how Google has minimal page – optimizes T4Chinese search engines – very busy page – but: they like it so busy
  • Only the last formula matters – ignore the restT2 Should be constant right??Green: CPU load increasesRed: The response time grows, then becomes unstableAt the end, it stabilizes, but the server is not serving anybodyNon-linear response: trouble!We don’t know the point where the server response becomes non-linear
  • Layer 4 stuffNumber 3 in the time comes from differentiation
  • Response times fluctuate – they are not deterministic
  • What do you do when there’s an unmovable rock in your lawn?Mow around it!Compensate in some other part of the E2EThink outside the boxWork with ISPsProvide services where neededWork with local ISPs
  • You can do this for yourself on a smaller scale at WebPageTest.org
  • Yslow turns 5 today!
  • Competitive testing is an art in itselfTry to measure task sequences not pagesBe prepared for lots of breakageBe respectful This is difficult and time-consuming!
  • Nothing we do about performance is more important than testingEverything upfront in a disciplined way. We need to establish the positive feedback loopsThat give developers the incentive to make it fast. Testing and theService-level requirements->test-refine->Service level agreement process are the big wins here.Tools and education play a big part in this.
  • What’s missing?Where are Xpath and Xquery? RDF?This specification is a kitchen sink full of technologies for the Web. – what wg website
  • One big issue for HTML5 right now is the difference in performance between the different browsers’ implementations of the new specs.
  • Web Performance BootCamp 2013

    1. 1. Web Performance Boot Camp Daniel Austin PayPal, Inc. HTML5 DevConf Oct. 24, 2013 V 2.2
    2. 2. Overture: Goals of the Class • Provide a basic understanding of Web performance for Architects, Developers, Designers, and Engineers • Empower YOU to identify and resolve performance problems and make your pages and applications faster!!! • Demonstrate and explain how to use common tools and techniques used in our industry to solve performance problems
    3. 3. Scope of Web Performance Anything that uses HTTP Always From the End User‟s Point of View Web Request/Response Only!
    4. 4. Current State of the Art • Web performance is both an Art and a Science (but it‟s not yet Engineering) • Multiple tools and methodologies, large ad hoc, contend in the marketplace (but little of it is well-thought out or based on scientific reasoning). • Things are getting better – W3C involvement and competitive pressures, as well as better infrastructure and the influx of new users in Asia is driving more attention to performance. There‟s hope.
    5. 5. Reading List • Performance by Design - Daniel A. Menasce (Safari) • The Practical Performance Analyst - Neil J. Gunther • Elements of Networking Style - M.A. Padlipsky • High Performance Web Sites – Steve Souders
    6. 6. Tools Used in This Class • Excel (or similar spreadsheet program) • Online Testing Tools – webspeedtest.org • Desktop Testing Tools – your browser, Firebug, Visual Round Trip Analyzer (VRTA), netmon, dig, ping others • Optional: R (and Rstudio), Mathematica, SPSS
    7. 7. Class Structure Schedule • • • • • Start: 9:00 AM Break: 10:30-1:45 Lunch: 12:30-1:15 Break: 2:30-2:45 End: 4:00 PM Agenda • • • • • • Section I – What Is Performance? Section I – Performance Basics Section III – The MPPC Model Section IV – Tools & Testing Section V – HTML5 & Peformance Section VI – Mobile Devices
    8. 8. Section I What is Performance?
    9. 9. What Problem Are We Trying To Solve? • World-class response times compared to our competitors • Reliable, predictable performance for users worldwide • Efficient use of resources: cost scales linearly with traffic • Delighted users!
    10. 10. Impact of Scalability on Business • Google – 500 ms reduces traffic to sites by 20% • Yahoo! – 400 ms reduces traffic y 5-9% • Amazon – 100 ms reduces revenue by 1% • Compuware – 1 sec delay reduces conversion by 7% 10
    11. 11. Who Needs Performance Anyway? Who needs ‘Performance’ measurements anyway? ?
    12. 12. A More Rational Approach?
    13. 13. Systemic Qualities In a Nutshell “Anything you can say about a black box – from the outside” • Systemic qualities are the “ilities” – physical features of the system such as capacity, performance, and scalability • The SQs correspond to different groups of stakeholders: users, developers, operators, organizations • SQs are the best measure of the quality of the user‟s experience of the system, regardless of the feature set
    14. 14. The Four Classes of Systemic Qualities Manifest Qualities - What the users see • Usability, Performance, Reliability, Availability, Accessibility Operational Qualities - What the system operators see • Throughput, Manageability, Security, Serviceability Developmental Qualities - What developers see • Buildability, Budgetability, Planability Evolutionary Qualities - How the system changes over time • Scalability, Maintainability, Extensibility, Reusability, Portability
    15. 15. The Manifest Systemic Qualities • Usability reflects the ease with which users can accomplish their goals • Performance reflects how much little time users must wait for actions to complete • Reliability measures how often the system fails • Availability measures uptime vs. downtime • Accessibility measures the systems ability to serve users regardless of location or physical condition (including I18N and L10N)
    16. 16. Performance is a Balancing Act Performance isn’t everything; sometimes we’re called on to make choices about which systemic qualities have priority over others. Security v. performance is a common tradeoff – what would you choose?
    18. 18. Section II Performance Basics
    19. 19. Scales of Measure
    20. 20. Statistics 101
    21. 21. Comparison of Mean, Median, and Mode Comparison of mean, median and mode of two log-normal distributions with different skewness.
    22. 22. Outliers, or Why We Use the Median • A: skewed to the left • B: skewed to the right • C: symmetrical
    23. 23. Statistical Distributions • Discrete or continuous? • Mean, median, sigma, 95%? • Is it reasonable? |mean – median| <= sigma • Does it correlate?
    24. 24. Understanding the Margin of Error The margin of error is a measure of how close the results are likely to be. • Margin of error at 99% confidence = 1.29/sqrt(n) • Margin of error at 95% confidence = 0.98/sqrt(n) • Margin of error at 90% confidence = 0.82/sqrt(n) (where n is the number of sample data points)
    25. 25. 5 Number Reports A simple way of summarizing a sample • • • • Shape of the distribution Extreme values Variance Skewness Median 1st quartile 3rd quartile Minimum Maximum This is how you get a sense of the data…
    26. 26. 5 Number Reports in Excel Excel functions: Min = MIN(Data Range) Q1 = QUARTILE(Data Range, 1) Q2 = QUARTILE(Data Range, 2) Q3 = QUARTILE(Data Range, 3) Max = MAX(Data Range)
    27. 27. 5 Number Reports in R Let‟s make a 5 number report in R: http://www.r-project.org/
    28. 28. Operational Research • Developed during WWII for managing armies and supply chains • A set of rules or „laws‟ that describe the operational aspects of a system. • Useful for understanding the performance of any system • Utilization Law • Forced Flow law • Little‟s Law • Response Time Law
    29. 29. Resources and Queues Ri Queue Si Resource Si: Service time Ri: Queue Residence time i: Queue length In general, systems consist of many combined queues and resources
    30. 30. The Utilization Law • The utilization (Ui) of resource i is the fraction of time that the resource is busy. • Xi: average throughput of queue i, i.e. average number of requests that complete from queue i per unit of time • Si: average service time of a request at queue i per visit to the resource Ui = Xi * Si
    31. 31. Interactive Response time law IMAGE COURTESY PROF. RAJ JAIN
    32. 32. Hands-on: Using the Response Time Law in the Real World Let‟s say Facebook‟s Web servers can process 10K „like‟ requests/second, and the number of concurrent users is 600K. If each user waits 5s between requests, how long will each request take? R = (N/X) –Z
    33. 33. Hands-on: Using the Response Time Law in the Real World Let‟s say Facebook‟s Web servers can process 10K „like‟ requests/second, and the number of concurrent users is 600K. If each user waits 5s between requests, how long will each request take? R = (N/X) –Z = 6 * 10e5 / (10 requests/ms) -5 * 10e4ms = 6 * 10e4 – 5* 10e4 ms = 1000ms Facebook needs more servers!
    34. 34. Antipattern: Keyhole optimization Problem: Optimizing your project Antipattern: Optimizing *your* project, at the expense of everyone else! Pattern: Your project is part of the system – optimize for the overall system performance, not just what you can see, even if that means your part is lessthan-perfectly optimized.
    35. 35. Section III The MPPC Model
    36. 36. Dimensions of Performance • • • • • Geography Network location Bandwidth Transport Type Browser/Device Type • RT Varies by as much as 50% • Page Composition • Client-side rendering and execution effects (JS, CSS) • Network Transport Effects • # of Connections, CDN Use
    37. 37. Hardware and Routing
    38. 38. The OSI Stack Model
    39. 39. OSI Functionality Summary Application Presentation Access network resources Translate, compress, encrypt Session Establish, manage, terminate sessions Transport Reliable message delivery end to end Network Transmit packets from source to destination Data Link Organize bits into frames Physical Transmit bits
    40. 40. All People Seem To Need Data Processing Application All Presentation People Session Transport Seem To Network Need Data Link Data Physical Processing
    41. 41. HTTP Connection Flow Server Client Connection setup Client‟s perceived response time Request Response Handshake time Request transmission time Estimated server processing time Response transmission time The more HTTP requests & network roundtrips you require, the slower your performance will be: Images, CSS, JS, DNS lookups, Redirects, #of packets
    42. 42. The MPPC Model Of Web Performance “Multiple Parallel Persistent Connections” Request Initiation by User S HTTP Request HTTP Request t1 This entire cycle, steps 1-4, is repeated once for each external reference on the page, so for a given page the total time is: T = S Dt1 + Dt2 + Dt3 + Dt4 n+1 End User DNS/Network Resolution Page Composition n+1 Where n is the number of external page requisites. Browser Rendering Time HTTP Response t4 Payload Delivery Time HTTP Response t3 T1 T2 T3 T4 Connection Time Server Duration Transport Time Render Time t2
    43. 43. T1 – Making the Connection 1= DNS + TCP+ SSL • Typically a larger part of the E2E than expected • Highly variable • SSL is slow!
    44. 44. Why DNS Matters • Nothing happens before DNS! • User does not see anything on their page  waiting time • Homework Assignment: create a host file for yourself. Try your favorite sites without DNS! • DNS has a great impact on user‟s perceptions in HTTP applications
    45. 45. Interacting with DNS: dig time
    46. 46. T2 – The Server Duration • Let ( ) r • U = ( r)[ r W] • X =U* • Navg = ( • … so 2 r [W( r)W+1 -(W+1)( r)W+1] = Navg/X (The response time law)
    47. 47. T3 – TCP Transport Time • Single Object: 3 = Sz/R+2RTT+ idle For persistent parallel connections: 3 = (M+1)Si/Ri+[M/kNh]*3 RTTi+ idle … for 1 base HTML page with M objects, with Si bits, at bandwidth Ri, k connections per host, and Nh unique hostnames
    48. 48. T4 - What the Browser Does n i=1 4 = off(i) off = time offset to parse the HTML, JS, CSS, and establish the individual connections (to different hostnames) 4 is especially significant for mobile devices!
    49. 49. Where are the delays?
    50. 50. Bandwidth Efficiency Bmax = 1.22*(L)^1/2 * MSS/RTT
    51. 51. Bandwidth, Latency, and All That
    52. 52. More Bandwidth? Less Latency!
    53. 53. Hands-on: Testing DNS Response times We‟ll use nslookup for this exercise 1. Run 10 nslookup commands for a site (e.g. www.facebook.com) 2. Observe the response time for the DNS lookup 3. Calculate statistics for the results • 5 number report (summary) • Sketch the distribution • What can you say about the response times for DNS?
    54. 54. Antipattern: That’s Outside My Control It‟s never the case that there is „nothing you can do‟ about a performance problem. Antipattern: avoiding solving a performance issue because you think it‟s outside your control. This path leads to despair. Pattern: Compensate in some other part of the E2E. Think outside the box
    55. 55. Section IV Tools and Testing
    56. 56. Let’s Talk Tools Site Performance Services – Gomez – Keynote – AlertSite – ThousandEyes • „Wholesale‟ Testing – Statistical data for many page views under different conditions – Operational testing – Best for understanding global and network effects Page Analysis Tools – YSlow – MS Virtual RoundTrip Analyzer, HTTPWatch, Many Others – F12 in your browser • „Retail‟ Testing – One Page or App – Diagnostic – Best for functional testing
    57. 57. Commercial Testing Services • Gomez, AlertSite, and Keynote toolsets are similar in many ways • Synthetic Test Setup • Test nodes in large datacenters and/or end user‟s machines • Statistical data about response times
    58. 58. Performance Testing Locations Your Data Centers QA & Test
    59. 59. HTTP Object Model Web Page(s) Page Objects A Test is a sequence of one or more URLs for which HTTP requests will be made. (or Components) A Monitor is a set of predefined Tests to be run at specific times and places Each Page Object has 4 associated time segments, t1, t2, t3, t4
    60. 60. Desktop Tools • Methodology – DOM Crawler and Packet Sniffer – More accurate – Analyzes components – Stats view • Implements the 14 18 22 105 YSlow Rules – All browsers except IE – Mobile bookmarklet – Best tool for page analysis
    61. 61. Unix Performance Testing Tools • • • • • Ping nslookup dig traceroute netstat
    62. 62. Task-Based Performance Thinking Welcome Inbox Bulk … Inbox (B) 61.77% Read Message (D) P(A,B) = 0.5168 10.04% 66.25% Verify (F) 34.52% Welcome (A) 9.85% Bulk (C) 58.07% 64.53% 22.30% Exercise: What % of users follow the path: A->E->F->G? 34.38% 35.02% Compose 28.04% (E) Send Confirm (G) 1400ms 2200ms 3200ms
    63. 63. Testing Your Competitors for Fun & Profit
    64. 64. Stormcat: Global Performance Testing • • Cannot compare performance data out-of-region There are many global factors involved in performance: • Bandwidth • ISP • Infrastructure • Secular cycles (weeks, holidays, usage patterns) • The best approach: use the „StormCat‟ system! • Best case (Northern California high broadband @3 AM) • Worst Case (rural Indonesia on VSNL @ 2PM local) • Divide the range into 5 categories equally spaced between the best & worst: some locales will be in Cat I, some in Cat II, some in Cat III, etc.
    65. 65. Hands-on: Analyzing Waterfall Diagrams http://www.webpagetest.org • Choose a location and a browser and test: http://www.yahoo.com
    66. 66. Cached v. Uncached
    67. 67. Antipattern: Design-time Failure Performance is a design time activity! Anti-Pattern: Releasing a new or modified product without testing its performance “Bake it in up front!”
    68. 68. Section V HTML5 & Performance
    69. 69. A Federated Model for HTML Core HTML5 Canvas 2D This is XHTML 1.1 HTML Markup Web Workers HTML Media Web Sockets IndexDB Web Storage Source: Sergey Mavrody c. 2013
    70. 70. The Co-Evolution of HTML, JS, CSS, and XML XML Core XSLT XSD Xpath/XQuery Document Object Model JavaScript… JSON… Source: Sergey Mavrody c. 2013
    71. 71. Tower of Babel: A Problem We Have Yet to Solve
    72. 72. The Current Browser Landscape
    73. 73. HTML5 Performance
    74. 74. W3C Resource Timing
    75. 75. W3C Navigation Timing
    76. 76. Hands-on Exercise: Testing Performance the W3C Way • Use the Navlet: http://code.google.com/p/navlet/ Make a bookmark or favorite using the code
    77. 77. Antipattern: We’ll Be Done With This Soon Performance is an ongoing activity, not fire and forget! Antipattern: Not treating performance as a property of the system, or only testing at release time. Pattern: establishing a long-term performance management plan as part of your cycle.
    78. 78. Section VI Mobile Devices
    79. 79. The Big Picture – Mobile is Growing
    80. 80. Native Apps v. HTML5 v. Desktop • Native Apps will run ~ 5x faster than HTML5 • Roughly 10x slower than desktop • HTML5 on the mobile device can be 50x slower – 10x from the ARM chip – 5x from JavaScript
    81. 81. Mobile Apps Are Slow If you‟re designing for mobile, it‟s safe to assume you‟re going to incur 2000ms of 3G latency.
    82. 82. Slow Compared to What? Since 2009 mobile browsers went from 30x to 5x slower than desktops – Better than Moore‟s Law improvement (!) – JavaScript v. Native code ~ 5x – 4g/LTE ~ 27% faster than 3g
    83. 83. Mobile Speeds by Example
    84. 84. Users Expect More. Now.
    85. 85. Mobile JavaScript Performance • That 5x is in the code interpretation? • Typed arrays • JSON layout • DOM Manipulation • Garbage Collection
    86. 86. How 3G/4G Networks Work
    87. 87. 3G to 4G Migration
    88. 88. HTML5: New Features for Mobile http://mobilehtml5.org/
    89. 89. Delay-tolerant Application Design • Plan for offline/intermittent connectivity • Caching local content – Local storage – Don‟t be afraid to use sessions – Use HTTP Caching headers wisely • Always have failure modes built-in
    90. 90. The Right Tool For the Right Job Source: Nick Zakas
    91. 91. Best Practices for Mobile • • • • • • • Tread lightly on the JavaScript Don‟t touch the DOM! CDNs are less effective due to network challenges TTFB is not a good measure of server duration Use Web Workers for preloading Test performance on different transport types Test battery consumption!
    92. 92. 4 Takeaways on Mobile Performance Mobile HTML apps are slow compared to native apps …but it‟s not all about JavaScript Mobile networking is a big challenge …so design for delay-tolerance HTML5 is designed for Mobile …so use it (wisely)! Use the right tool for the right job …including the right design patterns for Mobile
    93. 93. Tools for Mobile Testing • Speedtest/Ookla – Variability – Characteristics of different kinds of networks • iCurl – Simple HTTP Operations on your device • HTTPWatch Basic – Look at the Waterfall – Gather detailed data along with iCurl
    94. 94. Hands-on Exercise: Testing Mobile Performance
    95. 95. Antipattern: It’s The Application Stupid! T2 (server duration ~ 35% of total E2E – More on mobile however! Antipattern: Failing to recognize that the distribution of the Mobile E2E is very different from a desktop performance profile Pattern: Carefully analyze the MPPC numbers for your site and identify the problems that need to be solved and in what order.
    96. 96. Finale – Summing Up
    97. 97. The 7 Habits of Exceptional Performance 1. 2. 3. 4. 5. 6. 7. Make Performance a Priority Test, Measure, Test Again Learn about the Tools Balance Performance with Features Track Results Over Time Set Targets Ask Questions; Check It for Yourself! Thanks to Tenni Theurer
    98. 98. Yslow Rules! • Rule 1 - Make Fewer HTTP Requests • Rule 2 - Use a Content Delivery Network • Rule 3 - Add an Expires Header • Rule 4 - Gzip Components • Rule 5 - Put Style sheets at the Top • Rule 6 - Put Scripts at the Bottom • Rule 7 - Avoid CSS Expressions • Rule 8 - Make JavaScript and CSS External • Rule 9 - Reduce DNS Lookups • Rule 10 - Minify JavaScript • Rule 11 - Avoid Redirects • Rule 12 - Remove Duplicate Scripts • Rule 13 - Configure ETags • Rule 14 - Make AJAX Cacheable Source: Stevesouders.com
    99. 99. Every Tool Has Its Place in the Universe
    100. 100. The One Number of Truth “42”
    101. 101. About:HTML “…a single user-interface to many large classes of stored information such as reports, notes, data-bases, computer documentation and on-line systems help” WorldWideWeb: Proposal for a HyperText Project Berners-Lee & Caillau, 1990
    102. 102. Theme of the Work Ultimately, Performance is about Respect.
    103. 103. Thank You! Daniel Austin PayPal, Inc. HTML5 DevConf Oct. 24, 2013 @daniel_b_austin da@x.com