An unconventional loading strategy for YUI 3

11,630 views
11,571 views

Published on

This presentation outlines the unconventional YUI loading strategy employed by Yahoo! Search.

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
11,630
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

An unconventional loading strategy for YUI 3

  1. 1. YUI 3 Loading Strategies Yahoo! Search Julien Lecomte F2E Summit 2011
  2. 2. Terminology • SRP = Search Results Page • RTB = Round Trip Beacon (aka Boomerang) • YUI w/o version number will refer to YUI 3 -2- Yahoo! Confidential
  3. 3. What makes a good search results page? • Relevant results • Easy to scan quickly • Fast! -3- Yahoo! Confidential
  4. 4. The SRP, a very special page Performance is a business issue: If your web site is not responsive enough, you will lose revenue and customer loyalty! -4- Yahoo! Confidential
  5. 5. The SRP, a very special page Web Search is an extremely competitive arena, and it brings a significant amount of revenue to Yahoo! and its direct competitors. -5- Yahoo! Confidential
  6. 6. The SRP, a very special page Every millisecond and every byte counts! -6- Yahoo! Confidential
  7. 7. The SRP, a very special page Not only does the page need to be fast, it must feel fast  Perceived performance is critical! -7- Yahoo! Confidential
  8. 8. The SRP, a very special page Spinners and loading indicators are evil! -8- Yahoo! Confidential
  9. 9. The SRP, a very special page THIS IS EVIL! -9- Yahoo! Confidential
  10. 10. The SRP, a very special page THIS IS EVIL! - 10 - Yahoo! Confidential
  11. 11. The SRP, a very special page Reducing time to window.onload (without using dirty tricks*) is critical!(*) Don’t even think about lazy-loading the entire page! - 11 - Yahoo! Confidential
  12. 12. The SRP, a very special page Q: What does the overwhelming majority of users look for in a search results page? - 12 - Yahoo! Confidential
  13. 13. The SRP, a very special page A: The search results! - 13 - Yahoo! Confidential
  14. 14. The SRP, a very special page Although “fancy” features are a differentiating factor, they should not get in the way! - 14 - Yahoo! Confidential
  15. 15. The SRP, a very special page Well-designed “fancy” SRP features should: • either occupy minimal real estate footprint, • or appear only when the user needs it / wants it, • or appear towards the bottom of the page, • unless it is highly relevant, • and never slow down the page! - 15 - Yahoo! Confidential
  16. 16. Example of a “fancy” feature: “Super Wow” - 16 - Yahoo! Confidential
  17. 17. Example of a “fancy” feature: “Slide shows” - 17 - Yahoo! Confidential
  18. 18. Example of a “fancy” feature: “Search Direct” - 18 - Yahoo! Confidential
  19. 19. Example of a “fancy” feature: “Quick Apps” - 19 - Yahoo! Confidential
  20. 20. The SRP, a very special page Dynamic features require JavaScript. - 20 - Yahoo! Confidential
  21. 21. The SRP, a very special page It is unrealistic to develop complex features without a library nowadays… - 21 - Yahoo! Confidential
  22. 22. The SRP, a very special page YUI is the standard at Yahoo! - 22 - Yahoo! Confidential
  23. 23. The SRP, a very special page YUI is awesome… - 23 - Yahoo! Confidential
  24. 24. The SRP, a very special page …but it’s big!*(*) when compared to the amount of JavaScript we used to load on the SRP with YUI 2 - 24 - Yahoo! Confidential
  25. 25. The SRP, a very special page We load 70 KB compressed (gzip) / 210 KB uncompressed of JavaScript, most of which is YUI. - 25 - Yahoo! Confidential
  26. 26. The SRP, a very special page Advanced bootstrapping strategies mitigate the performance impact of loading a large library. - 26 - Yahoo! Confidential
  27. 27. Common YUI bootstrapping strategies
  28. 28. Bootstrap: YUI seed <script src="http://yui.yahooapis.com/3.3.0/build/yui/yui-min.js"></script> <script> YUI().use(node, function (Y) {...}); </script> YUI seed YUI loader “node” and dependencies…  8.30 seconds on a modem! (total time for the page to load + download of each JS file) - 28 - Yahoo! Confidential
  29. 29. Bootstrap: YUI seed + loader <script src="http://yui.yahooapis.com/combo?3.3.0/build/yui/yui- min.js&3.3.0/build/loader/loader-min.js"></script> <script> YUI().use(node, function (Y) {...}); </script> YUI seed + loader “node” and dependencies…  7.37 seconds on a modem! (total time for the page to load + download of each JS file) - 29 - Yahoo! Confidential
  30. 30. No bootstrap <script src="http://yui.yahooapis.com/combo?3.3.0/build/yui/yui-base- min.js&3.3.0/build/oop/oop-min.js&3.3.0/build/dom/dom-base- min.js&3.3.0/build/dom/selector-native-min.js&3.3.0/build/dom/selector-css2- min.js&3.3.0/build/event-custom/event-custom-base- min.js&3.3.0/build/event/event-base-min.js&3.3.0/build/pluginhost/pluginhost- min.js&3.3.0/build/dom/dom-style-min.js&3.3.0/build/dom/dom-style-ie- min.js&3.3.0/build/dom/dom-screen-min.js&3.3.0/build/node/node- min.js&3.3.0/build/event/event-base-ie-min.js&3.3.0/build/event/event-delegate- min.js"></script> <script> YUI().use(features, node, function (Y) {...}); </script> YUI seed + “node” and dependencies…  5.40 seconds on a modem! (total time for the page to load + download of each JS file) - 30 - Yahoo! Confidential
  31. 31. Lazy-load the YUI seed + loader YUI seed + loader “node” and dependencies…  7.43 seconds on a modem (total time for the page to load + download of each JS file)  “artificially” lower RTB but no overall improvement. - 31 - Yahoo! Confidential
  32. 32. Lazy-load all of the code… YUI seed + “node” and dependencies…  5.40 seconds on a modem (total time for the page to load + download of each JS file)  Same deal: “artificially” lower RTB but no overall improvement. - 32 - Yahoo! Confidential
  33. 33. Lazy-loading, the nitty gritty…window.onload = function () { var d = document, h = d.getElementsByTagName("head")[0], s = d.createElement("script"); function init () { YUI().use(node, function (Y) {...}); } s.src = ...; s.async = true; if (s.addEventListener) { s.addEventListener(load, init, false); } else { s.onreadystatechange = function () { if (s.readyState === loaded || s.readyState === complete) { s.onreadystatechange = null; init(); } }; } h.appendChild(s);}; - 33 - Yahoo! Confidential
  34. 34. YLS YLS (YUI Loader Service) is a new (awesome) way to load YUI modules. Check out Reid’s presentation: http://reid.github.com/decks/2011/bayjax/yls.html - 34 - Yahoo! Confidential
  35. 35. The YUI Loader
  36. 36. Old SRP skeleton <!doctype html> Contains following definitions: <html> YUI = ... ... Y = ... <body> ... <script src="srp-seed.js"></script> <script> Y.use(foo, function () { ... feature init code ... }); Y.use(bar, function () { ... feature init code ... }); ... </script> </body> </html> Note: we’re using a single global YUI instance (Y) - 36 - Yahoo! Confidential
  37. 37. YUI Loader – A trivial example var Y = YUI(); Y.use(json, function () { ... }); Y.use(profiler, function () { ... });  2 HTTP requests.  The profiler module will be loaded after the callback, passed to the first Y.use() call, has completed its execution! - 37 - Yahoo! Confidential
  38. 38. The problem we’re trying to solve The YUI loader is awesome, but it does not yet support parallel loading (it’s coming very soon!) - 38 - Yahoo! Confidential
  39. 39. The problem we’re trying to solve Even once the YUI loader supports parallel loading, we will still need to first load the seed, the loader and its meta-data… - 39 - Yahoo! Confidential
  40. 40. The problem we’re trying to solve That’s 16KB minified and compressed (gzip)… - 40 - Yahoo! Confidential
  41. 41. The problem we’re trying to solve …or 50KB uncompressed (15% of our traffic!) - 41 - Yahoo! Confidential
  42. 42. The problem we’re trying to solve We could lazy-load it… - 42 - Yahoo! Confidential
  43. 43. The problem we’re trying to solve …but we’re still stuck with sequentially loading seed + loader first, and then have the loader take care of the remainder of the code! - 43 - Yahoo! Confidential
  44. 44. The problem we’re trying to solve Sequential loading is evil! - 44 - Yahoo! Confidential
  45. 45. What we’re shooting for… Lazy-load the YUI seed in parallel with standard YUI modules and SRP-specific YUI modules*.(*) SRP features are implemented as YUI modules.These modules are “Y.use()’d” mostly from code output inline in the SRP. - 45 - Yahoo! Confidential
  46. 46. What we’re shooting for… YUI seed + standard YUI modules Standard YUI modules + SRP-specific YUI modules 3.17 seconds on a modem! (total time for the page to load + download of each JS file) lower RTB because everything is lazy loaded large overall improvement thanks to parallel download - 46 - Yahoo! Confidential
  47. 47. A word of warning… - 47 - Yahoo! Confidential
  48. 48. A word of warning… The tricks you are about to witness are not recommended practice! - 48 - Yahoo! Confidential
  49. 49. A word of warning… The YUI team does not support this (i.e. you’re pretty much on your own…) - 49 - Yahoo! Confidential
  50. 50. How to do this? • Split code into “bundles” of roughly similar size. • The YUI seed is included in one of these bundles. • A bundle contains several YUI modules (either standard or SRP-specific), and therefore is just a series of calls to YUI.add() • Lazy-load the bundles from onload handler. - 50 - Yahoo! Confidential
  51. 51. Huh, will that work? NO! The order in which the bundles are downloaded cannot be guaranteed i.e. YUI may be undefined when the code inside a bundle is evaluated. - 51 - Yahoo! Confidential
  52. 52. What about inline scripts? We want to make the loading process transparent to developers! - 52 - Yahoo! Confidential
  53. 53. Old SRP skeleton <!doctype html> Contains following definitions: <html> YUI = ... ... Y = ... <body> ... <script src="srp-seed.js"></script> <script> Y.use(foo, function () { ... feature init code ... }); Y.use(bar, function () { ... feature init code ... }); ... </script> </body> </html> Note: we’re using a single global YUI instance (Y) - 53 - Yahoo! Confidential
  54. 54. What about inline scripts? Developers should be able to safely output the following code inline: Y.use(foo, function () { ... feature init code ... }); - 54 - Yahoo! Confidential
  55. 55. Huh, will that work? NO! Since we want to lazy-load the YUI seed, we cannot have a YUI instance created at that point i.e. Y will be undefined. - 55 - Yahoo! Confidential
  56. 56. Our solution
  57. 57. New SRP skeleton <!doctype html> <html> ... <body> ... <script> YUI = ...; Y = ...; window.onload = function () { ... Lazy load bundles ... }; Y.use(foo, function () { ... feature init code ... }); Y.use(bar, function () { ... feature init code ... }); ... </script> </body> </html> - 57 - Yahoo! Confidential
  58. 58. The fake YUI object (output inline in the SRP) YUI = { Env: { mods: {} }, add: function (name, fn, version, details) { YUI.Env.mods[name] = { name: name, fn: fn, version: version, details: details || {} }; } }; - 58 - Yahoo! Confidential
  59. 59. The fake Y instance (output inline in the SRP) Y = { pending: [], use: function () { Y.pending.push(arguments); } }; - 59 - Yahoo! Confidential
  60. 60. The bundle containing the YUI seed • search/yui-override.js • yui-3.3.0/yui/yui-base.js • yui-3.3.0/yui/yui-later.js • yui-3.3.0/collection/array-extras.js • yui-3.3.0/yui/get.js • yui-3.3.0/yui/features.js • yui-3.3.0/oop/oop.js • yui-3.3.0/loader/loader.js • ... • search/srp-core.js - 60 - Yahoo! Confidential
  61. 61. yui-override.js if (typeof YUI !== undefined) { fakeYUI = YUI; YUI = undefined; } - 61 - Yahoo! Confidential
  62. 62. The real YUI instance Once the YUI seed has been downloaded, we create a real YUI instance. - 62 - Yahoo! Confidential
  63. 63. The real YUI instance (pseudo code) YUI({ bootstrap: false }).use(yui-base, function (Y) { 1) Save a local reference to the fake YUI instance (global Y) 2) Add the modules that may have been registered with the fake YUI object (fakeYUI) to the real YUI object (YUI) 3) Replace global Y variable by the real YUI instance passed to this function. 4) Process any pending calls to Y.use() }); - 63 - Yahoo! Confidential
  64. 64. The real YUI instance (pseudo code) YUI({ bootstrap: false }).use(yui-base, function (Y) { 1) Save a local reference to the fake YUI instance (global Y) 2) Add the modules that may have been registered with the fake YUI object (fakeYUI) to the real YUI object (YUI) 3) Replace global Y variable by the real YUI instance passed to this function. 4) Process any pending calls to Y.use() }); - 64 - Yahoo! Confidential
  65. 65. The real YUI instance (pseudo code) YUI({ bootstrap: false }).use(yui-base, function (Y) { 1) Save a local reference to the fake YUI instance (global Y) 2) Add the modules that may have been registered with the fake YUI object (fakeYUI) to the real YUI object (YUI) 3) Replace global Y variable by the real YUI instance passed to this function. 4) Process any pending calls to Y.use() }); - 65 - Yahoo! Confidential
  66. 66. The real YUI instance (pseudo code) YUI({ bootstrap: false }).use(yui-base, function (Y) { 1) Save a local reference to the fake YUI instance (global Y) 2) Add the modules that may have been registered with the fake YUI object (fakeYUI) to the real YUI object (YUI) 3) Replace global Y variable by the real YUI instance passed to this function. 4) Process any pending calls to Y.use() }); - 66 - Yahoo! Confidential
  67. 67. The real YUI instance (pseudo code) YUI({ bootstrap: false }).use(yui-base, function (Y) { 1) Save a local reference to the fake YUI instance (global Y) 2) Add the modules that may have been registered with the fake YUI object (fakeYUI) to the real YUI object (YUI) 3) Replace global Y variable by the real YUI instance passed to this function. 4) Process any pending calls to Y.use() }); - 67 - Yahoo! Confidential
  68. 68. Huh, will that work? NO! When the loader is disabled, the callback function passed to Y.use() is invoked, whether or not the dependencies are available (a bad design decision IMHO) - 68 - Yahoo! Confidential
  69. 69. Modifying the behavior of Y.use() YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var pending = []; ... Y.before(function () { If a dependency is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); ... }); - 69 - Yahoo! Confidential
  70. 70. Unblocking a pending call to Y.use() The availability of a new module (YUI.add) may unblock a pending call to Y.use() - 70 - Yahoo! Confidential
  71. 71. Modifying the behavior of YUI.add() YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { ... Y.after(function () { Process pending queue to see if this newly added module may unblock a pending call to Y.use() }, YUI, add); ... }); - 71 - Yahoo! Confidential
  72. 72. Loading/execution flow (1/2) Y.use() invoked Compute dependency tree. Missing Yes Append call to pending queue. dependency? No Execute standard Y.use() - 72 - Yahoo! Confidential
  73. 73. Loading/execution flow (1/2) Y.use() invoked Compute dependency tree. Missing Yes Append call to pending queue. dependency? No Execute standard Y.use() - 73 - Yahoo! Confidential
  74. 74. Loading/execution flow (1/2) Y.use() invoked Compute dependency tree. Missing Yes Append call to pending queue. dependency? No Execute standard Y.use() - 74 - Yahoo! Confidential
  75. 75. Loading/execution flow (1/2) Y.use() invoked Compute dependency tree. Missing Yes Append call to pending queue. dependency? No Execute standard Y.use() - 75 - Yahoo! Confidential
  76. 76. Loading/execution flow (1/2) Y.use() invoked Compute dependency tree. Missing Yes Append call to pending queue. dependency? No Execute standard Y.use() - 76 - Yahoo! Confidential
  77. 77. Loading/execution flow (2/2) JS bundle downloaded YUI.add() Yes Is a call to Y.use() Execute pending call. pending? - 77 - Yahoo! Confidential
  78. 78. Loading/execution flow (2/2) JS bundle downloaded YUI.add() Yes Is a call to Y.use() Execute pending call. pending? - 78 - Yahoo! Confidential
  79. 79. Loading/execution flow (2/2) JS bundle downloaded YUI.add() Yes Is a call to Y.use() Execute pending call. pending? - 79 - Yahoo! Confidential
  80. 80. Loading/execution flow (2/2) JS bundle downloaded YUI.add() Yes Is a call to Y.use() Execute pending call. pending? - 80 - Yahoo! Confidential
  81. 81. The code… Putting it together… - 81 - Yahoo! Confidential
  82. 82. srp-core.js (1/8) YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var fakeY = Y.config.win.Y, pending = []; Y.mix(YUI.Env, fakeYUI.Env, false, null, 0, true); Y.before(function () { If a dependency (direct or indirect) is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); Y.after(function () { Process queue to see if this newly added module unblocks a pending call to Y.use() }, YUI, add); Y.config.win.Y = Y; Y.each(fakeY.pending, function (args) { Y.use.apply(Y, args); }); fakeYUI = undefined; }); - 82 - Yahoo! Confidential
  83. 83. srp-core.js (2/8) YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var fakeY = Y.config.win.Y, pending = []; Y.mix(YUI.Env, fakeYUI.Env, false, null, 0, true); Y.before(function () { If a dependency (direct or indirect) is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); Y.after(function () { Process queue to see if this newly added module unblocks a pending call to Y.use() }, YUI, add); Y.config.win.Y = Y; Y.each(fakeY.pending, function (args) { Y.use.apply(Y, args); }); fakeYUI = undefined; }); - 83 - Yahoo! Confidential
  84. 84. srp-core.js (3/8) YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var fakeY = Y.config.win.Y, pending = []; Y.mix(YUI.Env, fakeYUI.Env, false, null, 0, true); Y.before(function () { If a dependency (direct or indirect) is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); Y.after(function () { Process queue to see if this newly added module unblocks a pending call to Y.use() }, YUI, add); Y.config.win.Y = Y; Y.each(fakeY.pending, function (args) { Y.use.apply(Y, args); }); fakeYUI = undefined; }); - 84 - Yahoo! Confidential
  85. 85. srp-core.js (4/8) YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var fakeY = Y.config.win.Y, pending = []; Y.mix(YUI.Env, fakeYUI.Env, false, null, 0, true); Y.before(function () { If a dependency is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); Y.after(function () { Process queue to see if this newly added module unblocks a pending call to Y.use() }, YUI, add); Y.config.win.Y = Y; Y.each(fakeY.pending, function (args) { Y.use.apply(Y, args); }); fakeYUI = undefined; }); - 85 - Yahoo! Confidential
  86. 86. srp-core.js (5/8) YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var fakeY = Y.config.win.Y, pending = []; Y.mix(YUI.Env, fakeYUI.Env, false, null, 0, true); Y.before(function () { If a dependency (direct or indirect) is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); Y.after(function () { Process queue to see if this newly added module unblocks a pending call to Y.use() }, YUI, add); Y.config.win.Y = Y; Y.each(fakeY.pending, function (args) { Y.use.apply(Y, args); }); fakeYUI = undefined; }); - 86 - Yahoo! Confidential
  87. 87. srp-core.js (6/8) YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var fakeY = Y.config.win.Y, pending = []; Y.mix(YUI.Env, fakeYUI.Env, false, null, 0, true); Y.before(function () { If a dependency (direct or indirect) is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); Y.after(function () { Process queue to see if this newly added module unblocks a pending call to Y.use() }, YUI, add); Y.config.win.Y = Y; Y.each(fakeY.pending, function (args) { Y.use.apply(Y, args); }); fakeYUI = undefined; }); - 87 - Yahoo! Confidential
  88. 88. srp-core.js (7/8) YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var fakeY = Y.config.win.Y, pending = []; Y.mix(YUI.Env, fakeYUI.Env, false, null, 0, true); Y.before(function () { If a dependency (direct or indirect) is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); Y.after(function () { Process queue to see if this newly added module unblocks a pending call to Y.use() }, YUI, add); Y.config.win.Y = Y; Y.each(fakeY.pending, function (args) { Y.use.apply(Y, args); }); fakeYUI = undefined; }); - 88 - Yahoo! Confidential
  89. 89. srp-core.js (8/8) YUI({ bootstrap: false }).use(yui-base, event-custom-base, function (Y) { var fakeY = Y.config.win.Y, pending = []; Y.mix(YUI.Env, fakeYUI.Env, false, null, 0, true); Y.before(function () { If a dependency (direct or indirect) is missing, do: pending.push(arguments); return new Y.Do.Prevent(); }, Y, use); Y.after(function () { Process queue to see if this newly added module unblocks a pending call to Y.use() }, YUI, add); Y.config.win.Y = Y; Y.each(fakeY.pending, function (args) { Y.use.apply(Y, args); }); fakeYUI = undefined; }); - 89 - Yahoo! Confidential
  90. 90. The results
  91. 91. The results… • RTB times lower by 40 to 50 msec on broadband (that’s considered a significant improvement :) • Over 1 second better on dialup connections! - 91 - Yahoo! Confidential
  92. 92. The holy grail of JavaScript loading on the SRP… • Adapt the number of bundles lazy-loaded in parallel to the user agent’s capabilities. • Reduce the amount of JavaScript we load for all page views by moving more of it to on- demand loading. - 92 - Yahoo! Confidential
  93. 93. Questions? • jlecomte@yahoo-inc.com • http://www.julienlecomte.net/ • Twitter: @powersander • Y!IM: julien.lecomte - 93 - Yahoo! Confidential

×