Meetup PerformanceGreg Whalin, CTO Meetup (@gwhalin), Justin Cataldo, Lead UI Engineer (@jcataldo), Will Howard, Lead UI E...
MeetupPlatform for local groupsMission is MEME (Meetup Everywhere About Most Everything)~6.2m members~70k Groups~500k grou...
General Architecture and Back-      end Performance     (just a tiny bit - this could and     should be another presentati...
DataMySQL (any RDBMs store) biggest pain   replication   use InnoDB!   smart indexing (take advantage of clustered indexes...
Storage  Over half a million photos uploaded to Meetup every month  Scaled and processed into 4 different sizes (plus orig...
Storage solutions  Options for growth include NAS, SAN, or something else  NAS and SAN are single point of failure and pos...
MogileFS developed by Brand Fitzpatrick (i.e. Memcached) OSS distributed filesystem (built in Perl) any hard drive on netw...
Much much much more going on       but...
UI Performance(much of our focus here)
Why does performance matter?
Why does performance matter?             Slow site
Why does performance matter?              Slow site         Bad User Experience
Why does performance matter?              Slow site         Bad User Experience           Drop in Member               Act...
Why focus on front end performance?  Back end only accounts for 10-15% of the response time  Less time and resources  Cost...
Case Study: Event Details
Event Details: Load Time  Load time = 6.321swww.webpagetest.org
Event Details: Requests        Lots of javascript being loaded
How do we improve  performance?
3 Steps to improving performance1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
3 Steps to improving performance1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
Why externalize scripts  Prevents blocking     Inline scripts prevent asynchronous downloads     Downloads must wait for t...
Pull out inline script<script type="text/javascript">                                                >if(typeof Meetup.Eve...
3 Steps to improving performance1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
Web:script    Custom tag built in house    Moves inline and external script to the bottom of the page    Allows UI enginee...
3 Steps to improving performance1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
Reduce Requests Concatenate as much as possible Load only what we need upfront
Concatenation using Sprockets Sprockets (www.getsprockets.com) Created by 37Signals Ruby library that preprocesses and con...
Concatenation using Sprockets (cont.)EventDetails.jsCommentDeleteConfirm.jsBubbleTips.jsPlacesManager.jsMaxCharactersEnfor...
Using Sprockets/******* Sprockets Directives *******///= require <CommentDeleteConfirm>//= require <DomDeco/BubbleTips>//=...
Lazy Loading Defer loading of javascript files until they are needed Reduces the initial upfront requests Helps reduce blo...
Lazy Loading: How it worksInserts scripts into the head dynamically var scriptNode = function(src) {   return createDOM("s...
Did it make a difference?
Reduced requests       Javascript requests cut by 50%
Event DetailsPage Load AfterOld Load Time = 6.321sNew Load time = 4.643s
Event DetailsPage Load AfterOld Load Time = 6.321sNew Load time = 4.643s   Load time  decreased by      27%!
But thats not all
Execute early       Execute when the dom is ready
Execute early: DOMReadyLibraries that have a DOM ready solution:   jQuery   YUI   Prototype   Pretty much every modern JS ...
Execute early: DOMReadyAnd by rolled our own, I mean were using the DeanEdwards/Matthias Miller/John Resig implementation....
Execute early: Even earlierDo you need to wait for the DOM to be ready?If you arent manipulating the DOM, theres no reason...
Automated Image OptimizationUsing smush.ithttp://developer.yahoo.com/yslow/smushit/Smusher Ruby gemhttp://github.com/gross...
Event DelegationRun less JavaScript up front
Event DelegationBut first, a little bit about event bubbling...  From: http://www.quirksmode.org/js/events_order.html
Event DelegationPros                             Cons                                      Does not work well with nested ...
Event Delegation: Meetup.Dispatcher <div id="C_page">   ...    <span class="meetup-topic"><a class="topic-id-7029 topic-li...
Event Delegation: Meetup.Dispatcher
Event Delegation: Meetup.Dispatcher <div id="C_page">   ...    <span class="meetup-topic"><a class="topic-id-7029 topic-li...
Speeding up DOM crawling with Sizzle                         sizzlejs.com  Internet Explorer 7  MochiKit: 6623.94ms  Sizzl...
Where do we go from here? More concatenation and lazy loading where it makes sense Defer image loading where it makes sens...
Deployment and Serving   As it pertains to css/html/js
LaunchesLaunch multiple times a day (sometimes)Need launches to be quick / no downtimeOptimize static resources only at de...
Deployment of static contentSprockets (reduce requests)YUICompressor for js (local mod to speed up optimizingmultiple file...
Link generation   Custom JSTL page function   http vs https   domain sharding and cookie free domain   content versioning<...
Versioning static content     MD5 checksum of contents of file     Run w/ each launch     Store versions in db tied to rel...
Static content serving    Served off CDN (reverse proxy)    Anycast DNS for hostname resolution    Origin servers running ...
Questions?Also, we need help! Hiring for:   Linux Systems Administrator   Software Engineers   UI Engineers   QA Engineers...
Upcoming SlideShare
Loading in...5
×

Meetup Performance

654

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
654
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Meetup Performance

  1. 1. Meetup PerformanceGreg Whalin, CTO Meetup (@gwhalin), Justin Cataldo, Lead UI Engineer (@jcataldo), Will Howard, Lead UI Engineer
  2. 2. MeetupPlatform for local groupsMission is MEME (Meetup Everywhere About Most Everything)~6.2m members~70k Groups~500k group joins every month~5 million Meetups have happened~53 million RSVPs
  3. 3. General Architecture and Back- end Performance (just a tiny bit - this could and should be another presentation)
  4. 4. DataMySQL (any RDBMs store) biggest pain replication use InnoDB! smart indexing (take advantage of clustered indexes) server side conn pools and short timeouts on connections smart about fs choice on Linux (we use XFS after benchmarking)Hardware - relatively expensive boxes multi-core (8 or 16) Opteron lots of ram (32/64GB) fast drives (lots of spindles, RAID10)Cache, cache, cache! innodb buffer cache memcache local app server memoryShrink data when possible archive unused data custom serialization when serializing data partitioning/sharding
  5. 5. Storage Over half a million photos uploaded to Meetup every month Scaled and processed into 4 different sizes (plus original)
  6. 6. Storage solutions Options for growth include NAS, SAN, or something else NAS and SAN are single point of failure and possibly $$$ Only postpones problem
  7. 7. MogileFS developed by Brand Fitzpatrick (i.e. Memcached) OSS distributed filesystem (built in Perl) any hard drive on network can easily be added to cluster scales easily and cheaply
  8. 8. Much much much more going on but...
  9. 9. UI Performance(much of our focus here)
  10. 10. Why does performance matter?
  11. 11. Why does performance matter? Slow site
  12. 12. Why does performance matter? Slow site Bad User Experience
  13. 13. Why does performance matter? Slow site Bad User Experience Drop in Member Activity
  14. 14. Why focus on front end performance? Back end only accounts for 10-15% of the response time Less time and resources Costs less http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html
  15. 15. Case Study: Event Details
  16. 16. Event Details: Load Time Load time = 6.321swww.webpagetest.org
  17. 17. Event Details: Requests Lots of javascript being loaded
  18. 18. How do we improve performance?
  19. 19. 3 Steps to improving performance1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
  20. 20. 3 Steps to improving performance1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
  21. 21. Why externalize scripts Prevents blocking Inline scripts prevent asynchronous downloads Downloads must wait for the script to be executed Caching Inline JavaScript is downloaded every time External scripts are cached by the browser Reduced overall page size Reusable Can use the same code somewhere else on the site easily
  22. 22. Pull out inline script<script type="text/javascript"> >if(typeof Meetup.EventDetails == undefined) Meetup.EventDetails = {};(function(){ EventDetails.js var self = Meetup.EventDetails;...})();</script>
  23. 23. 3 Steps to improving performance1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
  24. 24. Web:script Custom tag built in house Moves inline and external script to the bottom of the page Allows UI engineers to not have to worry about where they place scripts Compresses inline script using YUICompressor/***** Load External Script *****/<web:script src="/script/Meetup/packed/EventDetails.js" />/***** Load Inline Script *****/<web:script> Meetup.Copy.noMembersMarkedAttended = "<trn:message key="event.attendance.noMembersMarkedAttended">Nomembers have been marked attended</trn:message>"; Meetup.Copy.noMembersMarkedAttendedDynam = <trn:message key="event.attendance.noMembersMarkedAttendedDynam"><trn:param name="GROUPING">__GROUPING__</trn:param>No members in"{GROUPING}" have been marked attended</trn:message>; Meetup.Copy.noMembersMarkedAbsent = "<trn:message key="event.attendance.noMembersMarkedAbsent">Nomembers have been marked absent</trn:message>";</web:script>
  25. 25. 3 Steps to improving performance1. Externalize script2. Move scripts to the bottom of the page3. Reduce requests
  26. 26. Reduce Requests Concatenate as much as possible Load only what we need upfront
  27. 27. Concatenation using Sprockets Sprockets (www.getsprockets.com) Created by 37Signals Ruby library that preprocesses and concatenates JavaScript files Baked into our build process
  28. 28. Concatenation using Sprockets (cont.)EventDetails.jsCommentDeleteConfirm.jsBubbleTips.jsPlacesManager.jsMaxCharactersEnforcer.js > EventDetails.js
  29. 29. Using Sprockets/******* Sprockets Directives *******///= require <CommentDeleteConfirm>//= require <DomDeco/BubbleTips>//= require <DomDeco/PlacesManager>//= require <DomDeco/MaxCharactersEnforcer>/******* Begin Event Details Code *******/if(typeof(Meetup.EventDetails) == undefined) Meetup.EventDetails = {};(function(){ var self = Meetup.EventDetails;...........})();/******* Build Process *******/<exec executable="sprocketize" failonerror="true" output="${image.dir}/script/Meetup/packed/EventDetails.js"> <arg value="-I"/> <arg path="${image.dir}/script/Meetup/"/> <arg path="${image.dir}/script/Meetup/EventDetails.js"/></exec>
  30. 30. Lazy Loading Defer loading of javascript files until they are needed Reduces the initial upfront requests Helps reduce blocking by downloading files asynchronously Precaching
  31. 31. Lazy Loading: How it worksInserts scripts into the head dynamically var scriptNode = function(src) { return createDOM("script", { "type": "text/javascript", "src": src }); } var load = function(id, src, n) { var script = scriptNode(url); head.appendChild(script); script.onload = script.onreadystatechange = function() { if (!this.readyState || this.readyState == "loaded" || this.readyState == "complete") { script.onload = script.onreadystatechange = null; } } }Meetup.Script.include("http://static2.meetupstatic.com/script/Meetup/DomDeco/LinkDecorator.js",callback);
  32. 32. Did it make a difference?
  33. 33. Reduced requests Javascript requests cut by 50%
  34. 34. Event DetailsPage Load AfterOld Load Time = 6.321sNew Load time = 4.643s
  35. 35. Event DetailsPage Load AfterOld Load Time = 6.321sNew Load time = 4.643s Load time decreased by 27%!
  36. 36. But thats not all
  37. 37. Execute early Execute when the dom is ready
  38. 38. Execute early: DOMReadyLibraries that have a DOM ready solution: jQuery YUI Prototype Pretty much every modern JS library (not MochiKit) Meetup uses MochiKit, so we rolled our own.
  39. 39. Execute early: DOMReadyAnd by rolled our own, I mean were using the DeanEdwards/Matthias Miller/John Resig implementation.http://dean.edwards.name/weblog/2006/06/again/#comment5338With a few changes. Meetup.DOMReady.ready(function(){ Meetup.EventDetails.init(); if(Meetup.EventDetails.isCanceled != 4 && Meetup.EventDetails.rsvp != 0){ deletePopup = new Meetup.CommentDeleteConfirm(); deletePopup.pagerOffsetFieldName = "p_commentsList"; deletePopup._decorate(); } });
  40. 40. Execute early: Even earlierDo you need to wait for the DOM to be ready?If you arent manipulating the DOM, theres no reason towait until its ready.
  41. 41. Automated Image OptimizationUsing smush.ithttp://developer.yahoo.com/yslow/smushit/Smusher Ruby gemhttp://github.com/grosser/smusher (gem install smusher)BASH script that watches our image directories forchanges and executes smusher. ./filescan.sh /usr/local/meetup/static/img/ smusher -q @ jpg,png&
  42. 42. Event DelegationRun less JavaScript up front
  43. 43. Event DelegationBut first, a little bit about event bubbling... From: http://www.quirksmode.org/js/events_order.html
  44. 44. Event DelegationPros Cons Does not work well with nested elements Much faster on load (not Doesnt work with all events connecting DOM elements) Slight performance hit with execution No need to disconnect / reconnect with AJAX calls Fewer memory leaks A lot of JS libraries already have plug-ins for event delegation (jQuery, YUI, prototype). But, its pretty easy to write your own (we did).
  45. 45. Event Delegation: Meetup.Dispatcher <div id="C_page"> ... <span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href=" http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ... </div> var mdp = Meetup.Dispatcher.init("C_page", "onmouseover"); mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver); Meetup.UI.infoHover.mouseOver = function(e) { topicId = _getTopicId(e.target()); if (!topicId || topicId == "") return; _primeCache(topicId); var activeEl = _getActiveEl(e.target()); var pos = getElementPosition(activeEl); ... }
  46. 46. Event Delegation: Meetup.Dispatcher
  47. 47. Event Delegation: Meetup.Dispatcher <div id="C_page"> ... <span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href=" http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ... </div> // Inits a new instance of dispatcher // Connects a mouseover event to the parent container "C_page" var mdp = Meetup.Dispatcher.init("C_page", "onmouseover"); // Calls Meetup.UI.infoHover.mouseOver() when target element has "topic-info-hover" class. mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver); Meetup.UI.infoHover.mouseOver = function(e) { topicId = _getTopicId(e.target()); if (!topicId || topicId == "") return; _primeCache(topicId); var activeEl = _getActiveEl(e.target()); var pos = getElementPosition(activeEl); ... }
  48. 48. Speeding up DOM crawling with Sizzle sizzlejs.com Internet Explorer 7 MochiKit: 6623.94ms Sizzle: 306.03ms Firefox 3.5 MochiKit: 210.524ms Sizzle: 111.553ms
  49. 49. Where do we go from here? More concatenation and lazy loading where it makes sense Defer image loading where it makes sense Reduce DOM elements Reduce CSS and improved selector efficiency and more
  50. 50. Deployment and Serving As it pertains to css/html/js
  51. 51. LaunchesLaunch multiple times a day (sometimes)Need launches to be quick / no downtimeOptimize static resources only at deploy time and only ifmodified
  52. 52. Deployment of static contentSprockets (reduce requests)YUICompressor for js (local mod to speed up optimizingmultiple files)Pre-compress css and jspSet cache-control to be fresh for over a year (indefinite)All links on site generate programatically and versioned
  53. 53. Link generation Custom JSTL page function http vs https domain sharding and cookie free domain content versioning<link rel="stylesheet" href="${mfn:staticUrl( "/style/meetup.css", state.context.isSecure )}" type="text/css" /><img src="${mfn:imgUrl( /img/noPhoto_50.gif, state.context.isSecure )}" alt=""class="noPhoto" /><link rel="stylesheet" href="http://static1.meetupstatic.com/050991196173395491322880/style/meetup.css" type="text/css" /><img src="http://img1.meetupstatic.com/39194172310009655/img/noPhoto_50.gif" alt="" class="noPhoto"/>
  54. 54. Versioning static content MD5 checksum of contents of file Run w/ each launch Store versions in db tied to release mysql> select * from resource_version where filename = /style/base.css; | 394328 | /style/base.css | 39020083689267241 | 567 | | 398052 | /style/base.css | 8487620432388779772669 | 568 | | 401776 | /style/base.css | 357470606563045379 | 569 | | 405506 | /style/base.css | 3068234199748867 | 571 | | 409240 | /style/base.css | 024745310801291061590 | 572 | | 412974 | /style/base.css | 024745310801291061590 | 573 | | 416708 | /style/base.css | 09972542737049101325 | 574 |
  55. 55. Static content serving Served off CDN (reverse proxy) Anycast DNS for hostname resolution Origin servers running lighttpd Strip versioning from url using rewrite rules Cache set to 12 months out Compressexpire.url = ( "/" => "access plus 12 months")compress.cache-dir = "/var/cache/lighttpd/"compress.filetype = ("text/plain", "text/html", "text/css", "application/x-javascript", "text/javascript")url.rewrite-once = ( "^/script/(.*/)?[0-9]+/(.+).js$" => "/script/$1$2.js", "^/style/(.*/)?[0-9]+/(.+).css$" => "/style/$1$2.css", "^/img/(.*/)?[0-9]+/(.+).(gif|png|jpg)$" => "/img/$1$2.$3", "^/d+/script/(.+).js" => "/script/$1.js", "^/d+/style/(.+).css" => "/style/$1.css", "^/d+/img/(.+).(gif|jpg|png)" => "/img/$1.$2", "^/photos/(([^/]+)/.+/(.+).jpeg)(?:?.*)?$" => "/cgi-bin/photos.fcgi?type=$2&key=$3", "^/photos/([^/]+)/([^/]+).jpeg(?:?.*)?$" => "/cgi-bin/photos.fcgi?type=$1&key=$2", "^/file.*" => "/")
  56. 56. Questions?Also, we need help! Hiring for: Linux Systems Administrator Software Engineers UI Engineers QA Engineers Community Specialist PR Renegade Sponsorship Sales Account Coordinatorhttp://www.meetup.com/jobs/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×