Meetup Performance
Upcoming SlideShare
Loading in...5
×
 

Meetup Performance

on

  • 3,910 views

Presentation on how Meetup tackles web performance. Given on: ...

Presentation on how Meetup tackles web performance. Given on:
- Nov 17th, 2009 for the NY Web Performance Group (http://www.meetup.com/Web-Performance-NY/)
- Jan 26th, 2010 for NYC Tech Talks Meetup Group (http://www.meetup.com/NYC-Tech-Talks/)

Statistics

Views

Total Views
3,910
Views on SlideShare
3,739
Embed Views
171

Actions

Likes
4
Downloads
75
Comments
1

7 Embeds 171

http://www.techpresentations.org 123
http://www.linkedin.com 18
http://speakerrate.com 11
http://www.slideshare.net 6
http://www.meetup.com 5
http://www.lmodules.com 5
https://www.linkedin.com 3
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Great! Thanks.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Meetup Performance Meetup Performance Presentation Transcript

  • Meetup Performance Greg Whalin, CTO Meetup (@gwhalin), Justin Cataldo, Lead UI Engineer (@jcataldo), Will Howard, Lead UI Engineer
  • Meetup Platform for local groups Mission is MEME (Meetup Everywhere About Most Everything) ~6.2m members ~70k Groups ~500k group joins every month ~5 million Meetups have happened ~53 million RSVPs
  • General Architecture and Back- end Performance (just a tiny bit - this could and should be another presentation)
  • Data MySQL (any RDBMs store) biggest pain replication use InnoDB! smart indexing (take advantage of clustered indexes) server side conn pools and short timeouts on connections smart about fs choice on Linux (we use XFS after benchmarking) Hardware - relatively expensive boxes multi-core (8 or 16) Opteron lots of ram (32/64GB) fast drives (lots of spindles, RAID10) Cache, cache, cache! innodb buffer cache memcache local app server memory Shrink data when possible archive unused data custom serialization when serializing data partitioning/sharding
  • Storage Over half a million photos uploaded to Meetup every month Scaled and processed into 4 different sizes (plus original)
  • Storage solutions Options for growth include NAS, SAN, or something else NAS and SAN are single point of failure and possibly $$$ Only postpones problem
  • MogileFS developed by Brand Fitzpatrick (i.e. Memcached) OSS distributed filesystem (built in Perl) any hard drive on network can easily be added to cluster scales easily and cheaply
  • Much much much more going on but...
  • UI Performance (much of our focus here)
  • Why does performance matter?
  • Why does performance matter? Slow site
  • Why does performance matter? Slow site Bad User Experience
  • Why does performance matter? Slow site Bad User Experience Drop in Member Activity
  • Why focus on front end performance? Back end only accounts for 10-15% of the response time Less time and resources Costs less http://developer.yahoo.net/blog/archives/2007/03/high_performanc.html
  • Case Study: Event Details
  • Event Details: Load Time Load time = 6.321s www.webpagetest.org
  • Event Details: Requests Lots of javascript being loaded
  • How do we improve performance?
  • 3 Steps to improving performance 1. Externalize script 2. Move scripts to the bottom of the page 3. Reduce requests
  • 3 Steps to improving performance 1. Externalize script 2. Move scripts to the bottom of the page 3. Reduce requests
  • Why externalize scripts Prevents blocking Inline scripts prevent asynchronous downloads Downloads must wait for the script to be executed Caching Inline JavaScript is downloaded every time External scripts are cached by the browser Reduced overall page size Reusable Can use the same code somewhere else on the site easily
  • Pull out inline script <script type="text/javascript"> > if(typeof Meetup.EventDetails == 'undefined') Meetup.EventDetails = {}; (function(){ EventDetails.js var self = Meetup.EventDetails; ... })(); </script>
  • 3 Steps to improving performance 1. Externalize script 2. Move scripts to the bottom of the page 3. Reduce requests
  • Web:script Custom tag built in house Moves inline and external script to the bottom of the page Allows UI engineers to not have to worry about where they place scripts Compresses inline script using YUICompressor /***** Load External Script *****/ <web:script src="/script/Meetup/packed/EventDetails.js" /> /***** Load Inline Script *****/ <web:script> Meetup.Copy.noMembersMarkedAttended = "<trn:message key="event.attendance.noMembersMarkedAttended">No members have been marked attended</trn:message>"; Meetup.Copy.noMembersMarkedAttendedDynam = '<trn:message key="event.attendance. noMembersMarkedAttendedDynam"><trn:param name="GROUPING">__GROUPING__</trn:param>No members in "{GROUPING}" have been marked attended</trn:message>'; Meetup.Copy.noMembersMarkedAbsent = "<trn:message key="event.attendance.noMembersMarkedAbsent">No members have been marked absent</trn:message>"; </web:script>
  • 3 Steps to improving performance 1. Externalize script 2. Move scripts to the bottom of the page 3. Reduce requests
  • Reduce Requests Concatenate as much as possible Load only what we need upfront
  • Concatenation using Sprockets Sprockets (www.getsprockets.com) Created by 37Signals Ruby library that preprocesses and concatenates JavaScript files Baked into our build process
  • Concatenation using Sprockets (cont.) EventDetails.js CommentDeleteConfirm.js BubbleTips.js PlacesManager.js MaxCharactersEnforcer.js > EventDetails.js
  • Using Sprockets /******* Sprockets Directives *******/ //= require <CommentDeleteConfirm> //= require <DomDeco/BubbleTips> //= require <DomDeco/PlacesManager> //= require <DomDeco/MaxCharactersEnforcer> /******* Begin Event Details Code *******/ if(typeof(Meetup.EventDetails) == 'undefined') Meetup.EventDetails = {}; (function(){ var self = Meetup.EventDetails; ........... })(); /******* Build Process *******/ <exec executable="sprocketize" failonerror="true" output="${image.dir} /script/Meetup/packed/EventDetails.js"> <arg value="-I"/> <arg path="${image.dir}/script/Meetup/"/> <arg path="${image.dir}/script/Meetup/EventDetails.js"/> </exec>
  • Lazy Loading Defer loading of javascript files until they are needed Reduces the initial upfront requests Helps reduce blocking by downloading files asynchronously Precaching
  • Lazy Loading: How it works Inserts scripts into the head dynamically var scriptNode = function(src) { return createDOM("script", { "type": "text/javascript", "src": src }); } var load = function(id, src, n) { var script = scriptNode(url); head.appendChild(script); script.onload = script.onreadystatechange = function() { if (!this.readyState || this.readyState == "loaded" || this.readyState == "complete") { script.onload = script.onreadystatechange = null; } } } Meetup.Script.include("http://static2.meetupstatic.com/script/Meetup/DomDeco/LinkDecorator.js",callback);
  • Did it make a difference?
  • Reduced requests Javascript requests cut by 50%
  • Event Details Page Load After Old Load Time = 6.321s New Load time = 4.643s
  • Event Details Page Load After Old Load Time = 6.321s New Load time = 4.643s Load time decreased by 27%!
  • But that's not all
  • Execute early Execute when the dom is ready
  • Execute early: DOMReady Libraries that have a DOM ready solution: jQuery YUI Prototype Pretty much every modern JS library (not MochiKit) Meetup uses MochiKit, so we rolled our own.
  • Execute early: DOMReady And by rolled our own, I mean we're using the Dean Edwards/Matthias Miller/John Resig implementation. http://dean.edwards.name/weblog/2006/06/again/#comment5338 With a few changes. Meetup.DOMReady.ready(function(){ Meetup.EventDetails.init(); if(Meetup.EventDetails.isCanceled != 4 && Meetup.EventDetails.rsvp != 0){ deletePopup = new Meetup.CommentDeleteConfirm(); deletePopup.pagerOffsetFieldName = "p_commentsList"; deletePopup._decorate(); } });
  • Execute early: Even earlier Do you need to wait for the DOM to be ready? If you aren't manipulating the DOM, there's no reason to wait until it's ready.
  • Automated Image Optimization Using smush.it http://developer.yahoo.com/yslow/smushit/ Smusher Ruby gem http://github.com/grosser/smusher (gem install smusher) BASH script that watches our image directories for changes and executes smusher. ./filescan.sh /usr/local/meetup/static/img/ 'smusher -q @' 'jpg,png'&
  • Event Delegation Run less JavaScript up front
  • Event Delegation But first, a little bit about event bubbling... From: http://www.quirksmode.org/js/events_order.html
  • Event Delegation Pros Cons Does not work well with nested elements Much faster on load (not Doesn't work with all events connecting DOM elements) Slight performance hit with execution No need to disconnect / reconnect with AJAX calls Fewer memory leaks A lot of JS libraries already have plug-ins for event delegation (jQuery, YUI, prototype). But, it's pretty easy to write your own (we did).
  • Event Delegation: Meetup.Dispatcher <div id="C_page"> ... <span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href=" http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ... </div> var mdp = Meetup.Dispatcher.init("C_page", "onmouseover"); mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver); Meetup.UI.infoHover.mouseOver = function(e) { topicId = _getTopicId(e.target()); if (!topicId || topicId == "") return; _primeCache(topicId); var activeEl = _getActiveEl(e.target()); var pos = getElementPosition(activeEl); ... }
  • Event Delegation: Meetup.Dispatcher
  • Event Delegation: Meetup.Dispatcher <div id="C_page"> ... <span class="meetup-topic"><a class="topic-id-7029 topic-link J_onClick topic-info-hover" href=" http://javascript.meetup.com/cities/us/ny/brooklyn/">JavaScript</a></span> ... </div> // Inits a new instance of dispatcher // Connects a mouseover event to the parent container "C_page" var mdp = Meetup.Dispatcher.init("C_page", "onmouseover"); // Calls Meetup.UI.infoHover.mouseOver() when target element has "topic-info-hover" class. mdp.registerFunc("topic-info-hover", Meetup.UI.InfoHover.mouseOver); Meetup.UI.infoHover.mouseOver = function(e) { topicId = _getTopicId(e.target()); if (!topicId || topicId == "") return; _primeCache(topicId); var activeEl = _getActiveEl(e.target()); var pos = getElementPosition(activeEl); ... }
  • Speeding up DOM crawling with Sizzle sizzlejs.com Internet Explorer 7 MochiKit: 6623.94ms Sizzle: 306.03ms Firefox 3.5 MochiKit: 210.524ms Sizzle: 111.553ms
  • Where do we go from here? More concatenation and lazy loading where it makes sense Defer image loading where it makes sense Reduce DOM elements Reduce CSS and improved selector efficiency and more
  • Deployment and Serving As it pertains to css/html/js
  • Launches Launch multiple times a day (sometimes) Need launches to be quick / no downtime Optimize static resources only at deploy time and only if modified
  • Deployment of static content Sprockets (reduce requests) YUICompressor for js (local mod to speed up optimizing multiple files) Pre-compress css and jsp Set cache-control to be fresh for over a year (indefinite) All links on site generate programatically and versioned
  • Link generation Custom JSTL page function http vs https domain sharding and cookie free domain content versioning <link rel="stylesheet" href="${mfn:staticUrl( "/style/meetup.css", state.context.isSecure )}" type=" text/css" /> <img src="${mfn:imgUrl( '/img/noPhoto_50.gif', state.context.isSecure )}" alt="" class="noPhoto" /> <link rel="stylesheet" href="http://static1.meetupstatic. com/050991196173395491322880/style/meetup.css" type="text/css" /> <img src="http://img1.meetupstatic.com/39194172310009655/img/noPhoto_50.gif" alt="" class=" noPhoto"/>
  • Versioning static content MD5 checksum of contents of file Run w/ each launch Store versions in db tied to release mysql> select * from resource_version where filename = '/style/base.css'; | 394328 | /style/base.css | 39020083689267241 | 567 | | 398052 | /style/base.css | 8487620432388779772669 | 568 | | 401776 | /style/base.css | 357470606563045379 | 569 | | 405506 | /style/base.css | 3068234199748867 | 571 | | 409240 | /style/base.css | 024745310801291061590 | 572 | | 412974 | /style/base.css | 024745310801291061590 | 573 | | 416708 | /style/base.css | 09972542737049101325 | 574 |
  • Static content serving Served off CDN (reverse proxy) Anycast DNS for hostname resolution Origin servers running lighttpd Strip versioning from url using rewrite rules Cache set to 12 months out Compress expire.url = ( "/" => "access plus 12 months" ) compress.cache-dir = "/var/cache/lighttpd/" compress.filetype = ("text/plain", "text/html", "text/css", "application/x-javascript", "text/javascript") url.rewrite-once = ( "^/script/(.*/)?[0-9]+/(.+).js$" => "/script/$1$2.js", "^/style/(.*/)?[0-9]+/(.+).css$" => "/style/$1$2.css", "^/img/(.*/)?[0-9]+/(.+).(gif|png|jpg)$" => "/img/$1$2.$3", "^/d+/script/(.+).js" => "/script/$1.js", "^/d+/style/(.+).css" => "/style/$1.css", "^/d+/img/(.+).(gif|jpg|png)" => "/img/$1.$2", "^/photos/(([^/]+)/.+/(.+).jpeg)(?:?.*)?$" => "/cgi-bin/photos.fcgi?type=$2&key=$3", "^/photos/([^/]+)/([^/]+).jpeg(?:?.*)?$" => "/cgi-bin/photos.fcgi?type=$1&key=$2", "^/file.*" => "/" )
  • Questions? Also, we need help! Hiring for: Linux Systems Administrator Software Engineers UI Engineers QA Engineers Community Specialist PR Renegade Sponsorship Sales Account Coordinator http://www.meetup.com/jobs/