HIGH PERFORMANCEON DRUPAL 7 - ANANATOMY OF A SITEKalle VarisvirtaTechnology Director
Designing for highperformance The process is usually the same for major refactoring and building a new site for high performance It’s always easier to replace an existing site, because you have real data Creating a high performance site on some estimations from a customer might get you pretty far away from the actual needed solution
Designing for highperformance For this session, we’ll imagine a situation where we have an existing site with actual data available The recent case where we were working with this kind of a design was exactly that: a well matured site (running sine 1998!) going to be reincarnated for the fourth time
First look: identify theproblem When a site is not performing well, it can be caused by numerous different reasons Analyze it Profile under load Look at the logs Look at the server loads under load
First look: identify theproblem Make sure you’re not hitting some simple bottleneck Too many running services on a single hardware A crazy database query killing the site Broken router causing 3 sec delay to every request (seen that, for real) And many, many others
Problem identified When you’ve arrived to the conclusion that you actually have too much volume, then figure out of what? Too much content? I’ve seen 12 million nodes plus 60 million comments on a single installation, that’s a lot. Too many requests per second? Make sure they are page requests. Statics can be easily fixed, look at cache headers, aggregation, Varnish, Nginx, CDNs.
Problem identified Too many Drupal page requests per second? Anonymous? If anonymous, it’s usually easy to fix, as long as it’s cacheable. We’ll go into the whole “cacheable” thing later. If it’s cacheable, look at page cache, Boost, Varnish, CDNs. Logged in? Drupal cache is turning off, and the calls are bypassing all the caches This usually is a more difficult problem to solve
Problem identified: toomany logged in usersThere’s still one case that’s pretty common and stilleasy enough to solve: logged in users with small amount of personalized content (small in percentage of the CPU cost of building the page in Drupal)
Problem identified: toomany logged in users logged in as user highlights: content area: common content for common everybody your friends’ favorites
Problem identified: toomany logged in users Let’s make a couple pre-requisite conditions You’re running on your own environment You have Varnish configured in front of the Drupal site You have some skills in programming with Drupal You got all of this? Ok, let’s continue.
it’s time forCACHECONTROLdrupal.org/project/cache_control
What’s Cache Control It’s similar to ESI module with some benefits It’s mainly directed to cache blocks or block-like content on the page It needs some programming usually When dealing with an optimal problem for it, it’s the optimal solution and will make your site faster by magnitudes
Problem identified: toomany logged in users logged in as login box user highlights: content area: common content for common everybody your friends’ staff picks favorites
Benefits of Cache Control Burdens the back-end significantly less due to only loading the needed parts Loads multiple blocks and/or areas with a single request Gives the user something to look at while loading the hard parts of the page – and it does make the site feel faster Plays well with some other modules, like captcha etc.
What about ESI ESI (Edge Side Includes) is a partial loading technique supported by Varnish and some CDNs, e.g. Akamai It basically makes Varnish do the partial page loading Varnish first fetches the common version from cache Then it looks though the page to see any ESI markup Then it loads all the ESI marked parts of the page from cache or from the Drupal
How is Cache Controldifferent than ESI ESI needs to wait until the whole page is loaded before giving anything to the user ESI loads all the portions of the page (still in D7, this might change in D8) in separate http requests, thus burdening the server with even more bootstraps than without any cache
HEY… HOW ABOUT THATUSER GENERATEDCONTENT THAT MAKESVARNISH PURGEEVERYTHING ALL THETIME?
Different problem As stated, Cache Control works well for specific problems, but that also is in trouble when the Varnish cache gets purged all the time That usually happens on a really UGC (User Generated Content) oriented site
Different problem: UGCWhen a single page on a site gets new contentevery 2-30 seconds Caching is of no use, purging multiple pages on that rate makes no sense You need that data to have a way of refreshing even more frequently And we’re still talking about a page that doesn’t update after it has loaded (so no Socket.IO stuff on this slide deck, sorry)
Different problem: UGC logged in as user content area: common content for everybody highlights: common and this is getting updates every 30 seconds your friends’ favorites
Solution: A new cachelayer We add a new, fast-paced cache layer on the page We’ll try to purge and reload that cache as fast a humanly possible in Drupal We’ll minimize our efforts on the backend
Solution: A new cachelayer And the back-end? Exove has a module coming out to help get grouped and cached JSON outputs fast from Views It’s not something to be used for integrations but just for the faster cache layer Going to be released during this fall with a site using it Until that, just use Views and Views datasource
SO, WAIT A MINUTETHESE ARE ALL HACKS,RIGHT?Not quite.
Drupal doing highperformance You can’t really use Drupal for high performance out of the box Hacks, or actually extensions are needed and if done as proper contribs, are safe and convenient to use Drupal has been made extensible for this exact reason, it can be made better by extending it
What would we like to seein Drupal 8 We’d like to see a real JSON output from Drupal, preferably by piece by piece content We’d also like to see a thinner bootstrap with lazy-loading for pretty much everything REST interface for doing more stuff in the front, e.g. with JS frameworks
You can see a pattern here. This is allcovered by the WSSCI and Scotchiniatives. We’re waiting for Drupal 8 tobe a lot better.And Cache Control is going to rock on Drupal 8.
and there are always going to be hacksto get Drupal to do more
THANK YOU FOR YOURTIMEPS. We’re hiring. www.exove.fi/careers