Drupalcamp Estonia - High Performance Sites


Published on

Rami Järvinen's presentation about high-performance drupal sites at DrupalCamp Estonia.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • To tell about some concepts or building parts needed when crafting a high performance sites. Going not to dive deep into these topics, give some keywords for you digg up further.
  • From simple personal pages to large media site or community “platform”.
  • From technical point of view, back to 90’s, serving static pages while making content look like dynamic Most expensive thing a web site can do is to fire up the whole php stack, get something from the database and then render it. It takes time, processor time and memory – and it’s slow. (core is lightweight, but with tens modules on..) Avoid running Drupal’s bootstrap – in fact, try to avoid pulling anything from db or even from disk. We want to push generated or static content as close to the requesting client as possible. The greatest common factor (divisor) for every single user group or individual user. For 100% anon users site it means that every page looks the same for every user. For a site that has some personized content for logged in users, common factor would be the content around the user specific content. Aggregated CSS and JS Apache set up to compress its output with gzip
  • MySQL query cache, its out there, caching internally query results. Opcode cache, fundamental part of PHP configs. Drupal internal caching… block cache, page cache HTTP cache – hanging around on the edge of your server stack, serving content heavenly fast.
  • Surprise if APC its not enabled already..
  • Views caching Code level caching, cache_set / cache_get, static variables, all the fancy stuff happening on the micro level.
  • Never touch the database – image loading might touch because of using imagecache module but that’s for first-time loads too Cached versions are built when node is created or edited or when cron is being run. Configurable - Which node types will be cached, how long, which paths
  • Drupal cache API (cacherouter..), Up to one megabyte Cache bins are in db. Also version with a fallback to db Putting session data to memcache drops the amount of sql queries significantly
  • Drupal 7 or PressFlow needed State of the art Following the concept of bringing the cached data as close to user’s browser as possible. One ESI block may be just fine – but if there’s several of them one page load means several requests to the back end (and thus making bootstrap happen). theme_registry_alter hook
  • The API-caching backend Just a single request to the back-end – it knows by pageID what blocks of data are there and what needs to be filled - also this may be cached in varnish In project approval queue – link to sandbox with more details Comments appreciated
  • More memory, the better
  • When or preferrably before you get into trouble with site performance it’ll be wise to do some profiling. You get a cachegrind file which can viewed with KCacheGrind to analyse profiling data. Static variables for saving results for runtime. In a bad place, even a query taking one second will be troublesome. EXPLAIN, SHOW PROCESSLIST
  • Drupalcamp Estonia - High Performance Sites

    1. 1. High performance drupal sites Rami Järvinen, Exove
    2. 2. Agenda <ul><li>About myself </li></ul><ul><li>Caching </li></ul><ul><li>Scaling </li></ul><ul><li>Optimizing </li></ul>
    3. 3. About myself, Rami Järvinen <ul><li>Senior developer at Exove </li></ul><ul><li>Drupal experience from 2006. Been involved with a wide variety of different site projects. </li></ul>
    4. 4. Boosting up the performance <ul><li>Drupal ’ s internal architecture </li></ul><ul><ul><li>Single-controller </li></ul></ul><ul><ul><li>Loads a lot of code on every pageload </li></ul></ul><ul><ul><li>Tends to be slower than a pure MVC-model </li></ul></ul><ul><li>Caching </li></ul><ul><ul><li>Minimize the CPU usage </li></ul></ul><ul><ul><li>Minimize the amount of SQL queries </li></ul></ul><ul><ul><li>Ultimately – avoid running Drupal ’ s bootstrap </li></ul></ul><ul><li>Serving pages efficiently - it ’ s all about finding the greatest common factor </li></ul>
    5. 5. Caching layers <ul><li>MySQL query cache </li></ul><ul><li>PHP opcode cache </li></ul><ul><li>Drupal internal caching </li></ul><ul><li>HTTP cache (reverse proxy) </li></ul>
    6. 6. Caching layers – PHP opcode cache <ul><li>Alternative PHP Cache (APC) </li></ul><ul><li>Caches the compiled bytecode </li></ul><ul><li>Parsing and compiling PHP code is not needed, if the bytecode is in cache </li></ul><ul><li>Works generally everywhere and gives a major boost in performance </li></ul>
    7. 7. Caching layers – Drupal internal caching <ul><li>Block cache </li></ul><ul><ul><li>Global / per role / per user </li></ul></ul><ul><li>Page cache </li></ul><ul><ul><li>Anonymous users </li></ul></ul><ul><ul><li>Generally not used for logged in users </li></ul></ul><ul><li>Code level caching </li></ul><ul><li>Contrib modules </li></ul><ul><ul><li>Boost </li></ul></ul><ul><ul><li>Memcache API and Integration </li></ul></ul>
    8. 8. Caching layers - Boost <ul><li>Generated page HTML is saved as a static file </li></ul><ul><ul><li>Page loads never touch the database </li></ul></ul><ul><li>For anonymous traffic and sites with a little dynamic content </li></ul><ul><li>Easy to set up even on a cheap web hotel </li></ul><ul><ul><li>Enable the module, modify .htaccess and you ’ re done </li></ul></ul><ul><li>Highly configurable </li></ul><ul><li>For not yet cached content, serves the page first and saves HTML after that </li></ul><ul><li>Inbuilt crawler for cache warm-up </li></ul>
    9. 9. Caching layers – Memcached <ul><li>High-performance, distributed memory object caching system </li></ul><ul><li>In-memory key-value store for small chunks of arbitrary data </li></ul><ul><li>Drop-in replacement for changing Drupal cache backend </li></ul><ul><ul><li>Instead of saving cached data to DB, it goes to memcached </li></ul></ul><ul><ul><li>High-traffic sites really need to save the cache to memory </li></ul></ul><ul><li>Also for session data, Drupal variables etc. </li></ul>
    10. 10. <ul><li>Varnish Cache </li></ul><ul><li>Designed from the ground up as an HTTP accelerator </li></ul><ul><li>Stores data in virtual memory </li></ul><ul><li>Configurable with VCL (Varnish Configuration Language) </li></ul><ul><li>Edge Side Includes (ESI) </li></ul><ul><ul><li><esi include= “ /esi/some_content ” /> </li></ul></ul><ul><li>ESI integration module </li></ul><ul><ul><li>Block template will be changed to instruct Varnish to get block content from e.g. http://example.com/esi/block/xxxxxx </li></ul></ul>Caching layers – Reverse proxy
    11. 11. Cache Control module <ul><li>An alternative to ESI </li></ul><ul><li>Cheaper way to display user specific content </li></ul><ul><li>How it works </li></ul><ul><ul><li>For all users, we load the page with anonymous content hidden under a throbber </li></ul></ul><ul><ul><li>JS then checks if the user is logged in (w/ cookie) and (for anonymous users) set the anonymous content visible </li></ul></ul><ul><ul><li>For logged in users (after JS has checked the login status), it makes a single request to the backend to get the user-specific data for the page </li></ul></ul><ul><li>http://drupal.org/node/1155312 </li></ul>
    12. 12. Scaling Drupal <ul><li>MySQL </li></ul><ul><ul><li>Master-slave setup </li></ul></ul><ul><ul><li>Direct some of the SQL queries to slave </li></ul></ul><ul><ul><li>High-performance configurations. There are many good base configs available – start with them. </li></ul></ul><ul><li>Files </li></ul><ul><ul><li>Serve static files with Nginx or lighttpd </li></ul></ul><ul><ul><li>Or use reverse proxy to cache them </li></ul></ul><ul><li>Scaling by buying more hardware? </li></ul>
    13. 13. Hardware stack example Linux, Apache PHP MySQL master Server 1 memcached
    14. 14. Hardware stack example Linux, Apache PHP R/W MySQL master Front server 1 MySQL server 1 memcached
    15. 15. Hardware stack example Linux, Apache PHP R/W MySQL master Front server 1 MySQL server 1 Linux, Apache PHP Front server 2 MySQL slave MySQL server 2 Varnish HTTP cache 1 Varnish HTTP cache 2 Load balancer Front server R memcached memcached
    16. 16. Optimizing <ul><li>Profiling </li></ul><ul><ul><li>Xdebug or similar profiling tool to see what actually happens during a page load </li></ul></ul><ul><ul><li>Devel module to print a summary of all database queries executed for page request, including how many times each query was executed and how long each query took </li></ul></ul><ul><li>SQL bottlenecks </li></ul><ul><ul><li>Unnecessary repeating of same queries </li></ul></ul><ul><ul><li>Temporary tables and filesort </li></ul></ul><ul><ul><li>Table locking if using MyISAM engine in MySQL </li></ul></ul>
    17. 17. <ul><li>“ Is there a lot of logged in users or are most of them anonymous? ” </li></ul><ul><li>“ What kind of things my hosting environment allows me to do? ” </li></ul><ul><li>There ’ s no single best solution in performance matters </li></ul>
    18. 18. Thank you for your time <ul><li>Questions? </li></ul>