Improving Drupal Performances
Upcoming SlideShare
Loading in...5
×
 

Improving Drupal Performances

on

  • 2,160 views

Presentation from June 2013, Surrey, BC, Drupal Group meetup. ...

Presentation from June 2013, Surrey, BC, Drupal Group meetup.
- Some tips how to improve Drupal 7 performance.
- Get Drupal 7 working faster
- Optimize code in order to get proper responses
- Use cache (memcache, APC cache, entity cache, varnish)
- Scale Drupal horizontally in order to balance load

Statistics

Views

Total Views
2,160
Views on SlideShare
2,035
Embed Views
125

Actions

Likes
0
Downloads
15
Comments
0

6 Embeds 125

http://localhost 52
http://weblog.sialytix.com 25
http://burgerboydaddy.com 20
http://nodoquotidiano.blogspot.it 19
http://www.burgerboydaddy.com 7
http://drupal7.pierre.fr 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • The database logging modules writes all log messages to the database, when you have many errors, debugging information or modules that writes other log messages this can end up being many database inserts per page load. This then puts extra strain on your database server and cause performance issues. The recommendation here is to disable the database logging module and use the syslog module instead. Syslog also ships with Drupal core, but writes to the server log file, this will offer similar functionality at a fraction of the resources.The Statistics module is used to could how many times content has been viewed as well as collecting other data about user’s activity on the site, much like Google Analytics. This can cause multiple database writes per page load for both anonymous and authenticated users, which added unwanted load on the database. Also if using reverse proxy caching such as Varnish, statistics will not return accurate data. As the maintainer for the statistics module in Drupal core I am working to resolve these issues, and hope to have it solved in Drupal 8, and possibly rolled back to Drupal7, until then I would suggest using Google Analytics. The Google Analytics Reports module uses the API to fetch information from Google and make use of it in your site.The PHP filter module allows adding PHP code to content (nodes) and to blocks. This PHP code is stored in the database so when executed Drupal has to first load the code from the database before executing it. As you can imagine, this would be slower than just having the code in a file as a Drupal module. What makes it worse is that when the PHP filter module is used, none of the code executed gets cached. So please, put all of your code into custom modules.
  • Drupal site live or die by their database. – I think that every Drupal developer should have this printed and hanged over his desk.
  • While COUNT queries are blazingly fast on tables with MySQL's MyISAMengine. They are painfully slow when using InnoDB tables which is the recommended engine type for high traffic Drupal sites. The COUNT queries quickly degrade the more rows a table has.The Views Litepager module solves this problem for Views pagination by providing a pager option that does not require a COUNT query to be executed. This "Lite" pager is only slightly less useful than Drupal's core pager in that it does not allow you to navigate to the "last" page and does not show how many total pages of content there are. But for large sites, this small cost in features is worth the boost in performance by ridding your pages of the painfully slow (and sometimes crippling) COUNT queries.
  • After enabling caching, Drupal will begin storing database queries in a special table that allows for faster response times. One thing to note about caching is that Drupal is creating copies of the data in your database and these copies can get out of sync with the underlying data.You can help manage this by setting the 'Minimum cache lifetime' and 'Expiration of cached pages' options. And of course, the 'Clear all caches' button will force Drupal to retrieve the latest data as it rebuilds the cache.Also on the 'Performance' settings page are options to aggregate CSS and JavaScript files. This can help improve your website's speed as well, but having any of these settings active during development will cause a lot of headaches, so reserve them for use on your production site.
  • Clarification on MongoDB. You can't switch your complete database to MongoDB. MongoDB is something completely different than a relational DBMS (like MySQL). You can only replace certain pluggable components and use them to store a part of your data in MongoDB, for example fields, logs, blocks and so on. Think about MySQLMongoDB as a hybrid solution.
  • Fast 404All sites get “404 page not found” errors, although it is more common when you are launching a new site and paths to pages and images have changes. When loading a 404 page in Drupal it has to do a full “bootstrap”, load all modules, load settings, etc. If there were a few images missing on the page, this could end up with hundreds of megabytes of memory being used on the server, which doesn’t need to be used. The fast 404 module allows a very simple 404 page to be loaded which uses very little memory. Missing images, and 404 errors are not something that should be ignored.

Improving Drupal Performances Improving Drupal Performances Presentation Transcript

  •  About me  Get Drupal 7 working faster  Optimize code in order to get proper responses  Use cache (memcache, APC cache, entity cache, varnish)  Scale Drupal horizontally in order to balance load
  •  name: Vladimir Ilic  email: burger.boy.daddy@gmail.com  twitter: @burgerboydaddy  http://burgerboydaddy.com
  •  When it comes to attracting and keeping site visitors, the research is clear - fast sites win out over slower ones.  Users simply don‟t have much patience with sites that take too long to load and that impatience isn't measured in seconds, but milliseconds.  57% of mobile shoppers will wait 3 seconds or less before abandoning the site!
  •  Before, before start  Think about site performance before you start building site.  But if you already have a site up and running...  First measure how fast is site right now  https://developers.google.com/web-toolkit/speedtracer/  Yslow  Load test (blitz.io)  If it fast, responsive, and scale good under load  do not touch   If it is slow  time to change  Architecture  Software  Hardware (no matter how easy it looks like, leave HW as last one)  Oh, yes, there is MySQL; and that is most important point of your site (from every point of view including speed), but improving database performance is a part of all 3 points mentioned above.
  •  Drupal developers have tendency to add every module that exist on the planet, and you (as great Drupal developer) may want to install them all, but don‟t.  And many developers keep all of them enabled  How many times you noticed that some site has 150 – 200+ modules enabled, and only need 10-15 of them.  And every one (module) will add some execution time.
  •  Drupal core ships with some great modules but it also ships with some nasty ones. Here are 3 of the worst:  Database logging (dblog) (use syslog instead)  Statistics (use Google Analytics)  PHP Filter (put all of your code in modules!!!)
  •  Do not test module on a live production site.  For “how this module works” use some dummy development site created for those purpose only.  Properly disable / uninstall module  disable it  Go to “/admin/modules/uninstall “and uninstall it  Life-saver. Mistakenly deleted a module from the directory without disabling it first will kill site performance  SELECT name, filename FROM system WHERE type = 'module' AND status = 1 ORDER BY filename  And clean-up any modules that are still enabled but missing from the file system.
  •  Think about performance before starting to code  Think about  Modules that will be used by site (again: test somewhere else)  Is UI heavy loaded?  Do we need to connect to external data sources (web services, external databases, etc)?  Can we use CDN (content delivery network)?  Add expire headers (apache mod_expires)
  •  Drupal sites live or die by their database.  Is database optimized for speed  MySQL version,  InnoDB vs MyISAM, (phpMyAdmin still use as default MyISAM and developers do not care).  Can we tweak MySQL settings or they are locked by ISP  Can we install additional cache on server (do I have full access to server or it is on shared hosting environment)  Make sure the tables are properly indexed for faster searching.
  •  Custom tables  They are ok but  DO NOT FORGET INDEXES!!!  Use slow queries log (great to find system bottlenecks)  MyISAM vs. InnoDB My ISAM InnoDB Required full text Search Yes Require Transactions Yes frequent select queries Yes frequent insert, update, delete Yes Row Locking (multi processing on single table) Yes Relational base design Yes
  •  Tweak Your Site Images  No, you do not want to allow users to download 2-3MB image that will be scaled to 100x150px inside user browser.  Resize your images and compress them prior to uploading or use Image resize filter to do it.  Image resize filter is a smart piece of code that will automatically resize all the images on your website (not the dimensions), and makes Drupal performance much faster for your visitors.  Use Image Sprites (spritecow.com)
  •  What I build I can control.  Great one from Chris Baus “Talented, but inexperienced, developers tend to use every trick in their bag to show off their knowledge and capability. I've seen the harm that can be done when someone heads off into the wilds of a code base with such a mindset. It can be toxic, and it is often in the spirit of impressing other developers. Experienced developers make the difficult seem easy. New developers often make the routine look hard.” http://baus.net/you-cant-impress-developers/
  •  Be sure to check are there any PHP errors.  Avoid PHP errors that occur within nested foreach loops at all costs.  Once when site is in Production clean your watchdog logs calls. Every watchdog log consumes CPU resources on the web and database server (if you decided to keep logs in db  please don‟t). It also increases load time significantly.  But be sure to check your logs for warnings and errors. They are best source of info what‟s wrong.
  •  Code review is used it to improve code execution time, not how nice it looks.  Leave code formatting to automated tools.  Use your review time to improve code functionality and speed.  Parallel programming is ok if both heads are used to improve code.  Use recommended Drupal coding practices  Programming best practices (https://drupal.org/node/287350)  Coding Standards (https://drupal.org/coding-standards)
  •  When optimizing Views many of the same rules apply as optimizing database queries.  In the Views interface when you “Preview” a view it will show you the query it‟s generating and from there it may be clearer what is going on under the hood.  Be sure to optimize your views queries
  •  Example: SELECT node.nid AS nid, product.upcs AS product_upcs, product.image_url AS product_image_url, node.title AS node_title, product_price.points_only AS product_price_points_only, product_price.point_price_dollars AS product_price_point_price_dollars, product_price.point_price_points AS product_price_point_price_points, product.top_tag_line AS product_top_tag_line, product.brand_name AS product_brand_name, node.created AS node_created FROM {node} node LEFT JOIN {product} product ON node.nid = product.nid LEFT JOIN {product_price} product_price ON node.nid = product_price.nid WHERE (( (node.status = '1') AND (node.type IN ('product')) AND (product.end_date > 1371103484+0) AND (product.start_date <= 1371103484+0) )) ORDER BY product_top_tag_line ASC, node_created DESC LIMIT 8 OFFSET 0 Above query cannot be cached because where clause is bonded with current time. Do we really need that? - In this case not; For developer was easier to add current date/time but in real life we changed product start/end date only once a day!!!
  •  Look at ways not to use “distinct” and “count” in your queries.  Try a few different settings within Views, and a few different versions of the queries to see which ones load faster, you may be able to get much better performance by only slightly compromising on functionality.  Go to admin/structure/views/settings
  •  Check view caching (select as long as possible cache time).  Use Views lite pager (drupal.org/project/views_litepager/).  You do not need all items/articles loaded at the same time. 10-20 is sufficient in most cases.  Example: Product manger asked developer to remove pager and show all catalogue items during load (she hates to click “Show All” every time when she need to review all catalogue items). Young developer did that (to impress business user how he can do that in a second). Not long time after that clients started to complain about load time…
  •  No brainer: Cache your cache!!!  Always enable cache on site (at least default one for anonymous users).  Go to Configuration, Performance and enable your cache.  Once when you are there setup “Minimum cache life time”.  Compress and aggregate your JS, CSS files.  If you can leave compression to Apache server (deflate_module); it will do that faster.
  •  Drupal has a fantastic hook-able caching system, where any module can write to a standard cache table, or create a cache table, then use a specific API to write to these cache tables.  When using these cache tables it can save large complex PHP tasks or MySQL queries, but it can also create more slow queries for reading and writing the cache.  Memcache relieves that problem by storing all of these cache tables in memory. For many sites these reduces load on the server and increases the performance of the site.  Memcached has three components,  the Memcached software,  a PHP extension  and the Drupal Memcache module that work together to provide in-memory storage of database calls or rendered pages which makes it very fast.
  •  APC (Alternative PHP Cache) is a PHP OP code cache.  It is a very quick win when working with PHP and can offer a great performance boost when using Drupal. It is very much a “set it and forget it” type of application which can just be installed, enabled and left to do it‟s thing.  Many Drupal specific hosting companies will already have APC setup and running so you may even be using it without noticing.
  •  Install Drupal APC module (drupal.org/projects/apc)  Use APC for caches that do not change often and will not grow too big to avoid fragmentation.  The default setting of APC will allow you to store 32 MB for the opcode cache and the user cache combined.  Make sure you tweak this according to your website's needs.  An example configuration could be to cache 'cache' and 'cache_bootstrap' in APC; 'cache_field' and 'cache_menu' in Memcached and store 'cache_filter' in the database  Xcache and eaccelerator are other options
  •  When you have a lot of anonymous users reverse proxy cache can save you a lot of server load.  Varnish is one of the more popular solutions within the Drupal world.  Varnish sits in front of your web server application, for example Apache, nginx or lighttpd, and can run on the same server or a remote server.  It is often run on a load balancer in front of multiple web servers.  Varnish will cache pages for anonymous users, for as long as the “max_age” header is set.
  •  Varnish can be quiet complex to setup, the there are many Drupal focused tutorials.  In addition to the Varnish Cache software, you‟ll need the Drupal module (drupal.org/project/varnish).  According to the module page, Varnish will serve pages at a much faster rate than Apache - close to 3,000 page views per second!  It‟s advised to configure it to only bypass the cache for users with a cookie starting with “SESS” as these are given to authenticated Drupal users, but any module that sets “$_SESSION” in it‟s code will also set one of these cookies in Drupal, which will cause Varnish to be bypassed, and extra load to be added to the web server.  Also note that when a cached page is served from Varnish, no PHP code will get executed within Drupal, therefore things such as mobile detection, or geoip detection will not function.
  •  There are other ways of speeding up your site besides caching. MongoDB is a „NoSQL‟ type of database that can be used for your Drupal site. Yes, this means for some tables we will replace MySQL with MongoDB.  MongoDB avoids resource hogging JOIN statements by using document based records.  Another thing that really sets MongoDB apart is that the database is stored in memory, so writes are very fast. MongoDB does do occasional writes back to disk, but such traffic is greatly reduced. One criticism of keeping the database in memory is that some data would be lost in the event of a server crash. With the latest version, however, this issue can be solved with a simple configuration tweak.  Oh, and to get MongoDB up and running, you'll need the Drupal module.
  •  Content Delivery Networks are like alternative servers for your website that are spread all over the world- and serve pages to your visitors from the closest network to them.  CDNs are a segment of Cloud computing and provide lighting fast speed to websites.  A CDN is used to distribute static assets such as images, documents, CSS and JavaScript across many locations, so can be useful if you target an international audience.  If you only target a more local audience then serving your static assets from Varnish may actually work out faster.  Easy to setup using CDN module (drupal.org/project/cdn) and Amazon S3 as backend.
  •  Be Realistic About Your Hosting  If things still don‟t work then just stop where you are and think of increasing your hosting configuration. Lack of server resources is also one of the reasons for a slow website.  Shared or conventional hosting environment is known to be a performance barrier.  If you feel that even higher resource allocation is not doing any good, then you must choose some other decent hosting solution.  Cloud hosting environment best suits any type of Website. Its limitless resources and metered payment format suits the resource needs of all kinds of businesses.
  •  Extend mod_expires settings (make sure its on) in Drupal .htaccess  .htconfig move to httpd.conf eliminates Apache parse and search on every load  Search is resource intensive  Consider moving to Apache Solr  Disable “Update Manager” module in Production.  You do not want Drupal to waste time during every cron run to consult drupal.org about modules updates.  Disable developer modules (Views UI is included in that request)
  •  Disable automatic run of Cron job.  By default it runs every 3h, and it is attached to every page hit (check for passed time). Set to never and run your own cron job  Increase garbage collector run frequency for busy site inside settings.php  Session.gc_maxlifetime  Session.cache_expire  Cache warming  By using cache warmer module (https://drupal.org/project/cache_warmer) or  Manually using: #!/bin/bash wget -q -O - http://your-domain.com/sitemap.xml | egrep -o "http://your-domain.com[^<]+" | wget -q -i - -O -