This document provides information on improving Drupal performance through various techniques including performance testing, caching, optimizing database and server configurations, using tools like Varnish, load balancers, and CDNs, and addressing inefficient code. It also discusses a case study of using scalable cloud hosting and caching strategies to handle peak traffic for a site during major awards events.

1. Drupal Performance
Philip Norton

2. Blog at #! code (www.hashbangcode.com)
@philipnorton42 on Twitter
Creator of Vlad
Technical Lead at
Philip Norton

3. Not a sysadmin!

4. Improving
Performance

5. Improving
Performance

6. Performance
Testing

7. “Performance
enhancements without
benchmarks is little
more than guesswork.”
Probably heard it
at PHPNW...

8. Setting a benchmark.
“How does the site perform?”
Identifying bottlenecks.
“Where is the site slow?”
Comparing improvements with the benchmark.
“Has that change improved things?”
Gauging the limits of resources.
“If the current traffic doubles will the site fall over?”
1
2
3
4
Performance Testing Goals

9. Google Chrome
More than a
web browser.
Lots of profiling
tools available.
Measure all kinds of things like raw network speed,
page rendering, and JavaScript performance.

13. YSlow / ShowSlow
YSlow
YSlow is a Firebug plugin developed by Yahoo! It measures
factors that govern page speed and gives an overall score.
ShowSlow
ShowSlow can be used to store YSlow results and show
difference of scores over time.

17. Drupal Devel Module
Devel module can be used to render statistics on
the page load.
Shows all SQL queries run and how long they
took to execute.
Queries highlighted in red are slow and should
be addressed.
1
2
3

19. MySQL Process List
Provides a window on what the MySQL server is
doing, and can pinpoint problems.
> SHOW PROCESSLIST;
Keep an eye out for sleeping or lengthy processes
or processes that transmit lots of data.
Only provides a snapshot of processes.

20. MySQL Slow Query Log
Often overlooked but provides a valuable
information on queries that are slowing things down.
Set a minimum execution time for queries to be
entered into log.
Use EXPLAIN to figure out why queries are slow.

21. MySQL Slow Query Log
Add the following to my.cnf:
slow-query-log = 1
slow-query-log-file = /var/log/mysql/mysql-slow.log
long_query_time = 2
log-queries-not-using-indexes
Use mysqldumpslow to analyse the logs.

22. Xdebug Profiler
Part of the Xdebug tool.
Shows everything that happened during a page
request.
Use tools like KCachegrind and Webgrind to
view the information and identify bottlenecks.
Very resource heavy so run when needed (never
in production!).
1
2
3
4

23. XHProf
Generates profiling data for PHP.
Performant enough to run on production
environment.
Can profile 1 request in 100 or 1000 to get a
window on performance.
Use XH Gui to view XHProf results and compare
results over time.
1
2
3
4

26. Siege / JMeter / Loader.io
Benchmark a site by throwing traffic at it.
Find out how much traffic you can send before
you start getting errors.
Siege and JMeter are a local tools so all traffic
comes from one place.
Loader.io is a distributed service.
1
2
3
4

27. New Relic
Fully featured performance statistics package
for front and back end monitoring.
Has knowledge of Drupal and can identify slow
modules/views/hooks.
Lots of information available and shows
changes over time.
Paid service and can be a little expensive.
1
2
3
4

29. Improving Performance

30. There is no magic bullet!
Speed improvements are made through multiple
improvements and usually have diminishing returns.

31. Drupal Caching
Too obvious? Cache all the things!
Turn on anonymous page caching.
Use Block Cache Alter module to enable/force block
level caches.
https://www.drupal.org/project/blockcache_alter
1
2
3

32. Drupal Caching
Turn on Views query and render caching.
Use Entity Cache module to cache entities.
https://www.drupal.org/project/entitycache
1
2

33. Boost Module
Creates a static cache of page content.
Will also gzip content, including HTML, CSS,
JavaScript, XML.
https://www.drupal.org/project/boost
1
2

34. CSS / JavaScript Aggregation
Combines CSS and JavaScript files.
Less files being downloaded means:
- Less strain on the server.
- Quicker rendering times for users.
Drupal also removes comments from files which
makes them smaller.
1
2
3

35. Advanced CSS / JS Aggregation
Easy to install module.
Further reduces JS/CSS footprint by minifying and
compressing the files.
Not a substitute for slow JavaScript code.
https://www.drupal.org/project/advagg
1
2
3

36. Turn off database Logging
Drupal tends to log a lot of user activity.
PHP warnings and notices can bring a site down
if this module is active.
(You should really be removing these errors
where possible.)
Use syslog to log any errors to file based logs.
1
2
3

37. Statistics
Writes to the database on every page load.
Use an analytics package instead.
PHP Input Filter
Bypasses most caches. No. Just no.
1
2
Other Bad Modules

38. Custom Code
Where problems mostly stem from?
Cache where possible.
Try to avoid potentially heavy Drupal hooks like hook_init()
and hook_boot().
Profile and experiment, especially with lots of data. e.g. Test
with 5,000 or 50,000 nodes instead of 5.
1
2
3

39. Queues
Doing more than one thing in a
page request?
Stick that in a queue and process it later!
Built into Drupal 7, easily extendable.
Should form a key part of any batch process or API
service you implement.
1
2
3

40. PHP Micro Optimisations
Single quotes '' vs double quotes "".
Printing stuff using echo vs print.
Pre increment (++$i) vs post increment ($i++).
Creating arrays from strings using explode() vs
preg_split().

41. Micro optimizations are
basically pointless*
*unless you are Facebook.

42. PHP Micro Optimisations
Performance analysis and benchmarking should
feed into any code modifications.
Don’t optimise code this way unless you have
evidence that you need to.
Don’t sacrifice code maintainability to increase
site speed by 0.00001%.
1
2
3

43. Drupal Fields
Don’t overuse fields.
node_load() and user_load() will pull in all field
data (plus revisions) into memory.
50 fields on a node means 50 delete and 50
insert statements on every update.
Try to use shared fields where possible.
1
2
3
4

44. Cron
Don’t get your users to run cron.
Always do it via a crontab.
Use Elysia cron module to split the cron up.
Set cache clearing cron runs to hourly.
https://www.drupal.org/project/elysia_cron
1
2
3

45. PHP 5.5+
New versions of PHP are faster than ever.
Especially with objects.
You should be upgrading to take advantage of this.
Use PHP-FPM for high performance PHP.
1
2
3
4

46. PHP Opcode Cache
e.g. APC or Zend Cache.
Caches PHP scripts so they don’t have to be
recompiled when run again.
Easy to install and gives a speed increase
of up to 30%.
Make sure your cache size is big enough to
accommodate the number of files!
1
2
3
4

48. Add indexes to tables. Slow query log will tell you
where indexes are needed.
Add enough memory for MySQL to use (you don’t
want lots of read/writes to disk).
Use a master/slave setup - Drupal will use slave
server for expensive read only queries (eg. search).
MySQL Performance Tweaks
1
2
3

49. Apache Performance Tweaks
Turn off KeepAlive.
Set your max clients to a sensible value.
- Too high and server crashes when busy.
- Too low and you can’t serve traffic correctly.
Remove any modules that you don’t need.
Turn off AllowOverride and include .htaccess files in
server configuration.
1
2
3
4

50. Solr Search
Moves search out of your database server.
Much more efficient and has a bunch of nice
features (e.g. facets).
Hard to set up, but services exist to help you.
1
2
3

51. Memcache / Redis
Store your Drupal cache outside of a database.
Much faster than traditional database servers
(but only for this kind of data).
Can be scaled easily.
1
2
3

52. Split Site Up
Complex sites can be slow. Breaking them into sub-
sites allows for better experience.
Most users don’t care about the URL, as long as
they get a coherent experience.
Other sites can be hosted elsewhere.
Bakery module provides single sign on.
1
2
3
4

53. Server Side Caching
i.e. Varnish.
Provides a reverse proxy that caches all page
content.
Repeat requests never hit Drupal, Apache, or
MySQL and are instead served by Varnish.
Huge speed increase for little effort.
1
2
3
4

54. Load Balancers
Balances the load across multiple web servers.
More complex to setup, but some providers have
good solutions (Rackspace).
Stops Apache being the bottleneck.
Can have some negative effects on Drupal (e.g. file
uploads available on all servers).
1
2
3
4

55. Content Delivery Network (CDN)
Move the files or even the entire site out of your
hosting environment.
Pull vs push CDN.
CDN content nodes closer to users mean less
distance to travel for request/response.
Make sure you allow authentication traffic through.
1
2
3
4

56. Throw money at it!

57. Throw money at it!
Buying monster servers does not necessarily
mean performance improvements.
Can sometimes solve the wrong problem.
If you plaster over cracks, the cracks will still be
there.
“We’ve spent all this money on servers and the
site is still slow.” - An annoyed client
1
2
3
4

58. To Recap...
Server
PHP 5.5+ with APC
Optimised MySQL
Optimised Apache
Memcache/Redis
Solr
1
2
3
4
5
Drupal
Cache all the things
JS/CSS aggregation
Split complex sites up
Disable expensive
modules
1
2
3
4
External
Varnish
Load balancers
CDN
1
2
3

59. Essentially...

60. “Making a Drupal site fast
is mostly about removing
Drupal from the page
request.”
Philip Norton

61. Failing Gracefully
If at first you don’t succeed, fail!

62. Drupal Log Levels
Set a level that won’t print error logs to screen.

63. Maintenance theme
Config item in Drupal’s settings.php file.
Create a nice looking error page that fits in with
site theme.
Link out to social media channels.
Prevent Drupal from printing SQL error messages.
1
2
3
4

64. Varnish error page
If your web server doesn’t respond then Varnish
can be made to issue an error page.
Or even redirect traffic to another (static) site.
1
2

65. Static pages
Don’t be afraid to present users with a static page.
“We’re busy, please come back later” is better than
an error page.
With load balanced solutions you can give a
percentage of your users a static page and serve
the rest correctly.
1
2
3

66. Case Study
A quick case study about dealing with
peak traffic on awards nights

67. BAFTA: Peak traffic
High levels of traffic on major awards nights.
1,400% increase from normal traffic levels.
Needed a scalable hosting solution that could be
ramped up on selected evenings.
Rackspace cloud chosen as the hosting platform.
1
2
3
4

68. BAFTA: Peak traffic
Film and TV awards
Games Awards
Nominees Announced
Children’s Awards
1
2
4

69. BAFTA: Peak traffic
Sites are hosted on a number of web nodes.
Rackspace Load Balancer provides unified front end.
Each web node consists of a Varnish/Apache setup.
1
2
3

70. BAFTA: Peak traffic
In the run-up to awards night new nodes are
provisioned and added to the load balancer.
Caches on each web node are warmed and site is
load tested using loader.io service.
After awards night the nodes are scaled down to
normal service.
1
2
3

71. BAFTA: Peak traffic
Rackspace Load
Balancer
Web
Nodes
SolrNFSMySQL
Master/Slave

72. BAFTA: Peak traffic
Varnish
Apache
Load BalancerEach web node setup as
a Varnish/Apache pair.
Varnish configured to
produce nice error page
if Apache not available.
1
2

73. BAFTA: Peak traffic
Contingency plans put in place just in case things
went wrong.
Error page templates put in place for:
- Drupal
- Varnish
- Rackspace load balancer
Secure login module ready to disable member access
to reduce load on database server.
1
2
3

74. BAFTA: Peak traffic
Team Access on call during the weekend in case of
issues.
Rackspace also had a dedicated technician on hand.
Monitoring of servers during awards ceremony in
order to adapt to any problems.
1
2
3

75. BAFTA: Peak traffic
Cloudflare chosen as a CDN.
Caches content externally for anonymous traffic.
Cloudflare module used to clear external caches from
within Drupal.
>90% of all traffic served through Cloudflare.
1
2
3
4

76. BAFTA: Peak traffic

77. BAFTA: Results
No downtime.
Cloudflare served most of the traffic.
Site stayed responsive throughout the peak traffic
levels.
Client happy.
1
2
3
4

78. BAFTA: Peak traffic

79. Fin.