In this session we will present an overview from the point of view 'system that implementative on how to get the best performance from your drupal application. We will also show examples of use cases for drupal scalable infrastructure.

1. DRUPAL PERFORMANCE E SCALABILITÀ
STEFANO MAINARDI - STEFANO@TWINBIT.IT
MARCO GIACOMASSI - MARCO@TWINBIT.IT

2. About Stefano
• Drupal developer since 2007
• Twinbit co-founder
• ILDN founder
• I’m a geek and i love Open Source software
@stefanomainardi
stefano@twinbit.it

3. About Marco
• web and open source consultant and developer
• interested in knowledge management, GIS, and integration of
systems with CMS
• Twinbit co-founder
@marcogiaco
marco@twinbit.it

4. Agenda
• Why speed is so important
• Tips for frontend optimization
• Tips for backend optimization
• Q&A time (we hope!)

5. What is performance?

6. “The delay perceived
between an action (e.g. click)
and a meaningful response”
What is performance?

7. Performance is money
Why do we care about it?

8. Why speed is so
important?

9. some numbers

10. 1 Second
decrease page load time

11. Amazon's calculated that a
page load slowdown of just
one second could cost it
$1.6 billion in sales each
year.
Source: http://www.tagman.com/mdp-blog/2012/03/just-one-second-delay-in-page-load-can-cause-7-loss-in-customer-conversions/

12. Google has calculated that
by slowing its search results
by just four tenths of a
second they could lose 8
million searches per day
Source: http://www.tagman.com/mdp-blog/2012/03/just-one-second-delay-in-page-load-can-cause-7-loss-in-customer-conversions/

13. WHAT??

14. Performance golden rule

15. 80-90%of the end-user response time is spent on the frontend.

16. Start there.
80-90%of the end-user response time is spent on the frontend.
Source: http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/

17. But first, the horrible truth
about Drupal

19. Drupal is
Database intensive!
!
Memory intensive!
!
Can easily become a resource hog

20. Why most drupal sites are slow?
Poor frontend implementation!
!
Slow mysql queries!
!
Serving dynamic contents to anonymous !
users!
!
Module bloat (“open buffet” syndrome)

21. Let’s focus on frontend

22. 1. When we request a URL a DNS lookup is done
2. Download the html and start reads from top
3. CSS Block rendering. The browser starts rendering
once it has all the style declarations
4. Javascript blocks downloads. Make sure to serve
js at last
5. Circa 4/8 assets (images / js / css /fonts) are downloaded
in parallel from the same domain
www.twinbit.it ? IP = 188.40.59.145
How does a browser work?

23. Our goals
1. Put CSS on top (it blocks rendering)
2. Put JS on bottom (it blocks downloads)
3. Minimize the number of requests (CSS / js / images, fonts)
4. Send less data as possible
5. Spread your assets over several domains (CDN)

24. Put javascript on footer

25. Remove unneeded css
Using hook_css_alter() to limit amount of requests

26. Remove unneeded css
Using hook_css_alter() to limit amount of requests

27. Use aggregation
Built-in aggregation

28. Use aggregation
Advanced CSS/JS Aggregation module
http://drupal.org/project/advagg
or smart solution

29. Use aggregation
Advanced CSS/JS Aggregation module
http://drupal.org/project/advagg
Fully cached CSS/JS assets allow for zero file I/O if the
Aggregated file already exists. Results in better page generation
performance
On demand generation of CSS/JS Aggregates. If the file doesn't
exist it will be generated on demand.
Gzip support. All aggregated files can be pre-compressed into
a .gz file and served from Apache. This is faster then gzipping
the file on each request.

30. Use aggregation
Advanced CSS/JS Aggregation module
http://drupal.org/project/advagg
built-in support for !
!
HTTP Parallel Request & Threading Library
https://drupal.org/project/httprl

31. Image requests
1. Sprites: One file for all UI elements (1 request)
2. Use automatic tools like Compass
for generate sprites
http://compass-style.org/help/tutorials/spriting/
3. Optimize imageshttps://drupal.org/project/imageapi_optimize
4. Use font-based icons https://drupal.org/project/fontello

32. Keep in mind to send less data
to servers, ever!

33. SPREAD
your assets over several domains

34. use CDN module
https://drupal.org/project/cdn

35. 2 tips
1 . DNS Prefetching
2. Cookie-less domain
set in settings.php the variable
$cookie_domain = ‘www.twinbit.it’
<head>
…
<link rel=“dns-prefetch” href=“//script.google.com”>
…
</head>

36. Monitor your application

37. Monitor your application
YSlow
Chrome and Firefox developer tools, are your
best friends
Google page speed tools
or use external services like
New relic
and know the tools

38. take away
“Cache saves the cache”
cache everything at every level

39. Let’s focus on backend
thank you for now!

40. Caching

41. Anonymous vs authenticated
Identify the type of your application in
order to
apply appropriate caching policies.
!
The more static the application is the
more it will be served by the “client-
side” caches.
!
Highly dynamic application will need
efficient “application-side” caching.

42. Clustered architecture

43. Reverse proxy
A proxy server that retrieves resources on behalf of a client from
one or more servers. These resources are then returned to the
client as though they originated from the server itself.
!
Varnish is widely adopted, open source and natively supported
by Drupal 7.x
● from 300x to 1000x faster
● serves static pages and assets
● powerful configuration language
● ESI (https://drupal.org/project/esi)
● can act as load balancer
More links
https://drupal.org/node/1054886
https://drupal.org/project/varnish
https://www.varnish-cache.org/trac/wiki/VarnishAndDrupal
!

44. Web servers
Drupal configuration / coding
!
● use static caching $foo = &drupal_static(__FUNCTION__);
● use cache_set/cache_get
● disable unneeded modules
● avoid variable_set() in frontend pages
and cache invalidation
● keep app logic away from template, use
hook_preprocess functions
● don’t reinvent the wheel! api.drupal.org
is your best friend!
● keep drupal performance configuration
in settings.php
!
System
!
● configure Apache MaxClients
○ ((System RAM) – (RAM used by other processes)) /
(httpd process size) = MaxClients
○ KeepAliveTimeout < 5 sec
!
● use APC
○ apc.stat ~ 0
○ apc._num_files_hint > 1000
!
● use Nginx if no proxy or CND
More links
https://groups.drupal.org/high-performance

45. Web servers: more is better

46. !
● VM-C1: 1 cpu, 2 GB ram
● VM-C2: 2 cpu, 4 GB ram
● VM-C3: 4 cpu, 8 GB ram
Web servers: more is much better

47. Session management
Drupal is saving sessions to database. This can be used in a cluster
but we want to save database queries.
!
To do this we can use different solutions:
!
!
!
! Memcache Redis MongoDB
pros very easy and
fast
very fast native replication
cons no native
replication
no native replication cluster configuration
Drupal
compatibility
mature medium medium

48. Application caching
Drupal has several caches, by default stored in database. To avoid
loading the database we can still use different solutions:
!
● memcache / redis
● mongodb
!
!
More links
https://drupal.org/project/memcache
https://drupal.org/project/mongodb
http://www.mongodb.org/
!
!
!

49. Memcache tips
!
● exclude cache_form and cache_views unless different bucket size
● php mod: Memcache (3.x) vs Memcached
● bucket size!!!
!
● module settings
!
memcache.hash_strategy consistent
memcached.sess_consistent_hash 1
!
● Drupal settings.php
$conf['memcache_stampede_protection'] = TRUE;
$conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc';
Application caching: memcache

50. Database
MySQL
!
● 3% core queries goes to slaves
● tag queries to be slave query (also using query_alter)
● tag queries to be slave query also using views UI
!
Tune innodb buffer size
!
SELECT CEILING(Total_InnoDB_Bytes*1.6/POWER(1024,3)) RIBPS
FROM (SELECT SUM(data_length+index_length) Total_InnoDB_Bytes
FROM information_schema.tables WHERE engine='InnoDB' and
table_schema = ‘drupal_db’) A;
!
!
Use MySQL query cache!

51. Control servers
● control server: monitoring software (munin), phpmyadmin,
configuration engine (Puppet)
!
● staging server: should contain a copy of the production
environment. This is used for testing and deploying.
!
● file storage: user files
Interesting tools:
!
http://gatling-tool.org/
http://newrelic.com/
!
!
!
!
!
!

52. Performances
Anon users:
!
●proxy (varnish): 2000 req/s anonymous users
●60 apache processes per cpu core, 1 req/s per process
!
Auth users:
!
●web server (apache): 14 apache processes per cpu core in virtual env, 1 req/s per process
(consider clustering overhead). Could be much better using other hypervisors, with WMWARE
we reached18/20)
●web server (apache): 40MB per process

53. Example
Target utenti
(numero di sessioni utente aperte sul
sistema in un giorno)
300000
Peso medio hit in KB 100
Page hits per utente
(numero di pagine visitate da un utente)
10
Tot hits/giorno 3M

54. Carico
% utenti
intervallo ore intervallo
secondi
nell'intervallo
media (req/
sec)
triangolare
(req/sec)
normale (req/
sec)
100% 4.5 16200 185 370 332
Example

55. Infrastruttura
VM Tipo CPU cores RAM (GB) Numero
Tot. CPU
cores
Tot. RAM
(GB)
haproxy server 2 4 2 4 8
front-end servers
frontend, cpu
bounded 4 8 6 24 48
cache servers
memory
bound 2 4 3 6 12
mysql servers
memory
bound 4 14 2 8 28
control server 2 4 2 4 8
staging
environment 2 4 6 12 24
totale risorse 21 58 128
Processi (threads)
apache per cpu core
15
Drupal memory
footprint (MB)
64
Mysql connection
fooprint (MB)
16
Example

56. • caching is key to performance
• set your performance target (capacity plan)
• measure performance on changes
Conclusions

57. Grazie!

58. Grazie!