DRUPAL PERFORMANCE E SCALABILITÀ
STEFANO MAINARDI - STEFANO@TWINBIT.IT
MARCO GIACOMASSI - MARCO@TWINBIT.IT
About Stefano
• Drupal developer since 2007
• Twinbit co-founder
• ILDN founder
• I’m a geek and i love Open Source softwa...
About Marco
• web and open source consultant and developer
• interested in knowledge management, GIS, and integration of
s...
Agenda
• Why speed is so important
• Tips for frontend optimization
• Tips for backend optimization
• Q&A time (we hope!)
What is performance?
“The delay perceived
between an action (e.g. click)
and a meaningful response”
What is performance?
Performance is money
Why do we care about it?
Why speed is so
important?
some numbers
1 Second
decrease page load time
Amazon's calculated that a
page load slowdown of just
one second could cost it
$1.6 billion in sales each
year.
Source: ht...
Google has calculated that
by slowing its search results
by just four tenths of a
second they could lose 8
million searche...
WHAT??
Performance golden rule
80-90%of the end-user response time is spent on the frontend.
Start there.
80-90%of the end-user response time is spent on the frontend.
Source: http://www.stevesouders.com/blog/2012/0...
But first, the horrible truth
about Drupal
Drupal is
Database intensive!
!
Memory intensive!
!
Can easily become a resource hog
Why most drupal sites are slow?
Poor frontend implementation!
!
Slow mysql queries!
!
Serving dynamic contents to anonymou...
Let’s focus on frontend
1. When we request a URL a DNS lookup is done
2. Download the html and start reads from top
3. CSS Block rendering. The br...
Our goals
1. Put CSS on top (it blocks rendering)
2. Put JS on bottom (it blocks downloads)
3. Minimize the number of requ...
Put javascript on footer
Remove unneeded css
Using hook_css_alter() to limit amount of requests
Remove unneeded css
Using hook_css_alter() to limit amount of requests
Use aggregation
Built-in aggregation
Use aggregation
Advanced CSS/JS Aggregation module
http://drupal.org/project/advagg
or smart solution
Use aggregation
Advanced CSS/JS Aggregation module
http://drupal.org/project/advagg
Fully cached CSS/JS assets allow for z...
Use aggregation
Advanced CSS/JS Aggregation module
http://drupal.org/project/advagg
built-in support for !
!
HTTP Parallel...
Image requests
1. Sprites: One file for all UI elements (1 request)
2. Use automatic tools like Compass
for generate sprit...
Keep in mind to send less data
to servers, ever!
SPREAD
your assets over several domains
use CDN module
https://drupal.org/project/cdn
2 tips
1 . DNS Prefetching
2. Cookie-less domain
set in settings.php the variable
$cookie_domain = ‘www.twinbit.it’
<head>...
Monitor your application
Monitor your application
YSlow
Chrome and Firefox developer tools, are your
best friends
Google page speed tools
or use ex...
take away
“Cache saves the cache”
cache everything at every level
Let’s focus on backend
thank you for now!
Caching
Anonymous vs authenticated
Identify the type of your application in
order to
apply appropriate caching policies.
!
The mor...
Clustered architecture
Reverse proxy
A proxy server that retrieves resources on behalf of a client from
one or more servers. These resources are ...
Web servers
Drupal configuration / coding
!
● use static caching $foo = &drupal_static(__FUNCTION__);
● use cache_set/cach...
Web servers: more is better
!
● VM-C1: 1 cpu, 2 GB ram
● VM-C2: 2 cpu, 4 GB ram
● VM-C3: 4 cpu, 8 GB ram
Web servers: more is much better
Session management
Drupal is saving sessions to database. This can be used in a cluster
but we want to save database queri...
Application caching
Drupal has several caches, by default stored in database. To avoid
loading the database we can still u...
Memcache tips
!
● exclude cache_form and cache_views unless different bucket size
● php mod: Memcache (3.x) vs Memcached
●...
Database
MySQL
!
● 3% core queries goes to slaves
● tag queries to be slave query (also using query_alter)
● tag queries t...
Control servers
● control server: monitoring software (munin), phpmyadmin,
configuration engine (Puppet)
!
● staging serve...
Performances
Anon users:
!
●proxy (varnish): 2000 req/s anonymous users
●60 apache processes per cpu core, 1 req/s per pro...
Example
Target utenti
(numero di sessioni utente aperte sul
sistema in un giorno)
300000
Peso medio hit in KB 100
Page hit...
Carico
% utenti
intervallo ore intervallo
secondi
nell'intervallo
media (req/
sec)
triangolare
(req/sec)
normale (req/
sec...
Infrastruttura
VM Tipo CPU cores RAM (GB) Numero
Tot. CPU
cores
Tot. RAM
(GB)
haproxy server 2 4 2 4 8
front-end servers
f...
• caching is key to performance
• set your performance target (capacity plan)
• measure performance on changes
Conclusions
Grazie!
Grazie!
Drupal performance and scalability
Upcoming SlideShare
Loading in...5
×

Drupal performance and scalability

1,273

Published on

In this session we will present an overview from the point of view 'system that implementative on how to get the best performance from your drupal application.

We will also show examples of use cases for drupal scalable infrastructure.

Published in: Software, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,273
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
19
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Drupal performance and scalability

  1. 1. DRUPAL PERFORMANCE E SCALABILITÀ STEFANO MAINARDI - STEFANO@TWINBIT.IT MARCO GIACOMASSI - MARCO@TWINBIT.IT
  2. 2. About Stefano • Drupal developer since 2007 • Twinbit co-founder • ILDN founder • I’m a geek and i love Open Source software @stefanomainardi stefano@twinbit.it
  3. 3. About Marco • web and open source consultant and developer • interested in knowledge management, GIS, and integration of systems with CMS • Twinbit co-founder @marcogiaco marco@twinbit.it
  4. 4. Agenda • Why speed is so important • Tips for frontend optimization • Tips for backend optimization • Q&A time (we hope!)
  5. 5. What is performance?
  6. 6. “The delay perceived between an action (e.g. click) and a meaningful response” What is performance?
  7. 7. Performance is money Why do we care about it?
  8. 8. Why speed is so important?
  9. 9. some numbers
  10. 10. 1 Second decrease page load time
  11. 11. Amazon's calculated that a page load slowdown of just one second could cost it $1.6 billion in sales each year. Source: http://www.tagman.com/mdp-blog/2012/03/just-one-second-delay-in-page-load-can-cause-7-loss-in-customer-conversions/
  12. 12. Google has calculated that by slowing its search results by just four tenths of a second they could lose 8 million searches per day Source: http://www.tagman.com/mdp-blog/2012/03/just-one-second-delay-in-page-load-can-cause-7-loss-in-customer-conversions/
  13. 13. WHAT??
  14. 14. Performance golden rule
  15. 15. 80-90%of the end-user response time is spent on the frontend.
  16. 16. Start there. 80-90%of the end-user response time is spent on the frontend. Source: http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/
  17. 17. But first, the horrible truth about Drupal
  18. 18. Drupal is Database intensive! ! Memory intensive! ! Can easily become a resource hog
  19. 19. Why most drupal sites are slow? Poor frontend implementation! ! Slow mysql queries! ! Serving dynamic contents to anonymous ! users! ! Module bloat (“open buffet” syndrome)
  20. 20. Let’s focus on frontend
  21. 21. 1. When we request a URL a DNS lookup is done 2. Download the html and start reads from top 3. CSS Block rendering. The browser starts rendering once it has all the style declarations 4. Javascript blocks downloads. Make sure to serve js at last 5. Circa 4/8 assets (images / js / css /fonts) are downloaded in parallel from the same domain www.twinbit.it ? IP = 188.40.59.145 How does a browser work?
  22. 22. Our goals 1. Put CSS on top (it blocks rendering) 2. Put JS on bottom (it blocks downloads) 3. Minimize the number of requests (CSS / js / images, fonts) 4. Send less data as possible 5. Spread your assets over several domains (CDN)
  23. 23. Put javascript on footer
  24. 24. Remove unneeded css Using hook_css_alter() to limit amount of requests
  25. 25. Remove unneeded css Using hook_css_alter() to limit amount of requests
  26. 26. Use aggregation Built-in aggregation
  27. 27. Use aggregation Advanced CSS/JS Aggregation module http://drupal.org/project/advagg or smart solution
  28. 28. Use aggregation Advanced CSS/JS Aggregation module http://drupal.org/project/advagg Fully cached CSS/JS assets allow for zero file I/O if the Aggregated file already exists. Results in better page generation performance On demand generation of CSS/JS Aggregates. If the file doesn't exist it will be generated on demand. Gzip support. All aggregated files can be pre-compressed into a .gz file and served from Apache. This is faster then gzipping the file on each request.
  29. 29. Use aggregation Advanced CSS/JS Aggregation module http://drupal.org/project/advagg built-in support for ! ! HTTP Parallel Request & Threading Library https://drupal.org/project/httprl
  30. 30. Image requests 1. Sprites: One file for all UI elements (1 request) 2. Use automatic tools like Compass for generate sprites http://compass-style.org/help/tutorials/spriting/ 3. Optimize imageshttps://drupal.org/project/imageapi_optimize 4. Use font-based icons https://drupal.org/project/fontello
  31. 31. Keep in mind to send less data to servers, ever!
  32. 32. SPREAD your assets over several domains
  33. 33. use CDN module https://drupal.org/project/cdn
  34. 34. 2 tips 1 . DNS Prefetching 2. Cookie-less domain set in settings.php the variable $cookie_domain = ‘www.twinbit.it’ <head> … <link rel=“dns-prefetch” href=“//script.google.com”> … </head>
  35. 35. Monitor your application
  36. 36. Monitor your application YSlow Chrome and Firefox developer tools, are your best friends Google page speed tools or use external services like New relic and know the tools
  37. 37. take away “Cache saves the cache” cache everything at every level
  38. 38. Let’s focus on backend thank you for now!
  39. 39. Caching
  40. 40. Anonymous vs authenticated Identify the type of your application in order to apply appropriate caching policies. ! The more static the application is the more it will be served by the “client- side” caches. ! Highly dynamic application will need efficient “application-side” caching.
  41. 41. Clustered architecture
  42. 42. Reverse proxy A proxy server that retrieves resources on behalf of a client from one or more servers. These resources are then returned to the client as though they originated from the server itself. ! Varnish is widely adopted, open source and natively supported by Drupal 7.x ● from 300x to 1000x faster ● serves static pages and assets ● powerful configuration language ● ESI (https://drupal.org/project/esi) ● can act as load balancer More links https://drupal.org/node/1054886 https://drupal.org/project/varnish https://www.varnish-cache.org/trac/wiki/VarnishAndDrupal !
  43. 43. Web servers Drupal configuration / coding ! ● use static caching $foo = &drupal_static(__FUNCTION__); ● use cache_set/cache_get ● disable unneeded modules ● avoid variable_set() in frontend pages and cache invalidation ● keep app logic away from template, use hook_preprocess functions ● don’t reinvent the wheel! api.drupal.org is your best friend! ● keep drupal performance configuration in settings.php ! System ! ● configure Apache MaxClients ○ ((System RAM) – (RAM used by other processes)) / (httpd process size) = MaxClients ○ KeepAliveTimeout < 5 sec ! ● use APC ○ apc.stat ~ 0 ○ apc._num_files_hint > 1000 ! ● use Nginx if no proxy or CND More links https://groups.drupal.org/high-performance
  44. 44. Web servers: more is better
  45. 45. ! ● VM-C1: 1 cpu, 2 GB ram ● VM-C2: 2 cpu, 4 GB ram ● VM-C3: 4 cpu, 8 GB ram Web servers: more is much better
  46. 46. Session management Drupal is saving sessions to database. This can be used in a cluster but we want to save database queries. ! To do this we can use different solutions: ! ! ! ! Memcache Redis MongoDB pros very easy and fast very fast native replication cons no native replication no native replication cluster configuration Drupal compatibility mature medium medium
  47. 47. Application caching Drupal has several caches, by default stored in database. To avoid loading the database we can still use different solutions: ! ● memcache / redis ● mongodb ! ! More links https://drupal.org/project/memcache https://drupal.org/project/mongodb http://www.mongodb.org/ ! ! !
  48. 48. Memcache tips ! ● exclude cache_form and cache_views unless different bucket size ● php mod: Memcache (3.x) vs Memcached ● bucket size!!! ! ● module settings ! memcache.hash_strategy consistent memcached.sess_consistent_hash 1 ! ● Drupal settings.php $conf['memcache_stampede_protection'] = TRUE; $conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc'; Application caching: memcache
  49. 49. Database MySQL ! ● 3% core queries goes to slaves ● tag queries to be slave query (also using query_alter) ● tag queries to be slave query also using views UI ! Tune innodb buffer size ! SELECT CEILING(Total_InnoDB_Bytes*1.6/POWER(1024,3)) RIBPS FROM (SELECT SUM(data_length+index_length) Total_InnoDB_Bytes FROM information_schema.tables WHERE engine='InnoDB' and table_schema = ‘drupal_db’) A; ! ! Use MySQL query cache!
  50. 50. Control servers ● control server: monitoring software (munin), phpmyadmin, configuration engine (Puppet) ! ● staging server: should contain a copy of the production environment. This is used for testing and deploying. ! ● file storage: user files Interesting tools: ! http://gatling-tool.org/ http://newrelic.com/ ! ! ! ! ! !
  51. 51. Performances Anon users: ! ●proxy (varnish): 2000 req/s anonymous users ●60 apache processes per cpu core, 1 req/s per process ! Auth users: ! ●web server (apache): 14 apache processes per cpu core in virtual env, 1 req/s per process (consider clustering overhead). Could be much better using other hypervisors, with WMWARE we reached18/20) ●web server (apache): 40MB per process
  52. 52. Example Target utenti (numero di sessioni utente aperte sul sistema in un giorno) 300000 Peso medio hit in KB 100 Page hits per utente (numero di pagine visitate da un utente) 10 Tot hits/giorno 3M
  53. 53. Carico % utenti intervallo ore intervallo secondi nell'intervallo media (req/ sec) triangolare (req/sec) normale (req/ sec) 100% 4.5 16200 185 370 332 Example
  54. 54. Infrastruttura VM Tipo CPU cores RAM (GB) Numero Tot. CPU cores Tot. RAM (GB) haproxy server 2 4 2 4 8 front-end servers frontend, cpu bounded 4 8 6 24 48 cache servers memory bound 2 4 3 6 12 mysql servers memory bound 4 14 2 8 28 control server 2 4 2 4 8 staging environment 2 4 6 12 24 totale risorse 21 58 128 Processi (threads) apache per cpu core 15 Drupal memory footprint (MB) 64 Mysql connection fooprint (MB) 16 Example
  55. 55. • caching is key to performance • set your performance target (capacity plan) • measure performance on changes Conclusions
  56. 56. Grazie!
  57. 57. Grazie!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×