Drupal performance and scalability

Drupal performance and scalability
DRUPAL PERFORMANCE E SCALABILITÀ STEFANO MAINARDI - STEFANO@TWINBIT.IT MARCO GIACOMASSI - MARCO@TWINBIT.IT
About Stefano • Drupal developer since 2007 • Twinbit co-founder • ILDN founder • I’m a geek and i love Open Source software @stefanomainardi stefano@twinbit.it
About Marco • web and open source consultant and developer • interested in knowledge management, GIS, and integration of systems with CMS • Twinbit co-founder @marcogiaco marco@twinbit.it
Agenda • Why speed is so important • Tips for frontend optimization • Tips for backend optimization • Q&A time (we hope!)
What is performance?
“The delay perceived between an action (e.g. click) and a meaningful response” What is performance?
Performance is money Why do we care about it?
Why speed is so important?
some numbers
1 Second decrease page load time
Amazon's calculated that a page load slowdown of just one second could cost it $1.6 billion in sales each year. Source: http://www.tagman.com/mdp-blog/2012/03/just-one-second-delay-in-page-load-can-cause-7-loss-in-customer-conversions/
Google has calculated that by slowing its search results by just four tenths of a second they could lose 8 million searches per day Source: http://www.tagman.com/mdp-blog/2012/03/just-one-second-delay-in-page-load-can-cause-7-loss-in-customer-conversions/
WHAT??
Performance golden rule
80-90%of the end-user response time is spent on the frontend.
Start there. 80-90%of the end-user response time is spent on the frontend. Source: http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/
But first, the horrible truth about Drupal
Drupal is Database intensive! ! Memory intensive! ! Can easily become a resource hog
Why most drupal sites are slow? Poor frontend implementation! ! Slow mysql queries! ! Serving dynamic contents to anonymous ! users! ! Module bloat (“open buffet” syndrome)
Let’s focus on frontend
1. When we request a URL a DNS lookup is done 2. Download the html and start reads from top 3. CSS Block rendering. The browser starts rendering once it has all the style declarations 4. Javascript blocks downloads. Make sure to serve js at last 5. Circa 4/8 assets (images / js / css /fonts) are downloaded in parallel from the same domain www.twinbit.it ? IP = 188.40.59.145 How does a browser work?
Our goals 1. Put CSS on top (it blocks rendering) 2. Put JS on bottom (it blocks downloads) 3. Minimize the number of requests (CSS / js / images, fonts) 4. Send less data as possible 5. Spread your assets over several domains (CDN)
Put javascript on footer
Remove unneeded css Using hook_css_alter() to limit amount of requests
Remove unneeded css Using hook_css_alter() to limit amount of requests
Use aggregation Built-in aggregation
Use aggregation Advanced CSS/JS Aggregation module http://drupal.org/project/advagg or smart solution
Use aggregation Advanced CSS/JS Aggregation module http://drupal.org/project/advagg Fully cached CSS/JS assets allow for zero file I/O if the Aggregated file already exists. Results in better page generation performance On demand generation of CSS/JS Aggregates. If the file doesn't exist it will be generated on demand. Gzip support. All aggregated files can be pre-compressed into a .gz file and served from Apache. This is faster then gzipping the file on each request.
Use aggregation Advanced CSS/JS Aggregation module http://drupal.org/project/advagg built-in support for ! ! HTTP Parallel Request & Threading Library https://drupal.org/project/httprl
Image requests 1. Sprites: One file for all UI elements (1 request) 2. Use automatic tools like Compass for generate sprites http://compass-style.org/help/tutorials/spriting/ 3. Optimize imageshttps://drupal.org/project/imageapi_optimize 4. Use font-based icons https://drupal.org/project/fontello
Keep in mind to send less data to servers, ever!
SPREAD your assets over several domains
use CDN module https://drupal.org/project/cdn
2 tips 1 . DNS Prefetching 2. Cookie-less domain set in settings.php the variable $cookie_domain = ‘www.twinbit.it’ <head> … <link rel=“dns-prefetch” href=“//script.google.com”> … </head>
Monitor your application
Monitor your application YSlow Chrome and Firefox developer tools, are your best friends Google page speed tools or use external services like New relic and know the tools
take away “Cache saves the cache” cache everything at every level
Let’s focus on backend thank you for now!
Caching
Anonymous vs authenticated Identify the type of your application in order to apply appropriate caching policies. ! The more static the application is the more it will be served by the “client- side” caches. ! Highly dynamic application will need efficient “application-side” caching.
Clustered architecture
Reverse proxy A proxy server that retrieves resources on behalf of a client from one or more servers. These resources are then returned to the client as though they originated from the server itself. ! Varnish is widely adopted, open source and natively supported by Drupal 7.x ● from 300x to 1000x faster ● serves static pages and assets ● powerful configuration language ● ESI (https://drupal.org/project/esi) ● can act as load balancer More links https://drupal.org/node/1054886 https://drupal.org/project/varnish https://www.varnish-cache.org/trac/wiki/VarnishAndDrupal !
Web servers Drupal configuration / coding ! ● use static caching $foo = &drupal_static(__FUNCTION__); ● use cache_set/cache_get ● disable unneeded modules ● avoid variable_set() in frontend pages and cache invalidation ● keep app logic away from template, use hook_preprocess functions ● don’t reinvent the wheel! api.drupal.org is your best friend! ● keep drupal performance configuration in settings.php ! System ! ● configure Apache MaxClients ○ ((System RAM) – (RAM used by other processes)) / (httpd process size) = MaxClients ○ KeepAliveTimeout < 5 sec ! ● use APC ○ apc.stat ~ 0 ○ apc._num_files_hint > 1000 ! ● use Nginx if no proxy or CND More links https://groups.drupal.org/high-performance
Web servers: more is better
! ● VM-C1: 1 cpu, 2 GB ram ● VM-C2: 2 cpu, 4 GB ram ● VM-C3: 4 cpu, 8 GB ram Web servers: more is much better
Session management Drupal is saving sessions to database. This can be used in a cluster but we want to save database queries. ! To do this we can use different solutions: ! ! ! ! Memcache Redis MongoDB pros very easy and fast very fast native replication cons no native replication no native replication cluster configuration Drupal compatibility mature medium medium
Application caching Drupal has several caches, by default stored in database. To avoid loading the database we can still use different solutions: ! ● memcache / redis ● mongodb ! ! More links https://drupal.org/project/memcache https://drupal.org/project/mongodb http://www.mongodb.org/ ! ! !
Memcache tips ! ● exclude cache_form and cache_views unless different bucket size ● php mod: Memcache (3.x) vs Memcached ● bucket size!!! ! ● module settings ! memcache.hash_strategy consistent memcached.sess_consistent_hash 1 ! ● Drupal settings.php $conf['memcache_stampede_protection'] = TRUE; $conf['lock_inc'] = 'sites/all/modules/memcache/memcache-lock.inc'; Application caching: memcache
Database MySQL ! ● 3% core queries goes to slaves ● tag queries to be slave query (also using query_alter) ● tag queries to be slave query also using views UI ! Tune innodb buffer size ! SELECT CEILING(Total_InnoDB_Bytes*1.6/POWER(1024,3)) RIBPS FROM (SELECT SUM(data_length+index_length) Total_InnoDB_Bytes FROM information_schema.tables WHERE engine='InnoDB' and table_schema = ‘drupal_db’) A; ! ! Use MySQL query cache!
Control servers ● control server: monitoring software (munin), phpmyadmin, configuration engine (Puppet) ! ● staging server: should contain a copy of the production environment. This is used for testing and deploying. ! ● file storage: user files Interesting tools: ! http://gatling-tool.org/ http://newrelic.com/ ! ! ! ! ! !
Performances Anon users: ! ●proxy (varnish): 2000 req/s anonymous users ●60 apache processes per cpu core, 1 req/s per process ! Auth users: ! ●web server (apache): 14 apache processes per cpu core in virtual env, 1 req/s per process (consider clustering overhead). Could be much better using other hypervisors, with WMWARE we reached18/20) ●web server (apache): 40MB per process
Example Target utenti (numero di sessioni utente aperte sul sistema in un giorno) 300000 Peso medio hit in KB 100 Page hits per utente (numero di pagine visitate da un utente) 10 Tot hits/giorno 3M
Carico % utenti intervallo ore intervallo secondi nell'intervallo media (req/ sec) triangolare (req/sec) normale (req/ sec) 100% 4.5 16200 185 370 332 Example
Infrastruttura VM Tipo CPU cores RAM (GB) Numero Tot. CPU cores Tot. RAM (GB) haproxy server 2 4 2 4 8 front-end servers frontend, cpu bounded 4 8 6 24 48 cache servers memory bound 2 4 3 6 12 mysql servers memory bound 4 14 2 8 28 control server 2 4 2 4 8 staging environment 2 4 6 12 24 totale risorse 21 58 128 Processi (threads) apache per cpu core 15 Drupal memory footprint (MB) 64 Mysql connection fooprint (MB) 16 Example
• caching is key to performance • set your performance target (capacity plan) • measure performance on changes Conclusions
Grazie!
Grazie!

Check these out next

Drupal performance and scalability

Recommended

More Related Content

Slideshows for you(19)

Similar to Drupal performance and scalability(20)

More from Twinbit(10)

Recently uploaded(20)

Drupal performance and scalability