This document discusses principles for achieving high availability in Drupal applications. It recommends using version control for all code and configuration, deploying artifacts rather than code directly, and configuring infrastructure and monitoring automation through tools like Chef and Puppet. It also stresses the importance of redundancy across multiple availability zones for critical services and caching, dealing with issues like unique IDs, replication conflicts, and cache flushing across nodes. The document advocates measuring systems thoroughly with logging, metrics and alerts, and contributing optimization work back to the open source community.

1. High Available Drupal
A story from the trenches
Bram Vogelaar

2. ~$ whoami
● I used to be a Molecular Biologist
● Then became a Dev.
● Now an Ops
● Open Source Consultant @inuits.eu

3. (non-optional)
Reliability Principles

4. Use Version Control for everything

5. Deploy artifacts, not code
● Package once, deploy at will
● Native packages give you features build into every OS
● Little overhead when you automate
● Uniquely identify where files come from

6. Create artifacts from in-house resources
● Did I mention angry upstream developers yet?
● Corporate Firewalls
● Github fridays

7. The only way to get to prod are the (CI/CD) pipelines
●

8. Config != Code
● CONFIG DOES NOT BELONG IN A PACKAGE
● Config has different release cadence needs than code
● Introducing new features can now be a 2 step process
– Deploy code and feature switch

9. Config management for all other things
● Model your infrastructure, choose your poison e.g. Chef, puppet.
● Version and test your modules
● Dev/ test / UAT / prod for your infra
●A working service = automated ( Application Code + Infrastructure
Code + Security + Monitoring )

10. Measure all the Things
● Monitoring is too often an afterthought
– ENOBUDGET, ENOTIME
● We don’t go to Prod with meaningful log, metrics and alerts

11. N > 1 for all services
● Different hypervisor/Cluster?
● Different rack?
● Different SAN
● Different DC?
● Different supplier?
● Different continent?
● Whatever $customer is willing to pay for, adding 9’s will cost money

12. “ “●A working service == automated ( Application
Code + Infrastructure Code + Monitoring )

13. Surviving the
thundering herd

15. Shared Storage and Cache
●
Lots of small files on glusterfs are a really bad idea
●
Unique cache id are a great idea until they arent

16. Geo-spatial Considerations
●
There really is an optimum distance between replicas
●
Monitor your latency between DC’s
●
Look for distinct populations in your 95 percentile response time

17. Master - Master Replication woes
●
Mysql replication commonly uses staggered increments for primary
keys
●
Some drupal primary keys need to be unique and the same across all
database instances e.g semaphore
●
Have reconsilliation mechanisms in place

18. Redis Cache
●
No more DB destruction nor latency
●
This now becomes an orchestration problem
– HA vs Fail-over
– Need 2 flush cache on 2 nodes

19. Composer optimizations
●
Composer authoritative class map
composer install --no-interaction --classmap-authoritative
●
Proxy classes are not automatically added to this single source of
truth

20. Post-install Optimizations
● Deploys should be atomic, but for those small bits that aren’t we
allow for teams to define their own post-install scripts
● Review ofter and aggressively refactor
● Adding tasks that would run 2 seconds on your laptop .. but would
take 20+ minutes in prod tearing down the sites ..

21. Join us for
contribution opportunities
Thursday, October 31, 2019
9:00-18:00
Room: Europe Foyer 2
Mentored
Contribution
First Time
Contributor Workshop
General
Contribution
#DrupalContributions
9:00-14:00
Room: Diamond Lounge
9:00-18:00
Room: Europe Foyer 2

22. What did you think?
Locate this session at the DrupalCon Seattle website:
https://drupal.kuoni-congress.info/2019/program/
Take the Survey!
https://www.surveymonkey.com/r/DrupalConAmsterdam