Pavel Prischepa discusses how to build a fast Drupal backend for applications. He describes a project with 5 million nodes and 1 million daily users that achieved 0.4 second page loads. Drupal can power mobile apps, SPAs, and websites if tasks are processed asynchronously in the background by a queue worker. The talk provides best practices for application architecture including a service-oriented queue service and cache invalidation. The most important factors for performance are optimizing only critical paths and using technologies teams are familiar with.

1. FFFast Drupal
backend
No kidding!
Pavel Prischepa
1

2. Pavel Prischepa
Partner at i20 Group
CEO at DrupalJedi
5 years of work with Drupal (and Drupal only)
on an international level
• Development
• Project management
• Drupal PR, promotion
• Drupal audit
• Consulting
2

3. Is Drupal fast?
Browser Cache layer Drupal
3

4. Real project
• 5 000 000 nodes
• 1 000 000 unique visitors per day
• 0,4 seconds per page
• Cache: MongoDB, Redis, NginX, Varnish
• Dynamic blocks: ESI, AJAX
4

5. Drupal as backend
Drupal
Mobile
application
Single-page
application
Web sites
5

6. Drupal as backend
• DRUPAL_BOOTSTRAP_FULL for each request
• 30-100 database queries per request
• Slow server response
6

7. Silk Paints:
mobile application
for drawing
7

8. Silk Paints
• Users draw paintings, save them to server
• Backend on Drupal (Services)
• 500 000 users
• 5-20 requests per second
8

9. 9

10. Silk Paints:
launch of iOS version
• Likes
• Friend lists
• Pictures moderation
• 50-100 requests per second
10

11. 11

12. Looking for an
alternative to Drupal
• Node.js
• Python
• Zend
12

13. One does not simply
use a new framework
• Change production process
• Train / hire people
• “Nabit’ shishki”
• Meanwhile...project is uncontrollable
13

14. Back to Drupal
14

15. Back to Drupal
• Coding standards
• Config
• DB integration
• Cache
• Modules
• Design, Develop, Testing, Deploy processes
15

16. Tasks are processed
Запрос
Back to Drupal
in background
Drupal
Worker
Queue Worker
Quick response Worker
16

17. Запрос
Back to Drupal
Drupal
Module JS
Drush
Daemon API
Drupal
Queue API
Drush
Daemon API
Drush
Daemon API
17

18. Silk Paints: result
• 50 requests per second
• 5-10 requests to MySQL per request
• Response time less than 1 second
• Savings on development and support
• Predictability of development
18

19. Drupal as backend
Best practice
19

20. Application architecture
• Service Oriented
Architecture (SOA)
• Queue service
(middleware)
20

21. Queue service
Drupal
Core, Contrib
modules
Module 1
Module 2
21

22. Queue service
Service
(based on Module 1)
Drupal
Node.js
Python
Drupal
Module 2
Redis
as a Lock
storage
Core, Contrib
modules
External services
22

23. Queue
Worker 1
Producer
Worker 2
Worker 3
Work queues
(worker per task)
23

24. Queue
Subscriber 1
Publisher
Subscriber 2
Subscriber 3
Publish / subscribe
(one message for all subscribers)
24

25. Producer Worker
Queue
Task
Create task
Producer
RPC
Worker Producer
Queue
Result
Worker
Receive result
25

26. Best practice
What else?
26

27. Degradation of functionality
• If load increases, non-critical functionality could be
disabled
• Several cases for disabling
• Approach allows pass the peak load successfully
27

28. Cache
• Application should work perfectly without cache
• Cache invalidation scheme
• “Cold cache” problem
• Efficient of cache using (hit/miss correlation)
28

29. Database
• Take care of database
• Denormalize it!
• Check indexes
• Do not add other databases beforehand (like
MongoDB)
• Migrate to Postgres? (…, NoSQL, memory storage)
29

30. Database
replica, partition, sharding
CAP Theorem:
Consistency
Availability
Partition tolerance
30

31. The most important
• The main causes of low performance - “krivie ruki”
• Use technologies that you know well
• Remember about business
• Optimize only what prevents sells
31

32. Thank you!
Pavel Prischepa
CEO at DrupalJedi
pprischepa@drupaljedi.com
32