Kalle Varisvirta's session for accelerating Drupal performance using Node.js and MongoDB

1. Building a Node.JS
accelerator for your
headless Drupal backend
Kalle Varisvirta @kvirta
Technology Director
Exove @exove

2. Pre-requisites
Moderate understanding of PHP and Javascript
Moderate understanding of Drupal’s internals
Some idea of how NoSQL, SOLR and Varnish work

3. What you’ll learn
What’s headless Drupal
Why use headless Drupal
How to speed up headless Drupal content delivery with
Node.JS in real life
Why to speed up Drupal content delivery with Node.JS

4. About me
Kalle Varisvirta
Technology Director at Exove
Exove is a technology company based in Finland,
Estonia and UK with over 70 professionals

5. So, what’s a
headless
Drupal?

6. Drupal
Typically Drupal gets a
HTTP request and responds
with an HTML page
Theming is done in Drupal
No front-end framework is
used
Drupal
User

7. Headless Drupal
When you add a front-end
framework and let it do the
theming, you get headless
Drupal
Drupal will then deliver
content in JSON
Drupal
User
Front-end framework

8. Why headless Drupal?
If you want full control over the display of your content
If you want to create a app-like experience
If you want to create an app :-)

9. Why not headless
Drupal?
More manual labor than regular Drupal
Half of Drupal’s functionality is ignored, as it won’t
affect how the content is displayed

10. Headless Drupal
App, you say?
When just delivering
content, Drupal doesn’t care
what happens downstream
Drupal
iOS
app
Android
app
Smart TV
app
Drupal
site

11. Headless Drupal
Drupal is delivering content
in JSON via HTTP, so
caching is a valid option for
anonymous users
Using a Varnish layer to
cache the JSON will make
your site fast enough for
most setups
Drupal
Varnish
iOS
app
Android
app
Smart TV
app
Drupal
site

12. Authenticated users
Authenticated users in Drupal, especially with long
content listings from Views, makes things complicated
If you have a lot of authenticated users concurrently
using your headless Drupal, you’ll be in trouble fast

13. Solution

14. The architecture

15. Drupal content is
indexed on every
change using the
MongoDB indexer
module
(drupal.org/project/
mongodb_indexer)

16. We’re using MongoDB
here as a storage
We could be using any
NoSQL, including
search platforms like
Elasticsearch or Apache
SOLR
We prefer a real
database for fast
indexing

17. For delivery, we opt for
Node.JS
A framework like
Express or Koa is
suggested for better
structure

18. A deeper
look

19. Drupal
Focusing solely in content management, not delivering
any web pages outside the admin

20. MongoDB
indexer
MongoDB indexer is
just a small module we
created to index
Drupal’s content
outside, without
interfering the storage
engine of Drupal

21. MongoDB
indexer
MongoDB indexer is
just a small module we
created to index
Drupal’s content
outside, without
interfering the storage
engine of Drupal

22. MongoDB
We chose MongoDB over other NoSQL systems due
to the popularity and fast development
It’s had its issues, but they’ve fixed them all
Now expecting Wired Tiger storage engine that’ll
improve the locking mechanism

23. Delivery
We have optimized data in the MongoDB
We want to deliver it out, fast
We want to be able to do some logic while delivering it

24. Node.JS
Node.JS is Javascript running on the server
The Javascript is ran by Chrome’s Javascript engine,
the V8
It’s non-blocking, event-based and when used
correctly, blazing fast

25. RPS Avg response time
Drupal
Node.JS

26. Node.JS
Cluster (npm module)
Forever (npm module)

27. Node.JS frameworks
Express
Koa

28. Node.JS
For a PHP programmer, it’s quite a change
Due to the you need to understand parallel
programming
There are npm modules there to help with that, too
During the lifetime our big project (24 months) both
Node.JS and MongoDB have evolved quite a bit

29. Node.JS
The code is very simple as it’s mostly just passing
information out from the MongoDB
If you’re building a REST api, as you should, you can
add filters to the calls and pass them on to the
MongoDB
But… you can also check for user session and fetch
user-related content

30. REST API
REST API allows writing, too, so you’ll have to handle
the writes
Most of writing in these cases are left to the Node
app -level
Obviously, writes reaching the Drupal are also
possible

31. Real world
case?

32. Video platform project
Started three years ago
Started as a headless Drupal project
Originally was supposed to feed content straight from
Drupal

33. Drupal optimizations
It’s not like we didn’t think of the performance early on
The Drupal 7 was built on MongoDB field storage and
thus standing on a fast database
The Views feeds (JSON) were coming from SOLR
backend

34. Caching feeds
Downstream clients want integration feeds limited by
time
Time attribute by seconds of the last fetch
To deal with existing content
To deal with changed content
We tried to cache to Varnish, it didn’t work at all

35. We need a new approach
Indexing outside of Drupal felt like the only way out
First we decided we’ll go with SOLR via ApacheSOLR
integration
Distribution by a simple REST API
Later on, we wrote the MongoDB indexer and switched
to MongoDB fully

36. Video content management
system
(Drupal 7)
Linear television
data
(ERP)
Drupal 7 site
Drupal 7 site
Wordpress site
iOS app
Android app
Samsung
SmartTV app
MongoDB and Node.JS
RESTAPI
Indexing is done using a
Drupal module, MongoDB
indexer

37. Recap

38. Recap
Headless Drupal is when you use Drupal only for the
data and theme/display somewhere else
Headless Drupal needs performance that Drupal might
not be able to provide
MongoDB indexer will index your Drupal entities to
MongoDB
From MongoDB, it’s easy to distribute your data with
Node.JS and enjoy the high performance

39. Thank you!
Questions?