LNUG: Having Your Node.js Cake and Eating It Too

Copyright © 2017 M/Gateway Developments Ltd
Having Your Node.js Cake
and Eating It Too
Rob Tweed
M/Gateway Developments Ltd
Twitter: @rtweed
http://qewdjs.com

A bit of background
• Royal Marsden Hospital: 1980s
• Touche Ross Management Consultants
(now Deloitte): early 1990s
– NHS-wide Networking Project
• Independent consultant & software
developer since 1993
– Specialising in Web and associated
technologies, particularly in healthcare

A bit of background

A bit of background
• Node.js:
– Since August 2010 (v0.2 had just come out!)
– Gave one of the talks at LNUG inaugural
meeting

And So, 6 Years Later

Node.js Architecture
• Everything executes in a single process
– You might have 1000's of users doing stuff
concurrently
• They're all sharing the same process!

Node.js Architecture
• Non-blocking I/O
– No user activity must block the process, or everyone
grinds to a halt
• Event-driven, asynchronous logic
– Fire off the request, but don't wait for the results
• Carry on with the next task
– When results come back, handle them at the next
opportunity

Node.js = server-side JavaScript
• Ryan Dahl wasn't actually a big JavaScript fan
• However, he realised that it was a convenient
language for such an environment, as it's
designed for exactly the same kind of way of
working in the browser
• Google's V8 JavaScript engine was Open
Sourced, so he used it to provide the language
for Node.js
– (which is actually mostly written in C++)

Asynchronous Syntax in Other Languages?
• Node.js isn't unique in supporting
asynchronous logic and non-blocking I/O
• Available also in most other modern
languages
– Java
– Python, etc
• Allows efficient use of resources when
making parallel requests for resources
– Files, external web services etc

Asynchronous Syntax in Node.js
• Node.js is unique in that you have no
option but to use Asynchronous logic

Drawbacks of Node.js
• In every other language, zealots will
encourage you to use it for everything!
• But even Node's greatest exponents will
tell you not to use it for certain things

• Don't use Node.js if you have:
– CPU-intensive logic
– Complex database manipulation
• Particularly relational database handling

• Tie up the CPU and everyone else using the
process will be blocked

– Due to the limitations of asynchronous logic, eg you can't
chain functions
» Limits creation of very high-level database
abstractions

Main criticism of newbies to Node.js
• Asynchronous logic

Async is the New Sync?
• Node.js version 8 now supports
Async/Await which greatly improves the
syntax needed to handle asynchronous
logic
– Avoids "callback hell"
– Very like synchronous logic

Async/Await Doesn't solve:
• Node.js concurrency
– Still need to avoid CPU-intensive code
• High-level database abstractions
– Proper chaining of functions requires
synchronous logic

Usual Solutions
• Use a third-party queue to offload CPU-
intensive work to some other environment
– RabbitMQ
– ZeroMQ
• Use another language such as Rails for
database-intensive work

The down-sides
• Heterogeneous environment
• Multiple skill-sets
• Multiple moving parts
Node.js
RabbitMQ
Rails
Java
Database

What does Ryan Dahl Think?

"..within a single process we could handle many requests
by being completely asynchronous. I believed strongly in
this idea at the time, but over the past couple of years,
I think that’s probably not the be-all and end-all idea for
programming."

"..when I first started hearing about Go, which was around
2012, they had really easy to use abstractions, that make
"blocking I/O", because it’s all in green threads at the
interface between Go and the operating system.
I think it is actually all non-blocking I/O."

"I think Node is not the best system to build a massive server
web. I would definitely use Go for that.
And honestly, that’s basically the reason why I left Node.
It was the realization that: oh, actually, this is not the best
server side system ever."

What do I think?
I want to have my Node.js Cake and Eat it!

I want my Node.js Cake & Eat it
• I like JavaScript
• I want just one language for everything
– 1 skill set
– 1 set of moving parts
• No extra technologies, thank you
– Keep it simple
• And I want to be able to use Node.js for all
situations

The Problem is Concurrency
• All concurrent users in Node.js share the
same process

The Problem is Concurrency
• All concurrent users in Node.js share the
same process
• As it happens, Amazon Web Services
accidentally created a solution
– And nobody (including AWS) seems to have
realised the consequences of what they've
done

AWS Lambda
• "Function As A Service"
• AKA "Serverless"

AWS Lambda
• "Function As A Service"
• AKA "Serverless"
• You upload functions
• AWS will execute them
– You don't worry about how or on what physical
machine(s)
• You pay per invocation of your function(s)

AWS Lambda
• The first technology they supported was
Node.js

AWS Concurrency?
• Your function will be invoked in a private
computation container of some sort
• Your function has that container all to itself
for the duration of its execution
• No competition with other concurrent
users

Lambda Node.js Examples
• AWS examples show use of
asynchronous APIs
• Node.js users of Lambda use
asynchronous APIs and Async/Await

Lambda Node.js Examples
• AWS examples show use of
asynchronous APIs
• Node.js users of Lambda use
asynchronous APIs and Async/Await
• But unless your Lambda function really
needs to be asynchronous, why use
asynchronous logic?

Asynchronous Syntax in Other Languages?
• Available also in most other modern languages
– Java
– Python, etc
• Allows efficient use of resources when making
parallel requests for resources
– Files, external web services etc
• But no programmer in those languages
would use async logic if they didn't have to

• Why would Node.js developers use
asynchronous logic in Lambda functions if
they don't have to?
– Partly because that's what you do, right?

• Why would Node.js developers use
asynchronous logic in Lambda functions if
they don't have to?
– Partly because that's what you do, right?
– Partly because almost no synchronous APIs
exist, particularly for:
• Database integration
• Web/REST service access

Such APIs are possible

Any other way…
• To have your Node.js cake and eat it?
– Not everyone will want to use Lambda
• Would it be possible to create a locally-
available environment where my Node.js
code runs in an isolated container where
concurrency isn't an issue?

What Is QEWD?
• Essentially it's a multi-purpose Node.js-
based run-time Platform

What Is QEWD?
• Essentially it's a multi-purpose Node.js-
based run-time Platform
• Creates an isolated run-time container for
your message/request handler functions,
allowing:
– CPU-intensive work
– Database abstractions using synchronous
logic

QEWD's Architecture
• Master Process
• Pool of Worker Processes

QEWD's Architecture
• Master Process
– Handles and queues all incoming requests
from client
• HTTP/REST requests via Express or Koa.js
• WebSocket requests via socket.io
– Returns responses to client

QEWD's Architecture
• Pool of Persistent Worker Processes
– Where all the processing occurs
– A single queued request is dispatched to an
available worker process
– Each Worker process handles a single
request at a time

QEWD's Architecture
• The queue / dispatcher / worker-process
pool management part of QEWD is
handled by a module named ewd-qoper8

Master Node.js Process
Queue
Queue
processor/
dispatcher
Incoming
Requests
QEWD Architecture
Every incoming request
is passed from Express
and placed in a queue
No further processing
of requests occurs in
the master process
Express
or
Koa.js
socket.io
HTTP
REST
WebSocket

QEWD Architecture
Queue
Queue
processor/
dispatcher
Queue dispatcher is
invoked whenever a
request is added to
the queue

QEWD Architecture
Node.js Worker Process
Queue
Queue
processor/
dispatcher
Worker process
started if none
available

QEWD Architecture
Queue
Queue
processor/
dispatcher
Redis/Cache/GT.M
QEWD &
application-specific
Modules loaded
Custom
Worker
Module
Custom
Worker
Module

QEWD Architecture
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Custom
Worker
Module
Request passed
to worker
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Worker flagged as Unavailable
Custom
Worker
Module
Begin processing message
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Unavailable / processing
Another incoming
request Node.js Worker Process
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Custom
Worker
Module
Custom
Worker
Module
Custom
Worker
Module
If worker pool size not exceeded,
another worker is started
and request passed to it
Redis/Cache/GT.M
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M
If entire Worker Pool is busy:
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
If entire Worker Pool is busy:
New
requests
remain
in queue
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M
As soon as a worker is available again,
a queued message can be passed to it
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M
Available
Custom
Worker
Module
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Finished
Custom
Worker
Module
Custom
Worker
Module
Custom
Worker
Module
Redis/Cache/GT.M
A user's handler function signals
completion using the function:
finished(responseObject);
This returns the response
object to the master
process

QEWD Architecture
Queue
Queue
processor/
dispatcher
Finished
Custom
Worker
Module
Custom
Worker
Module
Custom
Worker
Module
And the response is
returned to the client
that sent the original request
(via Express/Koa/socket.io)
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Available
Custom
Worker
Module
The finished() function
also automatically returns
the worker process back
to the available pool
So it can now handle
the next queued request
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Available
Custom
Worker
Module
Worker processes, once started,
are persistent
No start-up / tear-down cost
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Node.js concurrency is handled
by the master process.
100% asynchronous logic
The master process does
almost nothing
The perfect Node.js application:
no CPU-intensive or long-
running tasks, so very
high-performance
Multiple
Concurrent
Incoming
requests

QEWD Architecture
Queue
Queue
processor/
dispatcher
Node.js concurrency is handled
by the master process.
100% asynchronous logic
The master process does
almost nothing
The perfect Node.js application:
no CPU-intensive or long-
running tasks, so very
high-performance
All the actual work happens in the
isolated worker processes
Multiple
Concurrent
Incoming
requests

QEWD Architecture
Queue
Queue
processor/
dispatcher
Available
Custom
Worker
Module
Worker processes only handle
a single request at a time
Completely isolated run-time
environment for handler functions
No need for concerns about
Node.js concurrency, so
synchronous APIs can be used
Redis/Cache/GT.M

QEWD Architecture
Queue
Queue
processor/
dispatcher
Available
Custom
Worker
Module
Redis/Cache/GT.M
Long-running or CPU-intensive
logic has no direct impact on
other worker processes

Why not just use child_process?
• To avoid the very high performance cost
of starting and tearing down each child
process
– At least 30ms to start a child_process
• Each QEWD worker is a child_process
– But keeps running, and is re-used

Why not just use Cluster?
• Cluster simply spreads the concurrent
requests across a pool of persistent
child_processes
– If one of them is CPU intensive, all other
requests in that cluster are held up
• QEWD forces each child_process to
handle a single request only
– Creates an isolated run-time environment for
each request

Why not just use RabbitMQ or ZeroMQ?
• I just want to just use a single technology:
Node.js / JavaScript
• Initial tests showed no performance
benefit in using ZeroMQ

Performance?
• ewd-qoper8 module in isolation:
– Handles the queue/master/worker architecture
• Raspberry Pi 3 Model B
– 4 core CPU
– Readily-available commodity item
– Low-cost
• Easily-replicable benchmark

Performance
• ewd-qoper8 benchmark script provided:
– How many workers? 3
– How many messages? 500,000
– Create a steady state of messages added to
the queue as they're being processed:
• Add a batch of 622 messages at a time
• Wait 100ms between each batch
– Messages are sent to workers which echo
them straight back

Performance
• 5,800 messages/second sustained
throughput
• Limiting factor is master process hitting
100% CPU
• Workers only 30% CPU, so plenty of
capacity to do real work at this rate

What can QEWD be Used For?
• Full-stack platform for browser & Native
mobile applications
– WebSocket or Ajax messaging
• Seamlessly swap between transports
• All automated and abstracted out of the way
– With security also automated for you too

• API server for REST applications
• Optional session management
– Automated token-based authentication
– Server-side (persistent JSON) session storage
• Or automated JSON Web Token based:
– Authentication
– Client-side session storage

• MicroService Platform
– With each MicroService supported by its own
shared or dedicated QEWD server
– Using JWTs
• Shared secret on each QEWD server
– Using socket.io to provide high-performance,
persistent connections between QEWD
servers
• Secured over HTTPS if required

QEWD MicroService Fabric
ewd-qoper8
queue
Express
Node.js
socket.io
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
ewd-qoper8
queue
Express
Node.js
socket.io
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
ewd-qoper8
queue
Express
Node.js
socket.io
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
User authentication
Demographics
Pharmacy
ewd-qoper8
queue
Express
Node.js
socket.io
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Cache
GT.M,
YottaDB
Redis
Node.js
Worker
Process
Client Orchestration
HTTPS
WebSocket
Connections

• Federation platform
– Providing a REST or WebSocket-based
middle tier to multiple REST-based back-end
servers
• of any type, not just QEWD servers
– One request in
• Routed to all back-end systems and responses
combined
• Performing a "dance" between federated servers

Federated Access to OpenEHR
Browser QEWD
GT.M or
Redis
ewd-qoper8
queue
qewd-ripple
Module
Express
OpenEHR
Server
AQL over
HTTP(S)
Worker
PulseTile
UI
OpenEHR
Server

• Interface to particular databases known as
Global Storage databases
– GT.M / YottaDB
– Cache / IRIS
– Redis (ewd-globals-redis module)
• All of which are abstracted as:
– Persistent JavaScript Objects
– Fine-grained Document Database
• Accessible at any level down to individual name/value pair
anywhere in a JSON object

QEWD.js
• Resilient / Audit mode
– When enabled, permanent record kept in the
database of:
• Queued requests
• Processing status
• Response(s)
– If QEWD restarted, database is examined for queued,
unprocessed requests
• Automatically re-queued
– Can specify retention period of database records

QEWD.js
• Supports all browser-side JavaScript
frameworks
• Some cool 3rd
-party tooling and support
available for React and Vue.js

Have Your Node.js Cake
And Eat It Too
http://qewdjs.com

LNUG: Having Your Node.js Cake and Eating It Too

LNUG: Having Your Node.js Cake and Eating It Too

Recommended

More Related Content

What's hot

What's hot(20)

Similar to LNUG: Having Your Node.js Cake and Eating It Too

Similar to LNUG: Having Your Node.js Cake and Eating It Too(20)

More from Rob Tweed

More from Rob Tweed(13)

Recently uploaded

Recently uploaded(20)

LNUG: Having Your Node.js Cake and Eating It Too