Copyright © 2016 M/Gateway Developments Ltd
EWD 3 Training Course
Part 2
Overview of EWD 3
Rob Tweed
Director, M/Gateway Developments Ltd
Twitter: @rtweed

Copyright © 2016 M/Gateway Developments Ltd
EWD 3 Design Aims
• Modularising previous-generation (EWD.js) into
independent components
• Mix and match "buffet" of components
• Interoperable with other Node.js modules, eg Express
• No core dependence on particular
databases
• Redis, Caché and GT.M can be used as embedded multi-
model databases
• But any other Node.js-interfaced database can be used too

Copyright © 2016 M/Gateway Developments Ltd
Core EWD 3 Component
• ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
• Very high-performance Node.js Message
Queue
• Master Node.js process:
• Queue & dispatch mechanism
• Pool of persistent Worker Node.js
processes
• These process the requests that are put on the queue

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
• Creates an isolated run-time environment
for each handler function
• Each worker process only handles a single request
at a time
• Comparable to AWS Lambda's run-time container
• Concurrency is handled by the master
process only
• Its sole purpose is to queue and dispatch incoming
requests to an available worker process, and
handle the response/results from the worker

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
• Worker processes are persistent
– Started on demand
– When a worker process finishes processing a
request, it is returned to the available pool,
ready to process another request
– No startup / tear-down cost

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
• Master process API
– this.addToQueue(messageObject)
• Adds a message to the queue
– this.on('response', function(responseObject, workerPid) {..}
• Handles the response from worker
– Usually simply returning the response to the client that
sent the original request message to the master process

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
• What the worker does to process a
message is up to you
• You define a custom worker module
• Worker API:
– this.on('message', function(messageObject, send, finished) {..}
• finished(responseObject):
– Mandatory function
– Sends final response to master process
– Releases Worker process back to available pool
• send(responseObject):
– Optional function
– Sends a response to master process
– Worker process remains unavailable

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Node.js Worker Process
Node.js Worker Process

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
addToQueue(obj)

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Unavailable
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Begin processing message

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Unavailable
worker.on('message', function(messageObj) {
// process message
});
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Begin processing message

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
Unavailable / processing
addToQueue(obj)
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Worker pool size not exceeded:

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Worker pool all busy:
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
New
request
remains
in queue

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
As soon as a worker is available again:
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Available
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Worker pool all busy:
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
Finished
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
finished(responseObject);

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
Finished
this.on('response', function(responseObj) {
// do something with the response
});
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
Available
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Worker automatically returned
to the available pool
NB: Workers are NOT destroyed
after use
Removes overhead of
continually starting worker
processes

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8
Master Node.js Process
Queue
Queue
processor/
dispatcher
Available
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8
Worker processes only handle
a single request at a time
Completely isolated run-time
environment for your handler functions
Worker process becomes immediately
available for the next queued request
when processing completed

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 is a building block
• Not really useful on its own
• Completely generic, general-purpose
message queue
• Core piece of infrastructure on which the
rest of EWD 3 is built
– ewd-qoper8 can also be used independently
of the other EWD 3 modules and integrated
with any other Node.js modules, databases
etc

Copyright © 2016 M/Gateway Developments Ltd
Express + ewd-qoper8
• Express is the standard Node.js web server
– REST, Web Services, interactive applications
– Express is a Node.js module
– Customised by defining middleware
– ewd-qoper8 can be configured as Express
middleware
• Requests sent to Express by client
• Routed to ewd-qoper8
• ewd-qoper8 queues and processes them
• ewd-qoper8 response returned via Express to client

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + Express
Queue
Queue
processor/
dispatcher
Express
HTTP(S)
Interface
WebSocket
socket.io
Interface
addToQueue(req)
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + Express
+ ewd-qoper8-express
Queue
Queue
processor/
dispatcher
Express
HTTP(S)
Interface
Automates and simplifies the use of
ewd-qoper8 with Express for HTTP
requests
Adds special Express router function
app.use('/qoper8', qx.router());
ewd-qoper8-express ewd-qoper8-
express
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + Express
+ ewd-qoper8-express
Queue
Queue
processor/
dispatcher
Express
HTTP(S)
Interface
Message constructed from Express
req object – all information needed
for back-end processing by Worker:
-path
-method
-headers
-query
-body
-etc
ewd-qoper8-express ewd-qoper8-
express
Node.js Worker Process
Custom
Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 Database Dependency?
• ewd-qoper8 itself does not have a
dependency on a database
– In-memory queue
– Master and Workers are just Node.js
processes
* see later parts of this course for details on how to add queue backup / resilience

Copyright © 2016 M/Gateway Developments Ltd
Using ewd-qoper8 with Caché
• Use a separate module:
– ewd-qoper8-cache
• Normally used by worker processes
• Makes use of cache.node interface module
– cache.node is a proprietary InterSystems module that is
included with Caché
– Allows access from JavaScript/Node.js to:
» Caché extrinsic functions
» Global storage
» Caché Objects & methods
– JavaScript API defined by InterSystems
» Low-level

Copyright © 2016 M/Gateway Developments Ltd
Using ewd-qoper8 with GT.M
• Use a separate module:
– ewd-qoper8-gtm
• Normally used by worker processes
• Makes use of NodeM interface module
– NodeM is an Open Source module for use with GT.M,
which implements the cache.node APIs
– Allows access from JavaScript/Node.js to:
» GT.M extrinsic functions
» Global storage
– JavaScript API originally defined by InterSystems
» Low-level

Copyright © 2016 M/Gateway Developments Ltd
Using ewd-qoper8 with Redis
• Use a separate module:
– ewd-qoper8-redis
• Normally used by worker processes
• Makes use of ewd-redis-globals module
– ewd-redis-globals is an Open Source module for use with
Redis, which:
» Implements Global Storage functionality
» implements the cache.node APIs
– Allows access from JavaScript/Node.js to:
» Global storage
» Uses the tcp-netx module to connect to Redis
– JavaScript API originally defined by InterSystems
» Low-level

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom Worker
Module
cache.node
ewd-qoper8-cache
Caché
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-gtm
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom Worker
Module
nodem
ewd-qoper8-gtm
GT.M
ewd-qoper8
Equivalent module for use with GT.M
Uses David Wicksell’s NodeM module
which emulates cache.node for GT.M

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-redis
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom Worker
Module
tcp-netx
ewd-qoper8-redis
Redis
ewd-qoper8
Equivalent module for use with Redis
Redis behaves as a Global Storage
Database via the same cache.node APIs
ewd-redis-globals

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
function
Custom Worker
Module
ewd-qoper8
For legacy Mumps or
Caché ObjectScript
Function integration

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-gtm
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
nodem
ewd-qoper8-gtm
GT.M
function
Custom Worker
Module
ewd-qoper8
For legacy Mumps
Function integration

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
Globals
Custom Worker
Module
ewd-qoper8
Basic low-level
Mumps-centric
API

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-gtm
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
nodem
ewd-qoper8-gtm
GT.M
Globals
Custom Worker
Module
ewd-qoper8
Basic low-level
Mumps-centric
API

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-redis
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom Worker
Module
tcp-netx
ewd-qoper8-redis
Redis
ewd-qoper8
ewd-redis-globals
Globals
Basic low-level
API

Copyright © 2016 M/Gateway Developments Ltd
JavaScript-centric database access
• ewd-qoper8-cache, ewd-qoper8-gtm and
ewd-qoper8-redis auto-install ewd-
document-store
• ewd-document-store provides a
JavaScript-oriented abstraction of Global
Storage database:
– "persistent JavaScript Objects"
– Document database
• Fine-grained storage of JSON

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache
+ ewd-document-store
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Custom Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache
+ ewd-document-store
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
Persistent JavaScript Objects
&
Fine-grained Document Database
Custom Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-gtm
+ ewd-document-store
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
nodem
ewd-qoper8-gtm
GT.M
ewd-document-store
Globals
Persistent JavaScript Objects
&
Fine-grained Document Database
Custom Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-redis
+ ewd-document-store
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom Worker
Module
tcp-netx
Redis
ewd-qoper8
ewd-redis-globals
Globals
ewd-qoper8-gtm ewd-document-store
Persistent JavaScript Objects
&
Fine-grained Document Database

Copyright © 2016 M/Gateway Developments Ltd
ewd-document-store
myDocument
“a” 123
“b”
“c2” “foo2”
“d”
“c1” “foo”
“e2”
“e1”
“f2” “bar2”
“f1” “bar1”
“f2” “bar2”
“f1” “bar1”
“f3” “bar3”
myDocument = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
}
d: {
e1: {
f1: 'bar1',
f2: 'bar2'
},
e2: {
f1: 'bar1',
f2: 'bar2',
f3: 'bar3'
}
}
}

Copyright © 2016 M/Gateway Developments Ltd
ewd-document-store
myDocument
“a” 123
“b”
“c2” “foo2”
“d”
“c1” “foo”
“e2”
“e1”
“f2” “bar2”
“f1” “bar1”
“f2” “bar2”
“f1” “bar1”
“f3” “bar3”
var f3node = new this.documentStore.DocumentNode('myDocument', ['d', 'e2', 'f3']);

Copyright © 2016 M/Gateway Developments Ltd
ewd-document-store
myDocument
“a” 123
“b”
“c2” “foo2”
“d”
“c1” “foo”
“e2”
“e1”
“f2” “bar2”
“f1” “bar1”
“f2” “bar2”
“f1” “bar1”
“f3” “bar3”
var value = f3Node.value; // 'bar3'

Copyright © 2016 M/Gateway Developments Ltd
ewd-document-store
myDocument
“a” 123
“b”
“c2” “foo2”
“d”
“c1” “foo”
“e2”
“e1”
“f2” “bar2”
“f1” “bar1”
“f2” “bar2”
“f1” “bar1”
“f3” “bar3”
var value = f3Node.parent.$('f2').value; // 'bar2'

Copyright © 2016 M/Gateway Developments Ltd
ewd-document-store
myDocument
“a” 123
“b”
“c2” “foo2”
“d”
“c1” “foo”
“e2”
“e1”
“f2” “bar2”
“f1” “bar1”
“f2” “bar2”
“f1” “bar1”
“f3” “bar3”
myDocument = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
}
d: {
e1: {
f1: 'bar1',
f2: 'bar2'
},
e2: {
f1: 'bar1',
f2: 'bar2',
f3: 'bar3'
}
}
}
var docOnDisk = new this.documentStore.DocumentNode('myDocument');
var localDoc = docOnDisk.getDocument();

Copyright © 2016 M/Gateway Developments Ltd
ewd-document-store
myDocument
“a” 123
“b”
“c2” “foo2”
“d”
“c1” “foo”
“e2”
“e1”
“f2” “bar2”
“f1” “bar1”
“f2” “bar2”
“f1” “bar1”
“f3” “bar3”
myDocument = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
}
d: {
e1: {
f1: 'bar1',
f2: 'bar2'
},
e2: {
f1: 'bar1',
f2: 'bar2',
f3: 'bar3'
}
}
}
var docOnDisk = new this.documentStore.DocumentNode('myDocument');
docOnDisk.setDocument(localDoc);

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache
+ ewd-document-store
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
This powerful & intuitive abstraction
requires synchronous access
ewd-qoper8 provides the isolated
run-time container to make this possible
Custom Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-gtm
+ ewd-document-store
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
nodem
ewd-qoper8-gtm
GT.M
ewd-document-store
Globals
Custom Worker
Module
ewd-qoper8
This powerful & intuitive abstraction
requires synchronous access
ewd-qoper8 provides the isolated
run-time container to make this possible

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-redis
+ ewd-document-store
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom Worker
Module
tcp-netx
Redis
ewd-qoper8
ewd-redis-globals
Globals
ewd-qoper8-gtm ewd-document-store
This powerful & intuitive abstraction
requires synchronous access
ewd-qoper8 provides the isolated
run-time container to make this possible

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 for synchronous
database access
• A worker process only handles one
request object at a time
• Any blocking database (or other) I/O has
no impact on the processing of other
requests
– Each request independently handled, isolated
in its own worker

Copyright © 2016 M/Gateway Developments Ltd
Sessions
• EWD 3 uses a stateless model for web access
• Each request from a client may be processed by
a different ewd-qoper8 worker process
• Usually you need to maintain state information
across a sequence of client requests
• EWD Sessions provide this capability
• The ewd-session module allows you to create
and maintain Sessions
– Sessions are automatically mapped to ewd-document
DocumentNode objects

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache
+ ewd-document-store + ewd-session
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
ewd-session
Session storage projected as DocumentNode objects
Custom Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-gtm
+ ewd-document-store + ewd-session
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
nodem
ewd-qoper8-gtm
GT.M
ewd-document-store
Globals
ewd-session
Session storage projected as DocumentNode objects
Custom Worker
Module
ewd-qoper8

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-redis
+ ewd-document-store + ewd-session
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom Worker
Module
tcp-netx
Redis
ewd-qoper8
ewd-redis-globals
Globals
ewd-qoper8-gtm ewd-document-store
ewd-session
Session storage projected as DocumentNode objects

Copyright © 2016 M/Gateway Developments Ltd
ewd-session APIs
• sessions.create(applicationName)
– Creates a new session, associated with the
specified application
– Returns a session token
• Randomly-generated uuid string
• sessions.authenticate(token)
– Checks that token is valid and has not yet
expired
– returns the session object for the token, or an
error

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache*+
ewd-document-store + ewd-session
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
ewd-session
var session = sessions.create('vista');
Send back session.token
Custom Worker
Module
ewd-qoper8
* Applies also to ewd-qoper8-gtm and ewd-qoper8-redis

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache*
+ ewd-document-store + ewd-session
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
ewd-session
var result = sessions.authenticate(token);
Returns result.session if successful / result.error if not
Custom Worker
Module
ewd-qoper8
* Applies also to ewd-qoper8-gtm and ewd-qoper8-redis

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache*
+ ewd-document-store + ewd-session
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
ewd-session
Session object extends GlobalNode object
Adds session-specific properties & methods
Custom Worker
Module
ewd-qoper8
* Applies also to ewd-qoper8-gtm and ewd-qoper8-redis

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8 + ewd-qoper8-cache
+ ewd-document-store + ewd-session
+ Express
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
ewd-session
Express
HTTP(S)
Interface
WebSocket
socket.io
Interface
Custom Worker
Module
ewd-qoper8
Functions

Copyright © 2016 M/Gateway Developments Ltd
Pre-packaged super-modules
• QEWD (formally known as ewd-xpress)
– Quick and Easy Web Development platform for:
• Interactive browser-based applications
– Web-sockets or Ajax
• Web and REST services
• Embedded persistent JSON store / session cache using:
– Caché, GT.M or Redis
• Compatible with all modern JavaScript front-end frameworks
• ewd-qoper8-vistarpc
– REST interface to VistA RPCs
• ewd-feder8
– Federation / integration platform
• Extension of QEWD
– Lightweight enterprise service bus functionality

Copyright © 2016 M/Gateway Developments Ltd
QEWD (using Caché)
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
ewd-xpress
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
ewd-session
Express
HTTP(S)
Interface
WebSocket
socket.io
Interface
browserewd-client
Custom Worker
Module
ewd-qoper8
Functions

Copyright © 2016 M/Gateway Developments Ltd
QEWD (using GT.M)
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
ewd-xpress
nodem
ewd-qoper8-gtm
GT.M
ewd-document-store
Globals
ewd-session
Express
HTTP(S)
Interface
WebSocket
socket.io
Interface
browserewd-client
Custom Worker
Module
ewd-qoper8
Functions

Copyright © 2016 M/Gateway Developments Ltd
QEWD (using Redis)
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
ewd-xpress
ewd-redis-globals
ewd-qoper8-redis
Redis
ewd-document-store
Globals
ewd-session
Express
HTTP(S)
Interface
WebSocket
socket.io
Interface
browserewd-client
Custom Worker
Module
ewd-qoper8
tcp-netx

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8-vistarpc (using Caché)
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
ewd-qoper8-
vistarpc
cache.node
ewd-qoper8-cache ewd-document-store
Globals
ewd-session
Express
HTTP(S)
Interface
REST
RPCs
Interface
function
Caché & VistA
Symbol
Table
Custom Worker
Module
ewd-qoper8
Symbol Table
Function

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8-vistarpc (using GT.M)
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
ewd-qoper8-
vistarpc
nodem
ewd-qoper8-gtm ewd-document-store
Globals
ewd-session
Express
HTTP(S)
Interface
REST
RPCs
Interface
function
GT.M & VistA
Symbol
Table
Custom Worker
Module
ewd-qoper8
Symbol Table
Function

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8-vistarpc
• Login: (POST)
– Access & verify codes
– XUS SIGNON SETUP
– XUS AV CODE
– Returns error or greeting object
– If successful:
• Creates new Session
• Symbol table saved to Session
– Contains DUZ etc
– No direct access by REST client
– Only Indirect access via Authorization token
• Return Session Token & VistA greeting
20 – 30ms round trip

Copyright © 2016 M/Gateway Developments Ltd
ewd-qoper8-vistarpc
• If logged in - runRPC:
– Use Session token in Authorization header
– POST:
• RPC name
• RPC arguments object
– Symbol table restored from Session
• Contains DUZ etc
• Determines whether user can run RPC
– Standard Mumps-side "business as usual" logic
– Return RPC result
– Save Symbol table to Session

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
ewd-feder8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
Globals
ewd-session
Express
HTTP(S)
Interface
Custom Worker
Module
ewd-qoper8
REST/
Web
Service
Client
ewd-xpress
Also works with GT.M and Redis

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8
Master Process
Express
REST/
Web
Service
Client
Worker Process ewd-feder8
ewd-qoper8
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8
Master Process
Express
REST/
Web
Service
Client
Worker Process ewd-feder8
ewd-qoper8
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Named
Group

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Copy of request
sent to all servers
in group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Responses
Received
asynchronously
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Last response
Received
Aggregate response
object created
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8
Master Process
Express
REST/
Web
Service
Client
Worker Process ewd-feder8
Custom Worker
Module
ewd-qoper8
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Named
Group
Aggregate response
Returned to client via
Master process

Copyright © 2016 M/Gateway Developments Ltd
Aggregate response
{
Server_1_name: {responseObject},
Server_2_name: {errorObject},
Server_3_name: {responseObject}
}

Copyright © 2016 M/Gateway Developments Ltd
Interception / Customisation
• ewd-feder8 emits events:
– When it receives a request from a client
– When it receives a response from an end-
point server
– When an aggregate response is ready for
return to a client
• These can be used to:
– Intercept the flow through ewd-feder8; and
– Customise the behaviour of ewd-feder8

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Intercept response
Possibly modify it
And/or determine
action based on
response

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Forward new request
to group of servers

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Intercept aggregate
Response
Decide what to do

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Send new request

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Intercept response
Create final response

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Return final
response to
client

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
Cache can be
Used to store/
build up final
response during
"dance"

Copyright © 2016 M/Gateway Developments Ltd
ewd-feder8 "dance"
Master Process
Express
REST/
Web
Service
Client
REST
Server
REST
Server
REST
Server
Web Service
Server
Web Service
Server
Named
Group
Worker Process ewd-feder8
ewd-qoper8
cache.node
ewd-qoper8-cache
Caché
ewd-document-store
ewd-session
EWD Sessions
can also be used –
potentially easier to
use and manage