EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Databases

Copyright © 2016 M/Gateway Developments Ltd
EWD 3 Training Course
Part 1
How Node.js Integrates with
Global Storage Databases
Rob Tweed
Director, M/Gateway Developments Ltd
Twitter: @rtweed

Global Storage Databases
• InterSystems Caché
– Proprietary, commercially-licensed
– Windows, Linux, OS X
– http://www.intersystems.com/our-products/cache/cache-overview/
• FIS GT.M
– Free, Open Source
– Linux
– https://www.fisglobal.com/Solutions/Banking-and-Wealth/Services/Database-Engine
• Redis, via ewd-redis-globals
– https://github.com/robtweed/ewd-redis-globals

Node.js
• Node.js:
– Server-side JavaScript
– Single thread of execution
– JavaScript is the world's most popular computer
language, by a considerable (and growing) margin

Caché
• High-performance multi-model NoSQL
database
– Hierarchical underlying data structure
• "Global Storage"
• Normally accessed via built-in language
– Cache ObjectScript
• Extension/ super-set of MUMPS language
– Considered obsolete by IT mainstream

GT.M
• High-performance multi-model NoSQL
database
– Hierarchical underlying data structure
• "Global Storage"
– Normally accessed via built-in language
• MUMPS
• Considered obsolete by IT mainstream

Redis / ewd-redis-globals
• Redis:
– Probably the world's most popular Open
Source NoSQL databases
– Also one of the fastest
– Key/Value storage primitive data structures
• ewd-redis-globals:
– Global storage functionality implemented on
top of Redis database key/value primitives
– Database only
• No dependency on Mumps language
• Open source equivalent to InterSystems'
GlobalsDB

Making Global Storage
mainstream
• Global Storage has proven to be a very
powerful, flexible, high-performance NoSQL
database model for real-world, demanding
applications
• It has been ignored by the mainstream by virtue
of its integrated access language being seen as
out-dated / obsolete
– "database baby has been thrown out with the
language bathwater"
• JavaScript as a primary language for accessing
Global Storage databases?

Node.js Interface
• Actually first implemented for free (but not Open
Source) GlobalsDB database
– Core engine from Caché
– No language parser/engine
– Now deprecated by InterSystems
• Accessed via the database engine's C++ call-in
interface
– Originally Java & .Net only
• Node.js interface developed by Chris Munt
• Interface module file: cache.node

Node.js Interface
• cache.node ported to Cache
• function() API added to allow
MUMPS/Cache Objectscript code to be
executed from JavaScript / Node.js
• Cache Object APIs also subsequently
added
• Network (TCP-based) interface is also
available

cache.node interface
Node.js
Process
Caché
Process
cache.node
module
C++call-ininterface
JavaScript
Globals Functions Objects

GT.M / NodeM
• cache.node APIs re-implemented by Open
Source community (David Wicksell) for
use with GT.M
– https://github.com/dlwicksell/nodem
– Interfaces with GT.M's C Call-in interface

NodeM interface
Node.js
Process
GT.M
Process
NodeM
module
Ccall-ininterface
JavaScript
Globals Functions

Redis / ewd-redis-globals
• cache.node APIs re-implemented as part
of the ewd-redis-globals module
– Interfaces with Redis via its standard TCP
connection

In-process Node.js Interfaces to
Cache & GT.M
• Very closely-coupled, intimate relationship
between Node.js and Caché / GT.M
– In-process
• The Node.js process and connected Caché / GT.M
process are one and the same
– Node.js process.pid === Caché/GT.M $job
– Node.js must be installed on the Caché or
GT.M server

cache.node interface
cache.node
module
C++call-ininterface
JavaScript
Globals Functions Objects
CachéNode.jsProcess

In-process Node.js Interface
• Very fast connection
– Significantly faster than a networked
connection between Node.js and Caché
– Currently between ⅓ and ¼ the performance
of native Mumps code when accessing Global
storage
– Currently limited by Google V8 API bottleneck
• https://bugs.chromium.org/p/v8/issues/detail?id=5144
• Potential for full native Mumps performance

Networked interfaces
• Available for Caché and Redis/ewd-redis-globals
• Allows database and Node.js to be on different
physical servers
• Trade-off between performance and flexibility /
redundancy

JavaScript access to Global
Storage
• The JavaScript / Node.js APIs for accessing
Global storage are identical, regardless of
database type and connection type
• Defined by the original cache.node APIs
• http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=BXJS

Installing cache.node
• Currently included with Cache build kits
– Not distributed separately
– Hoping that InterSystems will soon distribute via NPM
• Different Node.js versions have introduced API changes
– Different versions of cache.node for each main Node.js version
• 12.x
– cache0120.node
• 4.1.x
– cache410.node
• 4.2 and later 4.x versions
– Cache421.node
– Rename to cache.node and copy to main node_modules
directory
• Eg C:ewd3node_modulescache.node

Installing NodeM for GT.M
• Node.js Add-in module
• See:
– https://github.com/dlwicksell/nodem

ewd-redis-globals
• Assumes you have Redis installed
• ewd-redis-globals:
– Node.js module
• https://github.com/robtweed/ewd-redis-globals
– Implements Global Storage functionality using
Redis primitive data structures
– Provides the JavaScript APIs for accessing
Global Storage
• Complies with the cache.node API definitions

Connecting to Caché
var interface = require('cache'); // loads cache.node
var db = new interface.Cache();
// Change these parameters to match your Cache system:
var ok = db.open({
path: '/opt/cache/mgr',
username: '_SYSTEM',
password: 'SYS',
namespace: 'USER'
});
// confirm it connected OK
console.log('ok: ' + JSON.stringify(ok));
console.log(db.version());
Create a test
Script file, eg
C:ewd3test.js

Connecting to GT.M
var interface = require('nodem'); // loads NodeM
var db = new interface.Gtm();
// Assuming your process profile is automatically configured
// for GT.M:
var ok = db.open();
// once connected, the APIs for GT.M are identical to
// those for Cache
Create a test
Script file, eg
C:ewd3test.js

Connecting to Redis
var interface = require('ewd-redis-globals');
var db = new interface();
// Assuming Redis is listening on standard port and
// running on localhost:
var ok = db.open();
// once connected, the APIs for Redis are identical to
// those for Cache
Create a test
Script file, eg
C:ewd3test.js

Connecting to Global Storage
Run the test script file:
cd ewd3
node test
ok: {"ok":1,"result":1}
Node.js Adaptor for Cache: Version: 1.1.104 (CM); Cache Version: 2015.2 build 664
Connection is working!
The version string will be different for GT.M or ewd-redis-globals

Simple examples
• Set a Global Node, eg
– set ^test("foo","bar")="hello world"
var node = {
global: 'test',
subscripts: ['foo', 'bar'],
data: 'hello world'
};
db.set(node, function(error, result) {
// do something when node is created
});

Simple examples
• Get a Global Node, eg
– Set value = ^test("foo","bar")
var node = {
global: 'test',
subscripts: ['foo', 'bar']
};
db.get(node, function(error, result) {
console.log('value = ' + result.data);
});

cache.node APIs
• Open and close database
• Set, get, kill a global node
• Determine if a node is defined ($data)
• Equivalents to $order and $query (forwards and
backwards)
• Global directory listing
• Increment a global node
• Lock/unlock a global node
• Merge one global sub-tree into another

cache.node API Documentation
• http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=BXJS_refapi

cache.node also has APIs for Caché Objects
• invoke_classmethod
– invoke a class method
• create_instance
– create a new instance of an object
• open_instance
– open an existing instance of an object
• get_property
– retrieve the value of a property
• set_property
– set the value of a property
• invoke_method
– invoke a method
• save_instance
– save an instance
• close_instance
– close an instance
• Note: these are ONLY available for Caché!

cache.node APIs
• On all platforms, available as asynchronous &
synchronous versions
– If using a single Node.js process for multi-user
access, you must use the async APIs
– Synchronous APIs:
• Slightly faster
• Simpler and more intuitive to use
– Avoid call-back hell or use of Promises etc
• Allow higher-level database abstractions to be created
on top of the basic global storage primitives
– Multi-model NoSQL functionality via JavaScript
• Allow for "embedded" database functionality in Node.js

Synchronous cache.node APIs
• Normal limitations of synchronous APIs:
– They block I/O, so have disastrous results on
Node.js concurrency performance
– So, not normally usable with Node.js except
for single-user testing
• However, new run-time environments for
Node.js that provide an isolated run-time
container for a handler function can safely
use synchronous APIs, eg:
– AWS Lambda
– QEWD (previously known as ewd-xpress)

EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Databases

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Databases

Similar to EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Databases (20)

More from Rob Tweed

More from Rob Tweed (20)

Recently uploaded

Recently uploaded (20)

EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Databases