Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Copyright © 2016 M/Gateway Developments Ltd
EWD 3 Training Course
Part 1
How Node.js Integrates with
Global Storage Databa...
Copyright © 2016 M/Gateway Developments Ltd
Global Storage Databases
• InterSystems Caché
– Proprietary, commercially-lice...
Copyright © 2016 M/Gateway Developments Ltd
Node.js
• Node.js:
– Server-side JavaScript
– Single thread of execution
– Jav...
Copyright © 2016 M/Gateway Developments Ltd
Caché
• High-performance multi-model NoSQL
database
– Hierarchical underlying ...
Copyright © 2016 M/Gateway Developments Ltd
GT.M
• High-performance multi-model NoSQL
database
– Hierarchical underlying d...
Copyright © 2016 M/Gateway Developments Ltd
Redis / ewd-redis-globals
• Redis:
– Probably the world's most popular Open
So...
Copyright © 2016 M/Gateway Developments Ltd
Making Global Storage
mainstream
• Global Storage has proven to be a very
powe...
Copyright © 2016 M/Gateway Developments Ltd
Node.js Interface
• Actually first implemented for free (but not Open
Source) ...
Copyright © 2016 M/Gateway Developments Ltd
Node.js Interface
• cache.node ported to Cache
• function() API added to allow...
Copyright © 2016 M/Gateway Developments Ltd
cache.node interface
Node.js
Process
Caché
Process
cache.node
module
C++call-i...
Copyright © 2016 M/Gateway Developments Ltd
GT.M / NodeM
• cache.node APIs re-implemented by Open
Source community (David ...
Copyright © 2016 M/Gateway Developments Ltd
NodeM interface
Node.js
Process
GT.M
Process
NodeM
module
Ccall-ininterface
Ja...
Copyright © 2016 M/Gateway Developments Ltd
Redis / ewd-redis-globals
• cache.node APIs re-implemented as part
of the ewd-...
Copyright © 2016 M/Gateway Developments Ltd
In-process Node.js Interfaces to
Cache & GT.M
• Very closely-coupled, intimate...
Copyright © 2016 M/Gateway Developments Ltd
cache.node interface
cache.node
module
C++call-ininterface
JavaScript
Globals ...
Copyright © 2016 M/Gateway Developments Ltd
In-process Node.js Interface
• Very fast connection
– Significantly faster tha...
Copyright © 2016 M/Gateway Developments Ltd
Networked interfaces
• Available for Caché and Redis/ewd-redis-globals
• Allow...
Copyright © 2016 M/Gateway Developments Ltd
JavaScript access to Global
Storage
• The JavaScript / Node.js APIs for access...
Copyright © 2016 M/Gateway Developments Ltd
Installing cache.node
• Currently included with Cache build kits
– Not distrib...
Copyright © 2016 M/Gateway Developments Ltd
Installing NodeM for GT.M
• Node.js Add-in module
• See:
– https://github.com/...
Copyright © 2016 M/Gateway Developments Ltd
ewd-redis-globals
• Assumes you have Redis installed
• ewd-redis-globals:
– No...
Copyright © 2016 M/Gateway Developments Ltd
Connecting to Caché
var interface = require('cache'); // loads cache.node
var ...
Copyright © 2016 M/Gateway Developments Ltd
Connecting to GT.M
var interface = require('nodem'); // loads NodeM
var db = n...
Copyright © 2016 M/Gateway Developments Ltd
Connecting to Redis
var interface = require('ewd-redis-globals');
var db = new...
Copyright © 2016 M/Gateway Developments Ltd
Connecting to Global Storage
Run the test script file:
cd ewd3
node test
ok: {...
Copyright © 2016 M/Gateway Developments Ltd
Simple examples
• Set a Global Node, eg
– set ^test("foo","bar")="hello world"...
Copyright © 2016 M/Gateway Developments Ltd
Simple examples
• Get a Global Node, eg
– Set value = ^test("foo","bar")
var n...
Copyright © 2016 M/Gateway Developments Ltd
cache.node APIs
• Open and close database
• Set, get, kill a global node
• Det...
Copyright © 2016 M/Gateway Developments Ltd
cache.node API Documentation
• http://docs.intersystems.com/latest/csp/docbook...
Copyright © 2016 M/Gateway Developments Ltd
cache.node also has APIs for Caché Objects
• invoke_classmethod
– invoke a cla...
Copyright © 2016 M/Gateway Developments Ltd
cache.node APIs
• On all platforms, available as asynchronous &
synchronous ve...
Copyright © 2016 M/Gateway Developments Ltd
Synchronous cache.node APIs
• Normal limitations of synchronous APIs:
– They b...
Upcoming SlideShare
Loading in …5
×

EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Databases

2,693 views

Published on

This presentation is the first part of the EWD 3 Training Course. It provides an overview of how Node.js integrates with Global Storage Databases such as InterSystems' Cache, GT.M and the Redis-based ewd-redis-globals

Published in: Software

EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Databases

  1. 1. Copyright © 2016 M/Gateway Developments Ltd EWD 3 Training Course Part 1 How Node.js Integrates with Global Storage Databases Rob Tweed Director, M/Gateway Developments Ltd Twitter: @rtweed
  2. 2. Copyright © 2016 M/Gateway Developments Ltd Global Storage Databases • InterSystems Caché – Proprietary, commercially-licensed – Windows, Linux, OS X – http://www.intersystems.com/our-products/cache/cache-overview/ • FIS GT.M – Free, Open Source – Linux – https://www.fisglobal.com/Solutions/Banking-and-Wealth/Services/Database-Engine • Redis, via ewd-redis-globals – https://github.com/robtweed/ewd-redis-globals
  3. 3. Copyright © 2016 M/Gateway Developments Ltd Node.js • Node.js: – Server-side JavaScript – Single thread of execution – JavaScript is the world's most popular computer language, by a considerable (and growing) margin
  4. 4. Copyright © 2016 M/Gateway Developments Ltd Caché • High-performance multi-model NoSQL database – Hierarchical underlying data structure • "Global Storage" • Normally accessed via built-in language – Cache ObjectScript • Extension/ super-set of MUMPS language – Considered obsolete by IT mainstream
  5. 5. Copyright © 2016 M/Gateway Developments Ltd GT.M • High-performance multi-model NoSQL database – Hierarchical underlying data structure • "Global Storage" – Normally accessed via built-in language • MUMPS • Considered obsolete by IT mainstream
  6. 6. Copyright © 2016 M/Gateway Developments Ltd Redis / ewd-redis-globals • Redis: – Probably the world's most popular Open Source NoSQL databases – Also one of the fastest – Key/Value storage primitive data structures • ewd-redis-globals: – Global storage functionality implemented on top of Redis database key/value primitives – Database only • No dependency on Mumps language • Open source equivalent to InterSystems' GlobalsDB
  7. 7. Copyright © 2016 M/Gateway Developments Ltd Making Global Storage mainstream • Global Storage has proven to be a very powerful, flexible, high-performance NoSQL database model for real-world, demanding applications • It has been ignored by the mainstream by virtue of its integrated access language being seen as out-dated / obsolete – "database baby has been thrown out with the language bathwater" • JavaScript as a primary language for accessing Global Storage databases?
  8. 8. Copyright © 2016 M/Gateway Developments Ltd Node.js Interface • Actually first implemented for free (but not Open Source) GlobalsDB database – Core engine from Caché – No language parser/engine – Now deprecated by InterSystems • Accessed via the database engine's C++ call-in interface – Originally Java & .Net only • Node.js interface developed by Chris Munt • Interface module file: cache.node
  9. 9. Copyright © 2016 M/Gateway Developments Ltd Node.js Interface • cache.node ported to Cache • function() API added to allow MUMPS/Cache Objectscript code to be executed from JavaScript / Node.js • Cache Object APIs also subsequently added • Network (TCP-based) interface is also available
  10. 10. Copyright © 2016 M/Gateway Developments Ltd cache.node interface Node.js Process Caché Process cache.node module C++call-ininterface JavaScript Globals Functions Objects
  11. 11. Copyright © 2016 M/Gateway Developments Ltd GT.M / NodeM • cache.node APIs re-implemented by Open Source community (David Wicksell) for use with GT.M – https://github.com/dlwicksell/nodem – Interfaces with GT.M's C Call-in interface
  12. 12. Copyright © 2016 M/Gateway Developments Ltd NodeM interface Node.js Process GT.M Process NodeM module Ccall-ininterface JavaScript Globals Functions
  13. 13. Copyright © 2016 M/Gateway Developments Ltd Redis / ewd-redis-globals • cache.node APIs re-implemented as part of the ewd-redis-globals module – Interfaces with Redis via its standard TCP connection
  14. 14. Copyright © 2016 M/Gateway Developments Ltd In-process Node.js Interfaces to Cache & GT.M • Very closely-coupled, intimate relationship between Node.js and Caché / GT.M – In-process • The Node.js process and connected Caché / GT.M process are one and the same – Node.js process.pid === Caché/GT.M $job – Node.js must be installed on the Caché or GT.M server
  15. 15. Copyright © 2016 M/Gateway Developments Ltd cache.node interface cache.node module C++call-ininterface JavaScript Globals Functions Objects CachéNode.jsProcess
  16. 16. Copyright © 2016 M/Gateway Developments Ltd In-process Node.js Interface • Very fast connection – Significantly faster than a networked connection between Node.js and Caché – Currently between ⅓ and ¼ the performance of native Mumps code when accessing Global storage – Currently limited by Google V8 API bottleneck • https://bugs.chromium.org/p/v8/issues/detail?id=5144 • Potential for full native Mumps performance
  17. 17. Copyright © 2016 M/Gateway Developments Ltd Networked interfaces • Available for Caché and Redis/ewd-redis-globals • Allows database and Node.js to be on different physical servers • Trade-off between performance and flexibility / redundancy
  18. 18. Copyright © 2016 M/Gateway Developments Ltd JavaScript access to Global Storage • The JavaScript / Node.js APIs for accessing Global storage are identical, regardless of database type and connection type • Defined by the original cache.node APIs • http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=BXJS
  19. 19. Copyright © 2016 M/Gateway Developments Ltd Installing cache.node • Currently included with Cache build kits – Not distributed separately – Hoping that InterSystems will soon distribute via NPM • Different Node.js versions have introduced API changes – Different versions of cache.node for each main Node.js version • 12.x – cache0120.node • 4.1.x – cache410.node • 4.2 and later 4.x versions – Cache421.node – Rename to cache.node and copy to main node_modules directory • Eg C:ewd3node_modulescache.node
  20. 20. Copyright © 2016 M/Gateway Developments Ltd Installing NodeM for GT.M • Node.js Add-in module • See: – https://github.com/dlwicksell/nodem
  21. 21. Copyright © 2016 M/Gateway Developments Ltd ewd-redis-globals • Assumes you have Redis installed • ewd-redis-globals: – Node.js module • https://github.com/robtweed/ewd-redis-globals – Implements Global Storage functionality using Redis primitive data structures – Provides the JavaScript APIs for accessing Global Storage • Complies with the cache.node API definitions
  22. 22. Copyright © 2016 M/Gateway Developments Ltd Connecting to Caché var interface = require('cache'); // loads cache.node var db = new interface.Cache(); // Change these parameters to match your Cache system: var ok = db.open({ path: '/opt/cache/mgr', username: '_SYSTEM', password: 'SYS', namespace: 'USER' }); // confirm it connected OK console.log('ok: ' + JSON.stringify(ok)); console.log(db.version()); Create a test Script file, eg C:ewd3test.js
  23. 23. Copyright © 2016 M/Gateway Developments Ltd Connecting to GT.M var interface = require('nodem'); // loads NodeM var db = new interface.Gtm(); // Assuming your process profile is automatically configured // for GT.M: var ok = db.open(); // confirm it connected OK console.log('ok: ' + JSON.stringify(ok)); console.log(db.version()); // once connected, the APIs for GT.M are identical to // those for Cache Create a test Script file, eg C:ewd3test.js
  24. 24. Copyright © 2016 M/Gateway Developments Ltd Connecting to Redis var interface = require('ewd-redis-globals'); var db = new interface(); // Assuming Redis is listening on standard port and // running on localhost: var ok = db.open(); // confirm it connected OK console.log('ok: ' + JSON.stringify(ok)); console.log(db.version()); // once connected, the APIs for Redis are identical to // those for Cache Create a test Script file, eg C:ewd3test.js
  25. 25. Copyright © 2016 M/Gateway Developments Ltd Connecting to Global Storage Run the test script file: cd ewd3 node test ok: {"ok":1,"result":1} Node.js Adaptor for Cache: Version: 1.1.104 (CM); Cache Version: 2015.2 build 664 Connection is working! The version string will be different for GT.M or ewd-redis-globals
  26. 26. Copyright © 2016 M/Gateway Developments Ltd Simple examples • Set a Global Node, eg – set ^test("foo","bar")="hello world" var node = { global: 'test', subscripts: ['foo', 'bar'], data: 'hello world' }; db.set(node, function(error, result) { // do something when node is created });
  27. 27. Copyright © 2016 M/Gateway Developments Ltd Simple examples • Get a Global Node, eg – Set value = ^test("foo","bar") var node = { global: 'test', subscripts: ['foo', 'bar'] }; db.get(node, function(error, result) { console.log('value = ' + result.data); });
  28. 28. Copyright © 2016 M/Gateway Developments Ltd cache.node APIs • Open and close database • Set, get, kill a global node • Determine if a node is defined ($data) • Equivalents to $order and $query (forwards and backwards) • Global directory listing • Increment a global node • Lock/unlock a global node • Merge one global sub-tree into another
  29. 29. Copyright © 2016 M/Gateway Developments Ltd cache.node API Documentation • http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=BXJS_refapi
  30. 30. Copyright © 2016 M/Gateway Developments Ltd cache.node also has APIs for Caché Objects • invoke_classmethod – invoke a class method • create_instance – create a new instance of an object • open_instance – open an existing instance of an object • get_property – retrieve the value of a property • set_property – set the value of a property • invoke_method – invoke a method • save_instance – save an instance • close_instance – close an instance • Note: these are ONLY available for Caché!
  31. 31. Copyright © 2016 M/Gateway Developments Ltd cache.node APIs • On all platforms, available as asynchronous & synchronous versions – If using a single Node.js process for multi-user access, you must use the async APIs – Synchronous APIs: • Slightly faster • Simpler and more intuitive to use – Avoid call-back hell or use of Promises etc • Allow higher-level database abstractions to be created on top of the basic global storage primitives – Multi-model NoSQL functionality via JavaScript • Allow for "embedded" database functionality in Node.js
  32. 32. Copyright © 2016 M/Gateway Developments Ltd Synchronous cache.node APIs • Normal limitations of synchronous APIs: – They block I/O, so have disastrous results on Node.js concurrency performance – So, not normally usable with Node.js except for single-user testing • However, new run-time environments for Node.js that provide an isolated run-time container for a handler function can safely use synchronous APIs, eg: – AWS Lambda – QEWD (previously known as ewd-xpress)

×