SlideShare a Scribd company logo
Copyright © 2017 M/Gateway Developments Ltd
^ redux:
Data Persistence as a
Language Feature
Rob Tweed
M/Gateway Developments Ltd
Twitter: @rtweed
http://qewdjs.com
Copyright © 2017 M/Gateway Developments Ltd
A bit of background
• Royal Marsden Hospital: 1980s
• Touche Ross Management Consultants
(now Deloitte): early 1990s
– NHS-wide Networking Project
• Independent consultant & software
developer since 1993
– Specialising in Web and associated
technologies, particularly in healthcare
Copyright © 2017 M/Gateway Developments Ltd
Products
• WebLink:
– Co-creator of the original web connectivity for Caché
• WebLink Developer:
– Web Application Development framework
• pre-cursor to CSP
• eXtc:
– XML DOM framework
• EWD:
– 2nd
-generation web application development framework
– XML based custom pages
– Compiled to CSP or WebLink (or even PHP)
Copyright © 2017 M/Gateway Developments Ltd
Example Usage
Copyright © 2017 M/Gateway Developments Ltd
A bit of background
• Node.js:
– Since August 2010 (v0.2 had just come out!)
– Pioneered the integration of Caché and
Node.js
– Was instrumental in getting the cache.node
interface developed
• Originally for GlobalsDB
Copyright © 2017 M/Gateway Developments Ltd
Language or Database?
• Caché ObjectScript:
– programming language with integrated
database?
• Caché Database:
– Database with integrated scripting language
for its manipulation (Caché ObjectScript)?
Copyright © 2017 M/Gateway Developments Ltd
Language
• Caché ObjectScript shows its age
– Rightly or wrongly, opinion from IT
Mainstream developers is not favourable
– Tainted, in particular, by this:
• https://thedailywtf.com/articles/comments/A_Case_of_the_MUMPS/8
– As a result of the association with Mumps by,
eg:
– https://en.wikipedia.org/wiki/Cach%C3%A9_ObjectScript
Copyright © 2017 M/Gateway Developments Ltd
Language
• Caché ObjectScript typical objections:
– Limited
– Old-Fashioned
– To a modern developer, it's:
• A step back in time
• Lacking in many of the language capabilities they
expect
• Lacks an extensive ecosystem of pre-built
functions and modules
• A career-limiting move
Copyright © 2017 M/Gateway Developments Ltd
Database
• Uniquely powerful
• The original NoSQL database
– Pre-dated the NoSQL movement by several
decades
• "Universal NoSQL"
– AKA "Multi-Model database"
Copyright © 2017 M/Gateway Developments Ltd
Universal NoSQL
Published in 2010
Copyright © 2017 M/Gateway Developments Ltd
Database
• But in order to manipulate and use that
database to its fullest capabilities, you
need to learn its scripting language
– Caché ObjectScript
• A key reason why, as a database, it's
almost unknown in the mainstream of IT
– I've yet to meet a JavaScript developer who has even
heard of Caché
Copyright © 2017 M/Gateway Developments Ltd
Solution?
• Recurring suggestions within the
developer community that Caché
ObjectScript needs modernising
– New language capabilities
– Bring its syntax into line with modern
languages
– Create an ecosystem of functions & modules
Copyright © 2017 M/Gateway Developments Ltd
My Opinion
• This is the wrong approach
• Nobody in the mainstream of IT wants to
learn a new language just to access a
database
• They'd expect to be able to use a modern
language
Copyright © 2017 M/Gateway Developments Ltd
Language or Database?
• Is Caché first and foremost a database or
a language?
– Misses a unique feature of Caché which
provides its key power and which:
• We take for granted
• The mainstream of IT don't "get"
Copyright © 2017 M/Gateway Developments Ltd
What is a database?
• It's that thing "over there" that persists our
data
Copyright © 2017 M/Gateway Developments Ltd
What is a database?
• It's expected to be "over there"
• It's a separate server / sub-system
• It's accessed:
– over a network connection
– via some protocol and/or syntax
• REST
• SQL
Copyright © 2017 M/Gateway Developments Ltd
So what?
• Creates an explicit distinction between
stuff you do in your programming
language and persisted data
• Creates separate disciplines
– Programming experts
– Database experts
– Many people span both, but, to many
programmers, databases are a mystery
Copyright © 2017 M/Gateway Developments Ltd
The Unique Feature
• The underlying "Global Storage" of Caché
• Why?
– It blurs the distinction between what's permanently
persisted and what's in memory
– In memory: sparse arrays
– On disk: identical sparse arrays
– Only difference in syntax terms is a preceeding ^
Copyright © 2017 M/Gateway Developments Ltd
Revise the Definition
• Yes it's a database and a language, but it
warrants a new description:
• It's a language with data persistence:
– as a seamless language feature
Copyright © 2017 M/Gateway Developments Ltd
The Problem
• It's a language with data persistence:
– as a seamless language feature
• But the language is obsolete and a barrier
to new entry and mainstream adoption
Copyright © 2017 M/Gateway Developments Ltd
The Solution
• It's a language with data persistence:
– as a seamless language feature
• Add a modern language as a first-class
alternative to Caché ObjectScript
– With data persistence as a similarly seamless
feature
– An equivalent, in that language, to ^
Copyright © 2017 M/Gateway Developments Ltd
The Question
• What Language would fit the bill?
Copyright © 2017 M/Gateway Developments Ltd
From the Mainstream Perspective
• One of the currently "big" languages
– C / C++ too low level
– Java
– Ruby
– Go
– Python
– JavaScript
Copyright © 2017 M/Gateway Developments Ltd
From our Perspective
• Must be as fast as Native Caché
ObjectScript
– For general-purpose coding
– For database access
• > 1 million Global Nodes per second
Copyright © 2017 M/Gateway Developments Ltd
From both Perspectives
• Should be a language suitable for all the
kinds of application that Caché
ObjectScript is currently used for:
– Desktop
– Browser-based
– Mobile
– Batch processing / server-side heavy-lifting
Copyright © 2017 M/Gateway Developments Ltd
From both Perspectives
• Should map seamlessly between Global
Storage and a commonly-accepted native
in-memory representation with that
language
– No "impedance mismatch"
• That mapping should make sense within
the context of that language
– ie laguage-centric
Copyright © 2017 M/Gateway Developments Ltd
Global Storage Key Features
• Schema-Free
• Hierarchically-structured
• Dynamic creation / destruction
• Ideally suited for "late-bound" access
Copyright © 2017 M/Gateway Developments Ltd
Suitability: Java
Feature
Adoption Widespread
General-purpose Code Speed Excellent
Application suitability Not used in browser, but otherwise
widespread acceptance and adoption
Data persistence impedance
mismatch
Designed around pre-defined,
compiled classes / schema, ie early-
bound access
Copyright © 2017 M/Gateway Developments Ltd
Suitability: Ruby
Feature
Adoption Fairly limited by comparison to other
languages
Well-liked by its users
General-purpose Code Speed Poor
Application suitability Not used in browser. Poor choice for
mobile development
Data persistence impedance
mismatch
Good
Copyright © 2017 M/Gateway Developments Ltd
Suitability: Go
Feature
Adoption Fairly limited by comparison to other
languages. Well-liked by its users
General-purpose Code Speed Excellent
Application suitability Not used in browser. Otherwise very
good
Data persistence impedance
mismatch
Not so good – designed for pre-
defined, pre-compiled mappings
Miscellaneous Can we depend on Google
supporting it in the long term?
Copyright © 2017 M/Gateway Developments Ltd
Suitability: Python
Feature
Adoption Pretty high and growing. Well-liked
by its users
General-purpose Code Speed Poor
Application suitability Not used in browser. Not suitable for
mobile development
Data persistence impedance
mismatch
Good – dictionaries / JSON map well
Copyright © 2017 M/Gateway Developments Ltd
Suitability: JavaScript
Feature
Adoption High and still growing. Node.js now
massively popular. Language has
some quirks!
General-purpose Code Speed Node.js performance is excellent
Application suitability Used in browser. Ideal for mobile
development. Usual cited deficiency
is CPU-heavy processing, but that
can be addressed
Data persistence impedance
mismatch
JavaScript objects are dynamic and
map almost perfectly onto Global
Storage
Miscellaneous Only 1 language skill for entire
development stack, front and back
(Node.js at back-end)
Copyright © 2017 M/Gateway Developments Ltd
JavaScript as a Native
Language for Caché
• It's a very good fit, for all sorts of reasons:
– Interpreted
– Loosely-typed in a similar way
– JavaScript Objects (JSON)
• Schema-free
• Dynamic
• hierarchical
Copyright © 2017 M/Gateway Developments Ltd
Global Storage
myGlobal("a")=123
myGlobal("b","c1")="foo"
myGlobal("b","c2")="foo2"
myGlobal("d","e1","f1")="bar1"
myGlobal("d","e1","f2")="bar2"
myGlobal("d","e2","f1")="bar1"
myGlobal("d","e2","f2")="bar2"
myGlobal("d","e2","f3")="bar3"
myGlobal
"a" 123
"b"
"c2" "foo2"
"d"
"c1" "foo"
"e2"
"e1"
"f2" "bar2"
"f1" "bar1"
"f2" "bar2"
"f1" "bar1"
"f3" "bar3"
Hierarchical diagram representing a Global
Copyright © 2017 M/Gateway Developments Ltd
myGlobal = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
}
d: {
e1: {
f1: 'bar1',
f2: 'bar2'
},
e2: {
f1: 'bar1',
f2: 'bar2',
f3: 'bar3'
}
}
}
myGlobal("a")=123
myGlobal("b","c1")="foo"
myGlobal("b","c2")="foo2"
myGlobal("d","e1","f1")="bar1"
myGlobal("d","e1","f2")="bar2"
myGlobal("d","e2","f1")="bar1"
myGlobal("d","e2","f2")="bar2"
myGlobal("d","e2","f3")="bar3"
One to one correspondence between any tree of Global nodes
and a Javascript object
Copyright © 2017 M/Gateway Developments Ltd
myGlobal = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
}
d: {
e1: {
f1: 'bar1',
f2: 'bar2'
},
e2: {
f1: 'bar1',
f2: 'bar2',
f3: 'bar3'
}
}
}
myGlobal("a")=123
myGlobal("b","c1")="foo"
myGlobal("b","c2")="foo2"
myGlobal("d","e1","f1")="bar1"
myGlobal("d","e1","f2")="bar2"
myGlobal("d","e2","f1")="bar1"
myGlobal("d","e2","f2")="bar2"
myGlobal("d","e2","f3")="bar3"
Subscript <=> Property
Copyright © 2017 M/Gateway Developments Ltd
myGlobal("d","e2","f3") <=> myGlobal.d.e2.f3
Global Node JavaScript Object
Copyright © 2017 M/Gateway Developments Ltd
myGlobal("d","e2","f3") <=> myGlobal.d.e2.f3
Global Node JavaScript Object
But…
JavaScript Objects are in memory
Global Nodes are persistent, on disk
Copyright © 2017 M/Gateway Developments Ltd
myGlobal("d","e2","f3") <=> myGlobal.d.e2.f3
Global Node JavaScript Object
Could we abstract Global Nodes as
Persistent JavaScript Objects?
Copyright © 2017 M/Gateway Developments Ltd
ewd-document-store
• JavaScript abstraction of Global Storage
– https://github.com/robtweed/ewd-document-store
Copyright © 2017 M/Gateway Developments Ltd
ewd-document-store
• Abstracts Global Storage as:
– A document database
– Persistent JavaScript Objects
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode(name, subscripts);
Represents a node within a Global (aka Document)
May or may not physically exist at this point
ie this does NOT create a node
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode(name, subscripts);
Represents a node within a document's hierarchy
May or may not physically exist at this point
ie this does NOT create a node
Creates an instance of an object that represents that node, with
a set of methods and properties to manipulate and access that node
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']);
represents myDoc("d","e2")
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']);
represents myDoc("d","e2")
create: dnode.value = 'foo2';
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']);
represents myDoc("d","e2")
create: dnode.value = 'foo2';
get: var x = dnode.value;
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']);
represents myDoc("d","e2")
create: dnode.value = 'foo2';
get: var x = dnode.value;
value is a read/write property
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']);
represents myDoc("d","e2")
create: dnode.value = 'foo2';
get: var x = dnode.value;
delete: dnode.delete();
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']);
represents myDoc("d","e2")
create: dnode.value = 'foo2';
get: var x = dnode.value;
delete: dnode.delete();
Note: delete will also delete any lower-level DocumentNodes
in the underlying Global Storage tree
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']);
represents myDoc("d","e2")
create: dnode.value = 'foo2';
get: var x = dnode.value;
delete: dnode.delete();
exists?: var exists = dnode.exists; // true | false
DocumentNode Object
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d']);
dnode.hasChildren // true
dnode.hasValue // false
dnode.exists // true
myDoc("a")=123
myDoc("b","c1")="foo"
myDoc("b","c2")="foo2"
myDoc("d","e1","f1")="bar1"
myDoc("d","e1","f2")="bar2"
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2', 'f3']);
dnode.hasChildren // false
dnode.hasValue // true
dnode.exists // true
myDoc("a")=123
myDoc("b","c1")="foo"
myDoc("b","c2")="foo2"
myDoc("d","e1","f1")="bar1"
myDoc("d","e1","f2")="bar2"
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
Copyright © 2017 M/Gateway Developments Ltd
var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2', 'f4']);
dnode.hasChildren // false
dnode.hasValue // false
dnode.exists // false
Even though the node doesn’t physically exist, you can define a
DocumentNode Object for it
myDoc("a")=123
myDoc("b","c1")="foo"
myDoc("b","c2")="foo2"
myDoc("d","e1","f1")="bar1"
myDoc("d","e1","f2")="bar2"
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
Copyright © 2017 M/Gateway Developments Ltd
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']);
Traversing Documents
Copyright © 2017 M/Gateway Developments Ltd
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']);
e2Node.forEachChild(function(nodeName) {
// do something with the node name / subscript for this iteration
});
forEachChild() Method
Copyright © 2017 M/Gateway Developments Ltd
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']);
e2Node.forEachChild(function(nodeName) {
// 1st
iteration nodeName === 'f1’
});
forEachChild() Method
Copyright © 2017 M/Gateway Developments Ltd
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']);
e2Node.forEachChild(function(nodeName) {
// 2nd
iteration nodeName === 'f2’
});
forEachChild() Method
Copyright © 2017 M/Gateway Developments Ltd
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']);
e2Node.forEachChild(function(nodeName) {
// 3rd
iteration nodeName === 'f3’
});
forEachChild() Method
Copyright © 2017 M/Gateway Developments Ltd
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")="bar3"
var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']);
e2Node.forEachChild(function(nodeName) {
console.log(nodeName);
});
forEachChild() Method
f1
f2
f3
Copyright © 2017 M/Gateway Developments Ltd
getDocument()
var doc = new this.documentStore.DocumentNode('myDoc');
var myObj = doc.getDocument();
myDoc("a")=123
myDoc("b","c1")="foo"
myDoc("b","c2")="foo2"
myDoc("d","e1","f1a")="bar1a"
myDoc("d","e1","f2a")="bar2a"
myDoc("d","e2","f1b")="bar1b"
myDoc("d","e2","f2b")="bar2b"
myDoc("d","e2","f3b")="bar3b"
myObj = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
},
d: {
e1: {
f1a: 'bar1a',
f2a: 'bar2a'
},
e2: {
f1b: 'bar1b',
f2b: 'bar2b',
f3b: 'bar3b'
}
}
}
creates
Copyright © 2017 M/Gateway Developments Ltd
getDocument()
var doc = new this.documentStore.DocumentNode('myDoc');
var myObj = doc.getDocument();
myDoc("a")=123
myDoc("b","c1")="foo"
myDoc("b","c2")="foo2"
myDoc("d","e1","f1a")="bar1a"
myDoc("d","e1","f2a")="bar2a"
myDoc("d","e2","f1b")="bar1b"
myDoc("d","e2","f2b")="bar2b"
myDoc("d","e2","f3b")="bar3b"
myObj = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
},
d: {
e1: {
f1a: 'bar1a',
f2a: 'bar2a'
},
e2: {
f1b: 'bar1b',
f2b: 'bar2b',
f3b: 'bar3b'
}
}
}
myObj is a standard in-memory JavaScript object
containing a copy of the data that was held within
the DocumentNode's sub-tree of nodes
creates
Copyright © 2017 M/Gateway Developments Ltd
setDocument()
var myObj = { // create the object shown to the right };
myObj = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
},
d: {
e1: {
f1a: 'bar1a',
f2a: 'bar2a'
},
e2: {
f1b: 'bar1b',
f2b: 'bar2b',
f3b: 'bar3b'
}
}
}
Copyright © 2017 M/Gateway Developments Ltd
setDocument()
var myObj = { // create the object shown to the right };
var doc = new this.documentStore.DocumentNode('myDoc');
// doesn't yet exist on disk myObj = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
},
d: {
e1: {
f1a: 'bar1a',
f2a: 'bar2a'
},
e2: {
f1b: 'bar1b',
f2b: 'bar2b',
f3b: 'bar3b'
}
}
}
Copyright © 2017 M/Gateway Developments Ltd
setDocument()
var myObj = { // create the object shown to the right };
var doc = new this.documentStore.DocumentNode('myDoc');
doc.setDocument(myObj);
myDoc("a")=123
myDoc("b","c1")="foo"
myDoc("b","c2")="foo2"
myDoc("d","e1","f1a")="bar1a"
myDoc("d","e1","f2a")="bar2a"
myDoc("d","e2","f1b")="bar1b"
myDoc("d","e2","f2b")="bar2b"
myDoc("d","e2","f3b")="bar3b"
myObj = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
},
d: {
e1: {
f1a: 'bar1a',
f2a: 'bar2a'
},
e2: {
f1b: 'bar1b',
f2b: 'bar2b',
f3b: 'bar3b'
}
}
}
creates
Copyright © 2017 M/Gateway Developments Ltd
setDocument()
var myObj = { // create the object shown to the right };
var doc = new this.documentStore.DocumentNode('myDoc');
doc.setDocument(myObj);
myDoc("a")=123
myDoc("b","c1")="foo"
myDoc("b","c2")="foo2"
myDoc("d","e1","f1a")="bar1a"
myDoc("d","e1","f2a")="bar2a"
myDoc("d","e2","f1b")="bar1b"
myDoc("d","e2","f2b")="bar2b"
myDoc("d","e2","f3b")="bar3b"
myObj = {
a: 123,
b: {
c1: 'foo',
c2: 'foo2'
},
d: {
e1: {
f1a: 'bar1a',
f2a: 'bar2a'
},
e2: {
f1b: 'bar1b',
f2b: 'bar2b',
f3b: 'bar3b'
}
}
}
myDoc is created on disk, containing a copy of
the data that was held within the myObj
object, now mapped to Global Storage nodes
creates
Copyright © 2017 M/Gateway Developments Ltd
Making This Possible
• The cache.node interface
• Included with Caché (and IRIS)
• Runs in-process with Caché
• Connects to Caché's C++ Callin Interface
– Directly accessing the Caché database engine
– Bypassing the language interface
Copyright © 2017 M/Gateway Developments Ltd
cache.node interface
Node.js
Process
Caché
Process
cache.node
module
C++call-ininterface
JavaScript
Globals Functions Objects
Copyright © 2017 M/Gateway Developments Ltd
cache.node APIs
• Low-level
• Basic Global primitives
• "Global-centric"
• Not JavaScript-centric
Copyright © 2017 M/Gateway Developments Ltd
ewd-document-store
• Builds on top of the cache.node APIs
• Creates a JavaScript-centric abstraction of
Global Storage
– Mapping to/from JavaScript Objects
• The JavaScript equivalent of ^
– Local JavaScript objects (in-memory)
– Persistent JavaScript objects (on-disk)
Copyright © 2017 M/Gateway Developments Ltd
Node.js Architecture
• Everything executes in a single process
– You might have 1000's of users doing stuff
concurrently
• They're all sharing the same process!
Copyright © 2017 M/Gateway Developments Ltd
Brainchild of Ryan Dahl
Copyright © 2017 M/Gateway Developments Ltd
Node.js = server-side JavaScript
• Ryan Dahl wasn't actually a big JavaScript fan
– Node.js wasn't designed to be server-side JavaScript
• But he realised that it was a convenient
language for such an environment, as it's
designed for exactly the same kind of way of
working in the browser
• Google's V8 JavaScript engine was Open
Sourced, so he used it to provide the language
for Node.js
– (which is actually mostly written in C++)
Copyright © 2017 M/Gateway Developments Ltd
Node.js Architecture
• Non-blocking I/O
– No user activity must block the process, or everyone
grinds to a halt
• Event-driven, asynchronous logic
– Fire off the request, but don't wait for the results
• Carry on with the next task
– When results come back, handle them at the next
opportunity
Copyright © 2017 M/Gateway Developments Ltd
ewd-document-store requires
synchronous APIs
var myDoc = new this.documentStore.DocumentNode('myDoc');
var f3Node = myDoc.$('d').$('e2').$('f3');
myDoc.$d.$e2.$f3.value = 'New value';
myDoc("a")=123
myDoc("b","c1")="foo"
myDoc("b","c2")="foo2"
myDoc("d","e1","f1")="bar1"
myDoc("d","e1","f2")="bar2"
myDoc("d","e2","f1")="bar1"
myDoc("d","e2","f2")="bar2"
myDoc("d","e2","f3")=”New value"
Requires properly chainable functions
Copyright © 2017 M/Gateway Developments Ltd
The Problem is Concurrency
• All concurrent users in Node.js share the
same process
Copyright © 2017 M/Gateway Developments Ltd
Is there a way to avoid this?
• Would it be possible to create a locally-
available environment where my Node.js
code runs in an isolated container where
concurrency isn't an issue?
Copyright © 2017 M/Gateway Developments Ltd
Copyright © 2017 M/Gateway Developments Ltd
What Is QEWD?
• Essentially it's a multi-purpose Node.js-
based run-time Platform
Copyright © 2017 M/Gateway Developments Ltd
What Is QEWD?
• Essentially it's a multi-purpose Node.js-
based run-time Platform
• Creates an isolated run-time container for
your message/request handler functions,
allowing:
– CPU-intensive work
– Database abstractions using synchronous
logic
Copyright © 2017 M/Gateway Developments Ltd
QEWD's Architecture
• Master Process
• Pool of Worker Processes
Copyright © 2017 M/Gateway Developments Ltd
QEWD's Architecture
• Master Process
– Handles and queues all incoming requests
from client
• HTTP/REST requests via Express or Koa.js
• WebSocket requests via socket.io
– Returns responses to client
Copyright © 2017 M/Gateway Developments Ltd
QEWD's Architecture
• Pool of Persistent Worker Processes
– Where all the processing occurs
– A single queued request is dispatched to an
available worker process
– Each Worker process handles a single
request at a time
Copyright © 2017 M/Gateway Developments Ltd
QEWD's Architecture
• The queue / dispatcher / worker-process
pool management part of QEWD is
handled by a module named ewd-qoper8
Copyright © 2017 M/Gateway Developments Ltd
Master Node.js Process
Queue
Queue
processor/
dispatcher
Incoming
Requests
QEWD Architecture
Every incoming request
is passed from Express
and placed in a queue
No further processing
of requests occurs in
the master process
Express
or
Koa.js
socket.io
HTTP
REST
WebSocket
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Queue dispatcher is
invoked whenever a
request is added to
the queue
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Worker process
started if none
available
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Redis/Cache
QEWD &
application-specific
Modules loaded
Custom
Worker
Module
Custom
Worker
Module
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Request passed
to worker
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Worker flagged as Unavailable
Node.js Worker Process
Custom
Worker
Module
Begin processing message
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Unavailable / processing
Another incoming
request Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Node.js Worker Process
Master Node.js Process
Queue
Queue
processor/
dispatcher
Custom
Worker
Module
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
If worker pool size not exceeded,
another worker is started
and request passed to it
Redis/Cache
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
If entire Worker Pool is busy:
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
If entire Worker Pool is busy:
New
requests
remain
in queue
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
As soon as a worker is available again,
a queued message can be passed to it
Unavailable / processing
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
Available
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Finished
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Redis/Cache
A user's handler function signals
completion using the function:
finished(responseObject);
This returns the response
object to the master
process
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Finished
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
Node.js Worker Process
Custom
Worker
Module
And the response is
returned to the client
that sent the original request
(via Express/Koa/socket.io)
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Available
Node.js Worker Process
Custom
Worker
Module
The finished() function
also automatically returns
the worker process back
to the available pool
So it can now handle
the next queued request
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Available
Node.js Worker Process
Custom
Worker
Module
Worker processes, once started,
are persistent
No start-up / tear-down cost
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Node.js concurrency is handled
by the master process.
100% asynchronous logic
The master process does
almost nothing
The perfect Node.js application:
no CPU-intensive or long-
running tasks, so very
high-performance
Multiple
Concurrent
Incoming
requests
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Node.js concurrency is handled
by the master process.
100% asynchronous logic
The master process does
almost nothing
The perfect Node.js application:
no CPU-intensive or long-
running tasks, so very
high-performance
All the actual work happens in the
isolated worker processes
Multiple
Concurrent
Incoming
requests
Copyright © 2017 M/Gateway Developments Ltd
QEWD Architecture
Master Node.js Process
Queue
Queue
processor/
dispatcher
Available
Node.js Worker Process
Custom
Worker
Module
Worker processes only handle
a single request at a time
Completely isolated run-time
environment for handler functions
No need for concerns about
Node.js concurrency, so
synchronous APIs can be used
Redis/Cache
Copyright © 2017 M/Gateway Developments Ltd
QEWD + ewd-document-store +
cache.node
• JavaScript as a first-class language for
Caché
• Provides the JavaScript equivalent of ^
– Local JavaScript objects (in-memory)
– Persistent JavaScript objects (on-disk)
• JavaScript with data persistence as a
language feature
Copyright © 2017 M/Gateway Developments Ltd
Just one remaining issue
• Native Caché ObjectScript performance:
– Set: >1 million Global Nodes / sec
• cache.node
– Around 8 – 10% of that performance
Copyright © 2017 M/Gateway Developments Ltd
The reason?
• The Google V8 Engine's API that is used
by cache.node
– Has a bottleneck
• Usually thought to be insignificant
• Becomes significant for the kinds of things done by
ewd-document-store
Copyright © 2017 M/Gateway Developments Ltd
If it can be fixed
• We'd have the fastest Node.js database
on the planet
– By a significant margin
• We'd have data persistence as a feature
of JavaScript
– Until it's fully demonstrable, the IT mainstream
aren't going to "get it"
Copyright © 2017 M/Gateway Developments Ltd
Can It Be Fixed?
• Probably requires Google to sort it out
• Logged on their bug tracker
– But no sign of them doing anything to fix it
• Watch this space
• If it can be fixed, this is potentially huge
– And we all stand to benefit
Copyright © 2017 M/Gateway Developments Ltd
Additional Reading
• Universal NoSQL Paper:
– http://mgateway.com/docs/universalNoSQL.pdf
• QEWD.js:
– https://qewdjs.com
• JavaScript Abstraction of Global Storage:
– http://docs.qewdjs.com/qewd_training.html
• In particular, parts 17 – 27
• QEWD.js from a Node.js perspective:
– http://bit.ly/2gnnyh5
– http://bit.ly/2ikY1sS
Copyright © 2017 M/Gateway Developments Ltd
^ redux:
Data Persistence as a
Language Feature
Rob Tweed
M/Gateway Developments Ltd
Twitter: @rtweed
http://qewdjs.com

More Related Content

What's hot

Web Clients for Ruby and What they should be in the future
Web Clients for Ruby and What they should be in the futureWeb Clients for Ruby and What they should be in the future
Web Clients for Ruby and What they should be in the future
Toru Kawamura
 
HTML5 Real-Time and Connectivity
HTML5 Real-Time and ConnectivityHTML5 Real-Time and Connectivity
HTML5 Real-Time and Connectivity
Peter Lubbers
 
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWDEWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD
Rob Tweed
 
Plack basics for Perl websites - YAPC::EU 2011
Plack basics for Perl websites - YAPC::EU 2011Plack basics for Perl websites - YAPC::EU 2011
Plack basics for Perl websites - YAPC::EU 2011
leo lapworth
 
Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1
Henry S
 
XML and Web Services with Groovy
XML and Web Services with GroovyXML and Web Services with Groovy
XML and Web Services with Groovy
Paul King
 
Extending Build to the Client: A Maven User's Guide to Grunt.js
Extending Build to the Client: A Maven User's Guide to Grunt.jsExtending Build to the Client: A Maven User's Guide to Grunt.js
Extending Build to the Client: A Maven User's Guide to Grunt.js
Petr Jiricka
 
Docker meetup-20-apr-17-openshit
Docker meetup-20-apr-17-openshitDocker meetup-20-apr-17-openshit
Docker meetup-20-apr-17-openshit
Yusuf Hadiwinata Sutandar
 
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLHTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
Ulf Wendel
 
EWD 3 Training Course Part 5b: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5b: First Steps in Building a QEWD ApplicationEWD 3 Training Course Part 5b: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5b: First Steps in Building a QEWD Application
Rob Tweed
 
Docker for Java Developers
Docker for Java DevelopersDocker for Java Developers
Docker for Java Developers
Imesh Gunaratne
 
Ecom 1
Ecom 1Ecom 1
Java servlets and CGI
Java servlets and CGIJava servlets and CGI
Java servlets and CGI
lavanya marichamy
 
EWD 3 Training Course Part 19: The cache.node APIs
EWD 3 Training Course Part 19: The cache.node APIsEWD 3 Training Course Part 19: The cache.node APIs
EWD 3 Training Course Part 19: The cache.node APIs
Rob Tweed
 
EWD 3 Training Course Part 5a: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5a: First Steps in Building a QEWD ApplicationEWD 3 Training Course Part 5a: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5a: First Steps in Building a QEWD Application
Rob Tweed
 
Beginners Node.js
Beginners Node.jsBeginners Node.js
Beginners Node.js
Khaled Mosharraf
 
Hacking Docker the Easy way
Hacking Docker the Easy wayHacking Docker the Easy way
Hacking Docker the Easy way
Borg Han
 
PreJSS: the final point in the CSS war
PreJSS: the final point in the CSS warPreJSS: the final point in the CSS war
PreJSS: the final point in the CSS war
Denis Izmaylov
 
Gradle.Enemy at the gates
Gradle.Enemy at the gatesGradle.Enemy at the gates
Gradle.Enemy at the gates
Strannik_2013
 
From CoreOS to Kubernetes and Concourse CI
From CoreOS to Kubernetes and Concourse CIFrom CoreOS to Kubernetes and Concourse CI
From CoreOS to Kubernetes and Concourse CI
Denis Izmaylov
 

What's hot (20)

Web Clients for Ruby and What they should be in the future
Web Clients for Ruby and What they should be in the futureWeb Clients for Ruby and What they should be in the future
Web Clients for Ruby and What they should be in the future
 
HTML5 Real-Time and Connectivity
HTML5 Real-Time and ConnectivityHTML5 Real-Time and Connectivity
HTML5 Real-Time and Connectivity
 
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWDEWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD
 
Plack basics for Perl websites - YAPC::EU 2011
Plack basics for Perl websites - YAPC::EU 2011Plack basics for Perl websites - YAPC::EU 2011
Plack basics for Perl websites - YAPC::EU 2011
 
Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1
 
XML and Web Services with Groovy
XML and Web Services with GroovyXML and Web Services with Groovy
XML and Web Services with Groovy
 
Extending Build to the Client: A Maven User's Guide to Grunt.js
Extending Build to the Client: A Maven User's Guide to Grunt.jsExtending Build to the Client: A Maven User's Guide to Grunt.js
Extending Build to the Client: A Maven User's Guide to Grunt.js
 
Docker meetup-20-apr-17-openshit
Docker meetup-20-apr-17-openshitDocker meetup-20-apr-17-openshit
Docker meetup-20-apr-17-openshit
 
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQLHTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
HTTP, JSON, JavaScript, Map&Reduce built-in to MySQL
 
EWD 3 Training Course Part 5b: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5b: First Steps in Building a QEWD ApplicationEWD 3 Training Course Part 5b: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5b: First Steps in Building a QEWD Application
 
Docker for Java Developers
Docker for Java DevelopersDocker for Java Developers
Docker for Java Developers
 
Ecom 1
Ecom 1Ecom 1
Ecom 1
 
Java servlets and CGI
Java servlets and CGIJava servlets and CGI
Java servlets and CGI
 
EWD 3 Training Course Part 19: The cache.node APIs
EWD 3 Training Course Part 19: The cache.node APIsEWD 3 Training Course Part 19: The cache.node APIs
EWD 3 Training Course Part 19: The cache.node APIs
 
EWD 3 Training Course Part 5a: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5a: First Steps in Building a QEWD ApplicationEWD 3 Training Course Part 5a: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5a: First Steps in Building a QEWD Application
 
Beginners Node.js
Beginners Node.jsBeginners Node.js
Beginners Node.js
 
Hacking Docker the Easy way
Hacking Docker the Easy wayHacking Docker the Easy way
Hacking Docker the Easy way
 
PreJSS: the final point in the CSS war
PreJSS: the final point in the CSS warPreJSS: the final point in the CSS war
PreJSS: the final point in the CSS war
 
Gradle.Enemy at the gates
Gradle.Enemy at the gatesGradle.Enemy at the gates
Gradle.Enemy at the gates
 
From CoreOS to Kubernetes and Concourse CI
From CoreOS to Kubernetes and Concourse CIFrom CoreOS to Kubernetes and Concourse CI
From CoreOS to Kubernetes and Concourse CI
 

Similar to Data Persistence as a Language Feature

New DevOps for the DBA
New DevOps for the DBANew DevOps for the DBA
New DevOps for the DBA
Kellyn Pot'Vin-Gorman
 
Meetup. Technologies Intro for Non-Tech People
Meetup. Technologies Intro for Non-Tech PeopleMeetup. Technologies Intro for Non-Tech People
Meetup. Technologies Intro for Non-Tech People
IT Arena
 
Product Camp Silicon Valley 2018 - PM Technical Skills
Product Camp Silicon Valley 2018 - PM Technical SkillsProduct Camp Silicon Valley 2018 - PM Technical Skills
Product Camp Silicon Valley 2018 - PM Technical Skills
Sandeep Adwankar
 
Webinar: End-to-End CI/CD with GitLab and DC/OS
Webinar: End-to-End CI/CD with GitLab and DC/OSWebinar: End-to-End CI/CD with GitLab and DC/OS
Webinar: End-to-End CI/CD with GitLab and DC/OS
Mesosphere Inc.
 
Web development revolution
Web development revolutionWeb development revolution
Web development revolution
Ahmed El-Zekred
 
The Rise of DataOps: Making Big Data Bite Size with DataOps
The Rise of DataOps: Making Big Data Bite Size with DataOpsThe Rise of DataOps: Making Big Data Bite Size with DataOps
The Rise of DataOps: Making Big Data Bite Size with DataOps
Delphix
 
DevOps for the DBA- Jax Style!
DevOps for the DBA-  Jax Style!DevOps for the DBA-  Jax Style!
DevOps for the DBA- Jax Style!
Kellyn Pot'Vin-Gorman
 
Ten compelling reasons to learn .net framework
Ten compelling reasons to learn .net frameworkTen compelling reasons to learn .net framework
Ten compelling reasons to learn .net framework
JanBask Training
 
Developer Experience Overview
Developer Experience OverviewDeveloper Experience Overview
Developer Experience Overview
Ross Jimenez
 
MERN PPT
MERN PPTMERN PPT
Improve Your Business Standards with Backend Development .pdf
Improve Your Business Standards with Backend Development .pdfImprove Your Business Standards with Backend Development .pdf
Improve Your Business Standards with Backend Development .pdf
WPWeb Infotech
 
InterConnect 2017 : Programming languages in the enterprise: Which language s...
InterConnect 2017 : Programming languages in the enterprise: Which language s...InterConnect 2017 : Programming languages in the enterprise: Which language s...
InterConnect 2017 : Programming languages in the enterprise: Which language s...
DevOps for Enterprise Systems
 
Android Development: Approach for Agile Teams
Android Development: Approach for Agile TeamsAndroid Development: Approach for Agile Teams
Android Development: Approach for Agile Teams
Anay Kamat
 
11 Live Node.js CMS Frameworks
11 Live Node.js CMS Frameworks11 Live Node.js CMS Frameworks
11 Live Node.js CMS Frameworks
iScripts
 
Polyglot Persistence - Two Great Tastes That Taste Great Together
Polyglot Persistence - Two Great Tastes That Taste Great TogetherPolyglot Persistence - Two Great Tastes That Taste Great Together
Polyglot Persistence - Two Great Tastes That Taste Great Together
John Wood
 
OpenSlava 2013 - Dynamic Languages
OpenSlava 2013 - Dynamic LanguagesOpenSlava 2013 - Dynamic Languages
OpenSlava 2013 - Dynamic Languages
Oscar Renalias
 
Full stack development using javascript what and why - ajay chandravadiya
Full stack development using javascript   what and why - ajay chandravadiyaFull stack development using javascript   what and why - ajay chandravadiya
Full stack development using javascript what and why - ajay chandravadiya
ajayrcgmail
 
Top Programming Languages to Learn for Web Development in 2023
Top Programming Languages to Learn for Web Development in 2023Top Programming Languages to Learn for Web Development in 2023
Top Programming Languages to Learn for Web Development in 2023
priyanka rajput
 
Storage for containers and cloud-native deployments - Rancher Online Meetup -...
Storage for containers and cloud-native deployments - Rancher Online Meetup -...Storage for containers and cloud-native deployments - Rancher Online Meetup -...
Storage for containers and cloud-native deployments - Rancher Online Meetup -...
Shannon Williams
 
Empowering Agile Development with Containers
Empowering Agile Development with ContainersEmpowering Agile Development with Containers
Empowering Agile Development with Containers
Kellyn Pot'Vin-Gorman
 

Similar to Data Persistence as a Language Feature (20)

New DevOps for the DBA
New DevOps for the DBANew DevOps for the DBA
New DevOps for the DBA
 
Meetup. Technologies Intro for Non-Tech People
Meetup. Technologies Intro for Non-Tech PeopleMeetup. Technologies Intro for Non-Tech People
Meetup. Technologies Intro for Non-Tech People
 
Product Camp Silicon Valley 2018 - PM Technical Skills
Product Camp Silicon Valley 2018 - PM Technical SkillsProduct Camp Silicon Valley 2018 - PM Technical Skills
Product Camp Silicon Valley 2018 - PM Technical Skills
 
Webinar: End-to-End CI/CD with GitLab and DC/OS
Webinar: End-to-End CI/CD with GitLab and DC/OSWebinar: End-to-End CI/CD with GitLab and DC/OS
Webinar: End-to-End CI/CD with GitLab and DC/OS
 
Web development revolution
Web development revolutionWeb development revolution
Web development revolution
 
The Rise of DataOps: Making Big Data Bite Size with DataOps
The Rise of DataOps: Making Big Data Bite Size with DataOpsThe Rise of DataOps: Making Big Data Bite Size with DataOps
The Rise of DataOps: Making Big Data Bite Size with DataOps
 
DevOps for the DBA- Jax Style!
DevOps for the DBA-  Jax Style!DevOps for the DBA-  Jax Style!
DevOps for the DBA- Jax Style!
 
Ten compelling reasons to learn .net framework
Ten compelling reasons to learn .net frameworkTen compelling reasons to learn .net framework
Ten compelling reasons to learn .net framework
 
Developer Experience Overview
Developer Experience OverviewDeveloper Experience Overview
Developer Experience Overview
 
MERN PPT
MERN PPTMERN PPT
MERN PPT
 
Improve Your Business Standards with Backend Development .pdf
Improve Your Business Standards with Backend Development .pdfImprove Your Business Standards with Backend Development .pdf
Improve Your Business Standards with Backend Development .pdf
 
InterConnect 2017 : Programming languages in the enterprise: Which language s...
InterConnect 2017 : Programming languages in the enterprise: Which language s...InterConnect 2017 : Programming languages in the enterprise: Which language s...
InterConnect 2017 : Programming languages in the enterprise: Which language s...
 
Android Development: Approach for Agile Teams
Android Development: Approach for Agile TeamsAndroid Development: Approach for Agile Teams
Android Development: Approach for Agile Teams
 
11 Live Node.js CMS Frameworks
11 Live Node.js CMS Frameworks11 Live Node.js CMS Frameworks
11 Live Node.js CMS Frameworks
 
Polyglot Persistence - Two Great Tastes That Taste Great Together
Polyglot Persistence - Two Great Tastes That Taste Great TogetherPolyglot Persistence - Two Great Tastes That Taste Great Together
Polyglot Persistence - Two Great Tastes That Taste Great Together
 
OpenSlava 2013 - Dynamic Languages
OpenSlava 2013 - Dynamic LanguagesOpenSlava 2013 - Dynamic Languages
OpenSlava 2013 - Dynamic Languages
 
Full stack development using javascript what and why - ajay chandravadiya
Full stack development using javascript   what and why - ajay chandravadiyaFull stack development using javascript   what and why - ajay chandravadiya
Full stack development using javascript what and why - ajay chandravadiya
 
Top Programming Languages to Learn for Web Development in 2023
Top Programming Languages to Learn for Web Development in 2023Top Programming Languages to Learn for Web Development in 2023
Top Programming Languages to Learn for Web Development in 2023
 
Storage for containers and cloud-native deployments - Rancher Online Meetup -...
Storage for containers and cloud-native deployments - Rancher Online Meetup -...Storage for containers and cloud-native deployments - Rancher Online Meetup -...
Storage for containers and cloud-native deployments - Rancher Online Meetup -...
 
Empowering Agile Development with Containers
Empowering Agile Development with ContainersEmpowering Agile Development with Containers
Empowering Agile Development with Containers
 

More from Rob Tweed

EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService FunctionalityEWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
Rob Tweed
 
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.jsEWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
Rob Tweed
 
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST ServicesEWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
Rob Tweed
 
qewd-ripple: The Ripple OSI Middle Tier
qewd-ripple: The Ripple OSI Middle Tierqewd-ripple: The Ripple OSI Middle Tier
qewd-ripple: The Ripple OSI Middle Tier
Rob Tweed
 
EWD 3 Training Course Part 42: The QEWD Docker Appliance
EWD 3 Training Course Part 42: The QEWD Docker ApplianceEWD 3 Training Course Part 42: The QEWD Docker Appliance
EWD 3 Training Course Part 42: The QEWD Docker Appliance
Rob Tweed
 
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5
Rob Tweed
 
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4
Rob Tweed
 
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3
Rob Tweed
 
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2
Rob Tweed
 
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...
Rob Tweed
 
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
Rob Tweed
 
EWD 3 Training Course Part 35: QEWD Session Locking
EWD 3 Training Course Part 35: QEWD Session LockingEWD 3 Training Course Part 35: QEWD Session Locking
EWD 3 Training Course Part 35: QEWD Session Locking
Rob Tweed
 
EWD 3 Training Course Part 34: QEWD Resilient Mode
EWD 3 Training Course Part 34: QEWD Resilient ModeEWD 3 Training Course Part 34: QEWD Resilient Mode
EWD 3 Training Course Part 34: QEWD Resilient Mode
Rob Tweed
 
EWD 3 Training Course Part 33: Configuring QEWD to use CORS
EWD 3 Training Course Part 33: Configuring QEWD to use CORSEWD 3 Training Course Part 33: Configuring QEWD to use CORS
EWD 3 Training Course Part 33: Configuring QEWD to use CORS
Rob Tweed
 
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPSEWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS
Rob Tweed
 
EWD 3 Training Course Part 31: Using QEWD for Web and REST Services
EWD 3 Training Course Part 31: Using QEWD for Web and REST ServicesEWD 3 Training Course Part 31: Using QEWD for Web and REST Services
EWD 3 Training Course Part 31: Using QEWD for Web and REST Services
Rob Tweed
 
EWD 3 Training Course Part 30: Modularising QEWD Applications
EWD 3 Training Course Part 30: Modularising QEWD ApplicationsEWD 3 Training Course Part 30: Modularising QEWD Applications
EWD 3 Training Course Part 30: Modularising QEWD Applications
Rob Tweed
 
EWD 3 Training Course Part 29: Running QEWD as a Service
EWD 3 Training Course Part 29: Running QEWD as a ServiceEWD 3 Training Course Part 29: Running QEWD as a Service
EWD 3 Training Course Part 29: Running QEWD as a Service
Rob Tweed
 
EWD 3 Training Course Part 27: The QEWD Session
EWD 3 Training Course Part 27: The QEWD SessionEWD 3 Training Course Part 27: The QEWD Session
EWD 3 Training Course Part 27: The QEWD Session
Rob Tweed
 
EWD 3 Training Course Part 26: Event-driven Indexing
EWD 3 Training Course Part 26: Event-driven IndexingEWD 3 Training Course Part 26: Event-driven Indexing
EWD 3 Training Course Part 26: Event-driven Indexing
Rob Tweed
 

More from Rob Tweed (20)

EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService FunctionalityEWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
 
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.jsEWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
 
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST ServicesEWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
 
qewd-ripple: The Ripple OSI Middle Tier
qewd-ripple: The Ripple OSI Middle Tierqewd-ripple: The Ripple OSI Middle Tier
qewd-ripple: The Ripple OSI Middle Tier
 
EWD 3 Training Course Part 42: The QEWD Docker Appliance
EWD 3 Training Course Part 42: The QEWD Docker ApplianceEWD 3 Training Course Part 42: The QEWD Docker Appliance
EWD 3 Training Course Part 42: The QEWD Docker Appliance
 
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5
 
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4
 
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3
 
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2
 
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...
 
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
 
EWD 3 Training Course Part 35: QEWD Session Locking
EWD 3 Training Course Part 35: QEWD Session LockingEWD 3 Training Course Part 35: QEWD Session Locking
EWD 3 Training Course Part 35: QEWD Session Locking
 
EWD 3 Training Course Part 34: QEWD Resilient Mode
EWD 3 Training Course Part 34: QEWD Resilient ModeEWD 3 Training Course Part 34: QEWD Resilient Mode
EWD 3 Training Course Part 34: QEWD Resilient Mode
 
EWD 3 Training Course Part 33: Configuring QEWD to use CORS
EWD 3 Training Course Part 33: Configuring QEWD to use CORSEWD 3 Training Course Part 33: Configuring QEWD to use CORS
EWD 3 Training Course Part 33: Configuring QEWD to use CORS
 
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPSEWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS
 
EWD 3 Training Course Part 31: Using QEWD for Web and REST Services
EWD 3 Training Course Part 31: Using QEWD for Web and REST ServicesEWD 3 Training Course Part 31: Using QEWD for Web and REST Services
EWD 3 Training Course Part 31: Using QEWD for Web and REST Services
 
EWD 3 Training Course Part 30: Modularising QEWD Applications
EWD 3 Training Course Part 30: Modularising QEWD ApplicationsEWD 3 Training Course Part 30: Modularising QEWD Applications
EWD 3 Training Course Part 30: Modularising QEWD Applications
 
EWD 3 Training Course Part 29: Running QEWD as a Service
EWD 3 Training Course Part 29: Running QEWD as a ServiceEWD 3 Training Course Part 29: Running QEWD as a Service
EWD 3 Training Course Part 29: Running QEWD as a Service
 
EWD 3 Training Course Part 27: The QEWD Session
EWD 3 Training Course Part 27: The QEWD SessionEWD 3 Training Course Part 27: The QEWD Session
EWD 3 Training Course Part 27: The QEWD Session
 
EWD 3 Training Course Part 26: Event-driven Indexing
EWD 3 Training Course Part 26: Event-driven IndexingEWD 3 Training Course Part 26: Event-driven Indexing
EWD 3 Training Course Part 26: Event-driven Indexing
 

Recently uploaded

Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
dream girl
 
To Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance SheetsTo Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance Sheets
Task Tracker
 
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
jealousviolet
 
Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)
andrehoraa
 
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
Srinivas Dukka
 
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdfAI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
Daniel Zivkovic
 
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
45unexpected
 
Comprehensive Vulnerability Assessments Process _ Aardwolf Security.docx
Comprehensive Vulnerability Assessments Process _ Aardwolf Security.docxComprehensive Vulnerability Assessments Process _ Aardwolf Security.docx
Comprehensive Vulnerability Assessments Process _ Aardwolf Security.docx
Aardwolf Security
 
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
OnePlan Solutions
 
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
norina2645
 
welcome to presentation on Google Apps
welcome to   presentation on Google Appswelcome to   presentation on Google Apps
welcome to presentation on Google Apps
AsifKarimJim
 
Russian Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service ...
Russian Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service ...Russian Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service ...
Russian Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service ...
shanihomely
 
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
bhumivarma35300
 
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
confluent
 
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Deliverybangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
InflectraCON 360: Risk-Based Testing for Mission Critical Systems
InflectraCON 360: Risk-Based Testing for Mission Critical SystemsInflectraCON 360: Risk-Based Testing for Mission Critical Systems
InflectraCON 360: Risk-Based Testing for Mission Critical Systems
Inflectra
 
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
bahubalikumar09988
 
BATber53 AWS Modernize your applications with purpose-built AWS databases
BATber53 AWS Modernize your applications with purpose-built AWS databasesBATber53 AWS Modernize your applications with purpose-built AWS databases
BATber53 AWS Modernize your applications with purpose-built AWS databases
BATbern
 
Wired_2.0_Create_AmsterdamJUG_09072024.pptx
Wired_2.0_Create_AmsterdamJUG_09072024.pptxWired_2.0_Create_AmsterdamJUG_09072024.pptx
Wired_2.0_Create_AmsterdamJUG_09072024.pptx
SimonedeGijt
 
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
87tomato
 

Recently uploaded (20)

Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
 
To Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance SheetsTo Avoid Mistakes When Using Online Attendance Sheets
To Avoid Mistakes When Using Online Attendance Sheets
 
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
VVIP Girls Call Mumbai 9910780858 Provide Best And Top Girl Service And No1 i...
 
Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)
 
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
 
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdfAI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
 
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
 
Comprehensive Vulnerability Assessments Process _ Aardwolf Security.docx
Comprehensive Vulnerability Assessments Process _ Aardwolf Security.docxComprehensive Vulnerability Assessments Process _ Aardwolf Security.docx
Comprehensive Vulnerability Assessments Process _ Aardwolf Security.docx
 
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
Maximizing Efficiency and Profitability: Optimizing Data Systems, Enhancing C...
 
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
 
welcome to presentation on Google Apps
welcome to   presentation on Google Appswelcome to   presentation on Google Apps
welcome to presentation on Google Apps
 
Russian Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service ...
Russian Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service ...Russian Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service ...
Russian Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service ...
 
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
Independent Girls call Service Pune 000XX00000 Provide Best And Top Girl Serv...
 
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
 
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Deliverybangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
InflectraCON 360: Risk-Based Testing for Mission Critical Systems
InflectraCON 360: Risk-Based Testing for Mission Critical SystemsInflectraCON 360: Risk-Based Testing for Mission Critical Systems
InflectraCON 360: Risk-Based Testing for Mission Critical Systems
 
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...🚂🚘 Premium Girls Call Ranchi  🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
🚂🚘 Premium Girls Call Ranchi 🛵🚡000XX00000 💃 Choose Best And Top Girl Service...
 
BATber53 AWS Modernize your applications with purpose-built AWS databases
BATber53 AWS Modernize your applications with purpose-built AWS databasesBATber53 AWS Modernize your applications with purpose-built AWS databases
BATber53 AWS Modernize your applications with purpose-built AWS databases
 
Wired_2.0_Create_AmsterdamJUG_09072024.pptx
Wired_2.0_Create_AmsterdamJUG_09072024.pptxWired_2.0_Create_AmsterdamJUG_09072024.pptx
Wired_2.0_Create_AmsterdamJUG_09072024.pptx
 
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
Verified Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeli...
 

Data Persistence as a Language Feature

  • 1. Copyright © 2017 M/Gateway Developments Ltd ^ redux: Data Persistence as a Language Feature Rob Tweed M/Gateway Developments Ltd Twitter: @rtweed http://qewdjs.com
  • 2. Copyright © 2017 M/Gateway Developments Ltd A bit of background • Royal Marsden Hospital: 1980s • Touche Ross Management Consultants (now Deloitte): early 1990s – NHS-wide Networking Project • Independent consultant & software developer since 1993 – Specialising in Web and associated technologies, particularly in healthcare
  • 3. Copyright © 2017 M/Gateway Developments Ltd Products • WebLink: – Co-creator of the original web connectivity for Caché • WebLink Developer: – Web Application Development framework • pre-cursor to CSP • eXtc: – XML DOM framework • EWD: – 2nd -generation web application development framework – XML based custom pages – Compiled to CSP or WebLink (or even PHP)
  • 4. Copyright © 2017 M/Gateway Developments Ltd Example Usage
  • 5. Copyright © 2017 M/Gateway Developments Ltd A bit of background • Node.js: – Since August 2010 (v0.2 had just come out!) – Pioneered the integration of Caché and Node.js – Was instrumental in getting the cache.node interface developed • Originally for GlobalsDB
  • 6. Copyright © 2017 M/Gateway Developments Ltd Language or Database? • Caché ObjectScript: – programming language with integrated database? • Caché Database: – Database with integrated scripting language for its manipulation (Caché ObjectScript)?
  • 7. Copyright © 2017 M/Gateway Developments Ltd Language • Caché ObjectScript shows its age – Rightly or wrongly, opinion from IT Mainstream developers is not favourable – Tainted, in particular, by this: • https://thedailywtf.com/articles/comments/A_Case_of_the_MUMPS/8 – As a result of the association with Mumps by, eg: – https://en.wikipedia.org/wiki/Cach%C3%A9_ObjectScript
  • 8. Copyright © 2017 M/Gateway Developments Ltd Language • Caché ObjectScript typical objections: – Limited – Old-Fashioned – To a modern developer, it's: • A step back in time • Lacking in many of the language capabilities they expect • Lacks an extensive ecosystem of pre-built functions and modules • A career-limiting move
  • 9. Copyright © 2017 M/Gateway Developments Ltd Database • Uniquely powerful • The original NoSQL database – Pre-dated the NoSQL movement by several decades • "Universal NoSQL" – AKA "Multi-Model database"
  • 10. Copyright © 2017 M/Gateway Developments Ltd Universal NoSQL Published in 2010
  • 11. Copyright © 2017 M/Gateway Developments Ltd Database • But in order to manipulate and use that database to its fullest capabilities, you need to learn its scripting language – Caché ObjectScript • A key reason why, as a database, it's almost unknown in the mainstream of IT – I've yet to meet a JavaScript developer who has even heard of Caché
  • 12. Copyright © 2017 M/Gateway Developments Ltd Solution? • Recurring suggestions within the developer community that Caché ObjectScript needs modernising – New language capabilities – Bring its syntax into line with modern languages – Create an ecosystem of functions & modules
  • 13. Copyright © 2017 M/Gateway Developments Ltd My Opinion • This is the wrong approach • Nobody in the mainstream of IT wants to learn a new language just to access a database • They'd expect to be able to use a modern language
  • 14. Copyright © 2017 M/Gateway Developments Ltd Language or Database? • Is Caché first and foremost a database or a language? – Misses a unique feature of Caché which provides its key power and which: • We take for granted • The mainstream of IT don't "get"
  • 15. Copyright © 2017 M/Gateway Developments Ltd What is a database? • It's that thing "over there" that persists our data
  • 16. Copyright © 2017 M/Gateway Developments Ltd What is a database? • It's expected to be "over there" • It's a separate server / sub-system • It's accessed: – over a network connection – via some protocol and/or syntax • REST • SQL
  • 17. Copyright © 2017 M/Gateway Developments Ltd So what? • Creates an explicit distinction between stuff you do in your programming language and persisted data • Creates separate disciplines – Programming experts – Database experts – Many people span both, but, to many programmers, databases are a mystery
  • 18. Copyright © 2017 M/Gateway Developments Ltd The Unique Feature • The underlying "Global Storage" of Caché • Why? – It blurs the distinction between what's permanently persisted and what's in memory – In memory: sparse arrays – On disk: identical sparse arrays – Only difference in syntax terms is a preceeding ^
  • 19. Copyright © 2017 M/Gateway Developments Ltd Revise the Definition • Yes it's a database and a language, but it warrants a new description: • It's a language with data persistence: – as a seamless language feature
  • 20. Copyright © 2017 M/Gateway Developments Ltd The Problem • It's a language with data persistence: – as a seamless language feature • But the language is obsolete and a barrier to new entry and mainstream adoption
  • 21. Copyright © 2017 M/Gateway Developments Ltd The Solution • It's a language with data persistence: – as a seamless language feature • Add a modern language as a first-class alternative to Caché ObjectScript – With data persistence as a similarly seamless feature – An equivalent, in that language, to ^
  • 22. Copyright © 2017 M/Gateway Developments Ltd The Question • What Language would fit the bill?
  • 23. Copyright © 2017 M/Gateway Developments Ltd From the Mainstream Perspective • One of the currently "big" languages – C / C++ too low level – Java – Ruby – Go – Python – JavaScript
  • 24. Copyright © 2017 M/Gateway Developments Ltd From our Perspective • Must be as fast as Native Caché ObjectScript – For general-purpose coding – For database access • > 1 million Global Nodes per second
  • 25. Copyright © 2017 M/Gateway Developments Ltd From both Perspectives • Should be a language suitable for all the kinds of application that Caché ObjectScript is currently used for: – Desktop – Browser-based – Mobile – Batch processing / server-side heavy-lifting
  • 26. Copyright © 2017 M/Gateway Developments Ltd From both Perspectives • Should map seamlessly between Global Storage and a commonly-accepted native in-memory representation with that language – No "impedance mismatch" • That mapping should make sense within the context of that language – ie laguage-centric
  • 27. Copyright © 2017 M/Gateway Developments Ltd Global Storage Key Features • Schema-Free • Hierarchically-structured • Dynamic creation / destruction • Ideally suited for "late-bound" access
  • 28. Copyright © 2017 M/Gateway Developments Ltd Suitability: Java Feature Adoption Widespread General-purpose Code Speed Excellent Application suitability Not used in browser, but otherwise widespread acceptance and adoption Data persistence impedance mismatch Designed around pre-defined, compiled classes / schema, ie early- bound access
  • 29. Copyright © 2017 M/Gateway Developments Ltd Suitability: Ruby Feature Adoption Fairly limited by comparison to other languages Well-liked by its users General-purpose Code Speed Poor Application suitability Not used in browser. Poor choice for mobile development Data persistence impedance mismatch Good
  • 30. Copyright © 2017 M/Gateway Developments Ltd Suitability: Go Feature Adoption Fairly limited by comparison to other languages. Well-liked by its users General-purpose Code Speed Excellent Application suitability Not used in browser. Otherwise very good Data persistence impedance mismatch Not so good – designed for pre- defined, pre-compiled mappings Miscellaneous Can we depend on Google supporting it in the long term?
  • 31. Copyright © 2017 M/Gateway Developments Ltd Suitability: Python Feature Adoption Pretty high and growing. Well-liked by its users General-purpose Code Speed Poor Application suitability Not used in browser. Not suitable for mobile development Data persistence impedance mismatch Good – dictionaries / JSON map well
  • 32. Copyright © 2017 M/Gateway Developments Ltd Suitability: JavaScript Feature Adoption High and still growing. Node.js now massively popular. Language has some quirks! General-purpose Code Speed Node.js performance is excellent Application suitability Used in browser. Ideal for mobile development. Usual cited deficiency is CPU-heavy processing, but that can be addressed Data persistence impedance mismatch JavaScript objects are dynamic and map almost perfectly onto Global Storage Miscellaneous Only 1 language skill for entire development stack, front and back (Node.js at back-end)
  • 33. Copyright © 2017 M/Gateway Developments Ltd JavaScript as a Native Language for Caché • It's a very good fit, for all sorts of reasons: – Interpreted – Loosely-typed in a similar way – JavaScript Objects (JSON) • Schema-free • Dynamic • hierarchical
  • 34. Copyright © 2017 M/Gateway Developments Ltd Global Storage myGlobal("a")=123 myGlobal("b","c1")="foo" myGlobal("b","c2")="foo2" myGlobal("d","e1","f1")="bar1" myGlobal("d","e1","f2")="bar2" myGlobal("d","e2","f1")="bar1" myGlobal("d","e2","f2")="bar2" myGlobal("d","e2","f3")="bar3" myGlobal "a" 123 "b" "c2" "foo2" "d" "c1" "foo" "e2" "e1" "f2" "bar2" "f1" "bar1" "f2" "bar2" "f1" "bar1" "f3" "bar3" Hierarchical diagram representing a Global
  • 35. Copyright © 2017 M/Gateway Developments Ltd myGlobal = { a: 123, b: { c1: 'foo', c2: 'foo2' } d: { e1: { f1: 'bar1', f2: 'bar2' }, e2: { f1: 'bar1', f2: 'bar2', f3: 'bar3' } } } myGlobal("a")=123 myGlobal("b","c1")="foo" myGlobal("b","c2")="foo2" myGlobal("d","e1","f1")="bar1" myGlobal("d","e1","f2")="bar2" myGlobal("d","e2","f1")="bar1" myGlobal("d","e2","f2")="bar2" myGlobal("d","e2","f3")="bar3" One to one correspondence between any tree of Global nodes and a Javascript object
  • 36. Copyright © 2017 M/Gateway Developments Ltd myGlobal = { a: 123, b: { c1: 'foo', c2: 'foo2' } d: { e1: { f1: 'bar1', f2: 'bar2' }, e2: { f1: 'bar1', f2: 'bar2', f3: 'bar3' } } } myGlobal("a")=123 myGlobal("b","c1")="foo" myGlobal("b","c2")="foo2" myGlobal("d","e1","f1")="bar1" myGlobal("d","e1","f2")="bar2" myGlobal("d","e2","f1")="bar1" myGlobal("d","e2","f2")="bar2" myGlobal("d","e2","f3")="bar3" Subscript <=> Property
  • 37. Copyright © 2017 M/Gateway Developments Ltd myGlobal("d","e2","f3") <=> myGlobal.d.e2.f3 Global Node JavaScript Object
  • 38. Copyright © 2017 M/Gateway Developments Ltd myGlobal("d","e2","f3") <=> myGlobal.d.e2.f3 Global Node JavaScript Object But… JavaScript Objects are in memory Global Nodes are persistent, on disk
  • 39. Copyright © 2017 M/Gateway Developments Ltd myGlobal("d","e2","f3") <=> myGlobal.d.e2.f3 Global Node JavaScript Object Could we abstract Global Nodes as Persistent JavaScript Objects?
  • 40. Copyright © 2017 M/Gateway Developments Ltd ewd-document-store • JavaScript abstraction of Global Storage – https://github.com/robtweed/ewd-document-store
  • 41. Copyright © 2017 M/Gateway Developments Ltd ewd-document-store • Abstracts Global Storage as: – A document database – Persistent JavaScript Objects
  • 42. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode(name, subscripts); Represents a node within a Global (aka Document) May or may not physically exist at this point ie this does NOT create a node DocumentNode Object
  • 43. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode(name, subscripts); Represents a node within a document's hierarchy May or may not physically exist at this point ie this does NOT create a node Creates an instance of an object that represents that node, with a set of methods and properties to manipulate and access that node DocumentNode Object
  • 44. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']); represents myDoc("d","e2") DocumentNode Object
  • 45. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']); represents myDoc("d","e2") create: dnode.value = 'foo2'; DocumentNode Object
  • 46. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']); represents myDoc("d","e2") create: dnode.value = 'foo2'; get: var x = dnode.value; DocumentNode Object
  • 47. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']); represents myDoc("d","e2") create: dnode.value = 'foo2'; get: var x = dnode.value; value is a read/write property DocumentNode Object
  • 48. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']); represents myDoc("d","e2") create: dnode.value = 'foo2'; get: var x = dnode.value; delete: dnode.delete(); DocumentNode Object
  • 49. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']); represents myDoc("d","e2") create: dnode.value = 'foo2'; get: var x = dnode.value; delete: dnode.delete(); Note: delete will also delete any lower-level DocumentNodes in the underlying Global Storage tree DocumentNode Object
  • 50. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2']); represents myDoc("d","e2") create: dnode.value = 'foo2'; get: var x = dnode.value; delete: dnode.delete(); exists?: var exists = dnode.exists; // true | false DocumentNode Object
  • 51. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d']); dnode.hasChildren // true dnode.hasValue // false dnode.exists // true myDoc("a")=123 myDoc("b","c1")="foo" myDoc("b","c2")="foo2" myDoc("d","e1","f1")="bar1" myDoc("d","e1","f2")="bar2" myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3"
  • 52. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2', 'f3']); dnode.hasChildren // false dnode.hasValue // true dnode.exists // true myDoc("a")=123 myDoc("b","c1")="foo" myDoc("b","c2")="foo2" myDoc("d","e1","f1")="bar1" myDoc("d","e1","f2")="bar2" myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3"
  • 53. Copyright © 2017 M/Gateway Developments Ltd var dnode = new this.documentStore.DocumentNode('myDoc', ['d', 'e2', 'f4']); dnode.hasChildren // false dnode.hasValue // false dnode.exists // false Even though the node doesn’t physically exist, you can define a DocumentNode Object for it myDoc("a")=123 myDoc("b","c1")="foo" myDoc("b","c2")="foo2" myDoc("d","e1","f1")="bar1" myDoc("d","e1","f2")="bar2" myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3"
  • 54. Copyright © 2017 M/Gateway Developments Ltd myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3" var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']); Traversing Documents
  • 55. Copyright © 2017 M/Gateway Developments Ltd myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3" var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']); e2Node.forEachChild(function(nodeName) { // do something with the node name / subscript for this iteration }); forEachChild() Method
  • 56. Copyright © 2017 M/Gateway Developments Ltd myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3" var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']); e2Node.forEachChild(function(nodeName) { // 1st iteration nodeName === 'f1’ }); forEachChild() Method
  • 57. Copyright © 2017 M/Gateway Developments Ltd myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3" var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']); e2Node.forEachChild(function(nodeName) { // 2nd iteration nodeName === 'f2’ }); forEachChild() Method
  • 58. Copyright © 2017 M/Gateway Developments Ltd myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3" var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']); e2Node.forEachChild(function(nodeName) { // 3rd iteration nodeName === 'f3’ }); forEachChild() Method
  • 59. Copyright © 2017 M/Gateway Developments Ltd myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")="bar3" var e2Node = new this.documentStore.DocumentNode('myDoc', ['d', e2']); e2Node.forEachChild(function(nodeName) { console.log(nodeName); }); forEachChild() Method f1 f2 f3
  • 60. Copyright © 2017 M/Gateway Developments Ltd getDocument() var doc = new this.documentStore.DocumentNode('myDoc'); var myObj = doc.getDocument(); myDoc("a")=123 myDoc("b","c1")="foo" myDoc("b","c2")="foo2" myDoc("d","e1","f1a")="bar1a" myDoc("d","e1","f2a")="bar2a" myDoc("d","e2","f1b")="bar1b" myDoc("d","e2","f2b")="bar2b" myDoc("d","e2","f3b")="bar3b" myObj = { a: 123, b: { c1: 'foo', c2: 'foo2' }, d: { e1: { f1a: 'bar1a', f2a: 'bar2a' }, e2: { f1b: 'bar1b', f2b: 'bar2b', f3b: 'bar3b' } } } creates
  • 61. Copyright © 2017 M/Gateway Developments Ltd getDocument() var doc = new this.documentStore.DocumentNode('myDoc'); var myObj = doc.getDocument(); myDoc("a")=123 myDoc("b","c1")="foo" myDoc("b","c2")="foo2" myDoc("d","e1","f1a")="bar1a" myDoc("d","e1","f2a")="bar2a" myDoc("d","e2","f1b")="bar1b" myDoc("d","e2","f2b")="bar2b" myDoc("d","e2","f3b")="bar3b" myObj = { a: 123, b: { c1: 'foo', c2: 'foo2' }, d: { e1: { f1a: 'bar1a', f2a: 'bar2a' }, e2: { f1b: 'bar1b', f2b: 'bar2b', f3b: 'bar3b' } } } myObj is a standard in-memory JavaScript object containing a copy of the data that was held within the DocumentNode's sub-tree of nodes creates
  • 62. Copyright © 2017 M/Gateway Developments Ltd setDocument() var myObj = { // create the object shown to the right }; myObj = { a: 123, b: { c1: 'foo', c2: 'foo2' }, d: { e1: { f1a: 'bar1a', f2a: 'bar2a' }, e2: { f1b: 'bar1b', f2b: 'bar2b', f3b: 'bar3b' } } }
  • 63. Copyright © 2017 M/Gateway Developments Ltd setDocument() var myObj = { // create the object shown to the right }; var doc = new this.documentStore.DocumentNode('myDoc'); // doesn't yet exist on disk myObj = { a: 123, b: { c1: 'foo', c2: 'foo2' }, d: { e1: { f1a: 'bar1a', f2a: 'bar2a' }, e2: { f1b: 'bar1b', f2b: 'bar2b', f3b: 'bar3b' } } }
  • 64. Copyright © 2017 M/Gateway Developments Ltd setDocument() var myObj = { // create the object shown to the right }; var doc = new this.documentStore.DocumentNode('myDoc'); doc.setDocument(myObj); myDoc("a")=123 myDoc("b","c1")="foo" myDoc("b","c2")="foo2" myDoc("d","e1","f1a")="bar1a" myDoc("d","e1","f2a")="bar2a" myDoc("d","e2","f1b")="bar1b" myDoc("d","e2","f2b")="bar2b" myDoc("d","e2","f3b")="bar3b" myObj = { a: 123, b: { c1: 'foo', c2: 'foo2' }, d: { e1: { f1a: 'bar1a', f2a: 'bar2a' }, e2: { f1b: 'bar1b', f2b: 'bar2b', f3b: 'bar3b' } } } creates
  • 65. Copyright © 2017 M/Gateway Developments Ltd setDocument() var myObj = { // create the object shown to the right }; var doc = new this.documentStore.DocumentNode('myDoc'); doc.setDocument(myObj); myDoc("a")=123 myDoc("b","c1")="foo" myDoc("b","c2")="foo2" myDoc("d","e1","f1a")="bar1a" myDoc("d","e1","f2a")="bar2a" myDoc("d","e2","f1b")="bar1b" myDoc("d","e2","f2b")="bar2b" myDoc("d","e2","f3b")="bar3b" myObj = { a: 123, b: { c1: 'foo', c2: 'foo2' }, d: { e1: { f1a: 'bar1a', f2a: 'bar2a' }, e2: { f1b: 'bar1b', f2b: 'bar2b', f3b: 'bar3b' } } } myDoc is created on disk, containing a copy of the data that was held within the myObj object, now mapped to Global Storage nodes creates
  • 66. Copyright © 2017 M/Gateway Developments Ltd Making This Possible • The cache.node interface • Included with Caché (and IRIS) • Runs in-process with Caché • Connects to Caché's C++ Callin Interface – Directly accessing the Caché database engine – Bypassing the language interface
  • 67. Copyright © 2017 M/Gateway Developments Ltd cache.node interface Node.js Process Caché Process cache.node module C++call-ininterface JavaScript Globals Functions Objects
  • 68. Copyright © 2017 M/Gateway Developments Ltd cache.node APIs • Low-level • Basic Global primitives • "Global-centric" • Not JavaScript-centric
  • 69. Copyright © 2017 M/Gateway Developments Ltd ewd-document-store • Builds on top of the cache.node APIs • Creates a JavaScript-centric abstraction of Global Storage – Mapping to/from JavaScript Objects • The JavaScript equivalent of ^ – Local JavaScript objects (in-memory) – Persistent JavaScript objects (on-disk)
  • 70. Copyright © 2017 M/Gateway Developments Ltd Node.js Architecture • Everything executes in a single process – You might have 1000's of users doing stuff concurrently • They're all sharing the same process!
  • 71. Copyright © 2017 M/Gateway Developments Ltd Brainchild of Ryan Dahl
  • 72. Copyright © 2017 M/Gateway Developments Ltd Node.js = server-side JavaScript • Ryan Dahl wasn't actually a big JavaScript fan – Node.js wasn't designed to be server-side JavaScript • But he realised that it was a convenient language for such an environment, as it's designed for exactly the same kind of way of working in the browser • Google's V8 JavaScript engine was Open Sourced, so he used it to provide the language for Node.js – (which is actually mostly written in C++)
  • 73. Copyright © 2017 M/Gateway Developments Ltd Node.js Architecture • Non-blocking I/O – No user activity must block the process, or everyone grinds to a halt • Event-driven, asynchronous logic – Fire off the request, but don't wait for the results • Carry on with the next task – When results come back, handle them at the next opportunity
  • 74. Copyright © 2017 M/Gateway Developments Ltd ewd-document-store requires synchronous APIs var myDoc = new this.documentStore.DocumentNode('myDoc'); var f3Node = myDoc.$('d').$('e2').$('f3'); myDoc.$d.$e2.$f3.value = 'New value'; myDoc("a")=123 myDoc("b","c1")="foo" myDoc("b","c2")="foo2" myDoc("d","e1","f1")="bar1" myDoc("d","e1","f2")="bar2" myDoc("d","e2","f1")="bar1" myDoc("d","e2","f2")="bar2" myDoc("d","e2","f3")=”New value" Requires properly chainable functions
  • 75. Copyright © 2017 M/Gateway Developments Ltd The Problem is Concurrency • All concurrent users in Node.js share the same process
  • 76. Copyright © 2017 M/Gateway Developments Ltd Is there a way to avoid this? • Would it be possible to create a locally- available environment where my Node.js code runs in an isolated container where concurrency isn't an issue?
  • 77. Copyright © 2017 M/Gateway Developments Ltd
  • 78. Copyright © 2017 M/Gateway Developments Ltd What Is QEWD? • Essentially it's a multi-purpose Node.js- based run-time Platform
  • 79. Copyright © 2017 M/Gateway Developments Ltd What Is QEWD? • Essentially it's a multi-purpose Node.js- based run-time Platform • Creates an isolated run-time container for your message/request handler functions, allowing: – CPU-intensive work – Database abstractions using synchronous logic
  • 80. Copyright © 2017 M/Gateway Developments Ltd QEWD's Architecture • Master Process • Pool of Worker Processes
  • 81. Copyright © 2017 M/Gateway Developments Ltd QEWD's Architecture • Master Process – Handles and queues all incoming requests from client • HTTP/REST requests via Express or Koa.js • WebSocket requests via socket.io – Returns responses to client
  • 82. Copyright © 2017 M/Gateway Developments Ltd QEWD's Architecture • Pool of Persistent Worker Processes – Where all the processing occurs – A single queued request is dispatched to an available worker process – Each Worker process handles a single request at a time
  • 83. Copyright © 2017 M/Gateway Developments Ltd QEWD's Architecture • The queue / dispatcher / worker-process pool management part of QEWD is handled by a module named ewd-qoper8
  • 84. Copyright © 2017 M/Gateway Developments Ltd Master Node.js Process Queue Queue processor/ dispatcher Incoming Requests QEWD Architecture Every incoming request is passed from Express and placed in a queue No further processing of requests occurs in the master process Express or Koa.js socket.io HTTP REST WebSocket
  • 85. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Queue dispatcher is invoked whenever a request is added to the queue
  • 86. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Worker process started if none available
  • 87. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Redis/Cache QEWD & application-specific Modules loaded Custom Worker Module Custom Worker Module
  • 88. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Custom Worker Module Node.js Worker Process Custom Worker Module Request passed to worker Redis/Cache
  • 89. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Custom Worker Module Worker flagged as Unavailable Node.js Worker Process Custom Worker Module Begin processing message Redis/Cache
  • 90. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Unavailable / processing Another incoming request Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache
  • 91. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Node.js Worker Process Master Node.js Process Queue Queue processor/ dispatcher Custom Worker Module Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module If worker pool size not exceeded, another worker is started and request passed to it Redis/Cache Redis/Cache
  • 92. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache If entire Worker Pool is busy: Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache
  • 93. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher If entire Worker Pool is busy: New requests remain in queue Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache
  • 94. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache As soon as a worker is available again, a queued message can be passed to it Unavailable / processing Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache Available Node.js Worker Process Custom Worker Module Redis/Cache
  • 95. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Finished Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Redis/Cache A user's handler function signals completion using the function: finished(responseObject); This returns the response object to the master process
  • 96. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Finished Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module Node.js Worker Process Custom Worker Module And the response is returned to the client that sent the original request (via Express/Koa/socket.io) Redis/Cache
  • 97. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Available Node.js Worker Process Custom Worker Module The finished() function also automatically returns the worker process back to the available pool So it can now handle the next queued request Redis/Cache
  • 98. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Available Node.js Worker Process Custom Worker Module Worker processes, once started, are persistent No start-up / tear-down cost Redis/Cache
  • 99. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Node.js concurrency is handled by the master process. 100% asynchronous logic The master process does almost nothing The perfect Node.js application: no CPU-intensive or long- running tasks, so very high-performance Multiple Concurrent Incoming requests
  • 100. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Node.js concurrency is handled by the master process. 100% asynchronous logic The master process does almost nothing The perfect Node.js application: no CPU-intensive or long- running tasks, so very high-performance All the actual work happens in the isolated worker processes Multiple Concurrent Incoming requests
  • 101. Copyright © 2017 M/Gateway Developments Ltd QEWD Architecture Master Node.js Process Queue Queue processor/ dispatcher Available Node.js Worker Process Custom Worker Module Worker processes only handle a single request at a time Completely isolated run-time environment for handler functions No need for concerns about Node.js concurrency, so synchronous APIs can be used Redis/Cache
  • 102. Copyright © 2017 M/Gateway Developments Ltd QEWD + ewd-document-store + cache.node • JavaScript as a first-class language for Caché • Provides the JavaScript equivalent of ^ – Local JavaScript objects (in-memory) – Persistent JavaScript objects (on-disk) • JavaScript with data persistence as a language feature
  • 103. Copyright © 2017 M/Gateway Developments Ltd Just one remaining issue • Native Caché ObjectScript performance: – Set: >1 million Global Nodes / sec • cache.node – Around 8 – 10% of that performance
  • 104. Copyright © 2017 M/Gateway Developments Ltd The reason? • The Google V8 Engine's API that is used by cache.node – Has a bottleneck • Usually thought to be insignificant • Becomes significant for the kinds of things done by ewd-document-store
  • 105. Copyright © 2017 M/Gateway Developments Ltd If it can be fixed • We'd have the fastest Node.js database on the planet – By a significant margin • We'd have data persistence as a feature of JavaScript – Until it's fully demonstrable, the IT mainstream aren't going to "get it"
  • 106. Copyright © 2017 M/Gateway Developments Ltd Can It Be Fixed? • Probably requires Google to sort it out • Logged on their bug tracker – But no sign of them doing anything to fix it • Watch this space • If it can be fixed, this is potentially huge – And we all stand to benefit
  • 107. Copyright © 2017 M/Gateway Developments Ltd Additional Reading • Universal NoSQL Paper: – http://mgateway.com/docs/universalNoSQL.pdf • QEWD.js: – https://qewdjs.com • JavaScript Abstraction of Global Storage: – http://docs.qewdjs.com/qewd_training.html • In particular, parts 17 – 27 • QEWD.js from a Node.js perspective: – http://bit.ly/2gnnyh5 – http://bit.ly/2ikY1sS
  • 108. Copyright © 2017 M/Gateway Developments Ltd ^ redux: Data Persistence as a Language Feature Rob Tweed M/Gateway Developments Ltd Twitter: @rtweed http://qewdjs.com