EWD 3 Training Course Part 34: QEWD Resilient Mode

R
Rob TweedIT Consultant, Developer & Director/Founder at M/Gateway Developments Ltd
Copyright © 2016 M/Gateway Developments Ltd
EWD 3 Training Course
Part 34
QEWD Resilient Mode
Rob Tweed
Director, M/Gateway Developments Ltd
Twitter: @rtweed
Copyright © 2016 M/Gateway Developments Ltd
QEWD's Request Queue
• QEWD uses the ewd-qoper8 module to provide
the master process queue and worker dispatch
mechanism
• ewd-qoper8's queue is memory based
– A simple JavaScript array
• If the master process stops unexpectedly
or whilst the system is very busy, you
might lose some request messages
Copyright © 2016 M/Gateway Developments Ltd
QEWD Audit Information
• By default, QEWD does not keep any history or
audit information, eg:
– A history of incoming request messages
– A history of corresponding responses
Copyright © 2016 M/Gateway Developments Ltd
QEWD Resilient Mode
• QEWD's built in, optional Resilient Mode
addresses these concerns, at least for
WebSocket/Ajax messages
– Backs up incoming requests to your embedded
database
• Redis or Cache
• Not yet available for GT.M
– Backs up response objects to your embedded
database
– Flags completed messages in the database
• When your message handler's finished() function is invoked
Copyright © 2016 M/Gateway Developments Ltd
QEWD Recovery
• If the QEWD master process stops and is
restarted
– Any clients/browsers that were previously running an
interactive application (WebSockets or Ajax) will re-
register automatically
• Re-establishes the client's QEWD Session
– On re-registration, QEWD checks to see if that
Session had any uncompleted requests in the
Resilient Mode backup database
• If so, they are re-queued automatically
Copyright © 2016 M/Gateway Developments Ltd
QEWD Resilient Mode Storage
• The QEWD master process saves any incoming
messages asynchronously to your embedded
Global Storage database, using the Document
Store abstraction
– By default, saved in a document named ewdQueue
– You can opt to change this name
– This is done in parallel to ewd-qoper8's in-memory
queue
• ewdQueue is only used as a backup
• Resilient mode shouldn't significantly affect ewd-qoper8's
performance
Copyright © 2016 M/Gateway Developments Ltd
QEWD Audit Trail
• The ewdQueue Document can be used as
an audit trail. It contains a record of
– Every incoming request object
– Every outgoing response object
• You can determine the retention period
– Default is 1 hour
– Any records older than the retention period
are automatically purged by QEWD
Copyright © 2016 M/Gateway Developments Ltd
What about Web/REST Services?
• Web/REST Service HTTP requests are not
currently saved into the ewdQueue Document
• Reason:
– If the master process stops, the connection context
between the client and Express is lost
– So it would not be possible to return a response for a
re-queued HTTP request
• This may change in future
– Particularly where you've linked the client to a QEWD
Session: this would allow re-establishment of client
connection context
Copyright © 2016 M/Gateway Developments Ltd
Enabling Resilient Mode
• Simple change to your QEWD startup file
• Add the resilientMode property to the
config object
• Setting it to true applies the defaults:
– ewdQueue as the Document Name
– 1 hour retention period
• Can over-ride these via an object instead
Copyright © 2016 M/Gateway Developments Ltd
Edit your QEWD Startup File
var config = {
managementPassword: 'keepThisSecret!',
serverName: 'My QEWD Server',
port: 8080,
poolSize: 2,
database: {
type: 'cache',
params: {
path: 'c:InterSystemsCache2015-2mgr’
},
resilientMode: true
}
};
var qewd = require('qewd').master;
qewd.start(config);
Default mode
enabled
Copyright © 2016 M/Gateway Developments Ltd
Edit your QEWD Startup File
var config = {
managementPassword: 'keepThisSecret!',
serverName: 'My QEWD Server',
port: 8080,
poolSize: 2,
database: {
type: 'cache',
params: {
path: 'c:InterSystemsCache2015-2mgr’
},
resilientMode: {
keepPeriod: 3600 * 24 // keep for 24 hours
}
}
};
var qewd = require('qewd').master;
qewd.start(config);
Specify the required
retention period
in seconds
Copyright © 2016 M/Gateway Developments Ltd
Edit your QEWD Startup File
var config = {
managementPassword: 'keepThisSecret!',
serverName: 'My QEWD Server',
port: 8080,
poolSize: 2,
database: {
type: 'cache',
params: {
path: 'c:InterSystemsCache2015-2mgr’
},
resilientMode: {
queueDocumentName: 'myAuditTrail'
}
}
};
var qewd = require('qewd').master;
qewd.start(config);
Save the audit trail
information into a
Document named
myAuditTrail instead
Copyright © 2016 M/Gateway Developments Ltd
Edit your QEWD Startup File
var config = {
managementPassword: 'keepThisSecret!',
serverName: 'My QEWD Server',
port: 8080,
poolSize: 2,
database: {
type: 'cache',
params: {
path: 'c:InterSystemsCache2015-2mgr’
},
resilientMode: {
queueDocumentName: 'myAuditTrail',
keepPeriod: 3600*24*30 // 30 day retention
}
}
};
var qewd = require('qewd').master;
qewd.start(config);
Copyright © 2016 M/Gateway Developments Ltd
Restart QEWD
• Resilient Mode is now enabled
• Logging / backup of queued requests and
responses will now commence
Copyright © 2016 M/Gateway Developments Ltd
Queue Document Structure
{
'message': {
{{timeStamp}}: {
'token': //EWD Session token,
'content': //queued request message payload
}
},
'pending': {
sessionToken: { {{timeStamp}}: ''}
}
}
Where {{timeStamp}} is a unique index/property created from
the system clock time
Copyright © 2016 M/Gateway Developments Ltd
Time-stamp format
Logic used is:
var time = process.hrtime();
var ix = time[0] * 1e9 + time[1]; // time in nanoseconds
var timeStamp = new Date().getTime() + '-' + ix;
So the date/time for each audit record can be obtained from:
timeStamp.split('-')[0]
Copyright © 2016 M/Gateway Developments Ltd
Queue Document Structure
During message processing by worker process,
workerStatus information added:
{
'message': {
{{timeStamp}}: {
'token': //EWD Session token,
'content': //queued request message payload,
workerStatus: 'started' || 'error'
}
},
'pending': {
sessionToken: { {{timeStamp}}: ''}
}
}
Copyright © 2016 M/Gateway Developments Ltd
Queue Document Structure
When processing by a worker process is
completed, workerStatus will be finished
and pending index record is deleted
{
'message': {
{{timeStamp}}: {
'token': //EWD Session token,
'content': //queued request message payload,
workerStatus: 'finished'
}
}
}
Copyright © 2016 M/Gateway Developments Ltd
Queue Document Structure
Response objects are logged in the
response array
{
'message': {
{{timeStamp}}: {
'token': //EWD Session token,
'content': //queued request message payload,
workerStatus: 'finished',
response: [
// one or more response objects
]
}
}
}
Copyright © 2016 M/Gateway Developments Ltd
QEWD Audit Trail
• The ewdQueue Document is a standard
ewd-document-store Document, so it can
be accessed and analysed using the
standard DocumentStore / DocumentNode
APIs.
• You can also inspect it using the qewd-
monitor application
• It can be deleted or manipulated without
affecting QEWD's operation
1 of 20

Recommended

EWD 3 Training Course Part 4: Installing & Configuring QEWD by
EWD 3 Training Course Part 4: Installing & Configuring QEWDEWD 3 Training Course Part 4: Installing & Configuring QEWD
EWD 3 Training Course Part 4: Installing & Configuring QEWDRob Tweed
2.9K views31 slides
EWD 3 Training Course Part 30: Modularising QEWD Applications by
EWD 3 Training Course Part 30: Modularising QEWD ApplicationsEWD 3 Training Course Part 30: Modularising QEWD Applications
EWD 3 Training Course Part 30: Modularising QEWD ApplicationsRob Tweed
1.1K views85 slides
EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Data... by
EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Data...EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Data...
EWD 3 Training Course Part 1: How Node.js Integrates With Global Storage Data...Rob Tweed
3.9K views32 slides
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres... by
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...
EWD 3 Training Course Part 37: Building a React.js application with ewd-xpres...Rob Tweed
1.1K views53 slides
EWD 3 Training Course Part 3: Summary of EWD 3 Modules by
EWD 3 Training Course Part 3: Summary of EWD 3 ModulesEWD 3 Training Course Part 3: Summary of EWD 3 Modules
EWD 3 Training Course Part 3: Summary of EWD 3 ModulesRob Tweed
979 views23 slides
EWD 3 Training Course Part 8: Anatomy of the QEWD Messaging Cycle by
EWD 3 Training Course Part 8: Anatomy of the QEWD Messaging CycleEWD 3 Training Course Part 8: Anatomy of the QEWD Messaging Cycle
EWD 3 Training Course Part 8: Anatomy of the QEWD Messaging CycleRob Tweed
720 views20 slides

More Related Content

What's hot

EWD 3 Training Course Part 19: The cache.node APIs by
EWD 3 Training Course Part 19: The cache.node APIsEWD 3 Training Course Part 19: The cache.node APIs
EWD 3 Training Course Part 19: The cache.node APIsRob Tweed
905 views50 slides
EWD 3 Training Course Part 16: QEWD Services by
EWD 3 Training Course Part 16: QEWD ServicesEWD 3 Training Course Part 16: QEWD Services
EWD 3 Training Course Part 16: QEWD ServicesRob Tweed
671 views21 slides
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap... by
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...Rob Tweed
932 views45 slides
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality by
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService FunctionalityEWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService FunctionalityRob Tweed
2.1K views254 slides
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.js by
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.jsEWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.jsRob Tweed
1.5K views136 slides
EWD 3 Training Course Part 31: Using QEWD for Web and REST Services by
EWD 3 Training Course Part 31: Using QEWD for Web and REST ServicesEWD 3 Training Course Part 31: Using QEWD for Web and REST Services
EWD 3 Training Course Part 31: Using QEWD for Web and REST ServicesRob Tweed
3.3K views154 slides

What's hot(20)

EWD 3 Training Course Part 19: The cache.node APIs by Rob Tweed
EWD 3 Training Course Part 19: The cache.node APIsEWD 3 Training Course Part 19: The cache.node APIs
EWD 3 Training Course Part 19: The cache.node APIs
Rob Tweed905 views
EWD 3 Training Course Part 16: QEWD Services by Rob Tweed
EWD 3 Training Course Part 16: QEWD ServicesEWD 3 Training Course Part 16: QEWD Services
EWD 3 Training Course Part 16: QEWD Services
Rob Tweed671 views
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap... by Rob Tweed
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
EWD 3 Training Course Part 36: Accessing REST and Web Services from a QEWD ap...
Rob Tweed932 views
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality by Rob Tweed
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService FunctionalityEWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
EWD 3 Training Course Part 45: Using QEWD's Advanced MicroService Functionality
Rob Tweed2.1K views
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.js by Rob Tweed
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.jsEWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
EWD 3 Training Course Part 44: Creating MicroServices with QEWD.js
Rob Tweed1.5K views
EWD 3 Training Course Part 31: Using QEWD for Web and REST Services by Rob Tweed
EWD 3 Training Course Part 31: Using QEWD for Web and REST ServicesEWD 3 Training Course Part 31: Using QEWD for Web and REST Services
EWD 3 Training Course Part 31: Using QEWD for Web and REST Services
Rob Tweed3.3K views
qewd-ripple: The Ripple OSI Middle Tier by Rob Tweed
qewd-ripple: The Ripple OSI Middle Tierqewd-ripple: The Ripple OSI Middle Tier
qewd-ripple: The Ripple OSI Middle Tier
Rob Tweed720 views
EWD 3 Training Course Part 2: EWD 3 Overview by Rob Tweed
EWD 3 Training Course Part 2: EWD 3 OverviewEWD 3 Training Course Part 2: EWD 3 Overview
EWD 3 Training Course Part 2: EWD 3 Overview
Rob Tweed2.1K views
EWD 3 Training Course Part 11: Handling Errors in QEWD by Rob Tweed
EWD 3 Training Course Part 11: Handling Errors in QEWDEWD 3 Training Course Part 11: Handling Errors in QEWD
EWD 3 Training Course Part 11: Handling Errors in QEWD
Rob Tweed639 views
EWD 3 Training Course Part 5b: First Steps in Building a QEWD Application by Rob Tweed
EWD 3 Training Course Part 5b: First Steps in Building a QEWD ApplicationEWD 3 Training Course Part 5b: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5b: First Steps in Building a QEWD Application
Rob Tweed999 views
EWD 3 Training Course Part 42: The QEWD Docker Appliance by Rob Tweed
EWD 3 Training Course Part 42: The QEWD Docker ApplianceEWD 3 Training Course Part 42: The QEWD Docker Appliance
EWD 3 Training Course Part 42: The QEWD Docker Appliance
Rob Tweed1.3K views
QEWD.js, JSON Web Tokens & MicroServices by Rob Tweed
QEWD.js, JSON Web Tokens & MicroServicesQEWD.js, JSON Web Tokens & MicroServices
QEWD.js, JSON Web Tokens & MicroServices
Rob Tweed903 views
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD by Rob Tweed
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWDEWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD
EWD 3 Training Course Part 28: Integrating Legacy Mumps Code with QEWD
Rob Tweed1.2K views
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2 by Rob Tweed
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 2
Rob Tweed597 views
EWD 3 Training Course Part 5a: First Steps in Building a QEWD Application by Rob Tweed
EWD 3 Training Course Part 5a: First Steps in Building a QEWD ApplicationEWD 3 Training Course Part 5a: First Steps in Building a QEWD Application
EWD 3 Training Course Part 5a: First Steps in Building a QEWD Application
Rob Tweed1.4K views
EWD 3 Training Course Part 35: QEWD Session Locking by Rob Tweed
EWD 3 Training Course Part 35: QEWD Session LockingEWD 3 Training Course Part 35: QEWD Session Locking
EWD 3 Training Course Part 35: QEWD Session Locking
Rob Tweed532 views
EWD 3 Training Course Part 7: Applying the QEWD Messaging Pattern by Rob Tweed
EWD 3 Training Course Part 7: Applying the QEWD Messaging PatternEWD 3 Training Course Part 7: Applying the QEWD Messaging Pattern
EWD 3 Training Course Part 7: Applying the QEWD Messaging Pattern
Rob Tweed1.3K views
ewd-qoper8-vistarpc: Exposing VistA's RPCs as REST Services by Rob Tweed
ewd-qoper8-vistarpc: Exposing VistA's RPCs as REST Servicesewd-qoper8-vistarpc: Exposing VistA's RPCs as REST Services
ewd-qoper8-vistarpc: Exposing VistA's RPCs as REST Services
Rob Tweed333 views
EWD 3 Training Course Part 14: Using Ajax for QEWD Messages by Rob Tweed
EWD 3 Training Course Part 14: Using Ajax for QEWD MessagesEWD 3 Training Course Part 14: Using Ajax for QEWD Messages
EWD 3 Training Course Part 14: Using Ajax for QEWD Messages
Rob Tweed693 views
EWD 3 Training Course Part 12: QEWD Session Timeout Control by Rob Tweed
EWD 3 Training Course Part 12: QEWD Session Timeout ControlEWD 3 Training Course Part 12: QEWD Session Timeout Control
EWD 3 Training Course Part 12: QEWD Session Timeout Control
Rob Tweed561 views

Viewers also liked

EWD 3 Training Course Part 29: Running QEWD as a Service by
EWD 3 Training Course Part 29: Running QEWD as a ServiceEWD 3 Training Course Part 29: Running QEWD as a Service
EWD 3 Training Course Part 29: Running QEWD as a ServiceRob Tweed
1K views25 slides
EWD 3 Training Course Part 25: Document Database Capabilities by
EWD 3 Training Course Part 25: Document Database CapabilitiesEWD 3 Training Course Part 25: Document Database Capabilities
EWD 3 Training Course Part 25: Document Database CapabilitiesRob Tweed
905 views49 slides
EWD 3 Training Course Part 24: Traversing a Document's Leaf Nodes by
EWD 3 Training Course Part 24: Traversing a Document's Leaf NodesEWD 3 Training Course Part 24: Traversing a Document's Leaf Nodes
EWD 3 Training Course Part 24: Traversing a Document's Leaf NodesRob Tweed
622 views19 slides
EWD 3 Training Course Part 21: Persistent JavaScript Objects by
EWD 3 Training Course Part 21: Persistent JavaScript ObjectsEWD 3 Training Course Part 21: Persistent JavaScript Objects
EWD 3 Training Course Part 21: Persistent JavaScript ObjectsRob Tweed
779 views31 slides
EWD 3 Training Course Part 13: Putting Everything so far into Practice using ... by
EWD 3 Training Course Part 13: Putting Everything so far into Practice using ...EWD 3 Training Course Part 13: Putting Everything so far into Practice using ...
EWD 3 Training Course Part 13: Putting Everything so far into Practice using ...Rob Tweed
805 views14 slides
EWD 3 Training Course Part 20: The DocumentNode Object by
EWD 3 Training Course Part 20: The DocumentNode ObjectEWD 3 Training Course Part 20: The DocumentNode Object
EWD 3 Training Course Part 20: The DocumentNode ObjectRob Tweed
1K views41 slides

Viewers also liked(12)

EWD 3 Training Course Part 29: Running QEWD as a Service by Rob Tweed
EWD 3 Training Course Part 29: Running QEWD as a ServiceEWD 3 Training Course Part 29: Running QEWD as a Service
EWD 3 Training Course Part 29: Running QEWD as a Service
Rob Tweed1K views
EWD 3 Training Course Part 25: Document Database Capabilities by Rob Tweed
EWD 3 Training Course Part 25: Document Database CapabilitiesEWD 3 Training Course Part 25: Document Database Capabilities
EWD 3 Training Course Part 25: Document Database Capabilities
Rob Tweed905 views
EWD 3 Training Course Part 24: Traversing a Document's Leaf Nodes by Rob Tweed
EWD 3 Training Course Part 24: Traversing a Document's Leaf NodesEWD 3 Training Course Part 24: Traversing a Document's Leaf Nodes
EWD 3 Training Course Part 24: Traversing a Document's Leaf Nodes
Rob Tweed622 views
EWD 3 Training Course Part 21: Persistent JavaScript Objects by Rob Tweed
EWD 3 Training Course Part 21: Persistent JavaScript ObjectsEWD 3 Training Course Part 21: Persistent JavaScript Objects
EWD 3 Training Course Part 21: Persistent JavaScript Objects
Rob Tweed779 views
EWD 3 Training Course Part 13: Putting Everything so far into Practice using ... by Rob Tweed
EWD 3 Training Course Part 13: Putting Everything so far into Practice using ...EWD 3 Training Course Part 13: Putting Everything so far into Practice using ...
EWD 3 Training Course Part 13: Putting Everything so far into Practice using ...
Rob Tweed805 views
EWD 3 Training Course Part 20: The DocumentNode Object by Rob Tweed
EWD 3 Training Course Part 20: The DocumentNode ObjectEWD 3 Training Course Part 20: The DocumentNode Object
EWD 3 Training Course Part 20: The DocumentNode Object
Rob Tweed1K views
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS by Rob Tweed
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPSEWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS
EWD 3 Training Course Part 32: Configuring QEWD to use SSL/HTTPS
Rob Tweed608 views
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3 by Rob Tweed
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3
EWD 3 Training Course Part 39: Building a React.js application with QEWD, Part 3
Rob Tweed627 views
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4 by Rob Tweed
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4
EWD 3 Training Course Part 38: Building a React.js application with QEWD, Part 4
Rob Tweed718 views
EWD 3 Training Course Part 22: Traversing Documents using DocumentNode Objects by Rob Tweed
EWD 3 Training Course Part 22: Traversing Documents using DocumentNode ObjectsEWD 3 Training Course Part 22: Traversing Documents using DocumentNode Objects
EWD 3 Training Course Part 22: Traversing Documents using DocumentNode Objects
Rob Tweed732 views
EWD 3 Training Course Part 33: Configuring QEWD to use CORS by Rob Tweed
EWD 3 Training Course Part 33: Configuring QEWD to use CORSEWD 3 Training Course Part 33: Configuring QEWD to use CORS
EWD 3 Training Course Part 33: Configuring QEWD to use CORS
Rob Tweed1.1K views
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5 by Rob Tweed
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5
EWD 3 Training Course Part 41: Building a React.js application with QEWD, Part 5
Rob Tweed621 views

Similar to EWD 3 Training Course Part 34: QEWD Resilient Mode

Clug 2012 March web server optimisation by
Clug 2012 March   web server optimisationClug 2012 March   web server optimisation
Clug 2012 March web server optimisationgrooverdan
689 views30 slides
19servlets by
19servlets19servlets
19servletsAdil Jafri
173 views107 slides
Servlets by
ServletsServlets
ServletsSharon Cek
1.7K views54 slides
Servlets Java Slides & Presentation by
Servlets Java Slides & Presentation Servlets Java Slides & Presentation
Servlets Java Slides & Presentation Anas Sa
108 views54 slides
AppFabric Velocity by
AppFabric VelocityAppFabric Velocity
AppFabric VelocityDennis van der Stelt
997 views39 slides
cache concepts and varnish-cache by
cache concepts and varnish-cachecache concepts and varnish-cache
cache concepts and varnish-cacheMarc Cortinas Val
1.8K views29 slides

Similar to EWD 3 Training Course Part 34: QEWD Resilient Mode(20)

Clug 2012 March web server optimisation by grooverdan
Clug 2012 March   web server optimisationClug 2012 March   web server optimisation
Clug 2012 March web server optimisation
grooverdan689 views
Servlets Java Slides & Presentation by Anas Sa
Servlets Java Slides & Presentation Servlets Java Slides & Presentation
Servlets Java Slides & Presentation
Anas Sa108 views
Nginx Scalable Stack by Bruno Paiuca
Nginx Scalable StackNginx Scalable Stack
Nginx Scalable Stack
Bruno Paiuca222 views
High Availability Content Caching with NGINX by NGINX, Inc.
High Availability Content Caching with NGINXHigh Availability Content Caching with NGINX
High Availability Content Caching with NGINX
NGINX, Inc.3.2K views
[Hanoi-August 13] Tech Talk on Caching Solutions by ITviec
[Hanoi-August 13] Tech Talk on Caching Solutions[Hanoi-August 13] Tech Talk on Caching Solutions
[Hanoi-August 13] Tech Talk on Caching Solutions
ITviec2.6K views
ITB2017 - Nginx Effective High Availability Content Caching by Ortus Solutions, Corp
ITB2017 - Nginx Effective High Availability Content CachingITB2017 - Nginx Effective High Availability Content Caching
ITB2017 - Nginx Effective High Availability Content Caching
More Cache for Less Cash (DevLink 2014) by Michael Collier
More Cache for Less Cash (DevLink 2014)More Cache for Less Cash (DevLink 2014)
More Cache for Less Cash (DevLink 2014)
Michael Collier8K views
High Availability Content Caching with NGINX by Kevin Jones
High Availability Content Caching with NGINXHigh Availability Content Caching with NGINX
High Availability Content Caching with NGINX
Kevin Jones2.5K views
EWD 3 Training Course Part 6: What Happens when a QEWD Application is Started by Rob Tweed
EWD 3 Training Course Part 6: What Happens when a QEWD Application is StartedEWD 3 Training Course Part 6: What Happens when a QEWD Application is Started
EWD 3 Training Course Part 6: What Happens when a QEWD Application is Started
Rob Tweed1K views
GoldenGate and Oracle Data Integrator - A Perfect Match- Upgrade to 12c by Michael Rainey
GoldenGate and Oracle Data Integrator - A Perfect Match- Upgrade to 12cGoldenGate and Oracle Data Integrator - A Perfect Match- Upgrade to 12c
GoldenGate and Oracle Data Integrator - A Perfect Match- Upgrade to 12c
Michael Rainey883 views
0628阙宏宇 by zhu02
0628阙宏宇0628阙宏宇
0628阙宏宇
zhu021.1K views
WebLogic Stability; Detect and Analyse Stuck Threads by Maarten Smeets
WebLogic Stability; Detect and Analyse Stuck ThreadsWebLogic Stability; Detect and Analyse Stuck Threads
WebLogic Stability; Detect and Analyse Stuck Threads
Maarten Smeets3.9K views
How to Build a Multi-DC Cassandra Cluster in AWS with OpsCenter LCM by Anant Corporation
How to Build a Multi-DC Cassandra Cluster in AWS with OpsCenter LCMHow to Build a Multi-DC Cassandra Cluster in AWS with OpsCenter LCM
How to Build a Multi-DC Cassandra Cluster in AWS with OpsCenter LCM
Anant Corporation1.4K views
Managing your exchange architecture by Veeam Software
Managing your exchange architectureManaging your exchange architecture
Managing your exchange architecture
Veeam Software920 views
Less and faster – Cache tips for WordPress developers by Seravo
Less and faster – Cache tips for WordPress developersLess and faster – Cache tips for WordPress developers
Less and faster – Cache tips for WordPress developers
Seravo269 views

More from Rob Tweed

QEWD Update by
QEWD UpdateQEWD Update
QEWD UpdateRob Tweed
945 views32 slides
Data Persistence as a Language Feature by
Data Persistence as a Language FeatureData Persistence as a Language Feature
Data Persistence as a Language FeatureRob Tweed
1.3K views108 slides
LNUG: Having Your Node.js Cake and Eating It Too by
LNUG: Having Your Node.js Cake and Eating It TooLNUG: Having Your Node.js Cake and Eating It Too
LNUG: Having Your Node.js Cake and Eating It TooRob Tweed
2.8K views81 slides
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services by
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST ServicesEWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST ServicesRob Tweed
891 views93 slides
QEWD.js: Have your Node.js Cake and Eat It Too by
QEWD.js: Have your Node.js Cake and Eat It TooQEWD.js: Have your Node.js Cake and Eat It Too
QEWD.js: Have your Node.js Cake and Eat It TooRob Tweed
628 views76 slides
EWD 3 Training Course Part 23: Traversing a Range using DocumentNode Objects by
EWD 3 Training Course Part 23: Traversing a Range using DocumentNode ObjectsEWD 3 Training Course Part 23: Traversing a Range using DocumentNode Objects
EWD 3 Training Course Part 23: Traversing a Range using DocumentNode ObjectsRob Tweed
797 views35 slides

More from Rob Tweed(6)

QEWD Update by Rob Tweed
QEWD UpdateQEWD Update
QEWD Update
Rob Tweed945 views
Data Persistence as a Language Feature by Rob Tweed
Data Persistence as a Language FeatureData Persistence as a Language Feature
Data Persistence as a Language Feature
Rob Tweed1.3K views
LNUG: Having Your Node.js Cake and Eating It Too by Rob Tweed
LNUG: Having Your Node.js Cake and Eating It TooLNUG: Having Your Node.js Cake and Eating It Too
LNUG: Having Your Node.js Cake and Eating It Too
Rob Tweed2.8K views
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services by Rob Tweed
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST ServicesEWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
EWD 3 Training Course Part 43: Using JSON Web Tokens with QEWD REST Services
Rob Tweed891 views
QEWD.js: Have your Node.js Cake and Eat It Too by Rob Tweed
QEWD.js: Have your Node.js Cake and Eat It TooQEWD.js: Have your Node.js Cake and Eat It Too
QEWD.js: Have your Node.js Cake and Eat It Too
Rob Tweed628 views
EWD 3 Training Course Part 23: Traversing a Range using DocumentNode Objects by Rob Tweed
EWD 3 Training Course Part 23: Traversing a Range using DocumentNode ObjectsEWD 3 Training Course Part 23: Traversing a Range using DocumentNode Objects
EWD 3 Training Course Part 23: Traversing a Range using DocumentNode Objects
Rob Tweed797 views

Recently uploaded

Mobile App Development Company by
Mobile App Development CompanyMobile App Development Company
Mobile App Development CompanyRichestsoft
5 views6 slides
Top-5-production-devconMunich-2023-v2.pptx by
Top-5-production-devconMunich-2023-v2.pptxTop-5-production-devconMunich-2023-v2.pptx
Top-5-production-devconMunich-2023-v2.pptxTier1 app
8 views42 slides
Supercharging your Python Development Environment with VS Code and Dev Contai... by
Supercharging your Python Development Environment with VS Code and Dev Contai...Supercharging your Python Development Environment with VS Code and Dev Contai...
Supercharging your Python Development Environment with VS Code and Dev Contai...Dawn Wages
5 views51 slides
How Workforce Management Software Empowers SMEs | TraQSuite by
How Workforce Management Software Empowers SMEs | TraQSuiteHow Workforce Management Software Empowers SMEs | TraQSuite
How Workforce Management Software Empowers SMEs | TraQSuiteTraQSuite
6 views3 slides
The Path to DevOps by
The Path to DevOpsThe Path to DevOps
The Path to DevOpsJohn Valentino
5 views6 slides
How to build dyanmic dashboards and ensure they always work by
How to build dyanmic dashboards and ensure they always workHow to build dyanmic dashboards and ensure they always work
How to build dyanmic dashboards and ensure they always workWiiisdom
14 views13 slides

Recently uploaded(20)

Mobile App Development Company by Richestsoft
Mobile App Development CompanyMobile App Development Company
Mobile App Development Company
Richestsoft 5 views
Top-5-production-devconMunich-2023-v2.pptx by Tier1 app
Top-5-production-devconMunich-2023-v2.pptxTop-5-production-devconMunich-2023-v2.pptx
Top-5-production-devconMunich-2023-v2.pptx
Tier1 app8 views
Supercharging your Python Development Environment with VS Code and Dev Contai... by Dawn Wages
Supercharging your Python Development Environment with VS Code and Dev Contai...Supercharging your Python Development Environment with VS Code and Dev Contai...
Supercharging your Python Development Environment with VS Code and Dev Contai...
Dawn Wages5 views
How Workforce Management Software Empowers SMEs | TraQSuite by TraQSuite
How Workforce Management Software Empowers SMEs | TraQSuiteHow Workforce Management Software Empowers SMEs | TraQSuite
How Workforce Management Software Empowers SMEs | TraQSuite
TraQSuite6 views
How to build dyanmic dashboards and ensure they always work by Wiiisdom
How to build dyanmic dashboards and ensure they always workHow to build dyanmic dashboards and ensure they always work
How to build dyanmic dashboards and ensure they always work
Wiiisdom14 views
Quality Engineer: A Day in the Life by John Valentino
Quality Engineer: A Day in the LifeQuality Engineer: A Day in the Life
Quality Engineer: A Day in the Life
John Valentino7 views
Electronic AWB - Electronic Air Waybill by Freightoscope
Electronic AWB - Electronic Air Waybill Electronic AWB - Electronic Air Waybill
Electronic AWB - Electronic Air Waybill
Freightoscope 5 views
Ports-and-Adapters Architecture for Embedded HMI by Burkhard Stubert
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMI
Burkhard Stubert33 views
Understanding HTML terminology by artembondar5
Understanding HTML terminologyUnderstanding HTML terminology
Understanding HTML terminology
artembondar57 views
Bootstrapping vs Venture Capital.pptx by Zeljko Svedic
Bootstrapping vs Venture Capital.pptxBootstrapping vs Venture Capital.pptx
Bootstrapping vs Venture Capital.pptx
Zeljko Svedic15 views
Automated Testing of Microsoft Power BI Reports by RTTS
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI Reports
RTTS10 views

EWD 3 Training Course Part 34: QEWD Resilient Mode

  • 1. Copyright © 2016 M/Gateway Developments Ltd EWD 3 Training Course Part 34 QEWD Resilient Mode Rob Tweed Director, M/Gateway Developments Ltd Twitter: @rtweed
  • 2. Copyright © 2016 M/Gateway Developments Ltd QEWD's Request Queue • QEWD uses the ewd-qoper8 module to provide the master process queue and worker dispatch mechanism • ewd-qoper8's queue is memory based – A simple JavaScript array • If the master process stops unexpectedly or whilst the system is very busy, you might lose some request messages
  • 3. Copyright © 2016 M/Gateway Developments Ltd QEWD Audit Information • By default, QEWD does not keep any history or audit information, eg: – A history of incoming request messages – A history of corresponding responses
  • 4. Copyright © 2016 M/Gateway Developments Ltd QEWD Resilient Mode • QEWD's built in, optional Resilient Mode addresses these concerns, at least for WebSocket/Ajax messages – Backs up incoming requests to your embedded database • Redis or Cache • Not yet available for GT.M – Backs up response objects to your embedded database – Flags completed messages in the database • When your message handler's finished() function is invoked
  • 5. Copyright © 2016 M/Gateway Developments Ltd QEWD Recovery • If the QEWD master process stops and is restarted – Any clients/browsers that were previously running an interactive application (WebSockets or Ajax) will re- register automatically • Re-establishes the client's QEWD Session – On re-registration, QEWD checks to see if that Session had any uncompleted requests in the Resilient Mode backup database • If so, they are re-queued automatically
  • 6. Copyright © 2016 M/Gateway Developments Ltd QEWD Resilient Mode Storage • The QEWD master process saves any incoming messages asynchronously to your embedded Global Storage database, using the Document Store abstraction – By default, saved in a document named ewdQueue – You can opt to change this name – This is done in parallel to ewd-qoper8's in-memory queue • ewdQueue is only used as a backup • Resilient mode shouldn't significantly affect ewd-qoper8's performance
  • 7. Copyright © 2016 M/Gateway Developments Ltd QEWD Audit Trail • The ewdQueue Document can be used as an audit trail. It contains a record of – Every incoming request object – Every outgoing response object • You can determine the retention period – Default is 1 hour – Any records older than the retention period are automatically purged by QEWD
  • 8. Copyright © 2016 M/Gateway Developments Ltd What about Web/REST Services? • Web/REST Service HTTP requests are not currently saved into the ewdQueue Document • Reason: – If the master process stops, the connection context between the client and Express is lost – So it would not be possible to return a response for a re-queued HTTP request • This may change in future – Particularly where you've linked the client to a QEWD Session: this would allow re-establishment of client connection context
  • 9. Copyright © 2016 M/Gateway Developments Ltd Enabling Resilient Mode • Simple change to your QEWD startup file • Add the resilientMode property to the config object • Setting it to true applies the defaults: – ewdQueue as the Document Name – 1 hour retention period • Can over-ride these via an object instead
  • 10. Copyright © 2016 M/Gateway Developments Ltd Edit your QEWD Startup File var config = { managementPassword: 'keepThisSecret!', serverName: 'My QEWD Server', port: 8080, poolSize: 2, database: { type: 'cache', params: { path: 'c:InterSystemsCache2015-2mgr’ }, resilientMode: true } }; var qewd = require('qewd').master; qewd.start(config); Default mode enabled
  • 11. Copyright © 2016 M/Gateway Developments Ltd Edit your QEWD Startup File var config = { managementPassword: 'keepThisSecret!', serverName: 'My QEWD Server', port: 8080, poolSize: 2, database: { type: 'cache', params: { path: 'c:InterSystemsCache2015-2mgr’ }, resilientMode: { keepPeriod: 3600 * 24 // keep for 24 hours } } }; var qewd = require('qewd').master; qewd.start(config); Specify the required retention period in seconds
  • 12. Copyright © 2016 M/Gateway Developments Ltd Edit your QEWD Startup File var config = { managementPassword: 'keepThisSecret!', serverName: 'My QEWD Server', port: 8080, poolSize: 2, database: { type: 'cache', params: { path: 'c:InterSystemsCache2015-2mgr’ }, resilientMode: { queueDocumentName: 'myAuditTrail' } } }; var qewd = require('qewd').master; qewd.start(config); Save the audit trail information into a Document named myAuditTrail instead
  • 13. Copyright © 2016 M/Gateway Developments Ltd Edit your QEWD Startup File var config = { managementPassword: 'keepThisSecret!', serverName: 'My QEWD Server', port: 8080, poolSize: 2, database: { type: 'cache', params: { path: 'c:InterSystemsCache2015-2mgr’ }, resilientMode: { queueDocumentName: 'myAuditTrail', keepPeriod: 3600*24*30 // 30 day retention } } }; var qewd = require('qewd').master; qewd.start(config);
  • 14. Copyright © 2016 M/Gateway Developments Ltd Restart QEWD • Resilient Mode is now enabled • Logging / backup of queued requests and responses will now commence
  • 15. Copyright © 2016 M/Gateway Developments Ltd Queue Document Structure { 'message': { {{timeStamp}}: { 'token': //EWD Session token, 'content': //queued request message payload } }, 'pending': { sessionToken: { {{timeStamp}}: ''} } } Where {{timeStamp}} is a unique index/property created from the system clock time
  • 16. Copyright © 2016 M/Gateway Developments Ltd Time-stamp format Logic used is: var time = process.hrtime(); var ix = time[0] * 1e9 + time[1]; // time in nanoseconds var timeStamp = new Date().getTime() + '-' + ix; So the date/time for each audit record can be obtained from: timeStamp.split('-')[0]
  • 17. Copyright © 2016 M/Gateway Developments Ltd Queue Document Structure During message processing by worker process, workerStatus information added: { 'message': { {{timeStamp}}: { 'token': //EWD Session token, 'content': //queued request message payload, workerStatus: 'started' || 'error' } }, 'pending': { sessionToken: { {{timeStamp}}: ''} } }
  • 18. Copyright © 2016 M/Gateway Developments Ltd Queue Document Structure When processing by a worker process is completed, workerStatus will be finished and pending index record is deleted { 'message': { {{timeStamp}}: { 'token': //EWD Session token, 'content': //queued request message payload, workerStatus: 'finished' } } }
  • 19. Copyright © 2016 M/Gateway Developments Ltd Queue Document Structure Response objects are logged in the response array { 'message': { {{timeStamp}}: { 'token': //EWD Session token, 'content': //queued request message payload, workerStatus: 'finished', response: [ // one or more response objects ] } } }
  • 20. Copyright © 2016 M/Gateway Developments Ltd QEWD Audit Trail • The ewdQueue Document is a standard ewd-document-store Document, so it can be accessed and analysed using the standard DocumentStore / DocumentNode APIs. • You can also inspect it using the qewd- monitor application • It can be deleted or manipulated without affecting QEWD's operation