The following slides summarize and curate most of the knowledge and patterns gathered to date on Node error handling.
Without clear understanding and strategy, Node error handling might be the Achilles heel of your app – its unique single-threaded execution model and loose types raise challenges that don’t exist in any other frameworks. Node by itself doesn’t provide patterns for critical paths like where to put error handling code, even worst it suggest patterns that were rejected by the community like passing errors in callbacks.
It covers topics like promises, generators, callbacks, unhandled exceptions, APM products, testing errors, operational errors vs development errors and much more
2. In this presentation
YONI GOLDBERG2 |
Agenda: The following slides summarize and curate most of the knowledge and
patterns gathered to date on Node error handling. It contains more than 35 quotes,
code examples and diagrams from the highest ranked blog posts and
StackOverflow threads.
Why is this important: Node error handling embodies unique challenges and gotchas –
without clear understanding and strategy it might be the Achilles heel of your app
You may read the 10 top ranked blog posts on error handling OR watch the
following slides which summarizes them all
3. Summarizes and curates 6 blog posts
Use promises for async
error handling
1
1
2
3
4
5
6
doWork()
.then(doWork)
.then(doOtherWork)
.then((result) => doWork)
.catch((error) => throw error)
.then(verify);
4. TL;DR
YONI GOLDBERG4 |
Handling asynchronous errors in callback style is probably the fastest way to hell (a.k.a the pyramid of doom).
The best gift you can give to your code is using instead a reputable promise library which provides much compact
and familiar code syntax like try-catch
Code example – using promises to catch errors
1
2
3
4
5
6
doWork()
.then(doWork)
.then(doOtherWork)
.then((result) => doWork)
.catch((error) => throw error)
.then(verify);
5. Otherwise
YONI GOLDBERG5 |
Node.JS callback style, function(err, response), is a promising way to un-maintainable code due to the mix of error
handling with casual code and over-nested code patterns, see below
Anti pattern code example – callback style error handling
1
2
3
4
5
6
7
8
9
10
11
12
getData(someParameter, function(err, result){
if(err != null)
//do something like calling the given callback function and pass the error
getMoreData(a, function(err, result){
if(err != null)
//do something like calling the given callback function and pass the error
getMoreData(b, function(c){
getMoreData(d, function(e){
if(err != null)
//you get the idea?
});
});
6. One paragraph explainer
YONI GOLDBERG6 |
Callbacks don’t scale as they are not familiar to most programmers, force to check errors all over, deal with nasty code nesting and make it difficult to
reason about the code flow. Promise libraries like BlueBird, async, and Q pack a standard code style using RETURN and THROW to control the
program flow. Specifically, they support the favorite try-catch error handling style which allows freeing the main code path from dealing with errors in
every function
From the blog pouchdb.com, ranked 11 for the keywords “Node Promises”
…And in fact, callbacks do something even more sinister: they deprive us of the stack, which is something we usually take for
granted in programming languages. Writing code without a stack is a lot like driving a car without a brake pedal: you don’t realize
how badly you need it, until you reach for it and it’s not there. The whole point of promises is to give us back the language
fundamentals we lost when we went async: return, throw, and the stack. But you have to know how to use promises
correctly in order to take advantage of them.
“We have a problem with promises”
7. What other bloggers say
YONI GOLDBERG7 |
From the blog pouchdb.com, ranked 11 for the keywords “Node Promises”
…And in fact, callbacks do something even more sinister: they deprive us of the stack, which is something we usually take for
granted in programming languages. Writing code without a stack is a lot like driving a car without a brake pedal: you don’t realize
how badly you need it, until you reach for it and it’s not there. The whole point of promises is to give us back the language
fundamentals we lost when we went async: return, throw, and the stack. But you have to know how to use promises
correctly in order to take advantage of them.
Blog Quote: “We have a problem with promises”
From the blog gosquared.com, ranked 5 for the keywords “Node.JS error handling”
…The promises method is much more compact, clearer and quicker to write. If an error or exception occurs within any of the ops it is
handled by the single .catch() handler. Having this single place to handle all errors means you don’t need to write error
checking for each stage of the work.
Blog Quote: “The promises method is much more compact”
8. What other bloggers say
YONI GOLDBERG8 |
From the blog StrongLoop, ranked 7 for the keywords “Node.JS error handling”
…Callbacks have a lousy error-handling story. Promises are better. Marry the built-in error handling in Express with promises and
significantly lower the chances of an uncaught exception. Promises are native ES6, can be used with generators, and ES7
proposals like async/await through compilers like Babel
Blog Quote: “Promises are native ES6, can be used with generators”
From the blog Benno’s, ranked 13 for the keywords “Node.JS error handling”
…One of the best things about asynchronous, callback based programming is that basically all those regular flow control constructs
you are used to are completely broken. However, the one I find most broken is the handling of exceptions. Javascript provides a fairly
familiar try…catch construct for dealing with exceptions. The problems with exceptions is that they provide a great way of short-
cutting errors up a call stack, but end up being completely useless of the error happens on a different stack…
Blog Quote: “All those regular flow control constructs you are used to are completely broken”
9. Summarizes and quotes 6 sources
Use only the built-in
Error object
2
1
2
3
4
5
if(!productToAdd)
throw new Error(“invalidInput“ , 400,
“No product provided”);
10. TL;DR
YONI GOLDBERG10 |
Many throws errors as a string or as some custom type - this complicates the error handling logic and the
interoperability between modules. Whether you reject a promise, throw exception or emit error – using only the
built-in Error object will increases uniformity and prevents loss of information
Code example - doing it right
1
2
3
4
5
//'throwing' an Error from a Promise
return new promise(function (resolve, reject) {
Return DAL.getProduct(productToAdd.id).then((existingProduct) =>{
if(existingProduct != null)
reject(new Error("Why fooling us and trying to add an existing product?"));
)};
11. Otherwise
YONI GOLDBERG11 |
When invoking some component, being uncertain which type of errors come in return – makes it much harder to
handle errors properly. Even worth, using custom types to describe errors might lead to loss of critical error
information like the stack trace!
Code example - Anti Pattern
1
2
3
//throwing a String lacks any stack trace information and other important properties
if(!productToAdd)
throw ("How can I add new product when no value provided?");
//throwing as custom object also lack the stack trace
if(price < 0)
throw {errorType:invalidInput};
12. One paragraph explainer
YONI GOLDBERG12 |
The permissive nature of JS along with its variety code-flow options (e.g. EventEmitter, Callbacks, Promises, etc) pushes to great variance in how
developers raise errors – some use strings, other define their own custom types. Using Node.JS built-in Error object helps to keep uniformity within
your code and with 3rd party libraries, it also preserves significant information like the StackTrace. When raising the exception, it’s usually a good
practice to fill it with additional contextual properties like the error name and the associated HTTP error code. To achieve this uniformity and
practices, consider extending the Error object with additional properties (see code examples on next slides)
From the blog devthought.com, ranked 6 for the keywords “Node.JS error object”
…passing a string instead of an error results in reduced interoperability between modules. It breaks contracts with APIs that might
be performing instanceof Error checks, or that want to know more about the error. Error objects, as we’ll see, have very
interesting properties in modern JavaScript engines besides holding the message passed to the constructor…
“A string is not an error”
13. Code Example – extended Error object
YONI GOLDBERG13 |
Code example – doing it even better with extended error object
1
2
3
4
5
6
7
8
9
10
11
12
//centralized error object that derives from Node’s Error
function (name, httpCode, description, isOperational) {
Error.call(this);
Error.captureStackTrace(this);
this.name = name;
//...other properties assigned here
};
appError.prototype.__proto__ = Error.prototype;
module.exports.appError = appError;
//client throwing an exception
if(user == null)
throw new appError(commonErrors.resourceNotFound, commonHTTPErrors.notFound, "further explanation", true)
14. What other bloggers say
YONI GOLDBERG14 |
From Node.JS official documentation
…All JavaScript and System errors raised by Node.js inherit from, or are instances of, the standard JavaScript Error class and are
guaranteed to provide at least the properties available on that class. A generic JavaScript Error object that does not denote any
specific circumstance of why the error occurred. Error objects capture a “stack trace” detailing the point in the code at which the Error
was instantiated, and may provide a text description of the error. All errors generated by Node.js, including all System and
JavaScript errors, will either be instances of, or inherit from, the Error class…k.
Blog Quote: “All JavaScript and System errors raised by Node.js inherit from Error”
From the blog Ben Nadel, ranked 5 for the keywords “Node.JS error object”
Personally, I don’t see the value in having lots of different types of error objects – JavaScript, as a language, doesn’t seem to
cater to Constructor-based error-catching. As such, differentiating on an object property seems far easier than
differentiating on a Constructor type
Blog Quote: “I don’t see the value in having lots of different types”
15. What other bloggers say
YONI GOLDBERG15 |
From the blog machadogj, ranked 6 for the keywords “Node.JS error management”
…One problem that I have with the Error class is that is not so simple to extend. Of course you can inherit the class and create your
own Error classes like HttpError, DbError, etc. However that takes time, and doesn’t add too much value unless you are doing
something with types. Sometimes, you just want to add a message, and keep the inner error, and sometimes you might want to
extend the error with parameters, and such…
Blog Quote: “Inheriting from Error doesn’t add too much value”
16. Summarizes and quotes 4 sources
Distinguish operational
vs programmer errors
3
1
2
3
4
5
//error handling code within middleware
process.on('uncaughtException',
function(error) {
if(!error.isOperational)
process.exit(1);
17. TL;DR
YONI GOLDBERG
Operational errors (e.g. API received an invalid input) refer to known cases where the error impact is fully
understood and can be handled thoughtfully. On the other hand, programmer error (e.g. trying to read undefined
variable) refers to unknown code failures that dictate to gracefully restart the application
Code example – marking an error as operational (trusted)
1
2
3
//marking an error object as operational
var myError = new Error("How can I add new product when no value provided?");
myError.isOperational = true;
//killing the process only if an error is not trusted (not operational)
//error handling code within middleware
process.on('uncaughtException', function(error) {
if(!error.isOperational)
process.exit(1);
18. Otherwise
YONI GOLDBERG18 |
You may always restart the application when an error appear, but why letting ~5000 online users down because of
a minor, predicted, operational error? the opposite is also not ideal – keeping the application up when unknown
issue (programmer error) occurred might lead to an unpredicted behavior. Differentiating the two allows acting
tactfully and applying a balanced approach based on the given context
From the blog debugable.com, ranked 3 for the keywords “Node.JS uncaught exception”
…So, unless you really know what you are doing, you should perform a graceful restart of your service after receiving an
“uncaughtException” exception event. Otherwise you risk the state of your application, or that of 3rd party libraries to become
inconsistent, leading to all kinds of crazy bugs…
“Otherwise you risk the state of your application”
19. One paragraph explainer
YONI GOLDBERG19 |
Always distinguish between the two: operational errors refer to situations where you understand what happened and the impact of it – for example, a
query to some HTTP service failed due to connection problem. On the other hand, programmer errors refer to cases where you have no idea why and
sometimes where an error came from – it might be some code that tried to read undefined value or a DB connection pool that leaks memory.
Operational errors are relatively easy to handle – usually logging the error is enough. Things become hairy when a programmer errors pop-up, the
application might be in an inconsistent state and there’s nothing better you can do than restart gracefully + analyze quickly the reason for those failures
From the blog Joyent, ranked 1 for the keywords “Node.JS error handling”
…The best way to recover from programmer errors is to crash immediately. You should run your programs using a restarter that will
automatically restart the program in the event of a crash. With a restarter in place, crashing is the fastest way to restore reliable
service in the face of a transient programmer error…
“Programmer errors are bugs in the program”
20. Code Example – marking an error as operational
YONI GOLDBERG20 |
Code example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
//centralized error object that derives from Node’s Error
function (name, httpCode, description, isOperational) {
Error.call(this);
Error.captureStackTrace(this);
this.name = name;
//...other properties assigned here
};
appError.prototype.__proto__ = Error.prototype;
module.exports.appError = appError;
//marking an error object as operational
var myError = new Error(“invalidInput”, 400, How can I add new product when no value provided?“, true);
//error handling code within middleware
process.on('uncaughtException', function(error) {
if(!error.isOperational)
process.exit(1);
});
21. What other bloggers say
YONI GOLDBERG21 |
From Node.JS official documentation
Blog Quote: “No safe way to leave without creating some undefined brittle state”
…By the very nature of how throw works in JavaScript, there is almost never any way to safely “pick up where you left off”,
without leaking references, or creating some other sort of undefined brittle state. The safest way to respond to a thrown error
is to shut down the process. Of course, in a normal web server, you might have many connections open, and it is not reasonable to
abruptly shut those down because an error was triggered by someone else. The better approach is to send an error response to the
request that triggered the error, while letting the others finish in their normal time, and stop listening for new requests in that worker.
From the blog: JS Recipes”
…There are primarily three schools of thoughts on error handling:
1. Let the application crash and restart it.
2. Handle all possible errors and never crash.
3. Balanced approach between the two
Blog Quote: “There are three schools of thoughts on error handling”
22. Summarizes and quotes 4 sources
Handle errors centrally,
through but not within
middleware
4
1
2
3
4
module.exports.handler = new
errorHandler();
function errorHandler(){
this.handleError = function (error) {
23. YONI GOLDBERG
Error handling activities such as mail notification to admin and logging should be encapsulated in a dedicated and
centralized object that all end-points (e.g. Express middleware, cron jobs, lambda functions, unit-testing) call when
an error comes in
Code example – handling errors within a dedicated object
1
2
3
4
5
6
TL;DR
module.exports.handler = function (){
this.handleError = function (error) {
return logger.logError(err).then(sendMailToAdminIfCritical).then(saveInOpsQueueIfCritical).then(determineIfOperationalError);
}
24. Otherwise
YONI GOLDBERG24 |
Handling error within modules or HTTP routes (e.g. Express) will lead to duplicate code and greater chances of
errors that are handled improperly
Code example – Anti Pattern: handling errors within the middleware
1
2
3
4
5
6
7
8
//middleware handling the error directly, where will handle Cron jobs or AMQP subscriber errors?
app.use(function (err, req, res, next) {
logger.logError(err);
if(err.severity == errors.high)
mailer.sendMail(configuration.adminMail, "Critical error occured", err);
if(!err.isOperational)
next(err);
});
25. One paragraph explainer
YONI GOLDBERG25 |
Without one dedicated object for error handling, greater are the chances of important errors hiding under the radar due to improper handling. The
error handler object is responsible for making the error visible, for example by writing to a well-formatted logger, sending events to some monitoring
product or email to admin directly. A typical error flow might be: Some module throws an error -> API router catches the error -> it propagates the
error to the middleware (e.g. Express, KOA) who is responsible for catching errors -> a centralized error handler is called -> the middleware is being
told whether this error is untrusted error (not operational) so it can restart the app gracefully. Note that it’s a common, yet wrong, practice to handle
error within Express middleware – doing so will not cover errors that are thrown in non-web interfaces
From the blog Daily JS, ranked 14 for the keywords “Node.JS error handling”
…You should set useful properties in error objects, but use such properties consistently. And, don’t cross the streams: HTTP errors
have no place in your database code. Or for browser developers, Ajax errors have a place in code that talks to the server,
but not code that processes Mustache templates…
“HTTP errors have no place in your database code”
26. Code Example – a typical Error flow
YONI GOLDBERG26 |
Code example – handling errors within a dedicated object
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
//DAL layer, we don't handle errors here
DB.addDocument(newCustomer, (error, result) => {
if (error)
throw new Error("Great error explanation comes here", other useful parameters)
});
//API route code, we catch both sync and async errors and forward to the middleware
try {
customerService.addNew(req.body).then(function (result) {
res.status(200).json(result);
}).catch((error) => {
next(error)
});
}
catch (error) {
next(error);
}
//Error handling middleware, we delegate the handling to the centralized error handler
app.use(function (err, req, res, next) {
errorHandler.handleError(err).then((isOperationalError) => {
if (!isOperationalError)
next(err);
});
});
27. From the blog Joyent, ranked 1 for the keywords “Node.JS error handling”
…You may end up handling the same error at several levels of the stack. This happens when lower levels can’t do anything
useful except propagate the error to their caller, which propagates the error to its caller, and so on. Often, only the top-level caller
knows what the appropriate response is, whether that’s to retry the operation, report an error to the user, or something else. But that
doesn’t mean you should try to report all errors to a single top-level callback, because that callback itself can’t know in
what context the error occurred…
Blog Quote: “Sometimes lower levels can’t do anything useful except propagate the error
to their caller”
What other bloggers say
YONI GOLDBERG27 |
From the blog JS Recipes, ranked 17 for the keywords “Node.JS error handling”
…In Hackathon Starter api.js controller alone, there are over 79 occurrences of error objects. Handling each err individually would
result in tremendous amount of code duplication. The next best thing you can do is to delegate all error handling logic to an
Express middleware…
“Handling each err individually would result in tremendous duplication”
28. Summarizes and quotes 2 source
Document API errors
using Swagger
5
1
2
3
4
responses:
"405":
description: Validation exception
"404":
description: Pet not found
"400":
29. YONI GOLDBERG
Let your API callers know which errors might come in return so they can handle these thoughtfully without crashing.
This is best done with REST API documentation frameworks like Swagger
TL;DR
An example of API
errors documentation
using swagger
30. Otherwise
YONI GOLDBERG30 |
An API client might decide to crash and restart only because he received back an error he couldn’t understand.
Note: the caller of your API might be you (very typical in a microservices environment)
Code example – acting thoughtfully on a given HTTP error response
1
//Javascript example: treating an error thoughtfully improves the flow and UX
let newPet = {name:”Mike”, age:3};
let serviceURI = `http://myDoamin.com/api/pets/`;
httpRequest({method: 'POST', uri: serviceURI, resolveWithFullResponse: true, body: newPet, json: true}).then((response) => {
//http error 409 = a conflict
if(response.httpCode === 409)
notificationService.showError(“The given pet name already exist, kindly choose a new one?”);
});;
31. One paragraph explainer
YONI GOLDBERG31 |
REST APIs return results using HTTP code, it’s absolutely required for the API user to be aware not only about the API schema but also about
potential errors – the caller may then catch an error and tactfully handle it. For example, your API documentation might state in advanced that HTTP
status 409 is returned when the customer name already exist (assuming the API register new users) so the caller can correspondingly render the
best UX for the given situation. Swagger is a standard that defines the schema of API documentation with eco-system of tools that allow creating
documentation easily online, see prtscn screens below
From the blog Joyent, ranked 1 for the keywords “Node.JS logging”
We’ve talked about how to handle errors, but when you’re writing a new function, how do you deliver errors to the code
that called your function? …If you don’t know what errors can happen or don’t know what they mean, then your
program cannot be correct except by accident. So if you’re writing a new function, you have to tell your callers
what errors can happen and what they mean…
“You have to tell your callers what errors can happen”
33. Summarizes and quotes 4 sources
Shut the process
gracefully when a
stranger comes to town
6
1
2
3
4
process.on('uncaughtException', (error) =>{
//check if the error is safe (operational)
process.exit(1)
});
34. YONI GOLDBERG
When a non-operational error occurs (see best practice number #3) - there is uncertainty about the application
healthiness. A common practice suggests restarting the process carefully using a ‘restarter’ tool like Forever, PM2
or Linux systemd
Code example: deciding whether to crash
TL;DR
1
2
3
4
5
6
7
//deciding whether to crash when an uncaught exception arrives
//Assuming developers mark known operational errors with error.isOperational=true, read best practice #3
process.on('uncaughtException', function(error) {
errorManagement.handler.handleError(error);
if(!error.isOperational)
process.exit(1)
});
35. Otherwise
YONI GOLDBERG35 |
Some developer errors will lead to crazy and unpredicted behavior. For example, consider an event
emitter which is used globally and not firing events anymore due to some internal failure
Code example - swallowing errors is an anti-pattern
1
2
3
4
5
6
7
8
9
10
11
//error happened? let's swallow it and prevent crashes! (don't do that)
process.on('uncaughtException', function(error) {
logger.log(error)
});
36. One paragraph explainer
YONI GOLDBERG36 |
Somewhere within your code, an error handler object is responsible for deciding how to proceed when an error comes in – if the error is trusted (i.e.
operational error, see further explanation within best practice #3) then writing to log file might be enough. Things get hairy if the error is not familiar –
this means that some component might be in a fault state and all future requests are subject to failure. For example, a singleton, stateful token issuer
service that threw an exception and lost its state – from now it might behave unexpectedly and cause all requests to fail. Under this scenario, kill the
process and use a ‘Restarter tool’ (like Forever, PM2, etc) to start with a clean slate.
From the blog Joyent, ranked 1 for the keywords “Node.JS error handling”
…The best way to recover from programmer errors is to crash immediately. You should run your programs using a restarter that will
automatically restart the program in the event of a crash. With a restarter in place, crashing is the fastest way to restore reliable
service in the face of a transient programmer error…
“The best way is to crash”
37. Code Example – deciding when to crash
YONI GOLDBERG37 |
Code example – handling errors within a dedicated object
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
//deciding whether to crash when an uncaught exception arrives
//Assuming developers mark known operational errors with error.isOperational=true, read best practice #3
process.on('uncaughtException', function(error) {
errorManagement.handler.handleError(error);
if(!errorManagement.handler.isTrustedError(error))
process.exit(1)
});
//centralized error handler encapsulates error-handling related logic
function errorHandler(){
this.handleError = function (error) {
return logger.logError(err).then(sendMailToAdminIfCritical).then(saveInOpsQueueIfCritical).then(determineIfOperationalError);
}
this.isTrustedError = function(error)
{
return error.isOperational;
}
38. What other bloggers say
YONI GOLDBERG38 |
From the blog: JS Recipes”
…There are primarily three schools of thoughts on error handling:
1. Let the application crash and restart it.
2. Handle all possible errors and never crash.
3. Balanced approach between the two
Blog Quote: “There are three schools of thoughts on error handling”
From Node.JS official documentation
…By the very nature of how throw works in JavaScript, there is almost never any way to safely “pick up where you left off”, without
leaking references, or creating some other sort of undefined brittle state. The safest way to respond to a thrown error is to shut
down the process. Of course, in a normal web server, you might have many connections open, and it is not reasonable to
abruptly shut those down because an error was triggered by someone else. The better approach is to send an error response
to the request that triggered the error, while letting the others finish in their normal time, and stop listening for new requests in that
worker.
Blog Quote: “No safe way to leave without creating some undefined brittle state”
39. Summarizes and quotes 2 sources
Increase error visibility
using advanced logging
tools
7
1
2
3
4
5
6
7
8
9
//your centralized logger object
var logger = new winston.Logger({
level: 'info',
transports: [
new (winston.transports.Console)(),
new (winston.transports.File)({ filename:
'somefile.log' })
]
});
40. YONI GOLDBERG
A set of tools like mature logging libraries and log aggregators (Winston, Bunyan, ElasticSearch, AWS CloudWatch
etc) will speed-up error discovery and understanding. So forget about console.log.
Code example – Winston Logger in action
TL;DR
1
2
3
4
5
6
7
8
9
//your centralized logger object
var logger = new winston.Logger({
level: 'info',
transports: [
new (winston.transports.Console)(),
new (winston.transports.File)({ filename: 'somefile.log' })
]
});
//custom code somewhere using the logger
logger.log('info', 'Test Log Message with some parameter %s', 'some parameter', { anything: 'This is metadata' });
41. Otherwise
YONI GOLDBERG41 |
Skimming through console.logs or manually hunting an exception within messy text files in ~10 servers
might keep you busy at work until late
Code example – anti pattern, using console.log
//let's degrades performance and visibility (don't do that)
console.log("this log statement is not persistent and can't get
aggregated to a centralized log files repository");;
1
2
3
42. One paragraph explainer
YONI GOLDBERG42 |
We all loovve console.log but obviously a reputable and persisted Logger like Winston, Bunyan or L4JS is mandatory for serious projects. A set of
practices and tools will help to reason about errors much quicker – (1) log frequently using different levels (debug, info, error) (2) when logging,
provide contextual information as JSON objects, see example in next slides. (3) use a log aggregator solution like AWS CloudWatch, ELK, Splunk
(each cloud provider has its own aggregator service) that provide a unified view of all logs. Otherwise you'll have to hunt production bugs by SSH into
multiple servers (4) a dashboard that curates logs and provides insights like which errors happen most, which API endpoints are slower than others
and much more.
From the blog Strong Loop, ranked 1 for the keywords “Node.JS logging”
Lets identify a few requirements (for a logger):
1. Time stamp each log line. This one is pretty self explanatory – you should be able to tell when each log entry occured.
2. Logging format should be easily digestible by humans as well as machines.
3. Allows for multiple configurable destination streams. For example, you might be writing trace logs to one file but when an error is
encountered, write to the same file, then into error file and send an email at the same time…
“Logger Requirements”
43. CloudWatch - AWS service for viewing aggregated logs
YONI GOLDBERG43 |
Watch and filter
log entries that are
aggregated from
all servers using
AWS CloudWatch
service
44. Taking it higher – ELK logs dashboard view
YONI GOLDBERG44 |
A Kibana (part of
ELK) dashboard
that make sense of
logs
45. Code Example – querying log files using a logger library
YONI GOLDBERG45 |
Code example – handling errors within a dedicated object
1
2
3
4
5
6
7
8
var options = {
from: new Date - 24 * 60 * 60 * 1000, until: new Date, limit: 10, start: 0,
order: 'desc', fields: ['message']
};
// Find items logged between today and yesterday.
winston.query(options, function (err, results) {
//callback with results
});
46. Summarizes and quotes 1 source
Test error flows using
your favorite test
framework
8
1
2
3
describe("Facebook chat", () => {
it("Notifies on new chat message", () => {
var chatService = new chatService();
47. YONI GOLDBERG
Whether professional automated QA or plain manual developer testing – Ensure that your code not only satisfies
positive scenario but also handle and return the right errors. Testing framework like Mocha & Chai can handle this
easily (see code examples)
Code example – ensuring the right exception is thrown using Mocha & Chai
TL;DR
describe("Facebook chat", () => {
it("Notifies on new chat message", () => {
var chatService = new chatService();
chatService.participants = getDisconnectedParticipants();
expect(chatService.sendMessage.bind({message: "Hi"})).to.throw(ConnectionError);
});
});
1
2
3
4
5
6
7
48. Otherwise
YONI GOLDBERG48 |
Without testing, whether automatically or manually, you can’t rely on our code to return the right errors.
Without meaningful errors – there’s no error handling
Lightweight testing
option – Postman
(Chrome
extension) allows
testing HTTP API
in few minutes and
even export the
testing as a
console script that
can be included in
your CI process
49. One paragraph explainer
YONI GOLDBERG49 |
Testing ‘happy’ paths is no better than testing failures. Good testing code coverage demands to test exceptional paths. Otherwise, there is no trust
that exceptions are indeed handled correctly. Every unit testing framework, like Mocha & Chai, has a support for exception testing (code examples
below). If you find it tedious to test every inner function and exception – you may settle with testing only REST API HTTP errors.
Code example – ensuring API returns the right HTTP error code
1
2
3
4
5
6
7
8
9
10
it("Creates new Facebook group", function (done) {
var invalidGroupInfo = {};
httpRequest({method: 'POST', uri: "facebook.com/api/groups", resolveWithFullResponse: true, body: invalidGroupInfo, json: true
}).then((response) => {
//oh no if we reached here than no exception was thrown
}).catch(function (response) {
expect(400).to.equal(response.statusCode);
done();
});
});
50. Summarizes and quotes 2 sources
Discover errors and
downtime using APM
products
9
1
2
3
NPM install newrelic
//a single line of code is all what it
takes to benefit a dashboard that analyzes
your app performance
//app.js ->
Require(‘newrelic’);
51. YONI GOLDBERG
Monitoring and performance products (a.k.a APM) proactively gauge your codebase or API so they can auto-
magically highlight errors, crashes and slow parts that you were missing
TL;DR
Read at Wikipedia here
In the fields of information technology and systems management, Application Performance Management (APM) is the monitoring
and management of performance and availability of software applications. APM strives to detect and diagnose complex application
performance problems to maintain an expected level of service. APM is “the translation of IT metrics into business meaning ([i.e.]
value)
“Wikipedia about APM”
52. Otherwise
You might spend great effort on measuring API performance and downtimes, probably you’ll never be
aware which are your slowest code parts under real world scenario and how these affects the UX
Performance
monitoring for
example –
“newrelic”, a
commercial
product, highlights
the worst UX
experience in your
app by measuring
from the end-user
perspective
53. One paragraph explainer
YONI GOLDBERG53 |
Exception != Error. Traditional error handling assumes the existence of Exception but application errors might come in the form of slow code paths, API
downtime, lack of computational resources and more. This is where APM products come handy as they allow with minimal setup to detect a wide variety of
‘burried’ issues proactively. Among the common features of APM products are – alerting when HTTP API returns errors, detect when API response time drops
below some threshold, detection of ‘code smells’, monitor server resources, operational intelligence dashboard with IT metrics and many other useful features.
Most vendors offer a free plan.
“Major products and segments”
APM products constitues 3 major segments:
1. Website or API monitoring – external services that constantly monitor uptime and performance via HTTP requests. Can be setup
in few minutes. Following are few selected contenders: Pingdom, Uptime Robot, and New Relic
2. Code instrumetation – products family which require to embed an agent within the application to benefit feature slow code
detection, exceptions statistics, performance monitoring and many more. Following are few selected contenders: New Relic, App
Dynamics
3. Operational intelligence dashboard – these line of products are focused on fasciliatitating the ops team with metrics and curated
content that helps to easily stay on top of application peroformance. This is usually involves aggregating multiple sources of
information (application logs, DB logs, servers log, etc) and upfront dashboard design work. Following are few selected
contenders: Datadog, Splunk
54. Example: Up time monitoring using UpTimeRobot.Com
YONI GOLDBERG54 |
Up time
monitoring
products
specializes in
detecting
service
accessibility
issues
including high
latency
55. Example: performance monitoring with AppDynamic
YONI GOLDBERG55 |
Performance
monitoring
product takes
an holistic
approach of
gauging the
system
behavior from
multiple
angles
including from
the user’s
device
56. Summarizes and quotes 2 sources
Catch unhandled
promise rejections
10
1
2
3
4
5
6
DAL.getUserById(1).then((johnSnow) =>
{
//this error will just vanish!
if(johnSnow.isAlive == false)
throw new Error('ahhhh');
});
57. Anti pattern code example – Catching unresolved and rejected promises
1
2
3
4
5
6
7
8
9
10
process.on('unhandledRejection', function (reason, p) {
//I just caught an unhandled promise rejection, since we already have fallback handler for unhandled errors (see below), let throw and let him
handle that
throw reason;
});
process.on('uncaughtException', function (error) {
//I just received an error that was never handled, time to handle it and then decide whether a restart is needed
errorManagement.handler.handleError(error);
if (!errorManagement.handler.isTrustedError(error))
process.exit(1);
});
TL;DR
YONI GOLDBERG57 |
Any exception thrown within a promise will get swallowed and discarded unless a developer didn’t forget to explictly
handle. Even if you’re code is subscribed to process.uncaughtException! Overcome this by registering to the event
process.unhandledRejection
58. Otherwise*
58 |
Your errors will get swallowed and leave no trace. Nothing to worry about
Code example - Shockingly, these errors will leave no trace
1
2
3
4
5
6
DAL.getUserById(1).then((johnSnow) =>
{
//this error will just vanish
if(johnSnow.isAlive == false)
throw new Error('ahhhh');
});
*Update: As of Node 6.6, this behavior was partially improved and uncaught promises will get logged to the console. Though
this increases the chances of discovering errors, still you’re error handling code won’t have the chance of treating this error
like any other. Consequently, this practice is still valid and important
59. One paragraph explainer
YONI GOLDBERG59 |
Typically, most of modern Node.JS/Express application code runs within promises – whether within the .then handler, a function callback or in a
catch block. Surprisingly, unless a developer remembered to add a .catch clause, errors thrown at these places disappear without leaving any
trace(!). They will not get caught even by app.uncaughtException. The straightforward solution is to never forget adding .catch clause within each
promise chain call and redirect to a centralized error handler. However building your error handling strategy only on developer’s discipline is
somewhat fragile. Consequently, it’s highly recommended using a graceful fallback and subscribe to process.on(‘unhandledRejection’, callback) –
this will ensure that any promise error, if not handled locally, will get its treatment.
From the blog James Nelson
Let’s test your understanding. Which of the following would you expect to print an error to the console?
Promise.resolve(‘promised value’).then(function() {
throw new Error(‘error’);
});
Promise.reject(‘error value’).catch(function() {
throw new Error(‘error’);
});
The problem with being human is that if you can make a mistake, at some point you will. Keeping this in mind, it seems obvious that we should design
things in such a way that mistakes hurt as little as possible, and that means handling errors by default, not discarding them
“If you can make a mistake, at some point you will”
60. Summarizes and quotes 2 sources
Fail fast, validate
arguments using a
dedicated library
11
1
2
3
4
5
6
7
8
var memberSchema = Joi.object().keys({
password: Joi.string().regex(/^[a-zA-
Z0-9]{3,30}$/),
birthyear:
Joi.number().integer().min(1900).max(2
013)};
Joi.validate(newMember, memberSchema)
61. YONI GOLDBERG
This should be part of your endpoint best practices (Express, hapi, KOA) – Assert API input to avoid nasty bugs that
are much harder to track later. Validation code is usually tedious unless using a very cool helper libraries like Joi
Code example – validating complex JSON input using ‘Joi’
TL;DR
//Using Joi, a very popular NPM package, to define input schema and validate it
var memberSchema = Joi.object().keys({
password: Joi.string().regex(/^[a-zA-Z0-9]{3,30}$/),
birthyear: Joi.number().integer().min(1900).max(2013),
email: Joi.string().email()
});
function addNewMember(newMember)
{
//assertions comes first
if(Joi.validate(newMember), memberSchema, (err, value) => throw Error("Invalid input));
//other logic here
}
1
2
3
4
5
6
7
8
9
10
11
62. Otherwise
YONI GOLDBERG62 |
Consider this – your function expects a numeric argument “Discount” which the caller forgets to pass, later
on your code checks if Discount!=0 (amount of allowed discount is greater than zero), then it will allow the
user to enjoy a discount. Can you see the nasty bug hiding between the lines?
Anti-pattern: no validation yields nasty bugs
//if the discount is positive let's then redirect the user to print his discount coupons
function redirectToPrintDiscount(httpResponse, member, discount)
{
if(discount != 0)
httpResponse.redirect(`/discountPrintView/${member.id}`);
}
redirectToPrintDiscount(httpResponse, someMember);
//forgot to pass the parameter discount, why the heck was the user redirected to the discount screen?
1
2
3
4
5
6
7
8
9
63. One paragraph explainer
YONI GOLDBERG63 |
We all know how checking arguments and failing fast is important to avoid hidden bug. If not, read about explicit programming and defensive
programming. In reality, we tend to avoid it due to the annoyance of coding it (e.g. think of validating hierarchical JSON object with fields like email
and dates) – libraries like Joi and Validator turns this tedious task into a breeze.
From the blog: Joyent, ranked #1 in Google keywords “Node.JS error handling”
A degenerate case is where someone calls an asynchronous function but doesn’t pass a callback. You should throw these errors
immediately, since the program is broken and the best chance of debugging it involves getting at least a stack trace and ideally a
core file at the point of the error. To do this, we recommend validating the types of all arguments at the start of the function.
“You should throw these errors immediately”
64. Wikipedia: Defensive Programming
YONI GOLDBERG64 |
Read at Wikipedia
Defensive programming is an approach to improve software and source code, in terms of: General quality – reducing the number of
software bugs and problems. Making the source code comprehensible – the source code should be readable and understandable so
it is approved in a code audit. Making the software behave in a predictable manner despite unexpected inputs or user
actions.
“Wikipedia: Defensive Programming”
65. This best practice is obsolete now as not domains are
officially deprecated. It’s not recommended to use
domain for any scenario
[Deprecated] Use
Node.jS domain to
isolate errors
12
66. Thanks For Reading
See many others Node.JS best practices at my Twitter @nodepractices account or
my Facebook page at facebook.com/nodepractices
Be my guest at: www.goldbergyoni.com
Seek Node.JS consultation or training? I’m here for you at me@goldbergyoni.com