• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Defensive programming in Javascript and Node.js
 

Defensive programming in Javascript and Node.js

on

  • 2,356 views

Various factors to consider when trying to adopt a defensive programming mindset, methodology and process. This is especially useful for teams working with node.js.

Various factors to consider when trying to adopt a defensive programming mindset, methodology and process. This is especially useful for teams working with node.js.

Statistics

Views

Total Views
2,356
Views on SlideShare
2,354
Embed Views
2

Actions

Likes
8
Downloads
21
Comments
1

1 Embed 2

https://twitter.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial LicenseCC Attribution-NonCommercial License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Defensive programming in Javascript and Node.js Defensive programming in Javascript and Node.js Presentation Transcript

    • Defensive Programmingin Javascript & node.jsWednesday, May 29, 13
    • INTRODUCTIONRuben Tan Long Zheng 陈龙正VP of Engineering, OnApp CDN KLLead Engineer, 40 Square Sdn BhdJavascript > 5 years@roguejsOrganizer of Nodehack KLWednesday, May 29, 13
    • OVERVIEWDependency AwarenessJavascript MasteryMethodology ImprovementsWednesday, May 29, 13
    • SEASON 1DEPENDENCYAWARENESSWednesday, May 29, 13
    • Internal dependenciesLibrariesrequire(), include()External dependenciesServices, files, databases, etcsocket.connect(), db.open()DEPENDENCY TYPESWednesday, May 29, 13
    • NEVER ASSUME!Never assume a dependency is reliable!var db = require(‘database’);db.open();db.write(‘foo bar’, function (err, data) {// ... do something ...});Wednesday, May 29, 13
    • NEVER ASSUME!var db = require(‘database’);db.open();db.write(‘foo bar’, function (err, data) {// ... do something ...});What if this failed?will write() throw an error? will open() throwan exception?Wednesday, May 29, 13
    • NEVER ASSUME!var db = require(‘database’);db.open(function (err) {db.write(‘mr-big’, bigData, function (err, data) {// ... unrelated logicdb.close();});db.read(‘foo2’, function (err, data) {// ... some work done});});Accidents happen...Wednesday, May 29, 13
    • NEVER ASSUME!var db = require(‘database’);db.open(function (err) {db.write(‘mr-big’, bigData, function (err, data) {// ... unrelated logicdb.close();});db.read(‘foo2’, function (err, data) {// ... some work done});});close() might affect read()Wednesday, May 29, 13
    • A MORE COMPLEX EXAMPLE...Wednesday, May 29, 13
    • VIDEO STREAMING SERVICEVideoStreamerOriginStatsLoggerVODClientUserAccountingUploadLiveClientUserStreamStreamLogReportRenderRenderWednesday, May 29, 13
    • VIDEO STREAMING SERVICEVideoStreamerOriginStatsLoggerVODClientUserAccountingUploadLiveClientUserStreamStreamLogReportRenderRender1234567891011Wednesday, May 29, 13
    • DEPENDENCY AWARENESSWhat can fail, WILL FAIL!Never assume a dependency is reliable!Contingency plans - failover, redundancy, fail-fast, etcPro-active monitoringLoad test, stress test, chaos monkey, etcRemember, what can fail, WILL FAIL!Wednesday, May 29, 13
    • SEASON 2JAVASCRIPT MASTERYWednesday, May 29, 13
    • JAVASCRIPT MASTERYCode Execution OrderSanitization & ValidationScopeControl FlowWednesday, May 29, 13
    • I KNOW CODE-FU!Wednesday, May 29, 13
    • EXECUTION ORDERvar mq = require(‘mq’);mq.conn(...);mq.on(‘ready’, function () {mq.send(‘batman’);mq.on(‘message’, function (msg) {console.log(msg);mq.close();});});mq is never closed!send() executes before on()Wednesday, May 29, 13
    • DOIN’ IT RIGHT!var mq = require(‘mq’);mq.conn(...);mq.on(‘ready’, function () {mq.on(‘message’, function (msg) {console.log(msg);mq.close();});mq.send(‘batman’);});Swap placesWednesday, May 29, 13
    • SANITIZATION & VALIDATIONfunction foodForKittens(num) {return num * 10;}foodForKittens();num is not validated, is undefinedthis will fail!Wednesday, May 29, 13
    • TOO SIMPLE?Wednesday, May 29, 13
    • SANITIZATION & VALIDATIONvar db = require(‘database’);var conn = db.open(...);function writeToDb(conn, cb) {conn.write(bigData, function (err, res) {if (err) {cb(err);return;}cb(null, res);});});writeToDb(conn, ghostCallback);Wednesday, May 29, 13
    • Wednesday, May 29, 13
    • var db = require(‘database’);var conn = db.open(...);function writeToDb(conn, cb) {conn.write(bigData, function (err, res) {if (err) {cb(err);return;}cb(null, res);});});writeToDb(conn, ghostCallback);what if open() returned undefined?this will throw an exception!Wednesday, May 29, 13
    • var db = require(‘database’);var conn = db.open(...);function writeToDb(conn, cb) {conn.write(bigData, function (err, res) {if (err) {cb(err);return;}cb(null, res);});});writeToDb(conn, ghostCallback);What if ghostCallback is undefined?These will fail too!Wednesday, May 29, 13
    • DOIN’ IT RIGHT!var db = require(‘database’);var conn = db.open(...);function writeToDb(conn, cb) {if (typeof conn !== ‘object’) {// ... handle error ...}if (typeof cb !== ‘function’) {// ... handle error ...}conn.write(bigData, function (err, res) {if (err) {cb(err);return;}cb(null, res);});});writeToDb(conn, ghostCallback);Validate your input,especially when theyinvolve functions ormethods that you need toinvoke in your code.These are not the time tofail-fast!Wednesday, May 29, 13
    • DON’T GO OVERBOARD...Validate only necessary parametersMethod invocations (anObject.method())Function invocations (aFunction())Have a proper error/exception handling policyValidate for correctness, not existenceCorrectness: typeof a === ‘object’Existence: a !== undefinedWednesday, May 29, 13
    • SCOPE AWARENESSPlagues most callback-based codeBad practice leads to costly debugging wasteNew JS programmers not aware of scopingJS scoping is a simple but weird thing (to non-JSprogrammers)Wednesday, May 29, 13
    • SCOPE!!!var a = ‘outside’;if (true) {var a = ‘inside’;console.log(a);}console.log(a);What is the output?> node test.jsinsideinsideWednesday, May 29, 13
    • SCOPE!!!Non-JS programmers:a inside the if block is “inside”a outside the if block is “outside”JS programmers:they are both “inside”JS scope by functionWednesday, May 29, 13
    • SCOPE CHAINS!!!var avar = 1;(function outer1() {var avar = 2;(function inner1() {var avar = 3;console.log(avar); // outputs 3})();(function inner2() {console.log(avar); // outputs 2})();})();(function outer2() {(function inner3() {console.log(avar); // outputs 1})();})();inner1()local - found!inner2()local - nopeouter1() - found!inner3()local - nopeouter2() - nopeglobal - found!Wednesday, May 29, 13
    • HOISTING VARIABLESfunction () {for (var i = 0; i < 10; i++) {for (var j = 0; j < 10; j++) {// ... do something}}}function () {var i, j; // now the scope is clear for i & jfor (i = 0; i < 10; i++) {for (j = 0; j < 10; j++) {// ... do something}}}Below is far clearer what individual variable scopes are:Wednesday, May 29, 13
    • CONTROL FLOWNode.js’ async nature makes it unintuitive to predictcontrol flowI <3 async (github.com/caolan/async)Control flow is ugly. Welcome to Javascript.Async will save your life. Use it.Wednesday, May 29, 13
    • CONTROL FLOWvar fs;fs = require(‘fs’);fs.readFile(‘./myfile.txt’, function (err, data) {if (err) {console.log(err);return;}fs.writeFile(‘./myfile2.txt’, data, function (err) {if (err) {console.log(err);return;}// ... do stuff ...});})Wednesday, May 29, 13
    • CONTROL FLOWCallback hell!Step 1Step 2Step 3Step 4Step 5Wednesday, May 29, 13
    • mod.step1(function () {mod.step2(function () {mod.step3(function () {mod.step4(function () {mod.step5(function () {// ... too many levels ...});});}});});Wednesday, May 29, 13
    • CONTROL FLOWvar async, fs;async = require(‘async’);fs = require(‘fs’);async.waterfall([function step1(callback) {fs.readFile(‘./myfile.txt’, callback);},function step2(data, callback) {fs.writeFile(‘./myfile2.txt’, data, callback);}], function (err) {// ... execute something in the end ...});Wednesday, May 29, 13
    • SEASON 3METHODOLOGYIMPROVEMENTSWednesday, May 29, 13
    • GOLDEN RULESGolden Rules of Defensive ProgrammingProper error handling policyIntelligent loggingDesign for failureWednesday, May 29, 13
    • ERROR HANDLINGNever, ever HIDE errors> node app.js 2>&1 /dev/nullob.callback(function (err, data) {if (err) {}console.log(data);});socket.on(‘error’, function () {});Wednesday, May 29, 13
    • ERROR HANDLINGI WILL FIND YOUAND I WILL CRASH YOUWednesday, May 29, 13
    • ERROR HANDLINGStandardize error handling in the appLog to error DBOutput to error fileOutput error to a streamUse a logging libraryAsk a leprechaun to manage itetcWednesday, May 29, 13
    • LOGGINGHow do you feel if your “log” looks like this?> tail -f error.log[12:01:55] ERROR - General error detected[12:01:56] ERROR - General error detected[12:01:57] ERROR - General error detected[12:01:58] ERROR - General error detected[12:01:59] ERROR - General error detected[12:02:00] ERROR - General error detected[12:02:01] ERROR - General error detectedWednesday, May 29, 13
    • LOGGINGWednesday, May 29, 13
    • LOGGINGLogs are the first place you go to find out whathappenedStandardize a log location for each appMake logs easy to access for developersWednesday, May 29, 13
    • DESIGN FOR FAILURECommon steps to designing software:1 - what should it do?2 - how do I do it?3 - how do I deploy?4 - doneWednesday, May 29, 13
    • DESIGN FOR FAILUREProper steps in defensive programming:1 - what should it do?2 - how many ways can it fail?3 - how do I know when it fails?4 - how do I prevent it from failing?5 - write code accordinglyWednesday, May 29, 13
    • DESIGN FOR FAILURENothing is reliableTCP can failNetwork can go downServers can run out of memoryCows might fly through the sky crashing into yourdatacenter and flooding the server rooms with milkand destroying everythingWednesday, May 29, 13
    • DESIGN FOR FAILUREDesigning for failure mindset & methodologies:Identify SPOF (single point of failures)Redundancy, failover, monitoringFail-fast, start-fastPersist important dataReliability & Consistency > SpeedCode is liabilityWednesday, May 29, 13
    • ~ The End ~Wednesday, May 29, 13