NodeJS: the good parts? A skeptic’s view (devnexus2014)


Published on

JavaScript used to be confined to the browser. But these days, it becoming increasingly popular in server-side applications in the form of NodeJS. NodeJS provides event-driven, non-blocking I/O model that supposedly makes it easy to build scalable network application.

In this talk you will learn about the consequences of combining the event-driven programming model with a prototype-based, weakly typed, dynamic language. We will share our perspective as a server-side Java developer who wasn’t entirely happy about JavaScript in the browser, let alone on the server. You will learn how to use NodeJS effectively in modern, polyglot applications.

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

NodeJS: the good parts? A skeptic’s view (devnexus2014)

  1. 1. NodeJS: the good parts? A skeptic’s view Chris Richardson Author of POJOs in Action Founder of the original @crichardson @crichardson
  2. 2. Presentation goal How a grumpy, gray-haired server-side Java developer discovered an appreciation for NodeJS and JavaScript @crichardson
  4. 4. About Chris LispWorks 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 Z80 RPG 3 BCPL Pascal 6502 C Assembler Basic @crichardson
  5. 5. 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 C++ EJB @crichardson
  6. 6. 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 @crichardson
  7. 7. About Chris ? 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 @crichardson
  8. 8. Agenda Overview of NodeJS JavaScript: Warts and all The Reactor pattern: an event-driven architecture NodeJS: There is a module for that Building a front-end server with NodeJS @crichardson
  9. 9. What’s NodeJS? Designed for DIRTy apps @crichardson
  10. 10. Growing rapidly Busy! @crichardson
  11. 11. But it’s not everywhere... Primarily “edge” services: web apps, etc @crichardson
  12. 12. NodeJS Hello World app.js Load a module request handler $ node app.js $ curl http://localhost:1337 No complex configuration: simple! @crichardson
  13. 13. JavaScript NodeJS Modules Reactor pattern @crichardson
  14. 14. JavaScript NodeJS Modules Reactor pattern @crichardson
  15. 15. Dynamic and weakly-typed Dynamic: Types are associated with values - not variables Define new program elements at runtime Weakly typed: Leave out arguments to methods Read non-existent object properties Add new properties by simply setting them @crichardson
  16. 16. JavaScript is object-oriented > var fred = {name: “Fred”, gender: “Male”}; undefined > “Fred” > console.log("reading age=" + fred.age); reading age=undefined undefined > fred.age = 99; 99 > fred { name: 'Fred', gender: 'Male', age: 99 } > delete fred.age true > fred { name: 'Fred', gender: 'Male' } Unordered key-value pairs Keys = properties Add property Delete property @crichardson
  17. 17. JavaScript is a prototypal language Person __proto__ sayHello ... Prototype function ... inherited Chris __proto__ name nickname “Chris” “CER” overrides object specific @crichardson
  18. 18. Prototypal code Not defined here $ node > var person = { sayHello: function () { console.log("Hello " +; }}; [Function] > var chris = Object.create(person, {name: {value: "Chris"}}); undefined > var sarah = Object.create(person, {name: {value: "Sarah"}}); undefined > chris.sayHello(); Hello Chris create using undefined properties > sarah.sayHello(); prototype Hello Sarah undefined > chris.sayHello = function () { console.log("Hello mate: " +; }; [Function] > chris.sayHello(); Hello mate: Chris undefined @crichardson
  19. 19. JavaScript is Functional function makeGenerator(nextFunction) { Return a function closure var value = 0; return function() { var current = value; value = nextFunction(value); return current; }; Pass function (literal) as an argument } var inc = makeGenerator(function (x) {return x + 1; }); > inc() 0 > inc() 1 @crichardson
  20. 20. But JavaScript was created in a hurry @crichardson
  21. 21. Lots of flaws The ‘Java...’ name creates expectations that it can’t satisfy Fake classes: Hides prototypes BUT still seems weird global namespace scope of vars is confusing Missing return statement = confusion ‘function’ is really verbose ‘this’ is dynamically scoped Unexpected implicit conversions: 99 == “99”! truthy and falsy values 52-bit ints @crichardson
  22. 22. Dynamic + weakly-typed (+ event-driven) code + misspelt property names lots of time spent in the abyss Essential: Use IDE integrated with JSLint/JSHint + tests
  23. 23. Dynamic + weakly-typed code Refactoring is more difficult Understanding code/APIs is more difficult
  24. 24. Prototypal languages have benefits BUT Developers really like classes JavaScript prototypes lack the powerful features from the Self language e.g. Multiple (and dynamic) inheritance @crichardson
  25. 25. Verbose function syntax > var numbers = [1,2,3,4,5] > numbers.filter(function (n) { return n % 2 == 0; } ).map(function (n) { return n * n; }) [ 4, 16 ] > Versus Prelude> let numbers = [1,2,3,4,5] Prelude> map (n -> n * n) (filter (n -> mod n 2 == 0) numbers) [4,16] Or scala> val numbers = 1..5 scala> numbers filter { _ % 2 == 0} map { n => n * n } Vector(4, 16) @crichardson
  26. 26. Verbose DSLs describe('SomeEntity', function () { beforeEach(function () { ... some initialization ... }); Jasmine it('should do something', function () { ... expect(someExpression).toBe(someValue); }); }); Versus class SomeScalaTest ...{ Scalatest before { ... some initialization ... } it should "do something" in { ... someExpression should be(someValue) } @crichardson
  27. 27. JavaScript is the language of the web “You have to use the programming language you have, not the one that you might want” @crichardson
  28. 28. It works but the result is lost opportunities and impeded progress @crichardson
  29. 29. But if you think that this isn’t a problem then perhaps .... “Stockholm syndrome ... is a psychological phenomenon in which hostages ... have positive feelings toward their captors, sometimes to the point of defending them...” @crichardson
  30. 30. Martin Fowler once said: "...I'm one of those who despairs that a language with such deep flaws plays such an important role in computation. Still the consequence of this is that we must take javascript seriously as a first-class language and concentrate on how to limit the damage its flaws cause. ...." @crichardson
  31. 31. Use just the good parts Douglas Crockford @crichardson
  32. 32. Use a language that compiles to JavaScript TypeScript Classes and interfaces (dynamic structural typing) Typed parameters and fields Dart Class-based OO Optional static typing Bidirectional binding with DOM elements Less backwards compatibility with JavaScript Also has it’s own VM @crichardson
  33. 33. CoffeeScript Hello World Classes :-) http = require('http') Concise class HttpRequestHandler constructor: (@message) -> Bound method handle: (req, res) => res.writeHead(200, {'Content-Type': 'text/plain'}) res.end(@message + 'n') handler = new HttpRequestHandler "Hi There from CoffeeScript" server = http.createServer(handler.handle) server.listen(1338, '') console.log('Server running at') @crichardson
  34. 34. No escaping JavaScript @crichardson
  35. 35. JavaScript NodeJS Modules Reactor pattern @crichardson
  36. 36. About the Reactor pattern Defined by Doug Schmidt in 1995 Pattern for writing scalable servers Alternative to thread-per-connection model Single threaded event loop dispatches events on handles (e.g. sockets, file descriptors) to event handlers @crichardson
  37. 37. Reactor pattern structure Application register_handler(h1) register_handler(h2) handle_events() Initiation Dispatcher handle_events() handlers register_handler(h) uses select(handlers) for each h in handlers h.handle_event(type) end loop Event Handler handle_event(type) get_handle() owns Synchronous Event Demultiplexer select() notifies handle @crichardson
  38. 38. Benefits Separation of concerns - event handlers separated from low-level mechanism More efficient - no thread context switching Simplified concurrency - single threaded = no possibility of concurrent access to shared state @crichardson
  39. 39. Drawbacks Non-pre-emptive - handlers must not take a long time Difficult to understand and debug: Inverted flow of control Can’t single step through code easily Limited stack traces No stack-based context, e.g. thread locals, exception handlers How to enforce try {} finally {} behavior? @crichardson
  40. 40. NodeJS app = layers of event handlers Recurring events from Event Emitters Application code Event listener HTTP Callback function DB driver ... One time events: async operation completion Basic networking/file-system/etc. NodeJS event loop @crichardson
  41. 41. Async code = callback hell Difficult to implement common scenarios: Sequential: A B C Scatter/Gather: A and B C Code quickly becomes very messy @crichardson
  42. 42. Messy callback-based scatter/gather code The result of getProductDetails getProductDetails = (productId, callback) -> productId = req.params.productId result = {productId: productId} Propagate error makeCallbackFor = (key) -> (error, x) -> if error callback(error) else result[key] = x if (result.productInfo and result.recommendations and callback(undefined, result) Update result Gather getProductInfo(productId, makeCallbackFor('productInfo')) getRecommendations(productId, makeCallbackFor('recommendations')) getReviews(makeCallbackFor('reviews')) Scatter @crichardson
  43. 43. Simplifying code with Promises (a.k.a. Futures) Functions return a promise - no callback parameter A promise represents an eventual outcome Use a library of functions for transforming and composing promises Promises/A+ specification - when.js (part of cujo.js by SpringSource) is a popular implementation Crockford’s RQ library is another option @crichardson
  44. 44. Simpler promise-based code class ProductDetailsService getProductDetails: (productId) -> makeProductDetails = (productInfo, recommendations, reviews) -> productId: productId productDetails: productInfo.entity recommendations: recommendations.entity reviews: reviews.entity responses = [getProductInfo(productId), getRecommendations(productId), getReviews(productId)] getReviews(productId)] all(responses).spread(makeProductDetails) all(responses) spread(makeProductDetails) @crichardson
  45. 45. Not bad but lacks Scala’s syntactic sugar class ProductDetailsService .... { def getProductDetails(productId: Long) = { for (((productInfo, recommendations), reviews) <getProductInfo(productId) zip getProductInfo(productId) zip getRecommendations(productId) zip getRecommendations(productId) zip getReviews(productId) getReviews(productId)) yield ProductDetails(productInfo, recommendations, reviews) } } @crichardson
  46. 46. Long running computations Long running computation blocks event loop for other requests Need to run outside of main event loop Options: Community: web workers threads Built-in: NodeJS child processes @crichardson
  47. 47. Using child processes parent.js var child = require('child_process').fork('child.js'); function sayHelloToChild() { child.send({hello: "child"}); } Create child process Send message to child setTimeout(sayHelloToChild, 1000); child.on('message', function(m) { console.log('parent received:', m); }); function kill() { child.kill(); } setTimeout(kill, 2000); child.js process.on('message', function (m) { console.log("child received message=", m); process.send({ihateyou: "you ruined my life"}) }); @crichardson
  48. 48. Modern multi-core machines vs. single-threaded runtime Many components of many applications Don’t need the scalability of the Reactor pattern Request-level thread-based parallelism works fine There are other concurrency options Actors, Software transactional memory, ... Go goroutines, Erlang processes, ... Imposing a single-threaded complexity tax on the entire application is questionable @crichardson
  49. 49. JavaScript NodeJS Modules Reactor pattern @crichardson
  50. 50. Core built-in modules Basic networking HTTP(S) Filesystem Events Timers ...
  51. 51. Thousands of community developed modules web frameworks, SQL/NoSQL database drivers, messaging, utilities... @crichardson
  52. 52. What’s a module? foo.js One or more JavaScript files exports.sayHello = function () { console.log(“Hello”); } Optional native code: Compiled during installation JavaScript != systems programming language Package.json - metadata including dependencies @crichardson
  53. 53. Easy to install $ npm install package-name --save @crichardson
  54. 54. Easy to use Core module OR Path to file OR module in node_modules Module’s exports var http = require(“http”) var server = http.createServer... @crichardson
  55. 55. Developing with NodeJS modules Application code Your modules Community modules Core modules @crichardson
  56. 56. There is a module for that... Modules + glue code = rapid/easy application development AWESOME!... @crichardson
  57. 57. ... BUT Variable quality Multiple incomplete/competing modules, e.g. MySQL drivers without connection pooling! Often abandoned No notion of a Maven-style local repository/cache: repeated downloads for all transitive dependencies! Modules sometimes use native code that must be compiled ... @crichardson
  58. 58. To summarize Flawed and misunderstood JavaScript Rich but variable quality NodeJS Modules Scalable yet costly and often unnecessary Reactor pattern @crichardson
  59. 59. How will future history view NodeJS? C++ ? EJB @crichardson
  60. 60. Agenda Overview of NodeJS JavaScript: Warts and all The Reactor pattern: an event-driven architecture NodeJS: There is a module for that Building a front-end server with NodeJS @crichardson
  61. 61. So why care about NodeJS? Easy to write scalable network services Easy to push events to the browser Easy to get (small) stuff done It has a role to play in modern application architecture @crichardson
  62. 62. Evolving from a monolithic architecture.... WAR StoreFrontUI Product Info Service Recommendation Service Review Service Order Service @crichardson
  63. 63. ... to a micro-service architecture product info application Product Info Service recommendations application Store front application StoreFrontUI Recommendation Service reviews application Review Service orders application Order Service @crichardson
  64. 64. Presentation layer evolution.... WAR StoreFrontUI HTML / HTTP View Controller Browser + JavaScript Model @crichardson
  65. 65. ...Presentation layer evolution Browser View Web application Static content Controller JSON-REST Model HTML 5/JavaScript IOS/Android clients Events RESTful Endpoints Event publisher @crichardson
  66. 66. Directly connecting the front-end to the backend Chatty API View REST Product Info service REST Recommendation Service AMQP Controller Review service Model Traditional web application View Controller Model Browser/Native App Web unfriendly protocols @crichardson
  67. 67. NodeJS as an API gateway Single entry point Browser View Controller NodeJS Model HTML 5 - JavaScript Product Info service REST Recommendation Service AMQP Review service REST proxy Native App View API Gateway REST Controller Model Event publishing Optimized Client specific APIs Protocol translation @crichardson
  68. 68. Alternatively: Server-side MVC with micro-web apps Small, cohesive, independently deployable web apps Desktop /mobile Browser Product Catalog web app Account mgmt web app Order mgmt web app REST Product Info service REST Recommendation Service AMQP Review service Small footprint, nodeJS is ideal for micro-web apps @crichardson
  69. 69. Serving static content with the Express web framework var express = require('express') , http = require('http') , app = express() , server = http.createServer(app) ; From public sub directory app.configure(function(){ ... app.use(express.static(__dirname + '/public')); }); server.listen(8081); @crichardson
  70. 70. Templating with Dust.js http://localhost:8081/hello?title=DevNexus+2014 app.get('/hello', function (req, res) { res.render('hello', { title: (req.query.title || 'DevNexus') } ); }); <html> <body> <h1>Hello {title}</h1> </body> </html> Browser + Server templating! @crichardson
  71. 71. RESTful web services @crichardson
  72. 72. Proxying to backend server express = require('express') request = require('request') Returns a request handler that proxies to baseUrl app = express.createServer() proxyToBackend = (baseUrl) -> (req, res) -> callback = (error, response, body) -> console.log("error=", error) originRequest = request(baseUrl + req.url, callback) req.pipe(originRequest) originRequest.pipe(res) app.get('/productinfo/*', proxyToBackend('http://productinfo....')) app.get('/recommendations/*', proxyToBackend(''http://recommendations...')) app.get('/reviews/*', proxyToBackend('http://reviews...')) @crichardson
  73. 73. Implementing coarsegrained mobile API var express = require('express'), ...; app.get('/productdetails/:productId', function (req, res) { getProductDetails(req.params. productId).then( function (productDetails) { res.json(productDetails); } }); @crichardson
  74. 74. Delivering events to the browser @crichardson
  75. 75. server-side var express = require('express') , http = require('http') , amqp = require(‘amqp’) ....; server.listen(8081); ... var amqpCon = amqp.createConnection(...); Handle connection io.sockets.on('connection', function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { Republish var m = JSON.parse(; socket.emit(‘tick’, m); as }; event amqpCon.queue(“”, {}, function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); Subscribe to }); }); AMQP queue @crichardson
  76. 76. - client side <html> <body> The event is <span data-bind="text: ticker"></span> <script src="/"></script> <script src="/knockout-2.0.0.js"></script> <script src="/clock.js"></script> </body> </html> Bind to model Connect to server clock.js var socket = io.connect(location.hostname); function ClockModel() { self.ticker = ko.observable(1); socket.on('tick', function (data) { self.ticker(data); }); }; ko.applyBindings(new ClockModel()); Subscribe to tick event Update model @crichardson
  77. 77. NodeJS is also great for writing backend micro-services “Network elements” Simply ‘route, filter and transform packets’ Have minimal business logic @crichardson
  78. 78. NodeJS-powered home security FTP Server Log file FTP Server Upload directory Upload2S3 UploadQueue Processor S3 SQS Queue IpCamViewer Web App DynamoDB @crichardson
  79. 79. Summary JavaScript is a very flawed language The single-threaded, asynchronous model is often unnecessary; very constraining; and adds complexity BUT despite those problems Today, NodeJS is remarkably useful for building networkfocussed components @crichardson
  80. 80. @crichardson Questions? @crichardson