Matthew Eernisse gave a presentation on NodeJS at the Toster Conference in 2011. He discussed what NodeJS is, how it uses asynchronous and event-driven JavaScript, and some examples of using NodeJS at Yammer for applications like a file upload service and real-time document editing. He also covered challenges like debugging asynchronous code, developing for NodeJS, and operations considerations like monitoring and logging.

1. Intro to NodeJS
Matthew Eernisse
Toster Conference 2011-10-28
2001

2. Who am I?
Matthew Eernisse
Work at Yammer
@mde on Twitter

3. JavaScript at Yammer
• Browsers (yammer.com Web UI)
• Adobe AIR Desktop
• V8 in Rails via TheRubyRacer
• NodeJS

4. NodeJS:
“Evented I/O for V8 JavaScript”
http://nodejs.org/

5. Hello, NodeJS
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200,
{'Content-Type': 'text/plain'});
res.end('Hello Worldn');
}).listen(1337, "127.0.0.1");
console.log(
'Server running at http://127.0.0.1:1337/');

6. Server JS

7. •
History of SSJS
Netscape Enterprise Server (OG SSJS)
• Microsoft IIS
• Helma (now RingoJS)
• Whitebeam
• Zimki
• Jaxer
• Perservere
• Nitro
• Google App Engine
• CouchDB
• NodeJS

8. Why NodeJS now?
• Steve Yegge’s NBL post, 2007-02-10
• Competition in JavaScript
interpreters
• Simple, POSIX API
• Non-blocking from the ground up; so
are the libraries

9. What is NodeJS good for?
• Lightweight, networked apps
• Proxies with embedded logic
• Streaming data
• System scripting
• Evented realtime apps

10. NodeJS is not good at complex
database-backed Web
applications.
You can use Rails.

11. Geddy Web framework:
https://github.com/mde/geddy

12. • NodeJS at Yammer
Development proxy
• Jake for build and test
(https://github.com/mde/jake)
• Upload service for files and images (Geddy v0.2)
• Browserless tests with FooUnit
(https://github.com/foobarfighter/foounit)
• Realtime, collaborative document-editing service

13. Jake build tool
• https://github.com/mde/jake
• Similar to Make or Rake
• Tasks, prerequisites
• File tasks, directory tasks
• Namespaces
• PackageTasks
• Async task execution
• Just executable JavaScript

14. desc('This is the default task.');
task('default', function () {
console.log('This is the default task.');
console.dir(arguments);
});
namespace('foo', function () {
desc('This the foo:bar task');
task('bar', function () {
console.log('doing foo:bar task');
console.dir(arguments);
});
desc('This the foo:baz task');
task('baz', ['default', 'foo:bar'], function () {
console.log('doing foo:baz task');
console.dir(arguments);
});
});

15. desc('This is an asynchronous task.');
task('async', function () {
setTimeout(function () {
console.log('Hooray!');
complete();
}, 1000);
}, true);
desc('Calls the foo:bar task and its dependencies.');
task('invokeFooBar', function () {
// Calls foo:bar and its deps
jake.Task['foo:bar'].invoke();
// Does nothing
jake.Task['foo:bar'].invoke();
// Only re-runs foo:bar, but not its dependencies
jake.Task['foo:bar'].reenable();
jake.Task['foo:bar'].invoke();
});

16. var fs = require('fs')
, pkg = JSON.parse(
fs.readFileSync('package.json').toString())
, version = pkg.version
var t = new jake.PackageTask('jake', 'v' + version,
function () {
var fileList = [
'Makefile'
, 'Jakefile'
, 'README.md'
, 'package.json'
, 'lib/*'
, 'bin/*'
, 'tests/*'
];
this.packageFiles.include(fileList);
this.needTarGz = true;
this.needTarBz2 = true;
});

17. Remote upload service
• Minimal v1 in prod, Nov. 2010
• Redis, CORS XHR-push or JSONP for upload-progress
reporting
• Onboard thumbnailing, remote services for video and
document post-processing
• Three-machine cluster, not under a heavy load
• Large file sizes (e.g., 1.5GB)

18. Realtime, collaborative
doc-editing service
• In beta Oct. 21, 2011 (last week)
• NodeJS, Socket.io, PostgreSQL
• No production metrics yet for perf/scalability

19. Coding JS for Node

20. Awesome:
JavaScript is simple and
super-flexible

21. Horrible:
JavaScript is simple and
super-flexible

22. Asynchronous code
• Even shelling out is async?
• “1, 3, 2, go!” development
• Evented and callback-based control-flow
• A familiar model?
• Async patterns and libraries

23. 1, 3, 2, go!
var asyncFun = function () {
console.log('1');
setTimeout(function () {
console.log('3');
}, 0);
console.log('2');
console.log('go!');
};

24. Sync fetch-and-update
var fetchAndUpdate = function (params) {
var items = db.fetch(someQuery);
for (var i = 0, ii = items.length; i++) {
item.update(params);
}
return true;
};

25. Async fetch-and-update
var fetchAndUpdate = function (params, callback) {
db.fetch(someQuery, function (items) {
var count = 0;
for (var i = 0, ii = items.length; i++) {
item.update(params, function () {
count++;
if (count == ii) {
callback(true);
}
});
}
});
};

26. Is this familiar?
jQuery.ajax({
url: '/foo/bar.json'
, success: function () {
alert('yay!');
}
});
jQuery('#foo').bind('click', function (e) {
// Do some stuff
});

27. Async patterns and libs
• Queue
• Promise/deferred
• In-flight registry

28. Queue
var asyncQueueHandler = function (items,
handler, callback) {
var queue = items.slice()
, handleNextItem = function () {
var next = queue.pop();
if (next) {
handler(next, function () {
handleNextItem();
});
}
else {
callback();
}
};
handleNextItem();
};

29. Promise
var p = new yammer.util.Promise();
p.when('foo', 'bar', 'baz').then(
function () {
console.log('done!');
});
p.satisfy('foo');
p.satisfy('bar');
p.satisfy('baz');
p.then(function () {
console.log('still done!');
});

30. NodeJS in production

31. App dependencies
• Third-party modules still may change
rapidly
• Maintain forks, push back patches where
appropriate

32. Debugging NodeJS
• Callbacks in global scope have no stack
• Assume you’re fucked
• Default condition is a preemptible error
• In-flight registry with uncaughtException
handler

33. FlightCheck
• https://github.com/mde/flight_check
• Add items to in-flight registry
• Per-item timeout
• Configurable polling-interval
• Define a timeout-handler

34. In-flight registry
var FlightCheck = require('flight_check').FlightCheck;
var handler = function (req, resp) {
var checker = new FlightCheck(function (key) {
resp.writeHead(500);
resp.end('Oops, something bad happened.');
});
checker.add('foo', 10000);
doFoo(req, function (result) {
if (result.ok) {
checker.clear('foo');
// Do some other stuff
resp.writeHead(200);
resp.end('Hooray!');
}
});
};
process.on('uncaughtException', function (err) {
// Do some kind of logging
});

35. Visibility, metrics
• Measure everything
• Log everything
• https://github.com/mikejihbe/metrics

36. Ops
• Communicative, consultative dev
• Ask what is expected
• Play nicely with others

37. The future?
• JS interpreters will keep improving
• JS language will keep improving (see:
JS.next)
• NodeJS ecosystem will grow and mature
• Try NodeJS, you’ll like it

38. Matthew Eernisse
http://twitter.com/mde
Yammer Developer Center
http://developer.yammer.com/