1. node.js
asynchronous...
for the rest of us
Mike Brevoort
8.2.2011
DOSUG
code sample can be found at
https://github.com/mbrevoort/node.js-presentation
2. agenda
the case for node.js
developing with node
look at a few popular modules
lessons from the trenches
3.
4. typical n-tier
run-a-round
browser makes call to web server
(waits)
web server makes call to database
(waits)
web server returns result to browser
Response time is dominated by time waiting
7. “Most languages were designed to
solve computational problems, but
Node.js is different.
Node.js was designed from the
ground up to efficiently handle the
communication that is at the heart
of modern web applications.”
http://www.joyentcloud.com/products/smart-appliances/
node-js-smartmachine/
9. Runs Javascript, but isn’t
primarily Javascript
http://platformjs.wordpress.com/2010/11/24/node-js-under-
the-hood/
10. Why Javascript?
most widely used programing
language of the web
“never under estimate the power of familiarity and
friendliness” - Stacey Higginbotham, GigaOM
async by nature - the browser
is a single threaded event
loop
BAH! It’s a toy language!
11. the event loop
single threaded = no execution
concurrency
all execution initiated by an event
events may have zero to many
callbacks
events are executed in order
Tom Hughes-Croucher’s postman
analogy
12. Installing node
mac, linux or windows (with cygwin hell)
fear not! stable windows support coming
in v0.6.0 (~3wks)
# clone the git repo
git clone git://github.com/joyent/node.git
cd node
# checkout the version you want
git checkout v0.4.10
# build and install
./configure
make && sudo make install
14. Hello World HTTP
Server
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello Worldn');
}).listen(8080, "127.0.0.1");
console.log('Server running at http://127.0.0.1:8080/');
15. Not just HTTP
var net = require('net');
var server = net.createServer(function (socket) {
socket.write("Echo serverrn");
socket.pipe(socket);
});
server.listen(1337, "127.0.0.1");
http://nodejs.org/
24. npm install .
package.json isn’t just for
modules published to npm
npm can help you manage and
install dependencies in any
project
# from the same directory
# as package.json
npm install .
26. a very strong
community
nodejs.org
Google Group mailing list
IRC #node.js on freenode
Stack Overflow, LinkedIn groups
nodeconf, node summercamp, etc.
30. garbage collection
--trace-gc option to watch GC
behavior
V8 is a VM --> must GC
tuned for the browser
20Mb - 40Mb per tab
Large node heap sizes == :(
39. Socket.io
unified API for Comet style apps
transport negotiation
server and client libraries
feature rich, above and beyond what
the websocket protocol prescribes
heartbeats, timeouts, namespacing,
volatile messages, message
acknowledgements, etc.
43. uncaught errors
on error, node emits an ‘error’ event on
the corresponding object
if no listeners on object for ‘error’, a
top level exception is raised and the
process exits
>
var server = http.createServer(function (req, res) {});
prudent server.on('error', function(error) {
approach console.log("Caught error! Don't exit!");
});
nuclear
approach > process.on('uncaughtException', function(error) {
});
console.log("Kaboom.... handle " + error);
44. plan for multiple
processes from the start
each node process is bound to one core
many small processes better than one
big one
use Cluster
https://github.com/learnboost/cluster
var cluster = require('cluster');
cluster('app.js')
.set('workers', 16) // defaults to # of cores
.use(cluster.logger('logs'))
.use(cluster.stats())
.use(cluster.cli())
.use(cluster.repl(8888))
.listen('80')
45. keep the heap small
200Mb or less if you’re worried
about GC pause
move data out of the node process
instead use Redis, MongoDb, etc
encourages statelessness,
encourages scalability
reduces risk of losing a single
node process
46. everything you do
blocks
No CPU for YOU!
http://redriverpak.files.wordpress.com/
2010/08/vwtouareg-road-block.jpg
47. be weary of loops
for (var i=0, l=entries.length; i<l; i++) {
doSomething(entries[i]);
}
innocent enough?
if # entries = 10,000
doSomething() takes ~1ms
you block for 10 seconds!
48. non-blocking loops
// order matters
function processEntry(entries, index) {
index = index || 0;
if(index === entries.length) return done();
doSomething(entries[index]);
process.nextTick(function() {
processEntry(entries, index++)
});
}
processEntry(entries);
50. non-blocking loops
// order doesn't matter
// doSomething takes callback and is Async
// doSomethingAsync's happen in parallel
var leftToProcess = entries.length;
// doSomething's will be executed in parallel
for (var i=0, l=entries.length; i<l; i++) {
(function(foo) {
process.nextTick(function() {
doSomethingAsync(foo, function() {
if(--leftToProcess === 0) {
done();
}
});
});
})(entries[i]);
}
51. set ulimit -n
node can handles 1000’s of connections?
but your OS says...
Too many open files
default # file descriptors on most linux
systems is 1024
1 FD per socket means max open sockets < 1024
increase the max # of file descriptors
ulimit -n <max # FD>
ulimit -a to see current max
52. pooled outbound
connections
node pools outbound http(s) connections by default
for host + port combinations
default concurrent maxSockets per host + port is 5
is this what you want?
// for http as of node v0.4.10
require ('http')
.getAgent("api.twitter.com", 80)
.maxSockets = 100;
// for https as of node v0.4.10
require ('https')
.getAgent({ host:"docs.google.com", port: 443 })
.maxSockets = 100;
53. timeouts
expect that any callback could fail and
may not be called
anticipate conditions where both
inbound or outbound connections may
hang
use Mikeal Roger’s ‘request’ module
I contributed timeout functionality
should be part of node core ~v0.7/0.8
54. offload anything
computationally intensive
spawn a child process
require('child_process').spawn
call out to another system
more apt to handle heavy
lifting
use a job queue
55. be specific with
package dependencies
{
"name": "Foo Package",
"description": "my Foo package",
>
"version": "1.0.0",
"author": "Mike Brevoort <mikebre@ecollege.com>",
"dependencies": {
"express": "2.3.2",
"cluster": ">= 0.6.1", is this what
"mongodb": "0.9.x",
"connect-gzip": "~0.1",
you really
"underscore": "= latest" want? really?
},
"engines": { "node": "= 0.4.8" }
}
56. Let me tell you about my
friend node
he’s a great multi-tasker but
can only do one thing at a
time