NodeJS Concurrency

10,700 views

Published on

A discussion of concurrency features in NodeJS: the good and the less-than-good.

An overview of using Web Workers to address the less-than-good.

Published in: Technology
1 Comment
12 Likes
Statistics
Notes
  • Has featural development on this stopped? Well if it's complete it's complete I suppose! Though I can't get the thing to work. Both require('webworker') and require('../lib/webworker') result in 'object not found' or some such...
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
10,700
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
60
Comments
1
Likes
12
Embeds 0
No embeds

No notes for slide

NodeJS Concurrency

  1. 1. Concurrency in NodeJS PeterGriess, Yahoo! Inc. pg@std.in | @pgriess| http://blog.std.in
  2. 2. NodeJS Concurrency Model • Single process • Single thread • Non-preemptive • Conncurrency provided by multiple processes
  3. 3. The Bad • Can’t utilize multiple cores by default – Modern server-class hardware is at least 4x • Easy to stall request processing – Invoke a method that blocks (there are some) – Run some CPU-intensive code
  4. 4. The Good • No race conditions (almost) • No locking strategies to get wrong • Comparatively easy to reason about • Processes are a powerful sandbox • QoS: nice(2), ionice(1), etc. • Permissions: chroot(2), setuid(2), etc. • Fault isolation
  5. 5. child_process • Node has an API for creating child processes, child_process.spawn() • It has problems – Very UNIX; not friendly to JavaScript developers – IPC is DIY (transport? data format?)
  6. 6. Web Workers • A rich but (mostly) easy-to-use JavaScript API – Spawn new workers – Send and receive messages to/from workers – Worker lifecycle management (creation, termination, etc) • A Worker executes a JavaScript file/URL • Separate process and execution context • A W3C standard, part of the “HTML5” bucket; designed for browsers
  7. 7. Shameless plug: node-webworker • A Web Workers implementation for NodeJS • npm install webworker • Interested about the design? – http://blog.std.in/2010/07/08/nodejs-webworker- design/
  8. 8. Ping/pong example: parent.js var Worker = require(‘webworker/webworker’).Worker; varw = new Worker(‘/Users/griessp/child.js’); w.onmessage = function(e) { console.log(e.data); }; w.onerror = function(e) { console.log(‘Error: ‘ + e.message); w.terminate(); }; w.postMessage({ping : ‘pong’});
  9. 9. Ping/pong example: child.js onmessage = function(e) { console.log(e.data); postMessage([e.data.ping, {Hello : ‘world’}]); setTimeout(function() { throw new Error(‘Ohnoes!’); }, 1000); }; onclose = function() { console.log(‘Child: Shutting down.’); };
  10. 10. Ping/pong example: output % node ./parent.js { ping: 'pong' } [ 'pong', { Hello: 'world' } ] Parent error: Oh noes! Child: Shutting down.
  11. 11. Ping/pong example: hilights • Easily spawn a new process! • Pass complex messages! • Exceptions bubble up to the parent! • Graceful shutdown! • This code will work in a browser with (almost) no changes. Hooray standard APIs!
  12. 12. node-webworkers: Extensions • Gracefully shut down workers with onclose() • Watch for worker termination with onexit() • Send file descriptors with postMessage()

×