Your SlideShare is downloading. ×
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Node.js - async for the rest of us.
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Node.js - async for the rest of us.

12,498

Published on

presented at the Denver Open Source Users Group 8/2/2011

presented at the Denver Open Source Users Group 8/2/2011

Published in: Technology
0 Comments
16 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,498
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
196
Comments
0
Likes
16
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • Good Grief. Why do we need another...\n
  • the browser waits, the server waits - what’s the difference?\n
  • \n
  • John McCarthy\n
  • \n
  • FAST, handles many connections\n
  • \n
  • \n
  • \n
  • \n
  • CommonJS Modules\nhttp library - node docs http://nodejs.org/docs/v0.4.10/api/\nnode will exit if it has nothing to do\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. node.js asynchronous... for the rest of us Mike Brevoort 8.2.2011 DOSUG code sample can be found athttps://github.com/mbrevoort/node.js-presentation
    • 2. agendathe case for node.jsdeveloping with nodelook at a few popular moduleslessons from the trenches
    • 3. typical n-tier run-a-round browser makes call to web server (waits) web server makes call to database (waits) web server returns result to browserResponse time is dominated by time waiting
    • 4. typical i/o latency L1: 3 cycles L2: 14 cycles RAM: 250 cycles DISK: 41,000,000 cycles NETWORK: 240,000,000 cycles L1 L2 RAM DiskNetwork 0 60,000,000 120,000,000 180,000,000 240,000,000 300,000,000 http://nodejs.org/jsconf.pdf
    • 5. http://xach.livejournal.com/ 170311.html
    • 6. “Most languages were designed tosolve computational problems, but Node.js is different. Node.js was designed from theground up to efficiently handle thecommunication that is at the heart of modern web applications.” http://www.joyentcloud.com/products/smart-appliances/ node-js-smartmachine/
    • 7. node.jsAn Evented I/O network serverfor Javascript created by sponsored by Ryan Dahl
    • 8. Runs Javascript, but isn’t primarily Javascript http://platformjs.wordpress.com/2010/11/24/node-js-under- the-hood/
    • 9. Why Javascript?most widely used programinglanguage of the web“never under estimate the power of familiarity andfriendliness” - Stacey Higginbotham, GigaOMasync by nature - the browseris a single threaded eventloop BAH! It’s a toy language!
    • 10. the event loopsingle threaded = no executionconcurrencyall execution initiated by an eventevents may have zero to manycallbacksevents are executed in orderTom Hughes-Croucher’s postmananalogy
    • 11. Installing nodemac, linux or windows (with cygwin hell)fear not! stable windows support comingin v0.6.0 (~3wks) # clone the git repo git clone git://github.com/joyent/node.git cd node # checkout the version you want git checkout v0.4.10 # build and install ./configure make && sudo make install
    • 12. Running Node ~/ node > var x=1, y=2, z=3; > x+y+z REPL 6 > require(fs).statSync(./blocking.js) { dev: 234881026,Read-Eval-Print-Loop ino: 3162208, mode: 33188, nlink: 1, ..... node app.js arg1 arg2Invoke script process.argv[0] process.argv[1] === === "node" "app.js" process.argv[2] === "arg1" process.argv[3] === "arg2"
    • 13. Hello World HTTP Servervar http = require(http);http.createServer(function (req, res) { res.writeHead(200, {Content-Type: text/plain}); res.end(Hello Worldn);}).listen(8080, "127.0.0.1");console.log(Server running at http://127.0.0.1:8080/);
    • 14. Not just HTTPvar net = require(net);var server = net.createServer(function (socket) { socket.write("Echo serverrn"); socket.pipe(socket);});server.listen(1337, "127.0.0.1"); http://nodejs.org/
    • 15. simple irc demo
    • 16. Developing with Nodeinstall nodeinstall npmyour favorite text editor Sublime Text 2, Textmate, vim, Emacs, Eclipse, whatever
    • 17. npmnode package manager created by Isaac Schlueter (Joyent)
    • 18. npmpublish, install, discover,and develop node programsputs modules in a place wherenode can find themmanages dependencies
    • 19. npm search, install # install a module (copy to node_modules) npm install socket.io # install a specific version npm install socket.io@0.6.0 # install module globally npm install socket.io -g # search npm search socket #info npm info socket.io
    • 20. npm registry http://registry.npmjs.org/
    • 21. npmjs.org stats
    • 22. { "name": "express", package.json "description": "Sinatra inspired web development framework", "version": "3.0.0", "author": "TJ Holowaychuk <tj@vision-media.ca>", "contributors": [ { "name": "TJ Holowaychuk", "email": "tj@vision-media.ca" }, { "name": "Aaron Heckmann", "email": "aaron.heckmann+github@gmail.com" }, { "name": "Ciaran Jessup", "email": "ciaranj@gmail.com" }, { "name": "Guillermo Rauch", "email": "rauchg@gmail.com" } ], "dependencies": { "connect": ">= 1.5.2 < 2.0.0", "mime": ">= 0.0.1", "qs": ">= 0.3.0" }, "devDependencies": { "connect-form": "0.2.1", "ejs": "0.4.2", "expresso": "0.8.1", "hamljs": "0.5.1", "jade": "0.13.0", "stylus": "0.13.0", "should": "0.2.1", "express-messages": "0.0.2", "node-markdown": ">= 0.0.1", "connect-redis": ">= 0.0.1" }, "keywords": ["framework", "sinatra", "web", "rest", "restful"], "repository": "git://github.com/visionmedia/express", "main": "index", "bin": { "express": "./bin/express" }, "scripts": { "test": "make test", "prepublish" : "npm prune" }, "engines": { "node": ">= 0.4.9 < 0.7.0" }}
    • 23. npm install .package.json isn’t just formodules published to npmnpm can help you manage andinstall dependencies in anyproject # from the same directory # as package.json npm install .
    • 24. npm list~/ npm listmy_project@0.1.0 /Users/mikebre/my_project!"# cluster@0.6.9 invalid$ %"" log@1.2.0!"# connect-gzip@0.1.0$ !"# connect@1.4.6$ $ %"" qs@0.1.0$ %"" mime@1.2.2!"# date@1.0.1$ %"" require-kiss@1.0.5!"" docco@0.3.0 extraneous!"# express@2.3.2$ !"" connect@1.4.6$ !"" mime@1.2.2$ %"" qs@0.1.0!"" hbs@0.0.7!"" log@1.1.0 extraneous!"" metrics@0.1.1!"" request-forked@1.9.8!"" semver@1.0.6 invalid%"# xml2js@0.1.6 %"" sax@0.1.4
    • 25. a very strong communitynodejs.orgGoogle Group mailing listIRC #node.js on freenodeStack Overflow, LinkedIn groupsnodeconf, node summercamp, etc.
    • 26. debuggingndb - commandline debuggerEclipse debuggerplugin for V8node-inspectoris very nice!
    • 27. node-inspector uses WebKit Web Inspector# install with npmnpm -g install node-inspector# start node-inspectornode-inspector &# start node in debug modenode --debug app.js
    • 28. profilingnode-inspector optionallysupports the V8 profilercollects CPU and heapsnapshots Speaking of heaps...
    • 29. garbage collection --trace-gc option to watch GC behavior V8 is a VM --> must GC tuned for the browser 20Mb - 40Mb per tab Large node heap sizes == :(
    • 30. GC Demo
    • 31. several popular node modules http://splashinthepacific.files.wordpress.com/2010/10/looking-glass-721.jpg
    • 32. ExpressSinatra (Ruby) inspired webframework created by TJ Holowaychuk
    • 33. Expressrequest routingcontent negotiationview templating and partialssession supportstatic file servingfast, clean and powerful
    • 34. Express Demo
    • 35. a RESTful service?cute, terse. boringlet’s do something abit more interesting...
    • 36. streaming demo
    • 37. Socket.ioUnified API for Websockets +fallbacks created by Guillermo Rauch
    • 38. Socket.iounified API for Comet style appstransport negotiationserver and client librariesfeature rich, above and beyond whatthe websocket protocol prescribes heartbeats, timeouts, namespacing, volatile messages, message acknowledgements, etc.
    • 39. socket.io demo
    • 40. Lessonsfrom the trenches
    • 41. asynchronous learning curve app.get(/bar, function(req, res) { foo.fetchSomething(function(error, something) { if(!error) { foo.fetchSomeOne(something, function(error, someone) {easy to if(!error) { foo.fetchBar(function(error, bar) { if(!error) { write } res.send("we got bar: " + bar); code else { res.send(error.statusCode); } like } }); this else { res.send(error.statusCode); } }); } else { res.send(error.statusCode); } }); });
    • 42. uncaught errors on error, node emits an ‘error’ event on the corresponding object if no listeners on object for ‘error’, a top level exception is raised and the process exits > var server = http.createServer(function (req, res) {});prudent server.on(error, function(error) {approach console.log("Caught error! Dont exit!"); });nuclearapproach > process.on(uncaughtException, function(error) { }); console.log("Kaboom.... handle " + error);
    • 43. plan for multipleprocesses from the start each node process is bound to one core many small processes better than one big one use Cluster https://github.com/learnboost/cluster var cluster = require(cluster); cluster(app.js) .set(workers, 16) // defaults to # of cores .use(cluster.logger(logs)) .use(cluster.stats()) .use(cluster.cli()) .use(cluster.repl(8888)) .listen(80)
    • 44. keep the heap small 200Mb or less if you’re worried about GC pause move data out of the node process instead use Redis, MongoDb, etc encourages statelessness, encourages scalability reduces risk of losing a single node process
    • 45. everything you do blocks No CPU for YOU! http://redriverpak.files.wordpress.com/ 2010/08/vwtouareg-road-block.jpg
    • 46. be weary of loops for (var i=0, l=entries.length; i<l; i++) { doSomething(entries[i]); } innocent enough? if # entries = 10,000 doSomething() takes ~1ms you block for 10 seconds!
    • 47. non-blocking loops // order matters function processEntry(entries, index) { index = index || 0; if(index === entries.length) return done(); doSomething(entries[index]); process.nextTick(function() { processEntry(entries, index++) }); } processEntry(entries);
    • 48. non-blocking loops // order doesnt matter var leftToProcess = entries.length; for (var i=0, l=entries.length; i<l; i++) { (function(foo) { process.nextTick(function() { doSomething(foo); if(--leftToProcess === 0) { done(); } }); })(entries[i]); }
    • 49. non-blocking loops // order doesnt matter // doSomething takes callback and is Async // doSomethingAsyncs happen in parallel var leftToProcess = entries.length; // doSomethings will be executed in parallel for (var i=0, l=entries.length; i<l; i++) { (function(foo) { process.nextTick(function() { doSomethingAsync(foo, function() { if(--leftToProcess === 0) { done(); } }); }); })(entries[i]); }
    • 50. set ulimit -nnode can handles 1000’s of connections? but your OS says... Too many open filesdefault # file descriptors on most linuxsystems is 10241 FD per socket means max open sockets < 1024 increase the max # of file descriptors ulimit -n <max # FD> ulimit -a to see current max
    • 51. pooled outbound connectionsnode pools outbound http(s) connections by defaultfor host + port combinationsdefault concurrent maxSockets per host + port is 5is this what you want?// for http as of node v0.4.10require (http) .getAgent("api.twitter.com", 80) .maxSockets = 100;// for https as of node v0.4.10require (https) .getAgent({ host:"docs.google.com", port: 443 }) .maxSockets = 100;
    • 52. timeoutsexpect that any callback could fail andmay not be calledanticipate conditions where bothinbound or outbound connections mayhanguse Mikeal Roger’s ‘request’ module I contributed timeout functionality should be part of node core ~v0.7/0.8
    • 53. offload anythingcomputationally intensive spawn a child process require(child_process).spawn call out to another system more apt to handle heavy lifting use a job queue
    • 54. be specific withpackage dependencies{ "name": "Foo Package", "description": "my Foo package", > "version": "1.0.0", "author": "Mike Brevoort <mikebre@ecollege.com>", "dependencies": { "express": "2.3.2", "cluster": ">= 0.6.1", is this what "mongodb": "0.9.x", "connect-gzip": "~0.1", you really "underscore": "= latest" want? really? }, "engines": { "node": "= 0.4.8" }}
    • 55. Let me tell you about my friend nodehe’s a great multi-tasker but can only do one thing at a time
    • 56. Thank You!Questions? Mike Brevoort @mbrevoort mike [at] brevoort [dot] com

    ×