Non-blocking I/O, Event loops and node.js

23,539
-1

Published on

A lightning talk I held at a geek night at work about my recent experience with node.js, and my fledgling love story with javascript.

Published in: Technology

Non-blocking I/O, Event loops and node.js

  1. 1. This is a 15 minute presentation I did at work recently about threads, polling and node.js. It rests heavily on the shoulders of giants, especially Ryan Dahls original node.js talk at jsconf.eu.
  2. 2. non-blocking I/O, event loops, javascript and node.js.
  3. 3. Threads suck
  4. 4. Why do we use threads?
  5. 5. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
  6. 6. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
  7. 7. Policy p = cmServer.getPolicy(...); Blocking implies multiple execution stacks. Where else to go? System.out.println(p.getContentId().toString());
  8. 8. How to deal with blocking? Processes No shared memory Heavy OS Threads Can share memory Relatively cheap Green threads / Co-routines
  9. 9. Are threads cheaper than processes?
  10. 10. HTTP request latency; glibc 2.3.2 on Linux 2.6 HTTP-anropslatens; glibc 2.3.2 på Linux 2.6 1 thread per call 1 process per call http://bulk.fefe.de/scalable-networking.pdf Open connections
  11. 11. HTTP request latency; glibc 2.3.2 on Linux 2.6 1 thread per call 1 process per call Which is the reason why we use thread pools! Open connections
  12. 12. Are threads cheap at all?
  13. 13. nginx vs apache http://blog.webfaction.com/a-little-holiday-present
  14. 14. nginx vs apache http://blog.webfaction.com/a-little-holiday-present ∆ = Apaches context switching
  15. 15. lmbench ./lat_ctx -N 10 -s 4096 0 1 .. 20 80 74 68 62 56 50 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  16. 16. lmbench ./lat_ctx -N 10 -s 4096 0 1 .. 20 80 74 68 62 56 50 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 On a MBP, OS X 10.6 Context switch around 65 µs
  17. 17. 65 µs x 2 x 4000 = 520 ms
  18. 18. How about memory?
  19. 19. > ulimit -a | grep stack stack size (kbytes, -s) 8192
  20. 20. ...but at least 512+1 kb For OS X, source: ADC
  21. 21. nginx vs apache http://blog.webfaction.com/a-little-holiday-present
  22. 22. nginx vs apache http://blog.webfaction.com/a-little-holiday-present Difference: Threads and processes cost.
  23. 23. What about ngnix?
  24. 24. The cost of I/O L1-cache 3 cycles L2-cache 14 cycles RAM 250 cycles Disk 41 000 000 cycles Network 240 000 000 cycles
  25. 25. The weight of I/O
  26. 26. The weight of I/O L1-cache A big squirrel
  27. 27. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat
  28. 28. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically
  29. 29. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically Disk Like a 100 blue whales
  30. 30. The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically Disk Like a 100 blue whales Network Belarus yearly wheat import
  31. 31. Servers pretty I/O-intensive = Squirrelʼs burried in whales fat and wheat
  32. 32. Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());
  33. 33. cmServer.getPolicy(function (Policy p) { System.out.println( p.getContentId().toString() ); });
  34. 34. cmServer.getPolicy(function (Policy p) { System.out.println( Sleeps until p’s available p.getContentId().toString() ); });
  35. 35. cmServer.getPolicy(function (Policy p) { System.out.println( Sover tills dess p finns p.getContentId().toString() ); }); // And down here we can do other stuff
  36. 36. Non-blocking I/O... All code is run in a single thread Functions listen on events, act and sleep
  37. 37. ...is pretty demanding All I/O has to be non-blocking Callbacks are pretty ugly without HOF
  38. 38. V8 + node.js
  39. 39. “a purely evented, non-blocking infrastructure to script highly concurrent programs.”
  40. 40. V8 Chromes JS engine JS Built to be event driven node.js Queues built on /dev/epoll amongst other things
  41. 41. var http = require('http'); http.createServer(function (req, res) { setTimeout(function () { res.sendHeader(200, {'Content-Type': 'text/plain'}); res.sendBody('Hello World'); res.finish(); }, 2000); }).listen(8080);
  42. 42. % node hello.js % ab -n 1000 -c 200 http://127.0.0.1:8080/ ... Time per request: 2010.070 [ms] (mean) min mean[+/-sd] median max Connect: 0 2 1.9 1 8 Processing: 2001 2006 2.8 2005 2013 Waiting: 2000 2003 2.0 2003 2009 Total: 2002 2008 3.5 2006 2015 Percentage of the requests served within a certain time (ms) 50% 2006 66% 2008 75% 2012 80% 2013 90% 2013 95% 2014 98% 2014 99% 2014 100% 2015 (longest request)
  43. 43. var stat = require('posix').stat, puts = require('sys').puts; var promise = stat('/etc/passwd'); promise.addCallback(function (s) { puts('modified: ' + s.mtime); });
  44. 44. var web = require('./web'); web.server(function (route) { route.get("^/([a-z]+)$", function(parms, req, res) { res.sendHeader(200, {'Content-Type':'text/plain'}); res.sendBody("Hello " + parms[0]); res.finish(); }); }).listen(8080);
  45. 45. % node web.test.js % ab -n 10000 -c 250 http://127.0.0.1:8080/marcus ... Requests per second: 4851.93 [#/sec] (mean) Time per request: 51.526 [ms] (mean) Time per request: 0.206 [ms] (mean, across all c requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 14 111.8 1 999 Processing: 4 35 15.8 31 103 Waiting: 4 35 15.8 30 103 Total: 14 49 112.5 32 1035 Percentage of the requests served within a certain time (ms) 50% 32 66% 34 75% 35 80% 36 90% 57 95% 92 98% 102 99% 986 100% 1035 (longest request)
  46. 46. var web = require('./web'), posix = require('posix'); web.server(function (route) { route.get("^/([a-z]+)$", function(parms, req, res) { posix.cat(parms[0]).addCallback(function(text) { res.sendHeader(200, {'Content-Type':'text/plain'}); res.sendBody(text); res.finish(); }); }); }).listen(8080);
  47. 47. throughput 80 74 68 62 56 50 1 5 13 25 50 100 250
  48. 48. HEALTH
  49. 49. Distributed ab One configuration, multiple nodes REST interface Streams JSON over HTTP Decent performance
  50. 50. demo!

×