Non-blocking I/O, Event loops and node.js

This is a 15 minute presentation I did at work recently about threads, polling and node.js. It rests heavily on the shoulders of giants, especially Ryan Dahls original node.js talk at jsconf.eu.

non-blocking I/O, event loops, javascript and node.js.

Threads suck

Why do we use threads?

Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());

Policy p = cmServer.getPolicy(...); Blocking implies multiple execution stacks. Where else to go? System.out.println(p.getContentId().toString());

How to deal with blocking? Processes No shared memory Heavy OS Threads Can share memory Relatively cheap Green threads / Co-routines

Are threads cheaper than processes?

HTTP request latency; glibc 2.3.2 on Linux 2.6 HTTP-anropslatens; glibc 2.3.2 på Linux 2.6 1 thread per call 1 process per call http://bulk.fefe.de/scalable-networking.pdf Open connections

HTTP request latency; glibc 2.3.2 on Linux 2.6 1 thread per call 1 process per call Which is the reason why we use thread pools! Open connections

Are threads cheap at all?

nginx vs apache http://blog.webfaction.com/a-little-holiday-present

nginx vs apache http://blog.webfaction.com/a-little-holiday-present ∆ = Apaches context switching

lmbench ./lat_ctx -N 10 -s 4096 0 1 .. 20 80 74 68 62 56 50 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

lmbench ./lat_ctx -N 10 -s 4096 0 1 .. 20 80 74 68 62 56 50 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 On a MBP, OS X 10.6 Context switch around 65 µs

65 µs x 2 x 4000 = 520 ms

How about memory?

> ulimit -a | grep stack stack size (kbytes, -s) 8192

...but at least 512+1 kb For OS X, source: ADC

nginx vs apache http://blog.webfaction.com/a-little-holiday-present

nginx vs apache http://blog.webfaction.com/a-little-holiday-present Difference: Threads and processes cost.

What about ngnix?

The cost of I/O L1-cache 3 cycles L2-cache 14 cycles RAM 250 cycles Disk 41 000 000 cycles Network 240 000 000 cycles

The weight of I/O

The weight of I/O L1-cache A big squirrel

The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat

The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically

The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically Disk Like a 100 blue whales

The weight of I/O L1-cache A big squirrel L2-cache A medium-sized cat RAM Mattias, basically Disk Like a 100 blue whales Network Belarus yearly wheat import

Servers pretty I/O-intensive = Squirrelʼs burried in whales fat and wheat

Policy p = cmServer.getPolicy(...); System.out.println(p.getContentId().toString());

cmServer.getPolicy(function (Policy p) { System.out.println( p.getContentId().toString() ); });

cmServer.getPolicy(function (Policy p) { System.out.println( Sleeps until p’s available p.getContentId().toString() ); });

cmServer.getPolicy(function (Policy p) { System.out.println( Sover tills dess p finns p.getContentId().toString() ); }); // And down here we can do other stuff

Non-blocking I/O... All code is run in a single thread Functions listen on events, act and sleep

...is pretty demanding All I/O has to be non-blocking Callbacks are pretty ugly without HOF

V8 + node.js

“a purely evented, non-blocking infrastructure to script highly concurrent programs.”

V8 Chromes JS engine JS Built to be event driven node.js Queues built on /dev/epoll amongst other things

% node hello.js % ab -n 1000 -c 200 http://127.0.0.1:8080/ ... Time per request: 2010.070 [ms] (mean) min mean[+/-sd] median max Connect: 0 2 1.9 1 8 Processing: 2001 2006 2.8 2005 2013 Waiting: 2000 2003 2.0 2003 2009 Total: 2002 2008 3.5 2006 2015 Percentage of the requests served within a certain time (ms) 50% 2006 66% 2008 75% 2012 80% 2013 90% 2013 95% 2014 98% 2014 99% 2014 100% 2015 (longest request)

var stat = require('posix').stat, puts = require('sys').puts; var promise = stat('/etc/passwd'); promise.addCallback(function (s) { puts('modified: ' + s.mtime); });

% node web.test.js % ab -n 10000 -c 250 http://127.0.0.1:8080/marcus ... Requests per second: 4851.93 [#/sec] (mean) Time per request: 51.526 [ms] (mean) Time per request: 0.206 [ms] (mean, across all c requests) Connection Times (ms) min mean[+/-sd] median max Connect: 0 14 111.8 1 999 Processing: 4 35 15.8 31 103 Waiting: 4 35 15.8 30 103 Total: 14 49 112.5 32 1035 Percentage of the requests served within a certain time (ms) 50% 32 66% 34 75% 35 80% 36 90% 57 95% 92 98% 102 99% 986 100% 1035 (longest request)

throughput 80 74 68 62 56 50 1 5 13 25 50 100 250

HEALTH

Distributed ab One conﬁguration, multiple nodes REST interface Streams JSON over HTTP Decent performance

http://www.slideshare.net	86
http://spaces.fmr.com	50
http://www.techgig.com	14
http://beta.polopoly.com	4
http://a0.twimg.com	4
http://twitter.com	2
https://twitter.com	1
http://paper.li	1
http://us-w1.rockmelt.com	1

Non-blocking I/O, Event loops and node.js

by Marcus Frödin on Dec 10, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

9 Embeds 163

Statistics

Non-blocking I/O, Event loops and node.js — Presentation Transcript