Scaling with event-based webservers

Scaling with event– based webservers Morten Siebuhr Open Source Days March 5th 20111/66

whoami(1) Deﬁning “web–server work” Killing Apache Event–based servers Using Node.js Questions2/66

whoami(1)3/66

Computer Scientist Distributed Systems / Scientiﬁc Computing (Web–)developer @ One.com4/66

Web–development @ one.com: Rich Internet Applications e–mail, calendar, galleries, . . . + 900.000 customers5/66

Deﬁning “Web–server work”7/66

Typical web–server work8/66

Typical web–server work Serving static ﬁles8/66

Typical web–server work Serving static ﬁles Talking to databases8/66

Typical web–server work Serving static ﬁles Talking to databases Not doing lots of computations8/66

Typical web–server work Serving static ﬁles Talking to databases Not doing lots of computations ≈ 100 → 1000 connections served ASAP8/66

(A)typical web–server work9/66

(A)typical web–server work Persistent connections9/66

(A)typical web–server work Persistent connections = ∞ simultaneous connections9/66

(A)typical web–server work Persistent connections = ∞ simultaneous connections ≈ 10000 simultaneous connections9/66

(A)typical web–server work Persistent connections = ∞ simultaneous connections ≈ 10000 simultaneous connections (But we don’t care as much about latency. . . )9/66

waiting...10/66

sleep(1.0)11/66

Response time w. sleep(1). 25 s WSGI 20 s Average response time 15 s 10 s 5s 0s 0 100 200 300 400 500 600 700 800 900 1000 Concurrent requests13/66

Response time w. sleep(1). 25 s WSGI 20 s Average response time 15 s 10 s 5s 0s 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests14/66

Threading15/66

Use threads16/66

Wasted resources!17/66

Add threads18/66

Response time w. sleep(1). 25 s WSGI Apache 20 s Average response time 15 s 10 s 5s 0s 0 100 200 300 400 500 600 700 800 900 1000 Concurrent requests19/66

Response time w. sleep(1). 25 s WSGI Apache 20 s Average response time 15 s 10 s 5s 0s 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests20/66

Memory usage w. sleep(1). 1500 MB WSGI Apache 1250 MB 1000 MB Max Virtual Memory 750 MB 500 MB 250 MB 0 MB 0 100 200 300 400 500 600 700 800 900 1000 Concurrent requests25/66

Memory usage w. sleep(1). 1500 MB WSGI Apache 1250 MB 1000 MB Max Virtual Memory 750 MB 500 MB 250 MB 0 MB 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests26/66

A single connection27/66

A single connection = 1 thread27/66

A single connection = 1 thread = 1 C–stack & kernel data structures27/66

A single connection = 1 thread = 1 C–stack & kernel data structures + Locking & switching overhead &c. . .27/66

Add memory!29/66

Add memory! Memory is cheap29/66

Add memory! Memory is cheap Except, actually using it isn’t cheap. . .29/66

Memory Wall 1986 → 2000:30/66

Memory Wall 1986 → 2000: + 55% CPU speed P/A30/66

Memory Wall 1986 → 2000: + 55% CPU speed P/A + 10% RAM speed P/A (Source: http://www.cs.virginia.edu/papers/Hitting_Memory_Wall-wulf94.pdf)30/66

Threads and connections mapped 1:132/66

Threads and connections mapped 1:1 We use a lot of threads32/66

Threads and connections mapped 1:1 We use a lot of threads Which we don’t have the memory bandwidth to move around.32/66

Events!33/66

waiting...34/66

Event-basics37/66

Event-basics Event queue37/66

Event-basics Event queue Event loop37/66

Event-basics Event queue Event loop “The Event Loop”37/66

A single connection38/66

A single connection = 1 “Event”38/66

A single connection = 1 “Event” = Some variables & function38/66

A single connection = 1 “Event” = Some variables & function = Hash table & some pointers38/66

A single connection = 1 “Event” = Some variables & function = Hash table & some pointers + Queue of events waiting to be processed38/66

Why not earlier40/66

Why not earlier Simply not a problem!40/66

Why not earlier Simply not a problem! Traditional back–end programmers not used to thinking this way40/66

What we gain41/66

What we gain Relatively low memory use41/66

What we gain Relatively low memory use No threading issues41/66

What we gain Relatively low memory use No threading issues Client-side programmers have no preconditions.41/66

Node.js42/66

Node.js is a set of bindings to the V8 JavaScript engine for scripting network programs. — Ryan Dahl43/66

JavaScript44/66

JavaScript Built for event-based programming.44/66

JavaScript Built for event-based programming. Netscape needed something fast. . .44/66

JavaScript Built for event-based programming. Netscape needed something fast. . . . . . JavaScript “designed” and implemented in 14 days44/66

Google V8 JIT’ing JavaScript to native machine code45/66

Google V8 JIT’ing JavaScript to native machine code = Compiling server code45/66

Node.js Non–browser V8–bindings for various libraries Everything non–blocking46/66

A minimal webserver in Node.js. . . // Import HTTP package var http = require ( ’ http ’) ; // Set up basic server var server = http . createServer ( function ( req , res ) { res . writeHead (200 , { ’ Content - Type ’: ’ text / plain ’ }) ; res . end ( ’ Hello World n ’) ; }) ; // Start the server server . listen (8124 , " 127.0.0.1 " ) ; console . log ( ’ Server running at http : / / 1 2 7 . 0 . 0 . 1 : 8 1 2 4 / ’) ;47/66

. . . with “database” // Import HTTP package var http = require ( ’ http ’) ; // Set up basic server var server = http . createServer ( function ( req , res ) { setTimeout ( function () { res . writeHead (200 , { ’ Content - Type ’: ’ text / plain ’ }) ; res . end ( ’ Hello World n ’) ; } , 1000) ; }) ; // Start the server server . listen (8124 , " 127.0.0.1 " ) ; console . log ( ’ Server running at http : / / 1 2 7 . 0 . 0 . 1 : 8 1 2 4 / ’) ;48/66

Response time w. sleep(1). 25 s WSGI Apache Node.js 20 s Average response time 15 s 10 s 5s 0s 0 100 200 300 400 500 600 700 800 900 1000 Concurrent requests49/66

Response time w. sleep(1). 25 s WSGI Apache Node.js 20 s Average response time 15 s 10 s 5s 0s 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests50/66

Memory usage w. sleep(1). 1500 MB WSGI Apache Node.js 1250 MB 1000 MB Max Virtual Memory 750 MB 500 MB 250 MB 0 MB 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests51/66

Connections in Node.js52/66

Connections in Node.js = 1 “Event”52/66

Connections in Node.js = 1 “Event” = 1 Closure + callback function52/66

Connections in Node.js = 1 “Event” = 1 Closure + callback function = 1 struct + some pointers52/66

Connections in Node.js = 1 “Event” = 1 Closure + callback function = 1 struct + some pointers + Event queue operations + function calls52/66

Alternatives53/66

Alternatives Well–written thread program!53/66

Alternatives Well–written thread program! Erlang http://www.erlang.org/53/66

Alternatives Well–written thread program! Erlang http://www.erlang.org/ Google GO http://golang.org/53/66

Alternatives Well–written thread program! Erlang http://www.erlang.org/ Google GO http://golang.org/ Lua Event Machine https://github.com/esmil/lem53/66

Alternatives Well–written thread program! Erlang http://www.erlang.org/ Google GO http://golang.org/ Lua Event Machine https://github.com/esmil/lem Nginx http://wiki.nginx.org/53/66

Alternatives Well–written thread program! Erlang http://www.erlang.org/ Google GO http://golang.org/ Lua Event Machine https://github.com/esmil/lem Nginx http://wiki.nginx.org/ Varnish http://www.varnish-cache.org/53/66

Working with Node.js54/66

I’m new to JavaScript!55/66

JavaScript WTFs Always add ;! Function arguments. . . this scoped to callee object! Stacktraces + events sometimes look strange57/66

JSLint58/66

JSLint www.crockfordfacts.com58/66

Nice code! def process ( user ) : user = getAuth ( user ) if not user : return ’ Fail ’ db = getC onnnecti on ( params ) data = db . getUserData ( user ) if not data : return ’ Fail ’ return data59/66

Boomerang code function process ( user , callback ) { getAuth ( req . username , function ( err , user ) { if ( err ) return callback ( err ) ; g etDB Co n ne ct io n ( params , function ( err , db ) { if ( err ) return callback ( err ) ; db . getUserData ( user , function ( err , data ) { if ( err ) return callback ( err ) ; return callback ( null , data ) ; }) ; }) ; }) ; }60/66

NPM61/66

Express.js var app = require ( ’ express ’) . createServer () ; app . get ( ’/ ’ , function ( req , res ) { setTimeout ( function () { res . send ( ’ Hello World ’) ; } , 1000) ; }) ; app . listen (3000) ;62/66

Express.js - Chat server var app = require ( ’ express ’) . createServer () , chat = []; app . get ( ’/ ’ , function ( req , res ) { chat . push ( res ) ; }) ; app . get ( ’ /: msg ’ , function ( req , res ) { chat . forEach ( function ( conn ) { conn . write ( req . params . msg ) ; }) ; }) ; app . listen (3000) ;63/66

Questions (The end)64/66

http://nodejs.org/ https://github.com/joyent/node65/66

Scaling with event-based webservers

by Morten Siebuhr

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

Scaling with event-based webservers Presentation Transcript