Scaling with event–       based webservers       Morten Siebuhr       Open Source Days       March 5th 20111/66
whoami(1)       Defining “web–server work”            Killing Apache         Event–based servers             Using Node.js ...
whoami(1)3/66
Computer Scientist       Distributed Systems / Scientific Computing          (Web–)developer @ One.com4/66
Web–development @ one.com:         Rich Internet Applications         e–mail, calendar, galleries, . . .           + 900.0...
6/66
Defining       “Web–server work”7/66
Typical web–server work8/66
Typical web–server work            Serving static files8/66
Typical web–server work            Serving static files           Talking to databases8/66
Typical web–server work             Serving static files           Talking to databases       Not doing lots of computation...
Typical web–server work                 Serving static files                Talking to databases           Not doing lots o...
(A)typical web–server work9/66
(A)typical web–server work           Persistent connections9/66
(A)typical web–server work           Persistent connections        = ∞ simultaneous connections9/66
(A)typical web–server work            Persistent connections        = ∞ simultaneous connections       ≈ 10000 simultaneou...
(A)typical web–server work                  Persistent connections              = ∞ simultaneous connections            ≈ ...
waiting...10/66
sleep(1.0)11/66
12/66
Response time w. sleep(1).                                25 s                                                            ...
Response time w. sleep(1).                                25 s                                                            ...
Threading15/66
Use threads16/66
Wasted resources!17/66
Add threads18/66
Response time w. sleep(1).                                25 s                                                            ...
Response time w. sleep(1).                                25 s                                                            ...
21/66
22/66
23/66
Memory usage w. sleep(1).                             1500 MB                                                             ...
Memory usage w. sleep(1).                             1500 MB                                                             ...
A single connection27/66
A single connection                 =              1 thread27/66
A single connection                        =                    1 thread                        =        1 C–stack & kerne...
A single connection                         =                     1 thread                         =        1 C–stack & ke...
28/66
Add memory!29/66
Add memory!        Memory is cheap29/66
Add memory!                   Memory is cheap        Except, actually using it isn’t cheap. . .29/66
Memory Wall        1986 → 2000:30/66
Memory Wall           1986 → 2000:        + 55% CPU speed P/A30/66
Memory Wall                                 1986 → 2000:                           + 55% CPU speed P/A                    ...
31/66
Threads and connections mapped 1:132/66
Threads and connections mapped 1:1              We use a lot of threads32/66
Threads and connections mapped 1:1                         We use a lot of threads        Which we don’t have the memory b...
Events!33/66
waiting...34/66
35/66
36/66
Event-basics37/66
Event-basics         Event queue37/66
Event-basics         Event queue          Event loop37/66
Event-basics          Event queue           Event loop        “The Event Loop”37/66
A single connection38/66
A single connection                 =              1 “Event”38/66
A single connection                   =                1 “Event”                   =        Some variables & function38/66
A single connection                    =                1 “Event”                    =        Some variables & function   ...
A single connection                          =                       1 “Event”                          =              Som...
39/66
39/66
39/66
39/66
Why not earlier40/66
Why not earlier        Simply not a problem!40/66
Why not earlier                            Simply not a problem!        Traditional back–end programmers not used to think...
What we gain41/66
What we gain        Relatively low memory use41/66
What we gain        Relatively low memory use           No threading issues41/66
What we gain                  Relatively low memory use                     No threading issues        Client-side program...
Node.js42/66
Node.js is a set of bindings to the V8 JavaScript engine                     for scripting network programs.              ...
Node.js is a set of bindings to the V8 JavaScript engine                     for scripting network programs.              ...
JavaScript44/66
JavaScript        Built for event-based programming.44/66
JavaScript        Built for event-based programming.44/66
JavaScript        Built for event-based programming.        Netscape needed something fast. . .44/66
JavaScript                 Built for event-based programming.                   Netscape needed something fast. . .       ...
JavaScript                 Built for event-based programming.                   Netscape needed something fast. . .       ...
Google V8        JIT’ing JavaScript to native machine code45/66
Google V8        JIT’ing JavaScript to native machine code                = Compiling server code45/66
Node.js        Non–browser V8–bindings for various libraries                  Everything non–blocking46/66
A minimal webserver in Node.js. . .        // Import HTTP package        var http = require ( ’ http ’) ;        // Set up...
. . . with “database”        // Import HTTP package        var http = require ( ’ http ’) ;        // Set up basic server ...
Response time w. sleep(1).                                25 s                                                            ...
Response time w. sleep(1).                                25 s                                                            ...
Memory usage w. sleep(1).                             1500 MB                                                             ...
Connections in Node.js52/66
Connections in Node.js                  =               1 “Event”52/66
Connections in Node.js                     =                 1 “Event”                     =        1 Closure + callback f...
Connections in Node.js                     =                 1 “Event”                     =        1 Closure + callback f...
Connections in Node.js                          =                      1 “Event”                          =             1 ...
Alternatives53/66
Alternatives        Well–written thread program!53/66
Alternatives        Well–written thread program!                   Erlang            http://www.erlang.org/53/66
Alternatives        Well–written thread program!                   Erlang            http://www.erlang.org/               ...
Alternatives        Well–written thread program!                    Erlang            http://www.erlang.org/              ...
Alternatives        Well–written thread program!                    Erlang            http://www.erlang.org/              ...
Alternatives        Well–written thread program!                    Erlang            http://www.erlang.org/              ...
Working with Node.js54/66
I’m new to JavaScript!55/66
56/66
JavaScript WTFs                         Always add ;!                  Function arguments. . .                this   scope...
JSLint58/66
JSLint        www.crockfordfacts.com58/66
Nice code!        def process ( user ) :          user = getAuth ( user )          if not user :            return ’ Fail ...
Boomerang code        function process ( user , callback ) {          getAuth ( req . username , function ( err , user ) {...
NPM61/66
Express.js        var app = require ( ’ express ’) . createServer () ;        app . get ( ’/ ’ , function ( req , res ) { ...
Express.js - Chat server        var app = require ( ’ express ’) . createServer () ,            chat = [];        app . ge...
Questions          (The end)64/66
http://nodejs.org/        https://github.com/joyent/node65/66
function process ( user , callback ) {          var user = null , db = null ;          function done_ () {            if (...
Scaling with event-based webservers
Upcoming SlideShare
Loading in …5
×

Scaling with event-based webservers

2,015 views
1,921 views

Published on

Slides from my talk at Open Source Days 2011 on Event-based webservers.

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,015
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
23
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

Scaling with event-based webservers

  1. 1. Scaling with event– based webservers Morten Siebuhr Open Source Days March 5th 20111/66
  2. 2. whoami(1) Defining “web–server work” Killing Apache Event–based servers Using Node.js Questions2/66
  3. 3. whoami(1)3/66
  4. 4. Computer Scientist Distributed Systems / Scientific Computing (Web–)developer @ One.com4/66
  5. 5. Web–development @ one.com: Rich Internet Applications e–mail, calendar, galleries, . . . + 900.000 customers5/66
  6. 6. 6/66
  7. 7. Defining “Web–server work”7/66
  8. 8. Typical web–server work8/66
  9. 9. Typical web–server work Serving static files8/66
  10. 10. Typical web–server work Serving static files Talking to databases8/66
  11. 11. Typical web–server work Serving static files Talking to databases Not doing lots of computations8/66
  12. 12. Typical web–server work Serving static files Talking to databases Not doing lots of computations ≈ 100 → 1000 connections served ASAP8/66
  13. 13. (A)typical web–server work9/66
  14. 14. (A)typical web–server work Persistent connections9/66
  15. 15. (A)typical web–server work Persistent connections = ∞ simultaneous connections9/66
  16. 16. (A)typical web–server work Persistent connections = ∞ simultaneous connections ≈ 10000 simultaneous connections9/66
  17. 17. (A)typical web–server work Persistent connections = ∞ simultaneous connections ≈ 10000 simultaneous connections (But we don’t care as much about latency. . . )9/66
  18. 18. waiting...10/66
  19. 19. sleep(1.0)11/66
  20. 20. 12/66
  21. 21. Response time w. sleep(1). 25 s WSGI 20 s Average response time 15 s 10 s 5s 0s 0 100 200 300 400 500 600 700 800 900 1000 Concurrent requests13/66
  22. 22. Response time w. sleep(1). 25 s WSGI 20 s Average response time 15 s 10 s 5s 0s 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests14/66
  23. 23. Threading15/66
  24. 24. Use threads16/66
  25. 25. Wasted resources!17/66
  26. 26. Add threads18/66
  27. 27. Response time w. sleep(1). 25 s WSGI Apache 20 s Average response time 15 s 10 s 5s 0s 0 100 200 300 400 500 600 700 800 900 1000 Concurrent requests19/66
  28. 28. Response time w. sleep(1). 25 s WSGI Apache 20 s Average response time 15 s 10 s 5s 0s 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests20/66
  29. 29. 21/66
  30. 30. 22/66
  31. 31. 23/66
  32. 32. Memory usage w. sleep(1). 1500 MB WSGI Apache 1250 MB 1000 MB Max Virtual Memory 750 MB 500 MB 250 MB 0 MB 0 100 200 300 400 500 600 700 800 900 1000 Concurrent requests25/66
  33. 33. Memory usage w. sleep(1). 1500 MB WSGI Apache 1250 MB 1000 MB Max Virtual Memory 750 MB 500 MB 250 MB 0 MB 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests26/66
  34. 34. A single connection27/66
  35. 35. A single connection = 1 thread27/66
  36. 36. A single connection = 1 thread = 1 C–stack & kernel data structures27/66
  37. 37. A single connection = 1 thread = 1 C–stack & kernel data structures + Locking & switching overhead &c. . .27/66
  38. 38. 28/66
  39. 39. Add memory!29/66
  40. 40. Add memory! Memory is cheap29/66
  41. 41. Add memory! Memory is cheap Except, actually using it isn’t cheap. . .29/66
  42. 42. Memory Wall 1986 → 2000:30/66
  43. 43. Memory Wall 1986 → 2000: + 55% CPU speed P/A30/66
  44. 44. Memory Wall 1986 → 2000: + 55% CPU speed P/A + 10% RAM speed P/A (Source: http://www.cs.virginia.edu/papers/Hitting_Memory_Wall-wulf94.pdf)30/66
  45. 45. 31/66
  46. 46. Threads and connections mapped 1:132/66
  47. 47. Threads and connections mapped 1:1 We use a lot of threads32/66
  48. 48. Threads and connections mapped 1:1 We use a lot of threads Which we don’t have the memory bandwidth to move around.32/66
  49. 49. Events!33/66
  50. 50. waiting...34/66
  51. 51. 35/66
  52. 52. 36/66
  53. 53. Event-basics37/66
  54. 54. Event-basics Event queue37/66
  55. 55. Event-basics Event queue Event loop37/66
  56. 56. Event-basics Event queue Event loop “The Event Loop”37/66
  57. 57. A single connection38/66
  58. 58. A single connection = 1 “Event”38/66
  59. 59. A single connection = 1 “Event” = Some variables & function38/66
  60. 60. A single connection = 1 “Event” = Some variables & function = Hash table & some pointers38/66
  61. 61. A single connection = 1 “Event” = Some variables & function = Hash table & some pointers + Queue of events waiting to be processed38/66
  62. 62. 39/66
  63. 63. 39/66
  64. 64. 39/66
  65. 65. 39/66
  66. 66. Why not earlier40/66
  67. 67. Why not earlier Simply not a problem!40/66
  68. 68. Why not earlier Simply not a problem! Traditional back–end programmers not used to thinking this way40/66
  69. 69. What we gain41/66
  70. 70. What we gain Relatively low memory use41/66
  71. 71. What we gain Relatively low memory use No threading issues41/66
  72. 72. What we gain Relatively low memory use No threading issues Client-side programmers have no preconditions.41/66
  73. 73. Node.js42/66
  74. 74. Node.js is a set of bindings to the V8 JavaScript engine for scripting network programs. — Ryan Dahl43/66
  75. 75. Node.js is a set of bindings to the V8 JavaScript engine for scripting network programs. — Ryan Dahl43/66
  76. 76. JavaScript44/66
  77. 77. JavaScript Built for event-based programming.44/66
  78. 78. JavaScript Built for event-based programming.44/66
  79. 79. JavaScript Built for event-based programming. Netscape needed something fast. . .44/66
  80. 80. JavaScript Built for event-based programming. Netscape needed something fast. . . . . . JavaScript “designed” and implemented in 14 days44/66
  81. 81. JavaScript Built for event-based programming. Netscape needed something fast. . . . . . JavaScript “designed” and implemented in 14 days44/66
  82. 82. Google V8 JIT’ing JavaScript to native machine code45/66
  83. 83. Google V8 JIT’ing JavaScript to native machine code = Compiling server code45/66
  84. 84. Node.js Non–browser V8–bindings for various libraries Everything non–blocking46/66
  85. 85. A minimal webserver in Node.js. . . // Import HTTP package var http = require ( ’ http ’) ; // Set up basic server var server = http . createServer ( function ( req , res ) { res . writeHead (200 , { ’ Content - Type ’: ’ text / plain ’ }) ; res . end ( ’ Hello World n ’) ; }) ; // Start the server server . listen (8124 , " 127.0.0.1 " ) ; console . log ( ’ Server running at http : / / 1 2 7 . 0 . 0 . 1 : 8 1 2 4 / ’) ;47/66
  86. 86. . . . with “database” // Import HTTP package var http = require ( ’ http ’) ; // Set up basic server var server = http . createServer ( function ( req , res ) { setTimeout ( function () { res . writeHead (200 , { ’ Content - Type ’: ’ text / plain ’ }) ; res . end ( ’ Hello World n ’) ; } , 1000) ; }) ; // Start the server server . listen (8124 , " 127.0.0.1 " ) ; console . log ( ’ Server running at http : / / 1 2 7 . 0 . 0 . 1 : 8 1 2 4 / ’) ;48/66
  87. 87. Response time w. sleep(1). 25 s WSGI Apache Node.js 20 s Average response time 15 s 10 s 5s 0s 0 100 200 300 400 500 600 700 800 900 1000 Concurrent requests49/66
  88. 88. Response time w. sleep(1). 25 s WSGI Apache Node.js 20 s Average response time 15 s 10 s 5s 0s 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests50/66
  89. 89. Memory usage w. sleep(1). 1500 MB WSGI Apache Node.js 1250 MB 1000 MB Max Virtual Memory 750 MB 500 MB 250 MB 0 MB 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Concurrent requests51/66
  90. 90. Connections in Node.js52/66
  91. 91. Connections in Node.js = 1 “Event”52/66
  92. 92. Connections in Node.js = 1 “Event” = 1 Closure + callback function52/66
  93. 93. Connections in Node.js = 1 “Event” = 1 Closure + callback function = 1 struct + some pointers52/66
  94. 94. Connections in Node.js = 1 “Event” = 1 Closure + callback function = 1 struct + some pointers + Event queue operations + function calls52/66
  95. 95. Alternatives53/66
  96. 96. Alternatives Well–written thread program!53/66
  97. 97. Alternatives Well–written thread program! Erlang http://www.erlang.org/53/66
  98. 98. Alternatives Well–written thread program! Erlang http://www.erlang.org/ Google GO http://golang.org/53/66
  99. 99. Alternatives Well–written thread program! Erlang http://www.erlang.org/ Google GO http://golang.org/ Lua Event Machine https://github.com/esmil/lem53/66
  100. 100. Alternatives Well–written thread program! Erlang http://www.erlang.org/ Google GO http://golang.org/ Lua Event Machine https://github.com/esmil/lem Nginx http://wiki.nginx.org/53/66
  101. 101. Alternatives Well–written thread program! Erlang http://www.erlang.org/ Google GO http://golang.org/ Lua Event Machine https://github.com/esmil/lem Nginx http://wiki.nginx.org/ Varnish http://www.varnish-cache.org/53/66
  102. 102. Working with Node.js54/66
  103. 103. I’m new to JavaScript!55/66
  104. 104. 56/66
  105. 105. JavaScript WTFs Always add ;! Function arguments. . . this scoped to callee object! Stacktraces + events sometimes look strange57/66
  106. 106. JSLint58/66
  107. 107. JSLint www.crockfordfacts.com58/66
  108. 108. Nice code! def process ( user ) : user = getAuth ( user ) if not user : return ’ Fail ’ db = getC onnnecti on ( params ) data = db . getUserData ( user ) if not data : return ’ Fail ’ return data59/66
  109. 109. Boomerang code function process ( user , callback ) { getAuth ( req . username , function ( err , user ) { if ( err ) return callback ( err ) ; g etDB Co n ne ct io n ( params , function ( err , db ) { if ( err ) return callback ( err ) ; db . getUserData ( user , function ( err , data ) { if ( err ) return callback ( err ) ; return callback ( null , data ) ; }) ; }) ; }) ; }60/66
  110. 110. NPM61/66
  111. 111. Express.js var app = require ( ’ express ’) . createServer () ; app . get ( ’/ ’ , function ( req , res ) { setTimeout ( function () { res . send ( ’ Hello World ’) ; } , 1000) ; }) ; app . listen (3000) ;62/66
  112. 112. Express.js - Chat server var app = require ( ’ express ’) . createServer () , chat = []; app . get ( ’/ ’ , function ( req , res ) { chat . push ( res ) ; }) ; app . get ( ’ /: msg ’ , function ( req , res ) { chat . forEach ( function ( conn ) { conn . write ( req . params . msg ) ; }) ; }) ; app . listen (3000) ;63/66
  113. 113. Questions (The end)64/66
  114. 114. http://nodejs.org/ https://github.com/joyent/node65/66
  115. 115. function process ( user , callback ) { var user = null , db = null ; function done_ () { if ( user && db ) { db . getUserData ( user , function ( err , data ) { if ( err ) return callback ( err ) ; return callback ( null , data ) ; }) ; } } getAuth ( req . username , function ( err , retuser ) { if ( err ) return callback ( err ) ; user = retuser ; done_ () ; }) ; g et DBCo n ne ct i on ( params , function ( err , retdb ) { if ( err ) return callback ( err ) ; db = retdb ; done_ () ; }) ; }66/66

×