PLANNING FOR THE                                  HORIZONTAL                                     SCALING NODE.JS APPLICATI...
ME                        HOSTING, DATA, STATS FOR NODE.JS                                   modulus.ioThursday, April 4, 13
WHEN TO SCALE?                        1. RESPONSE TIMES                        2. CPU USAGE                        3. CONC...
NODEFLY.COMThursday, April 4, 13
STARTING POINT                           mydomain.com                               > NODE                              SE...
NODE TWEAKS                                     CONCURRENT OUTGOING                                       CONNECTION LIMIT...
HURTLE: THE SERVER                        LINUX CONFIGURATION                          1. FILE-MAX                        ...
FILE-MAX                        SYSTEM FILE DESCRIPTOR LIMIT                        1. Run sysctl -w fs.file-max=65535    ...
SOMAXCONN                             SOCKET LISTEN QUEUE LENGTH                        1. Run sysctl -w net.core.somaxcon...
ULIMIT                        PER PROCESS FILE DESCRIPTOR LIMIT                          1. Edit /etc/security/limits.conf...
RUNNING SMOOTH                            mydomain.com                                > NODE                              ...
HURTLE: THE CPU                           BUY A BIGGER BOX                         > NODE         > NODE                  ...
MULTICORE NODE                           100%                        USAGE                                  1   2      3  ...
CLUSTER MODULE                           mydomain.com                            > NODE    > NODE                         ...
CLUSTER EXAMPLE             var cluster = require(cluster);            The Cluster Module             var http = require(h...
CLUSTER EXAMPLE             var cluster = require(cluster);             var http = require(http);             var numCPUs ...
CLUSTER EXAMPLE             var cluster = require(cluster);             var http = require(http);             var numCPUs ...
CLUSTER LISTEN                                 listen(...)                        WORKER                 MASTER           ...
ROLLING UPDATES                        1. UPDATE SCRIPT                        2. WORKER -> STOP LISTENING                ...
CLUSTER MODULE                           mydomain.com                            > NODE    > NODE                         ...
HURTLE: SHARED STATE                                          > NODE    > NODE                        NO SHARED STATE     ...
INSTALL REDIS                           > NODE    > NODE                           > NODE    > NODE                       ...
EXAMPLE 1: SESSION                                     MEMORY STORE                          var express = require(express...
EXAMPLE 1: SESSION                                               REDIS STORE                        var express = require(...
EXAMPLE 2: SOCKET.IO                        var   RedisStore = require(socket.io/lib/stores/redis)                        ...
RUNNING SMOOTH                            mydomain.com                            > NODE    > NODE                        ...
LAST HURTLE: HORIZONTAL                           > NODE    > NODE   > NODE    > NODE                           > NODE    ...
SEPARATE REDIS                        > NODE    > NODE            > NODE    > NODE                        > NODE    > NODE...
LOAD BALANCING                   mydomain.com              LOAD BALANCER                                                  ...
LOAD BALANCING                         1. MANAGED                         2. INSTALL ONE                         3. WRITE ...
NGINX                        http {                          upstream mydomain_com {                            server hos...
WRITE ONE                        https://github.com/substack/bouncyThursday, April 4, 13
BOUNCY              var bouncy = require(bouncy);                    bouncy module              var hosts = [             ...
BOUNCY              var bouncy = require(bouncy);              var hosts = [                                      Server c...
BOUNCY              var bouncy = require(bouncy);              var hosts = [                 host1.mydomain.com,          ...
BOUNCY              var bouncy = require(bouncy);              var hosts = [                 host1.mydomain.com,          ...
AFFINITY                        SESSION AFFINITY                        STICKY SESSIONS                          SEND THE ...
NGINX AFFINITY                        http {                          upstream mydomain_com {                            s...
CUSTOM AFFINITY                        req.headers[x-forwarded-for]                        req.connection.remoteAddressThu...
RUNNING SMOOTH                   mydomain.com              LOAD BALANCER                                                  ...
ROLLING UPDATES                    1. REMOVE APP SERVER FROM LOAD BALANCER                    2. UPGRADE APP SERVER       ...
SSL                        TERMINATE EARLYThursday, April 4, 13
SSL                                    LB               SSL                SSL TERMINATOR                                 ...
SSL                        mydomain.com                              80   443                         LB             SSL  ...
STUD                           EXAMPLE CONFIG FILE                        frontend = [*]:443                        backen...
RUNNING SMOOTH W/SSL                   mydomain.com               LB               SSL                                    ...
HUGE                        LB            SSL            LB            SSL                             SERVER             ...
DNS                        ROUND-ROBIN DNS                         MULTIPLE RECORDS,                           ONE DOMAINT...
ROUND-ROBIN DNS                        CLIENT 1   1. xxx.xxx.xxx.x                                   2. xxx.xxx.xxx.y     ...
RUNNING SMOOTH                          LB            SSL            LB            SSL                               SERVE...
BIG ENOUGH                        > NODE                        SERVERThursday, April 4, 13
BIG ENOUGH                        > NODE                        SERVERThursday, April 4, 13
Upcoming SlideShare
Loading in...5
×

Planning for the Horizontal: Scaling Node.js Applications

3,223

Published on

Slides from Brandon Cannaday's talk at HTML5 Developer Conference on how to horizontally scale Node.js applications. Covers everything from overview of architecture to specific linux settings.

Published in: Technology

Planning for the Horizontal: Scaling Node.js Applications

  1. 1. PLANNING FOR THE HORIZONTAL SCALING NODE.JS APPLICATIONS TheReddest Brandon Cannaday brandon@modulus.ioThursday, April 4, 13
  2. 2. ME HOSTING, DATA, STATS FOR NODE.JS modulus.ioThursday, April 4, 13
  3. 3. WHEN TO SCALE? 1. RESPONSE TIMES 2. CPU USAGE 3. CONCURRENT CONNECTIONSThursday, April 4, 13
  4. 4. NODEFLY.COMThursday, April 4, 13
  5. 5. STARTING POINT mydomain.com > NODE SERVERThursday, April 4, 13
  6. 6. NODE TWEAKS CONCURRENT OUTGOING CONNECTION LIMIT http.globalAgent.maxSockets = Number.MAX_VALUE;Thursday, April 4, 13
  7. 7. HURTLE: THE SERVER LINUX CONFIGURATION 1. FILE-MAX 2. SOMAXCONN 3. ULIMITThursday, April 4, 13
  8. 8. FILE-MAX SYSTEM FILE DESCRIPTOR LIMIT 1. Run sysctl -w fs.file-max=65535 2. Run sysctl -pThursday, April 4, 13
  9. 9. SOMAXCONN SOCKET LISTEN QUEUE LENGTH 1. Run sysctl -w net.core.somaxconn=65535 2. Run sysctl -pThursday, April 4, 13
  10. 10. ULIMIT PER PROCESS FILE DESCRIPTOR LIMIT 1. Edit /etc/security/limits.conf 2. Add the following: * soft nofile 65535 * hard nofile 65535 root soft nofile 65535 root hard nofile 65535Thursday, April 4, 13
  11. 11. RUNNING SMOOTH mydomain.com > NODE SERVERThursday, April 4, 13
  12. 12. HURTLE: THE CPU BUY A BIGGER BOX > NODE > NODE SERVER SERVER 1 CORE 4 CORESThursday, April 4, 13
  13. 13. MULTICORE NODE 100% USAGE 1 2 3 4 COREThursday, April 4, 13
  14. 14. CLUSTER MODULE mydomain.com > NODE > NODE > NODE > NODE SERVERThursday, April 4, 13
  15. 15. CLUSTER EXAMPLE var cluster = require(cluster); The Cluster Module var http = require(http); var numCPUs = require(os).cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end(Hello World.); }).listen(80); }Thursday, April 4, 13
  16. 16. CLUSTER EXAMPLE var cluster = require(cluster); var http = require(http); var numCPUs = require(os).cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); Fork Children } } else { http.createServer(function(req, res) { res.writeHead(200); res.end(Hello World.); }).listen(80); }Thursday, April 4, 13
  17. 17. CLUSTER EXAMPLE var cluster = require(cluster); var http = require(http); var numCPUs = require(os).cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { Handle Requests res.writeHead(200); res.end(Hello World.); }).listen(80); }Thursday, April 4, 13
  18. 18. CLUSTER LISTEN listen(...) WORKER MASTER HandleThursday, April 4, 13
  19. 19. ROLLING UPDATES 1. UPDATE SCRIPT 2. WORKER -> STOP LISTENING 3. KILL WORKER 4. CALL FORK() AGAINThursday, April 4, 13
  20. 20. CLUSTER MODULE mydomain.com > NODE > NODE > NODE > NODE SERVERThursday, April 4, 13
  21. 21. HURTLE: SHARED STATE > NODE > NODE NO SHARED STATE > NODE > NODE SERVERThursday, April 4, 13
  22. 22. INSTALL REDIS > NODE > NODE > NODE > NODE REDIS SERVERThursday, April 4, 13
  23. 23. EXAMPLE 1: SESSION MEMORY STORE var express = require(express), app = express(); app.use(express.cookieParser()); app.use(express.session({ secret: My Cookie Signing Secret })); app.get(/, function(req, res) { req.session.somekey = some value; });Thursday, April 4, 13
  24. 24. EXAMPLE 1: SESSION REDIS STORE var express = require(express), RedisStore = require(connect-redis)(express), app = express(); app.use(express.cookieParser()); app.use(express.session({ store: new RedisStore({ host: localhost, port: 6379 }), secret: My Cookie Signing Secret })); app.get(/, function(req, res) { req.session.somekey = some value; });Thursday, April 4, 13
  25. 25. EXAMPLE 2: SOCKET.IO var RedisStore = require(socket.io/lib/stores/redis) , redis = require(socket.io/node_modules/redis) , pub = redis.createClient() , sub = redis.createClient() , client = redis.createClient(); io.set(store, new RedisStore({ redisPub : pub , redisSub : sub , redisClient : client })); https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IOThursday, April 4, 13
  26. 26. RUNNING SMOOTH mydomain.com > NODE > NODE > NODE > NODE REDIS SERVERThursday, April 4, 13
  27. 27. LAST HURTLE: HORIZONTAL > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE REDIS REDIS APP SERVER A APP SERVER BThursday, April 4, 13
  28. 28. SEPARATE REDIS > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVERThursday, April 4, 13
  29. 29. LOAD BALANCING mydomain.com LOAD BALANCER SERVER > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVERThursday, April 4, 13
  30. 30. LOAD BALANCING 1. MANAGED 2. INSTALL ONE 3. WRITE YOUR OWNThursday, April 4, 13
  31. 31. NGINX http { upstream mydomain_com { server host1.mydomain.com:80; server host2.mydomain.com:80; } LOAD BALANCER server { SERVER listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } }Thursday, April 4, 13
  32. 32. WRITE ONE https://github.com/substack/bouncyThursday, April 4, 13
  33. 33. BOUNCY var bouncy = require(bouncy); bouncy module var hosts = [ host1.mydomain.com, host2.mydomain.com ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80);Thursday, April 4, 13
  34. 34. BOUNCY var bouncy = require(bouncy); var hosts = [ Server collection host1.mydomain.com, host2.mydomain.com ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80);Thursday, April 4, 13
  35. 35. BOUNCY var bouncy = require(bouncy); var hosts = [ host1.mydomain.com, host2.mydomain.com ]; var count = 0; var server = bouncy(function(req, res, bounce) { Create server count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80);Thursday, April 4, 13
  36. 36. BOUNCY var bouncy = require(bouncy); var hosts = [ host1.mydomain.com, host2.mydomain.com ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); Bounce request }); server.listen(80);Thursday, April 4, 13
  37. 37. AFFINITY SESSION AFFINITY STICKY SESSIONS SEND THE SAME PERSON BACK TO THE SAME SERVERThursday, April 4, 13
  38. 38. NGINX AFFINITY http { upstream mydomain_com { sticky; server host1.mydomain.com:80; server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } }Thursday, April 4, 13
  39. 39. CUSTOM AFFINITY req.headers[x-forwarded-for] req.connection.remoteAddressThursday, April 4, 13
  40. 40. RUNNING SMOOTH mydomain.com LOAD BALANCER SERVER > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVERThursday, April 4, 13
  41. 41. ROLLING UPDATES 1. REMOVE APP SERVER FROM LOAD BALANCER 2. UPGRADE APP SERVER 3. ADD BACK 4. REPEATThursday, April 4, 13
  42. 42. SSL TERMINATE EARLYThursday, April 4, 13
  43. 43. SSL LB SSL SSL TERMINATOR SERVER > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVERThursday, April 4, 13
  44. 44. SSL mydomain.com 80 443 LB SSL SERVERThursday, April 4, 13
  45. 45. STUD EXAMPLE CONFIG FILE frontend = [*]:443 backend = [127.0.0.1]:80 ssl = on pem-file = "myCert.pem" https://github.com/bumptech/studThursday, April 4, 13
  46. 46. RUNNING SMOOTH W/SSL mydomain.com LB SSL SERVER > NODE > NODE > NODE > NODE > NODE > NODE > NODE > NODE APP SERVER A APP SERVER B REDIS SERVERThursday, April 4, 13
  47. 47. HUGE LB SSL LB SSL SERVER SERVER REDIS SERVERThursday, April 4, 13
  48. 48. DNS ROUND-ROBIN DNS MULTIPLE RECORDS, ONE DOMAINThursday, April 4, 13
  49. 49. ROUND-ROBIN DNS CLIENT 1 1. xxx.xxx.xxx.x 2. xxx.xxx.xxx.y CLIENT 2 1. xxx.xxx.xxx.y 2. xxx.xxx.xxx.xThursday, April 4, 13
  50. 50. RUNNING SMOOTH LB SSL LB SSL SERVER SERVER REDIS SERVERThursday, April 4, 13
  51. 51. BIG ENOUGH > NODE SERVERThursday, April 4, 13
  52. 52. BIG ENOUGH > NODE SERVERThursday, April 4, 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×