Planning for the Horizontal: Scaling Node.js Applications

PLANNING FOR THE
HORIZONTAL
SCALING NODE.JS APPLICATIONS

TheReddest Brandon Cannaday brandon@modulus.io

Thursday, April 4, 13

ME

HOSTING, DATA, STATS FOR NODE.JS

modulus.io


WHEN TO SCALE?

1. RESPONSE TIMES
2. CPU USAGE
3. CONCURRENT CONNECTIONS


NODEFLY.COM


STARTING POINT
mydomain.com

> NODE

SERVER


NODE TWEAKS

CONCURRENT OUTGOING
CONNECTION LIMIT

http.globalAgent.maxSockets = Number.MAX_VALUE;


HURTLE: THE SERVER

LINUX CONFIGURATION
1. FILE-MAX
2. SOMAXCONN
3. ULIMIT


FILE-MAX

SYSTEM FILE DESCRIPTOR LIMIT

1. Run sysctl -w fs.file-max=65535
2. Run sysctl -p


SOMAXCONN

SOCKET LISTEN QUEUE LENGTH

1. Run sysctl -w net.core.somaxconn=65535
2. Run sysctl -p


ULIMIT
PER PROCESS FILE DESCRIPTOR LIMIT
1. Edit /etc/security/limits.conf
2. Add the following:
* soft nofile 65535
* hard nofile 65535
root soft nofile 65535
root hard nofile 65535

RUNNING SMOOTH

mydomain.com

> NODE

SERVER


HURTLE: THE CPU

BUY A BIGGER BOX

> NODE > NODE

SERVER
SERVER
1 CORE
4 CORES


MULTICORE NODE

100%

USAGE

1 2 3 4
CORE


CLUSTER MODULE
mydomain.com

> NODE > NODE

> NODE > NODE

SERVER


CLUSTER EXAMPLE

var cluster = require('cluster'); The Cluster Module
var http = require('http');
var numCPUs = require('os').cpus().length;

if(cluster.isMaster) {
for(var i = 0; i < numCPUs; i++) {
cluster.fork();
}
}
else {
http.createServer(function(req, res) {
res.writeHead(200);
res.end('Hello World.');
}).listen(80);
}


CLUSTER EXAMPLE

var cluster = require('cluster');

cluster.fork(); Fork Children
}
}
else {
http.createServer(function(req, res) {
res.writeHead(200);
}).listen(80);
}


CLUSTER EXAMPLE

var cluster = require('cluster');

cluster.fork();
}
}
else {
http.createServer(function(req, res) { Handle Requests
res.writeHead(200);
}).listen(80);
}


CLUSTER LISTEN

listen(...)

WORKER MASTER

Handle


ROLLING UPDATES

1. UPDATE SCRIPT
2. WORKER -> STOP LISTENING
3. KILL WORKER
4. CALL FORK() AGAIN


HURTLE: SHARED STATE

> NODE > NODE

NO SHARED STATE
> NODE > NODE

SERVER


INSTALL REDIS

> NODE > NODE

> NODE > NODE

REDIS

SERVER


EXAMPLE 1: SESSION
MEMORY STORE

var express = require('express'),
app = express();

app.use(express.cookieParser());
app.use(express.session({
secret: 'My Cookie Signing Secret'
}));

app.get('/', function(req, res) {
req.session.somekey = 'some value';
});


EXAMPLE 1: SESSION
REDIS STORE

var express = require('express'),
RedisStore = require('connect-redis')(express),
app = express();

app.use(express.cookieParser());
app.use(express.session({
store: new RedisStore({ host: 'localhost', port: 6379 }),
secret: 'My Cookie Signing Secret'
}));

app.get('/', function(req, res) {
req.session.somekey = 'some value';
});


EXAMPLE 2: SOCKET.IO

var RedisStore = require('socket.io/lib/stores/redis')
, redis = require('socket.io/node_modules/redis')
, pub = redis.createClient()
, sub = redis.createClient()
, client = redis.createClient();

io.set('store', new RedisStore({
redisPub : pub
, redisSub : sub
, redisClient : client
}));

https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO


RUNNING SMOOTH
mydomain.com

> NODE > NODE

> NODE > NODE

REDIS

SERVER


LAST HURTLE: HORIZONTAL

> NODE > NODE > NODE > NODE


REDIS REDIS

APP SERVER A APP SERVER B


SEPARATE REDIS




REDIS

SERVER


LOAD BALANCING

mydomain.com LOAD BALANCER

SERVER




REDIS

SERVER


LOAD BALANCING

1. MANAGED
2. INSTALL ONE
3. WRITE YOUR OWN


NGINX

http {
upstream mydomain_com {
server host1.mydomain.com:80;
} LOAD BALANCER

server {
SERVER
listen 80;
server_name www.mydomain.com;
location / {
proxy_pass http://mydomain_com;
}
}
}


WRITE ONE

https://github.com/substack/bouncy


BOUNCY
var bouncy = require('bouncy'); bouncy module
var hosts = [
'host1.mydomain.com',
'host2.mydomain.com'
];

var count = 0;

var server = bouncy(function(req, res, bounce) {

count++;
var host = hosts[count % hosts.length];

bounce(host, 80);

});

server.listen(80);


BOUNCY
var bouncy = require('bouncy');

var hosts = [ Server collection
];

var count = 0;


count++;

bounce(host, 80);

});

server.listen(80);


BOUNCY

var hosts = [
];

var count = 0;

var server = bouncy(function(req, res, bounce) { Create server
count++;

bounce(host, 80);

});

server.listen(80);


BOUNCY

var hosts = [
];

var count = 0;


count++;

bounce(host, 80); Bounce request
});

server.listen(80);


AFFINITY

SESSION AFFINITY
STICKY SESSIONS
SEND THE SAME PERSON
BACK TO THE SAME SERVER


NGINX AFFINITY

http {
upstream mydomain_com {
sticky;
}

server {
listen 80;
server_name www.mydomain.com;
location / {
proxy_pass http://mydomain_com;
}
}
}


CUSTOM AFFINITY

req.headers['x-forwarded-for']
req.connection.remoteAddress


RUNNING SMOOTH

mydomain.com LOAD BALANCER

SERVER




REDIS

SERVER


ROLLING UPDATES

1. REMOVE APP SERVER FROM LOAD BALANCER
2. UPGRADE APP SERVER
3. ADD BACK
4. REPEAT


SSL

TERMINATE EARLY


SSL

LB SSL SSL TERMINATOR

SERVER




REDIS

SERVER


SSL

mydomain.com
80 443

LB SSL

SERVER


STUD
EXAMPLE CONFIG FILE

frontend = [*]:443
backend = [127.0.0.1]:80
ssl = on
pem-file = "myCert.pem"

https://github.com/bumptech/stud


RUNNING SMOOTH W/SSL

mydomain.com LB SSL

SERVER




REDIS

SERVER


HUGE

LB SSL LB SSL

SERVER SERVER

REDIS

SERVER


DNS

ROUND-ROBIN DNS
MULTIPLE RECORDS,
ONE DOMAIN


ROUND-ROBIN DNS

CLIENT 1 1. xxx.xxx.xxx.x
2. xxx.xxx.xxx.y

CLIENT 2 1. xxx.xxx.xxx.y
2. xxx.xxx.xxx.x


RUNNING SMOOTH

LB SSL LB SSL

SERVER SERVER

REDIS

SERVER


BIG ENOUGH

> NODE

SERVER


Planning for the Horizontal: Scaling Node.js Applications

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (15)

Similar to Planning for the Horizontal: Scaling Node.js Applications

Similar to Planning for the Horizontal: Scaling Node.js Applications (7)

Recently uploaded

Recently uploaded (20)

Planning for the Horizontal: Scaling Node.js Applications