1. Multi-tiered Node.js
Architectures
Tom Hughes-Croucher
@sh1mmer
tom@joyent.com

2. Scalable Server-Side Code with JavaScript
Node Up and Running
Tom Hughes-Croucher

3. Goal 1.
Minimise client
response time

5. Goal 2.
Maximise resource
efficiency on server

7. Client → Server Server → DB
Computation Computation

8. Pre-allocate resources
to the client for the
entire duration

9. Get efficiency within
each connection

10. Pre-allocation = sacrificing
resource efficiency for
connection response times

12. Server

13. Request

15. In order to serve the
fastest request memory
is heavily wasted

16. When you run out of
memory the server
gets overwhelmed

17. And that sucks.

19. Don’t pre-allocate
memory

20. Create a place-holder
for ‘slow’ tasks

21. Place-
holder

23. Shared
Work
Resources

24. Deal with tasks when
they are ready in a
FIFO way

25. App
(event loop)
App
Events
Event
OS Stack
Events

26. If we block the event
loop then we can’t
serve requests

27. If we do “big” work in the event
loop then we can only serve
requests after the work is
finished

28. Blocking the main event
loop is bad. Mmmkay.

29. Event-loop harassment Pan-da

30. Multi-tiered
Architectures

31. The obvious
architecture

32. Client Front-end Database
User Node DB

33. What about this?

34. Front-end JSDom
Client Database
Farm Render Farm
User Node
Node Node
Node DB
Node
Node Node
Node

35. Why an architecture
like this?

36. Parallelisation?

37. Client → Server Server → DB
Computation Computation

38. Non-blocking I/O
removes resource
allocation during latency

39. Pushing client-facing processing
off the main process does not
improve response time

40. Better multi-tier
architectures

41. Front-end
Client Database
Farm
DB
User Node
Node
Node
Node Big Slow
Logging farm
Disk
Node
Node Disk

42. Move work which isn’t
user-facing out of main
event loop

43. Use pre-forking to
distribute work across
processors into ‘farms’

44. Distributing work

45. Cluster

46. children.js:
var server = require('http')
.createServer(
function(req, res) {
res.writeHead(200, {});
res.end("Sever stuff");
});

47. Run

48. var cluster = require('cluster');
cluster('children').listen(80);

49. Use load balancers to
distribute across
servers

50. node-proxy
squid
varnish
haproxy

51. Sharding

52. Disclaimer:
Sharding is hard

53. Node processes can
hold many clients so
you can do big shards

54. Inter-process
communication on
machine

55. Use file descriptor
sharing to do on
machine multicast

56. Load Front-end
Client Database
balancers Shards
Node
Node
Node
Node
User Node
Node DB
Node
Node
Node
Node
Node
Node

57. Unix FDs have much
lower unbounded
latency than network

58. Summary

59. 1. Don’t sweat client facing
CPU time
2. Move non client-facing away
3. Cluster to use all processors

60. Fin.
(You should follow me,
@sh1mmer
on Twitter. Like now!)

61. Bonus