Node.js in action presented at Mongo Dilli meetup

1. node.js in action
Learnings from Cisco
Karan & Sneha
ThoughtWorks
Saturday, 24 August 13

2. node.js ?
• JavaScript is not jQuery
• Created by Ryan Dahl utilizing the Google
V8 JavaScript engine
• Has evented I/O as part of its arsenal
Saturday, 24 August 13

3. Tech Stack
• Express (web framework)
• Sequelize (ORM)
• Gruntjs (task runner)
• coffee-resque (background jobs and queues)
• Q (callback sanity)
• Mocha/Sinon/Chai (TDD)
Saturday, 24 August 13

4. Callback Hell!
• Avoid like the plague
• Can suck all the fun out of writing
JavaScript (everywhere)
• Waiting for returns across logical
boundaries is almost impossible
• Senthil/Kuldeep cover this area in a lot
more detail right after this talk
Saturday, 24 August 13

5. Example
Saturday, 24 August 13

6. TDD in JS?!?
• With Q/promises, there is no reason not to
• We used Mocha/Sinon/Chai(and their -as-
promised brothers) to be able to assert on
methods returning promises
• This converts each of your tests into
promises which either resolve to success
or failure
Saturday, 24 August 13

7. Example
Saturday, 24 August 13

8. socket.io
• Library in a state of constant flux
• Was a major cause of performance slow down in
our application
• First we make a normal GET request to get a
handshake token
• The second request using the token then gets
upgraded to a full duplex conn
• Had to integrate with a (k,v) store like Redis to help
coordinate the creation of the handshake tokens
Saturday, 24 August 13

9. Performance
• Double-edged sword:
• node.js comes with a promise of speed
• the promised land is full of land mines
which can literally blow you away if you
don’t hit dead center the first time around
• Gatling proved a very capable companion in
figuring out the bottlenecks
Saturday, 24 August 13

10. Performance contd.
• The JS native MySQL driver turned out to be the
sore point
• Arriving at this conclusion took a lot of doing:
• newrelic didn’t help
• nodetime didn’t help
• running around wildly didn’t help
• Intuition strikes again: Performance was inversely
proportional to the number of round trips to MySQL
Saturday, 24 August 13

11. Background Jobs
• Anything which doesn’t belong in your web
flow, should be scaled separately
• A simple Resqueisque background jobs
queue solved the problem for us
• Redis again chimed in with its queue data
structure
Saturday, 24 August 13

12. Example
Saturday, 24 August 13

13. Clustering
• Utilizing all the available CPU resource
from within a single node.js application is
hard
• Eventually, you will need to fire up all the
cylinders
• The “cluster” module helps out grandly in
this regard; however it must be done right
to ensure a stable production environment
Saturday, 24 August 13

14. Example
Saturday, 24 August 13

15. Deployment
• Source based deployment is easy to do, but hard to get right
• “npm install” can bring in different versions of your
dependencies if you have sliding dependencies in your
package.json file
• Have to have a full compiler suite to handle native
extensions (e.g. libmysqlclient)
• Doing a node.js upgrade can silently fail if you do not “rm -
rf” node_modules first
• Doing a RPM (name your favorite packager) based setup
might not be such a bad idea afterall
Saturday, 24 August 13

16. Deployment contd.
• Things we take for granted in the Rails
world is not so simple here
• Had to write our own asset-combiner to
get around shoddy implementation in
connect-assetmanager
• A must if the site is going live with a CDN
Saturday, 24 August 13

17. Monitoring
• Clustering (which splits your application
into master/worker processes) makes it
difficult to keep the application up
• We finally ended up using “mon” (a highly
simplified cmd line based monit) as the
usual techniques (forever, etc.) were not
playing ball
Saturday, 24 August 13

18. Conclusion
• It was a rewarding experience
• Not a magic bullet
• Avoiding callback hell helped keep our
sanity
• Be prepared to spend sleepless nights
“when” the performance bugs hit
Saturday, 24 August 13

19. ThankYou
Questions???
Contact us at karanm@thoughtworks.com and snehakot@thoughtworks.com
Saturday, 24 August 13