The document details the author's process of converting a medium-sized Rails project to Node.js, emphasizing the challenges and changes made in the application's structure and code. It outlines the transition from using Rails components like HAML, SCSS, and Sidekiq to utilizing Express, Jade, and custom models in Node.js. The author shares insights on the difficulties faced during the conversion and future steps to improve the application.

1. CONVERTING TO NODE
How I converted a medium sized Rails project to Node.js

2. BACKGROUND
•
CTO of Ideal Candidate
•
MVP existed when I joined
•
Author of Haraka and a few
other Node.js libraries
•
Background in Perl and
Anti-Spam

3. THE RAILS APP
•
Unicorn + Thin + Rails
•
CoffeeScript with Angular.JS
•
SASS+Compass for stylesheets
•
HAML for templates
•
MySQL + MongoDB
•
Sidekiq for background tasks

4. WHY?

5. APP SIZE
•
5700 Lines of Ruby (170 files)
•
2500 Lines of CoffeeScript (119 files)
•
3800 Lines of SASS (42 files)
•
1500 Lines of HAML (100 files)

6. FINAL GOAL
•
Express + minimal middleware
•
PostgreSQL hand coded with referential integrity
•
Jade templates
•
SCSS for CSS
•
Javascript + Angular.js frontend

7. HOW?
It’s all about the information, Marty

8. RAILS GIVES YOU
•
Routes
•
Middleware
•
DB abstraction layer (aka Model)
•
Views
•
Controllers
•
Configuration
•
Plugins
•
A directory structure
•
Unit tests

9. EXPRESS GIVES YOU
•
Routes
•
Middleware
•
That’s it
•
(This scares a lot of people)

10. DIRECTORY STRUCTURE
lib
- general JS libraries
⌞ model - database accessors
public
- files that get served to the front end
⌞ assets - files that are considered part of the app
⌞ javascripts
⌞ stylesheets
⌞ images
routes
- files that provide express routes for HTML
⌞ api/v1 - files providing REST API endpoints (JSON)
views
- Jade templates
app.js
- entry point

11. RAILS ASSET PIPELINE
•
Rails does a lot for you, but it’s f ’ing confusing
•
HAML Template Example:
•
= javascript_include_tag :application
•
You’d think this adds <script
src=“application.js”>, but no.

12. RAILS ASSET PIPELINE
•
Easy if you understand it
•
Too much magic for my liking
•
But… the overall effect is good. So I copied some
of it.

13. JADE EQUIVALENT
each script in javascripts
script(src=script, type=“text/javascript”)

14. APP.JS:
var js = find.fileSync(/.(coffee|js)$/, __dirname +
‘/public/assets/javascripts/controllers')
.map(function (i) {
return i.replace(/^.*/assets/, ‘/assets')
.replace(/.coffee$/, ‘.js')
});
app.all('*', function (req, res, next) {
res.locals.javascripts = js;
next();
});

15. SERVING COFFEESCRIPT
// development
app.use(require('coffee-middleware')({
src: __dirname + '/public',
compress: false,
}));
!
// Production
var all_js = coffee_compiler.generate(js, true);
!
var shasum = crypto.createHash('sha256');
shasum.update(all_js);
var js_sha = shasum.digest('hex');
!
js = ['/assets/javascripts/application-' + js_sha + '.js'];
!
app.get('/assets/javascripts/application-' + js_sha + '.js', function (req, res) {
res.type('js');
res.setHeader('Cache-Control', 'public, max-age=31557600');
res.end(all_js);
});

16. SERVING SCSS
•
Node-sass module serves .scss files
•
Doesn’t serve up .sass files (weird, huh?)
!
# mass-convert.bash
for F in `find . -name *.sass`; do
O=“${F/.sass/.scss}”
sass-convert -F sass -T scss “$F” “$O”
git rm -f “$F”
git add “$O”
done

17. SCSS APP.JS
function compile_css () {
console.log("Recompiling CSS");
resources.watchers.forEach(function (w) {
w.close();
});
resources.watchers = [];
!
!
!
!
}
resources.css = sass.renderSync({
file: __dirname + '/public/assets/stylesheets/application.scss',
includePaths: [__dirname + '/public/assets/stylesheets'],
});
var shasum = crypto.createHash('sha256');
shasum.update(resources.css);
var css_sha = shasum.digest('hex');
resources.stylesheets[0] = '/assets/stylesheets/application-' + css_sha + '.css';
resources.stylesheets[1] = '//fonts.googleapis.com/css?family=Open+Sans:400,700,800,300,600';
find.fileSync(cssregexp, __dirname + '/public/assets/stylesheets').forEach(function (f) {
resources.watchers.push(fs.watch(f, compile_css));
})

18. SCSS APP.JS
compile_css();
!
// Dev and Production the same
app.get(//assets/stylesheets/application-(w+).css/, function (req, res) {
res.type('css');
res.setHeader('Cache-Control', 'public, max-age=31557600');
res.end(resources.css);
});

19. ADDING COMPASS
•
Ruby CSS framework
•
Luckily only the CSS3 part used
•
CSS3 code is just SASS files
•
Once I figured this out, copied them into my
project, et voila!

20. CONVERTING HAML TO JADE
•
Both indent based
•
HAML:
•
•
%tag{attr: “Value”}
Some Text
.person
.name
Bob
JADE:
tag(attr=“Value”) Some Text
.person
.name Bob
Other subtle differences too

21. CONVERSION TOOL: SUBLIME
TEXT
•
Afterthought: I should have written something in Perl
•
Regexp Find/Replace
•
^(s*)%
=> $1 (fix tags)
•
(w){(.*)} => $1($2) (attribute curlys)
•
(w):s*([“‘]) => $1=$2 (attribute key: value)

22. THINGS LEFT TO FIX
•
Helpers: = some_helper
•
Text on a line on its own - Jade treats these as tags
•
Nested attributes:
•
->
%tag{ng:{style:’…’,click:’…’},class:’foo’}
tag(ng-style=‘…’, ng-click=‘…’, class=‘foo’)
Making sure the output matched the Ruby/HAML
version was HARD - HTML Diff tools suck

23. HELPERS BECAME MIXINS
•
Standard Rails Helpers:
= form_for @person do |f|
f.label :first_name
f.text_field :first_name
%br
!
f.label :last_name
f.text_field :last_name
%br
!
f.submit
•
Custom Rails helpers stored in app/helpers/ folder
•
http://api.rubyonrails.org/classes/ActionView/Helpers/FormHelper.html

24. JADE MIXINS
•
Very powerful. Poorly documented.
•
Standalone or can have block contents
// implementation
mixin form(action)
form(accept-charset="UTF-8", action=action, method="POST")
input(name="utf8" type="hidden" value="&#x2713;")
block
// usage:
+form(‘/post/to/here’)
input(type=“text”,name=“hello”,value=“world”)
•
Supports js code
mixin app_error(field)
if (errors && errors[field])
div(class="err_msg " + field)
each error in errors[field]
span= error

25. JADE: NO DYNAMIC
INCLUDES
•
HAML/Helpers can do:
= popup ‘roles/new'
•
Jade needed:
+popup(‘roles/new’)
include ../roles/new

26. CREATING THE MODEL
•
Hand coded db.js layer developed over time from previous
projects
•
One file per table (generally) in lib/model/*.js
"use strict";
var db = require('./db');
exports.get = function get (id, cb) {
db.get_one_row("SELECT * FROM Answers WHERE id=$1", [id], cb);
}
exports.get_by_submission_id = function (submission_id, cb) {
db.query("SELECT * FROM Answers
WHERE submission_id=$1
ORDER BY question_id", [submission_id], cb);
}

27. WHY CONTROL THE SQL?
exports.get_avg_team_size = function (company_id, role_id, months, cb) {
var role_sql = role_id ? ' AND role_id=$2' : ' AND $2=$2';
!
!
}
if (months == 0 || months == '0') {
months = '9000'; // OVER NINE THOUSAND
}
var sql = "SELECT avg(c) as value
FROM (
SELECT g.month, count(e.*)
FROM generate_series(
date_trunc('month', now() - CAST($3 AS INTERVAL)),
now(),
INTERVAL '1 month') g(month)
LEFT JOIN employees e ON e.company_id=$1" + role_sql + "
AND (start_date, COALESCE(end_date, 'infinity'))
OVERLAPS
(g.month, INTERVAL '1 month')
GROUP BY g.month
HAVING count(e.*) > 0
) av(month,c)";
db.get_one_row(sql, [company_id, role_id, months + " months"], cb);

28. AND FINALLY…
•
Routes
•
Run Rails version of App
•
Open Chrome Dev Console Network tools
•
Hit record
•
Find all routes and implement them

29. CREATING ROUTES/MODELS
•
While I glossed over this, it was the bulk of the
work
•
Each endpoint was painstakingly re-created
•
This allowed me to get a view of the DB layout
•
And fix design bugs in the DB layout
find.fileSync(/.js$/, __dirname + '/routes').forEach(function (route_file) {
require(route_file);
});

30. BACKGROUND TASKS
•
Currently using Sidekiq, which uses Redis as a queue
•
Used for downloading slow data feeds
•
Node.js doesn’t care if downloads are slow
•
So I punted on background tasks for now
•
If I need them later I will use Kue (see npmjs.org)

31. DEPLOYMENT
•
Linode + Nginx + Postgres 9.3.1 + Runit +
Memcached
•
/var/apps and deploy_to_runit for github autodeploy
•
Monitoring via Zabbix
•
60M used vs 130M for Rails

32. NEXT STEPS
•
Convert coffeescript code to plain JS - I find
coffeescript too much of a pain
•
Implement graceful restarts using cluster
•
Consider porting CSS to Bootstrap so we get
mobile support