CONVERTING TO NODE
How I converted a medium sized Rails project to Node.js
BACKGROUND
•

CTO of Ideal Candidate	


•

MVP existed when I joined	


•

Author of Haraka and a few
other Node.js librar...
THE RAILS APP
•

Unicorn + Thin + Rails	


•

CoffeeScript with Angular.JS	


•

SASS+Compass for stylesheets	


•

HAML f...
WHY?
APP SIZE
•

5700 Lines of Ruby (170 files)	


•

2500 Lines of CoffeeScript (119 files)	


•

3800 Lines of SASS (42 files)	
...
FINAL GOAL
•

Express + minimal middleware	


•

PostgreSQL hand coded with referential integrity	


•

Jade templates	


...
HOW?
It’s all about the information, Marty
RAILS GIVES YOU
•

Routes	


•

Middleware	


•

DB abstraction layer (aka Model)	


•

Views	


•

Controllers	


•

Confi...
EXPRESS GIVES YOU
•

Routes	


•

Middleware	


•

That’s it	


•

(This scares a lot of people)
DIRECTORY STRUCTURE
lib
- general JS libraries	

⌞ model - database accessors	

public
- files that get served to the front...
RAILS ASSET PIPELINE
•

Rails does a lot for you, but it’s f ’ing confusing	


•

HAML Template Example:	

•

= javascript...
RAILS ASSET PIPELINE
•

Easy if you understand it	


•

Too much magic for my liking	


•

But… the overall effect is good...
JADE EQUIVALENT
	

	

each script in javascripts	
script(src=script, type=“text/javascript”)
APP.JS:
var js = find.fileSync(/.(coffee|js)$/, __dirname + 	
	 ‘/public/assets/javascripts/controllers')	
	 .map(function...
SERVING COFFEESCRIPT
// development	
app.use(require('coffee-middleware')({	
src: __dirname + '/public',	
compress: false,...
SERVING SCSS
•

Node-sass module serves .scss files	


•

Doesn’t serve up .sass files (weird, huh?)	


!
# mass-convert.bas...
SCSS APP.JS
function compile_css () {	
console.log("Recompiling CSS");	
resources.watchers.forEach(function (w) {	
w.close...
SCSS APP.JS

compile_css();	

!
// Dev and Production the same	
app.get(//assets/stylesheets/application-(w+).css/, functi...
ADDING COMPASS
•

Ruby CSS framework	


•

Luckily only the CSS3 part used	


•

CSS3 code is just SASS files	


•

Once I ...
CONVERTING HAML TO JADE
•

Both indent based	


•

HAML:	


	
	
	
	
	

•
	
	
	

•

	
	
	
	
	

%tag{attr: “Value”}	
	
Some ...
CONVERSION TOOL: SUBLIME
TEXT
•

Afterthought: I should have written something in Perl	


•

Regexp Find/Replace	

•

^(s*...
THINGS LEFT TO FIX
•

Helpers: = some_helper	


•

Text on a line on its own - Jade treats these as tags	


•

Nested attr...
HELPERS BECAME MIXINS
•

Standard Rails Helpers:	


= form_for @person do |f|	
f.label :first_name	
f.text_field :first_na...
JADE MIXINS
•

Very powerful. Poorly documented.	


•

Standalone or can have block contents	

// implementation	
mixin fo...
JADE: NO DYNAMIC
INCLUDES
•

HAML/Helpers can do:	


	 	

= popup ‘roles/new'	

•

Jade needed:	


	
	

	
	

+popup(‘roles...
CREATING THE MODEL
•

Hand coded db.js layer developed over time from previous
projects	


•

One file per table (generally...
WHY CONTROL THE SQL?
exports.get_avg_team_size = function (company_id, role_id, months, cb) {	
var role_sql = role_id ? ' ...
AND FINALLY…
•

Routes	

•

Run Rails version of App	


•

Open Chrome Dev Console Network tools	


•

Hit record	


•

Fi...
CREATING ROUTES/MODELS
•

While I glossed over this, it was the bulk of the
work	


•

Each endpoint was painstakingly re-...
BACKGROUND TASKS
•

Currently using Sidekiq, which uses Redis as a queue	

•

Used for downloading slow data feeds	


•

N...
DEPLOYMENT
•

Linode + Nginx + Postgres 9.3.1 + Runit +
Memcached	


•

/var/apps and deploy_to_runit for github autodeplo...
NEXT STEPS
•

Convert coffeescript code to plain JS - I find
coffeescript too much of a pain	


•

Implement graceful resta...
Upcoming SlideShare
Loading in …5
×

Converting a Rails application to Node.js

15,063 views

Published on

This presentation details the process I went through converting a medium sized Ruby on Rails application to Node.js. This included converting SASS to SCSS, Converting HAML to Jade, and building a model and routing framework to match the features in Rails.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
15,063
On SlideShare
0
From Embeds
0
Number of Embeds
7,133
Actions
Shares
0
Downloads
17
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • Knowledge
    Hiring
    Lack of “Magic”
    Performance
    Queryability
    PostgreSQL
    Referential integrity
    Pre-Customers so zero downtime
  • “Little ones and zeros, it’s all just electrons”.
    Web apps are a flow of information. HTML+JS goes to the client, XHR queries come back, XHR responses get sent, POST and PUT requests modify data. Monitor all these, watch the flow, and you have your app. The backend is unimportant at that level.
  • And probably more
  • Let’s start with a solid directory structure. I always do this when I’m creating express apps as it helps me stay grounded and not create too many hacks.
    My app.js has evolved a bit over time to include many things. Happy to share it.
  • What it actually does:
    Looks for your application.js. If it doesn’t find it, looks for application.coffee.
    It then scans that file for comments containing =require or =require_tree
    In production it loads all those files, coffee script compiles them if necessary, minimises, and delivers via a <script src=“application-$md5.js”>.
    In dev, it puts in as many <script> tags as required for each file and delivers them, coffee script compiled.
    Rails does the same for SASS stylesheets.
    FML.
  • I wrote coffee_compiler - but it’s very simple
  • Did this by installing the gem, finding the .sass files, and copying them over to my project.
  • Helpers are functions to simplify HTML generation. Rails comes with a bunch that help with creating forms (and some others).
    These had to be ported to become Jade mixins.
  • HAML version creates some HTML and loads the template ‘views/roles/new’ to be part of the content
    Jade cannot do that, so we manually include the template
  • Slightly more code than ActiveRecord, but gives you full control over SQL
    Only returns plain objects, doesn’t support dynamic updates
    Considering using getters/setters for next iteration, but not sure how they serialize
  • Converting a Rails application to Node.js

    1. 1. CONVERTING TO NODE How I converted a medium sized Rails project to Node.js
    2. 2. BACKGROUND • CTO of Ideal Candidate • MVP existed when I joined • Author of Haraka and a few other Node.js libraries • Background in Perl and Anti-Spam
    3. 3. THE RAILS APP • Unicorn + Thin + Rails • CoffeeScript with Angular.JS • SASS+Compass for stylesheets • HAML for templates • MySQL + MongoDB • Sidekiq for background tasks
    4. 4. WHY?
    5. 5. APP SIZE • 5700 Lines of Ruby (170 files) • 2500 Lines of CoffeeScript (119 files) • 3800 Lines of SASS (42 files) • 1500 Lines of HAML (100 files)
    6. 6. FINAL GOAL • Express + minimal middleware • PostgreSQL hand coded with referential integrity • Jade templates • SCSS for CSS • Javascript + Angular.js frontend
    7. 7. HOW? It’s all about the information, Marty
    8. 8. RAILS GIVES YOU • Routes • Middleware • DB abstraction layer (aka Model) • Views • Controllers • Configuration • Plugins • A directory structure • Unit tests
    9. 9. EXPRESS GIVES YOU • Routes • Middleware • That’s it • (This scares a lot of people)
    10. 10. DIRECTORY STRUCTURE lib - general JS libraries ⌞ model - database accessors public - files that get served to the front end ⌞ assets - files that are considered part of the app ⌞ javascripts ⌞ stylesheets ⌞ images routes - files that provide express routes for HTML ⌞ api/v1 - files providing REST API endpoints (JSON) views - Jade templates app.js - entry point
    11. 11. RAILS ASSET PIPELINE • Rails does a lot for you, but it’s f ’ing confusing • HAML Template Example: • = javascript_include_tag :application • You’d think this adds <script src=“application.js”>, but no.
    12. 12. RAILS ASSET PIPELINE • Easy if you understand it • Too much magic for my liking • But… the overall effect is good. So I copied some of it.
    13. 13. JADE EQUIVALENT each script in javascripts script(src=script, type=“text/javascript”)
    14. 14. APP.JS: var js = find.fileSync(/.(coffee|js)$/, __dirname + ‘/public/assets/javascripts/controllers') .map(function (i) { return i.replace(/^.*/assets/, ‘/assets') .replace(/.coffee$/, ‘.js') }); app.all('*', function (req, res, next) { res.locals.javascripts = js; next(); });
    15. 15. SERVING COFFEESCRIPT // development app.use(require('coffee-middleware')({ src: __dirname + '/public', compress: false, })); ! // Production var all_js = coffee_compiler.generate(js, true); ! var shasum = crypto.createHash('sha256'); shasum.update(all_js); var js_sha = shasum.digest('hex'); ! js = ['/assets/javascripts/application-' + js_sha + '.js']; ! app.get('/assets/javascripts/application-' + js_sha + '.js', function (req, res) { res.type('js'); res.setHeader('Cache-Control', 'public, max-age=31557600'); res.end(all_js); });
    16. 16. SERVING SCSS • Node-sass module serves .scss files • Doesn’t serve up .sass files (weird, huh?) ! # mass-convert.bash for F in `find . -name *.sass`; do O=“${F/.sass/.scss}” sass-convert -F sass -T scss “$F” “$O” git rm -f “$F” git add “$O” done
    17. 17. SCSS APP.JS function compile_css () { console.log("Recompiling CSS"); resources.watchers.forEach(function (w) { w.close(); }); resources.watchers = []; ! ! ! ! } resources.css = sass.renderSync({ file: __dirname + '/public/assets/stylesheets/application.scss', includePaths: [__dirname + '/public/assets/stylesheets'], }); var shasum = crypto.createHash('sha256'); shasum.update(resources.css); var css_sha = shasum.digest('hex'); resources.stylesheets[0] = '/assets/stylesheets/application-' + css_sha + '.css'; resources.stylesheets[1] = '//fonts.googleapis.com/css?family=Open+Sans:400,700,800,300,600'; find.fileSync(cssregexp, __dirname + '/public/assets/stylesheets').forEach(function (f) { resources.watchers.push(fs.watch(f, compile_css)); })
    18. 18. SCSS APP.JS compile_css(); ! // Dev and Production the same app.get(//assets/stylesheets/application-(w+).css/, function (req, res) { res.type('css'); res.setHeader('Cache-Control', 'public, max-age=31557600'); res.end(resources.css); });
    19. 19. ADDING COMPASS • Ruby CSS framework • Luckily only the CSS3 part used • CSS3 code is just SASS files • Once I figured this out, copied them into my project, et voila!
    20. 20. CONVERTING HAML TO JADE • Both indent based • HAML: • • %tag{attr: “Value”} Some Text .person .name Bob JADE: tag(attr=“Value”) Some Text .person .name Bob Other subtle differences too
    21. 21. CONVERSION TOOL: SUBLIME TEXT • Afterthought: I should have written something in Perl • Regexp Find/Replace • ^(s*)% => $1 (fix tags) • (w){(.*)} => $1($2) (attribute curlys) • (w):s*([“‘]) => $1=$2 (attribute key: value)
    22. 22. THINGS LEFT TO FIX • Helpers: = some_helper • Text on a line on its own - Jade treats these as tags • Nested attributes: • -> %tag{ng:{style:’…’,click:’…’},class:’foo’} tag(ng-style=‘…’, ng-click=‘…’, class=‘foo’) Making sure the output matched the Ruby/HAML version was HARD - HTML Diff tools suck
    23. 23. HELPERS BECAME MIXINS • Standard Rails Helpers: = form_for @person do |f| f.label :first_name f.text_field :first_name %br ! f.label :last_name f.text_field :last_name %br ! f.submit • Custom Rails helpers stored in app/helpers/ folder • http://api.rubyonrails.org/classes/ActionView/Helpers/FormHelper.html
    24. 24. JADE MIXINS • Very powerful. Poorly documented. • Standalone or can have block contents // implementation mixin form(action) form(accept-charset="UTF-8", action=action, method="POST") input(name="utf8" type="hidden" value="✓") block // usage: +form(‘/post/to/here’) input(type=“text”,name=“hello”,value=“world”) • Supports js code mixin app_error(field) if (errors && errors[field]) div(class="err_msg " + field) each error in errors[field] span= error
    25. 25. JADE: NO DYNAMIC INCLUDES • HAML/Helpers can do: = popup ‘roles/new' • Jade needed: +popup(‘roles/new’) include ../roles/new
    26. 26. CREATING THE MODEL • Hand coded db.js layer developed over time from previous projects • One file per table (generally) in lib/model/*.js "use strict"; var db = require('./db'); exports.get = function get (id, cb) { db.get_one_row("SELECT * FROM Answers WHERE id=$1", [id], cb); } exports.get_by_submission_id = function (submission_id, cb) { db.query("SELECT * FROM Answers WHERE submission_id=$1 ORDER BY question_id", [submission_id], cb); }
    27. 27. WHY CONTROL THE SQL? exports.get_avg_team_size = function (company_id, role_id, months, cb) { var role_sql = role_id ? ' AND role_id=$2' : ' AND $2=$2'; ! ! } if (months == 0 || months == '0') { months = '9000'; // OVER NINE THOUSAND } var sql = "SELECT avg(c) as value FROM ( SELECT g.month, count(e.*) FROM generate_series( date_trunc('month', now() - CAST($3 AS INTERVAL)), now(), INTERVAL '1 month') g(month) LEFT JOIN employees e ON e.company_id=$1" + role_sql + " AND (start_date, COALESCE(end_date, 'infinity')) OVERLAPS (g.month, INTERVAL '1 month') GROUP BY g.month HAVING count(e.*) > 0 ) av(month,c)"; db.get_one_row(sql, [company_id, role_id, months + " months"], cb);
    28. 28. AND FINALLY… • Routes • Run Rails version of App • Open Chrome Dev Console Network tools • Hit record • Find all routes and implement them
    29. 29. CREATING ROUTES/MODELS • While I glossed over this, it was the bulk of the work • Each endpoint was painstakingly re-created • This allowed me to get a view of the DB layout • And fix design bugs in the DB layout find.fileSync(/.js$/, __dirname + '/routes').forEach(function (route_file) { require(route_file); });
    30. 30. BACKGROUND TASKS • Currently using Sidekiq, which uses Redis as a queue • Used for downloading slow data feeds • Node.js doesn’t care if downloads are slow • So I punted on background tasks for now • If I need them later I will use Kue (see npmjs.org)
    31. 31. DEPLOYMENT • Linode + Nginx + Postgres 9.3.1 + Runit + Memcached • /var/apps and deploy_to_runit for github autodeploy • Monitoring via Zabbix • 60M used vs 130M for Rails
    32. 32. NEXT STEPS • Convert coffeescript code to plain JS - I find coffeescript too much of a pain • Implement graceful restarts using cluster • Consider porting CSS to Bootstrap so we get mobile support

    ×