Going crazy with  Node.js  and  CakePHP CakeFest 2011 Manchester, UK Mariano Iglesias @mgiglesias
Hello world! Hailing from Miramar, Argentina CakePHP developer since 2006 Worked in countless projects Contact me if you are looking for work gigs! A FOSS supporter, and contributor CakePHP 1.3 book recently published Survived Node Knockout 2011
Node.js... that's not CakePHP! If there's something I'd like you to learn it'd be... There are  different  solutions to different problems! CakePHP Python Node.js C++ NGINx / Lighttpd
What's the problem? What's an app normally doing? What can I do then? Add  caching Add  workers Faster  DB Vertical  scale: add more resources Horizontal  scale: add more servers Still can't get n10K concurrent users?
Threads vs events http://blog.wefaction.com/a-little-holiday-present
What is Node.js? In a nutshell, it's  JavaScript  on the  server V8 JavaScript engine Evented I/O + =
V8 Engine Property access through hidden classes Machine code Garbage collection Performance is king http://code.google.com/apis/v8/design.html
Evented I/O libeio : async I/O libev : event loop libuv : wrapper for libev and  IOCP db. query (). select ( '*' ). from ( 'users' ). execute ( function () { fs. readFile ( 'settings.json' ,   function () { // ... }); });
Libuv == Node.exe http_simple (/bytes/1024) over 1-gbit network, with 700 concurrent connections: windows-0.5.4  : 3869 r/s windows-latest  : 4990 r/s linux-latest-legacy  : 5215 r/s linux-latest-uv  : 4970 r/s
More stuff buffer : large portions of data c-ares : async DNS child_process : spawn(), exec(), fork() (0.5.x) crypto : OpenSSL  http_parser : high performance HTTP parser timer : setTimeout(), setInterval()
Should I throw away CakePHP? Remember... There are  different  solutions to different problems!
First node.js server var   http   =   require ( 'http' ); http. createServer ( function (req,   res) { res. writeHead ( 200 , { 'Content-type' :   'text/plain' }); res. end ( 'Hello world!' ); }). listen ( 1337 ); console. log ( 'Server running at http://localhost:1337' );
Understanding the event loop There is a  single thread  running in Node.js No parallel execution... for  YOUR  code var   http   =   require ( 'http' ); http. createServer ( function (req,   res) { console. log ( 'New request' ); // Block for five seconds var   now   =  new   Date (). getTime (); while ( new   Date (). getTime () <   now   +   5000 ) ; // Response res. writeHead ( 200 , {  'Content-type' :   'text/plain'  }); res. end ( 'Hello world!' ); }). listen ( 1337 ); console. log ( 'Server running at http://localhost:1337' );
What about multiple cores? The  load balancer  approach :1337 :1338 :1339 The  OS  approach var  http   =   require ( 'http' ), cluster = ...; var  server = http. createServer ( function (req,   res) { res. writeHead ( 200 , {  'Content-type' :  'text/plain'  }); res. end ( 'Hello world!' ); }); cluster (server). listen ( 1337 );
Packaged modules $ curl http://npmjs.org/install.sh | sh $ npm install db-mysql There are more than  3350  packages, and more than  14  are added each day
Packaged modules var   m   =   require ( './module' ); m. sum ( 1 ,   3 ,   function (err,   res) { if   (err) { return   console. log ( 'ERROR: '   +   err); } console. log ( 'RESULT IS: '   +   res); }); exports.sum   =   function (a,   b,   callback) { if   ( isNaN (a)   ||   isNaN (b)) { return   callback ( new   Error ( 'Invalid parameter' )); } callback ( null ,   a+b); };
Frameworks are everywhere Multiple environments Middleware Routing View rendering Session support http://expressjs.com
Multiple environments var   express   =   require ( 'express' ); var   app   =   express. createServer (); app. get ( '/' ,   function (req,   res) { res. send ( 'Hello world!' ); }); app. listen ( 3000 ); console. log ( 'Server listening in http://localhost:3000' ); app. configure ( function () { app. use (express. bodyParser ()); }); app. configure ( 'dev' ,   function () { app. use (express. logger ()); }); $ NODE_ENV=dev node app.js
Middleware function   getUser (req,   res,   next) { if   (!req.params.id) { return   next (); }   else if   (!users[req.params.id]) { return   next ( new   Error ( 'Invalid user' )); } req.user   =   users[req.params.id]; next (); } app. get ( '/users/:id?' ,   getUser,   function (req,   res,   next) { if   (!req.user) { return   next (); } res. send (req.user); });
View rendering views/index.jade app. configure ( function () { app. set ( 'views' ,   __dirname   +   '/views' ); app. set ( 'view engine' ,   'jade' ); }); app. get ( '/users/:id?' ,   function (req,   res,   next) { if   (!req.params.id) { return   next (); } if   (!users[req.params.id]) { return   next ( new   Error ( 'Invalid user' )); } res. send (users[req.params.id]); }); app. get ( '/users' ,   function (req,   res) { res. render ( 'index' , {   layout:   false , locals: {   users:   users   } }); }); html body h1 Node.js ROCKS ul - each user, id in users li a(href='/users/#{id}') #{user.name}
node-db What's the point? Supported databases Queries Manual API JSON types Buffer http://nodejsdb.org
node-db var  mysql   =   require ( 'db-mysql' ); new   mysql. Database ({ hostname:   'localhost' , user:   'root' , password:   'password' , database:   'db' }). connect ( function (err) { if   (err) { return   console. log ( 'CONNECT error: ' ,   err); } this . query (). select ([ 'id' ,   'email' ]). from ( 'users' ). where ( 'approved = ? AND role IN ?' , [ true , [   'user' ,   'admin'   ] ]). execute ( function (err,   rows,   cols) { if   (err) { return   console. log ( 'QUERY error: ' ,   err); } console. log (rows,   cols); }); });
Let's get to work
Sample application Basic CakePHP 2.0 app JSON endpoint for latest messages
Why are we doing this? CakePHP: 442.90 trans/sec Node.js: 610.09 trans/sec Node.js & Pool: 727.19 trans/sec Node.js & Pool & Cluster: 846.61 trans/sec $ siege -d1 -r10 -c25
Sample application CREATE TABLE `users`( `id` char(36) NOT NULL, `email` varchar(255) NOT NULL, `password` text NOT NULL, `name` varchar(255) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `email` (`email`) ); CREATE TABLE `messages` (  `id` char(36) NOT NULL, `from_user_id` char(36) NOT NULL, `to_user_id` char(36) NOT NULL, `message` text NOT NULL, `created` datetime NOT NULL, PRIMARY KEY (`id`), KEY `from_user_id` (`from_user_id`), KEY `to_user_id` (`to_user_id`), CONSTRAINT `messages_from_user` FOREIGN KEY (`from_user_id`) REFERENCES `users` (`id`), CONSTRAINT `messages_to_user` FOREIGN KEY (`to_user_id`) REFERENCES `users` (`id`) );
Sample application http://cakefest3.loc/messages/incoming/4e4c2155-e030-477e-985d-18b94c2971a2 [ { &quot;Message&quot;: { &quot;id&quot;:&quot;4e4d8cf1-15e0-4b87-a3fc-62aa4c2971a2&quot;, &quot;message&quot;:&quot;Hello Mariano!&quot; }, &quot;FromUser&quot;: { &quot;id&quot;:&quot;4e4c2996-f964-4192-a084-19dc4c2971a2&quot;, &quot;name&quot;:&quot;Jane Doe&quot; }, &quot;ToUser&quot;: {&quot;name&quot;:&quot;Mariano Iglesias&quot;} }, { &quot;Message&quot;: { &quot;id&quot;:&quot;4e4d8cf5-9534-49b9-8cba-62bf4c2971a2&quot;, &quot;message&quot;:&quot;How are you?&quot; }, &quot;FromUser&quot;: { &quot;id&quot;:&quot;4e4c2996-f964-4192-a084-19dc4c2971a2&quot;, &quot;name&quot;:&quot;Jane Doe&quot; }, &quot;ToUser&quot;: {&quot;name&quot;:&quot;Mariano Iglesias&quot;} } ]
CakePHP code class   MessagesController   extends   AppController   { public function   incoming ( $userId ) { $since   = ! empty ( $this ->request->query[ 'since' ])   ?   urldecode ( $this ->request->query[ 'since' ]) :   null; if   ( empty ( $since )   ||   ! preg_match ( '/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}$/' ,   $since ) ) { $since   =   '0000-00-00 00:00:00' ; } $messages   = ... $this ->autoRender   =   false; $this ->response-> type ( 'json' ); $this ->response-> body ( json_encode ( $messages )); $this ->response-> send (); $this -> _stop (); } }
CakePHP code $messages   =   $this ->Message-> find ( 'all' ,   array ( 'fields'   =>   array ( 'Message.id' , 'Message.message' , 'FromUser.id' , 'FromUser.name' , 'ToUser.name' ), 'joins'   =>   array ( array ( 'type'   =>   'INNER' , 'table'   =>   'users' , 'alias'   =>   'FromUser' , 'conditions'   =>   array ( 'FromUser.id = Message.from_user_id' ) ), array ( 'type'   =>   'INNER' , 'table'   =>   'users' , 'alias'   =>   'ToUser' , 'conditions'   =>   array ( 'ToUser.id = Message.to_user_id' ) ), ), 'conditions'   =>   array ( 'Message.to_user_id'   =>   $userId , 'Message.created >='   =>   $since ), 'order'   =>   array ( 'Message.created'   =>   'asc' ), 'recursive'   => - 1 ));
Node.js code: express var   express   =   require ( 'express' ), mysql   =   require ( 'db-mysql' ), port   =   1337 ; var   app   =   express. createServer (); app. get ( '/messages/incoming/:id' ,   function (req,   res){ var   r   = ... var   userId   =   req.params.id; if   (!userId) { return   r ( new   Error ( 'No user ID provided' )); } var   since   =   req.query.since ? req.query.since   :   false ; if   (!since ||   !/^\d{ 4 }-\d{ 2 }-\d{ 2 }   \d{ 2 }:\d{ 2 }:\d{ 2 }$/. test (since)) { since   =   '0000-00-00 00:00:00' ; } new   mysql. Database (...). connect ( function (err) { if   (err) {   return   r (err); } ... }); }); app. listen (port); console. log ( 'Server running at http://localhost:'   +   port);
Node.js code: express Avoids the typical: XMLHttpRequest cannot load URL. Origin URL is not allowed by Access-Control-Allow-Origin var   r   =   function (err,   data) { if   (err) { console. log ( 'ERROR: '   +   err); res. writeHead ( 503 ); return   res. end (); } res.charset   =   'UTF-8' ; res. contentType ( 'application/json' ); res. header ( 'Access-Control-Allow-Origin' ,   '*' ); res. send (data); };
Node.js code: node-db db. query (). select ({ 'Message_id' :   'Message.id' ,   'Message_message' :   'Message.message' ,   'FromUser_id' :   'FromUser.id' , 'FromUser_name' :   'FromUser.name' ,   'ToUser_name' :   'ToUser.name' }). from ({ 'Message' :   'messages' }). join ({ type:   'INNER' , table:   'users' , alias:   'FromUser' , conditions:   'FromUser.id = Message.from_user_id' }). join ({ type:   'INNER' , table:   'users' , alias:   'ToUser' , conditions:   'ToUser.id = Message.to_user_id' }). where ( 'Message.to_user_id = ?' , [   userId   ]). and ( 'Message.created >= ?' , [   since   ]). order ({ 'Message.created' :   'asc' }). execute ( function (err,   rows) { ... });
Node.js code: node-db function (err,   rows) { db. disconnect (); if   (err) { return   r (err); } for   ( var   i= 0 ,   limiti=rows.length;   i   <   limiti;   i++) { var   row   = {}; for   ( var   key   in   rows [i] ) { var   p   =   key. indexOf ( '_' ), model   =   key. substring ( 0 ,   p), field   =   key. substring (p+ 1 ); if   (!row [model] ) { row [model]   = {}; } row [model][field]   =   rows [i][key] ; } rows [i]   =   row; } r ( null ,   rows); }
Long polling Reduce HTTP requests Open one request and wait for response function   fetch () { $. ajax ({ url: ..., async:   true , cache:   false , timeout:   60   *   1000 , success:   function (data) { ... setTimeout ( fetch (),   1000 ); }, error: ... }); }
Bonus tracks
#1 Pooling connections
Pooling connections https://github.com/coopernurse/node-pool var   mysql   =   require ( 'db-mysql' ), generic_pool   =   require ( 'generic-pool' ); var   pool   =   generic_pool. Pool ({ name:   'mysql' , max:   30 , create:   function (callback) { new   mysql. Database ({ ... }). connect ( function (err) { callback (err,   this ); }); }, destroy:   function (db) { db. disconnect (); } }); pool. acquire ( function (err,   db) { if   (err) { return   r (err); } ... pool. release (db); });
#2 Clustering express
Clustering express http://learnboost.github.com/cluster var   cluster   =   require ( 'cluster' ), port   =   1337 ; cluster ( 'app' ). on ( 'start' ,   function () { console. log ( 'Server running at http://localhost:'   +   port); }). on ( 'worker' ,   function (worker) { console. log ( 'Worker #'   +   worker.id   +   ' started' ); }). listen (port); var   express   =   require ( 'express' ), generic_pool   =   require ( 'generic-pool' ); var   pool   =   generic_pool. Pool ({ ... }); module.exports   =   express. createServer (); module.exports. get ( '/messages/incoming/:id' ,   function (req,   res) { pool. acquire ( function (err,   db) { ... }); });
Clustering express
#3 Dealing with parallel tasks
Dealing with parallel tasks Asynchronous code can get complex to manage Async offers utilities for collections Control flow series (tasks, [callback]) parallel (tasks, [callback]) waterfall (tasks, [callback]) https://github.com/caolan/async
Dealing with parallel tasks var   async   =   require ( 'async' ); async. waterfall ([ function (callback) { callback ( null ,   4 ); }, function (id,   callback) { callback ( null , { id:   id, name:   'Jane Doe' }); }, function (user,   callback) { console. log ( 'USER: ' ,   user); callback ( null ); } ]); $ node app.js USER:  { id: 4, name: 'Jane Doe' }
#4 Unit testing
Unit testing Export tests from a module Uses node's  assert  module: ok (value) equal (value, expected) notEqual (value, expected) throws (block, error) doesNotThrow (block, error) The  expect () and  done () functions https://github.com/caolan/nodeunit
Unit testing var  nodeunit =  require (' nodeunit' ); exports[ 'group1' ] =   nodeunit. testCase ({ setUp:   function (cb) { cb (); }, tearDown:   function (cb) { cb (); }, test1:   function (test) { test. equals ( 1 + 1 ,   2 ); test. done (); }, test2:   function (test) { test. expect ( 1 ); ( function () { test. equals ( 'a' ,   'a' ); })(); test. done (); } }); $ nodeunit tests.js nodeunit.js ✔  group1 – test1 ✔  group1 – test2
Questions?
Thanks! You rock! @mgiglesias http://marianoiglesias.com.ar

Going crazy with Node.JS and CakePHP

  • 1.
    Going crazy with Node.js and CakePHP CakeFest 2011 Manchester, UK Mariano Iglesias @mgiglesias
  • 2.
    Hello world! Hailingfrom Miramar, Argentina CakePHP developer since 2006 Worked in countless projects Contact me if you are looking for work gigs! A FOSS supporter, and contributor CakePHP 1.3 book recently published Survived Node Knockout 2011
  • 3.
    Node.js... that's notCakePHP! If there's something I'd like you to learn it'd be... There are different solutions to different problems! CakePHP Python Node.js C++ NGINx / Lighttpd
  • 4.
    What's the problem?What's an app normally doing? What can I do then? Add caching Add workers Faster DB Vertical scale: add more resources Horizontal scale: add more servers Still can't get n10K concurrent users?
  • 5.
    Threads vs eventshttp://blog.wefaction.com/a-little-holiday-present
  • 6.
    What is Node.js?In a nutshell, it's JavaScript on the server V8 JavaScript engine Evented I/O + =
  • 7.
    V8 Engine Propertyaccess through hidden classes Machine code Garbage collection Performance is king http://code.google.com/apis/v8/design.html
  • 8.
    Evented I/O libeio: async I/O libev : event loop libuv : wrapper for libev and IOCP db. query (). select ( '*' ). from ( 'users' ). execute ( function () { fs. readFile ( 'settings.json' , function () { // ... }); });
  • 9.
    Libuv == Node.exehttp_simple (/bytes/1024) over 1-gbit network, with 700 concurrent connections: windows-0.5.4 : 3869 r/s windows-latest : 4990 r/s linux-latest-legacy : 5215 r/s linux-latest-uv : 4970 r/s
  • 10.
    More stuff buffer: large portions of data c-ares : async DNS child_process : spawn(), exec(), fork() (0.5.x) crypto : OpenSSL http_parser : high performance HTTP parser timer : setTimeout(), setInterval()
  • 11.
    Should I throwaway CakePHP? Remember... There are different solutions to different problems!
  • 12.
    First node.js servervar http = require ( 'http' ); http. createServer ( function (req, res) { res. writeHead ( 200 , { 'Content-type' : 'text/plain' }); res. end ( 'Hello world!' ); }). listen ( 1337 ); console. log ( 'Server running at http://localhost:1337' );
  • 13.
    Understanding the eventloop There is a single thread running in Node.js No parallel execution... for YOUR code var http = require ( 'http' ); http. createServer ( function (req, res) { console. log ( 'New request' ); // Block for five seconds var now = new Date (). getTime (); while ( new Date (). getTime () < now + 5000 ) ; // Response res. writeHead ( 200 , { 'Content-type' : 'text/plain' }); res. end ( 'Hello world!' ); }). listen ( 1337 ); console. log ( 'Server running at http://localhost:1337' );
  • 14.
    What about multiplecores? The load balancer approach :1337 :1338 :1339 The OS approach var http = require ( 'http' ), cluster = ...; var server = http. createServer ( function (req, res) { res. writeHead ( 200 , { 'Content-type' : 'text/plain' }); res. end ( 'Hello world!' ); }); cluster (server). listen ( 1337 );
  • 15.
    Packaged modules $curl http://npmjs.org/install.sh | sh $ npm install db-mysql There are more than 3350 packages, and more than 14 are added each day
  • 16.
    Packaged modules var m = require ( './module' ); m. sum ( 1 , 3 , function (err, res) { if (err) { return console. log ( 'ERROR: ' + err); } console. log ( 'RESULT IS: ' + res); }); exports.sum = function (a, b, callback) { if ( isNaN (a) || isNaN (b)) { return callback ( new Error ( 'Invalid parameter' )); } callback ( null , a+b); };
  • 17.
    Frameworks are everywhereMultiple environments Middleware Routing View rendering Session support http://expressjs.com
  • 18.
    Multiple environments var express = require ( 'express' ); var app = express. createServer (); app. get ( '/' , function (req, res) { res. send ( 'Hello world!' ); }); app. listen ( 3000 ); console. log ( 'Server listening in http://localhost:3000' ); app. configure ( function () { app. use (express. bodyParser ()); }); app. configure ( 'dev' , function () { app. use (express. logger ()); }); $ NODE_ENV=dev node app.js
  • 19.
    Middleware function getUser (req, res, next) { if (!req.params.id) { return next (); } else if (!users[req.params.id]) { return next ( new Error ( 'Invalid user' )); } req.user = users[req.params.id]; next (); } app. get ( '/users/:id?' , getUser, function (req, res, next) { if (!req.user) { return next (); } res. send (req.user); });
  • 20.
    View rendering views/index.jadeapp. configure ( function () { app. set ( 'views' , __dirname + '/views' ); app. set ( 'view engine' , 'jade' ); }); app. get ( '/users/:id?' , function (req, res, next) { if (!req.params.id) { return next (); } if (!users[req.params.id]) { return next ( new Error ( 'Invalid user' )); } res. send (users[req.params.id]); }); app. get ( '/users' , function (req, res) { res. render ( 'index' , { layout: false , locals: { users: users } }); }); html body h1 Node.js ROCKS ul - each user, id in users li a(href='/users/#{id}') #{user.name}
  • 21.
    node-db What's thepoint? Supported databases Queries Manual API JSON types Buffer http://nodejsdb.org
  • 22.
    node-db var mysql = require ( 'db-mysql' ); new mysql. Database ({ hostname: 'localhost' , user: 'root' , password: 'password' , database: 'db' }). connect ( function (err) { if (err) { return console. log ( 'CONNECT error: ' , err); } this . query (). select ([ 'id' , 'email' ]). from ( 'users' ). where ( 'approved = ? AND role IN ?' , [ true , [ 'user' , 'admin' ] ]). execute ( function (err, rows, cols) { if (err) { return console. log ( 'QUERY error: ' , err); } console. log (rows, cols); }); });
  • 23.
  • 24.
    Sample application BasicCakePHP 2.0 app JSON endpoint for latest messages
  • 25.
    Why are wedoing this? CakePHP: 442.90 trans/sec Node.js: 610.09 trans/sec Node.js & Pool: 727.19 trans/sec Node.js & Pool & Cluster: 846.61 trans/sec $ siege -d1 -r10 -c25
  • 26.
    Sample application CREATETABLE `users`( `id` char(36) NOT NULL, `email` varchar(255) NOT NULL, `password` text NOT NULL, `name` varchar(255) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `email` (`email`) ); CREATE TABLE `messages` ( `id` char(36) NOT NULL, `from_user_id` char(36) NOT NULL, `to_user_id` char(36) NOT NULL, `message` text NOT NULL, `created` datetime NOT NULL, PRIMARY KEY (`id`), KEY `from_user_id` (`from_user_id`), KEY `to_user_id` (`to_user_id`), CONSTRAINT `messages_from_user` FOREIGN KEY (`from_user_id`) REFERENCES `users` (`id`), CONSTRAINT `messages_to_user` FOREIGN KEY (`to_user_id`) REFERENCES `users` (`id`) );
  • 27.
    Sample application http://cakefest3.loc/messages/incoming/4e4c2155-e030-477e-985d-18b94c2971a2[ { &quot;Message&quot;: { &quot;id&quot;:&quot;4e4d8cf1-15e0-4b87-a3fc-62aa4c2971a2&quot;, &quot;message&quot;:&quot;Hello Mariano!&quot; }, &quot;FromUser&quot;: { &quot;id&quot;:&quot;4e4c2996-f964-4192-a084-19dc4c2971a2&quot;, &quot;name&quot;:&quot;Jane Doe&quot; }, &quot;ToUser&quot;: {&quot;name&quot;:&quot;Mariano Iglesias&quot;} }, { &quot;Message&quot;: { &quot;id&quot;:&quot;4e4d8cf5-9534-49b9-8cba-62bf4c2971a2&quot;, &quot;message&quot;:&quot;How are you?&quot; }, &quot;FromUser&quot;: { &quot;id&quot;:&quot;4e4c2996-f964-4192-a084-19dc4c2971a2&quot;, &quot;name&quot;:&quot;Jane Doe&quot; }, &quot;ToUser&quot;: {&quot;name&quot;:&quot;Mariano Iglesias&quot;} } ]
  • 28.
    CakePHP code class MessagesController extends AppController { public function incoming ( $userId ) { $since = ! empty ( $this ->request->query[ 'since' ]) ? urldecode ( $this ->request->query[ 'since' ]) : null; if ( empty ( $since ) || ! preg_match ( '/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}$/' , $since ) ) { $since = '0000-00-00 00:00:00' ; } $messages = ... $this ->autoRender = false; $this ->response-> type ( 'json' ); $this ->response-> body ( json_encode ( $messages )); $this ->response-> send (); $this -> _stop (); } }
  • 29.
    CakePHP code $messages = $this ->Message-> find ( 'all' , array ( 'fields' => array ( 'Message.id' , 'Message.message' , 'FromUser.id' , 'FromUser.name' , 'ToUser.name' ), 'joins' => array ( array ( 'type' => 'INNER' , 'table' => 'users' , 'alias' => 'FromUser' , 'conditions' => array ( 'FromUser.id = Message.from_user_id' ) ), array ( 'type' => 'INNER' , 'table' => 'users' , 'alias' => 'ToUser' , 'conditions' => array ( 'ToUser.id = Message.to_user_id' ) ), ), 'conditions' => array ( 'Message.to_user_id' => $userId , 'Message.created >=' => $since ), 'order' => array ( 'Message.created' => 'asc' ), 'recursive' => - 1 ));
  • 30.
    Node.js code: expressvar express = require ( 'express' ), mysql = require ( 'db-mysql' ), port = 1337 ; var app = express. createServer (); app. get ( '/messages/incoming/:id' , function (req, res){ var r = ... var userId = req.params.id; if (!userId) { return r ( new Error ( 'No user ID provided' )); } var since = req.query.since ? req.query.since : false ; if (!since || !/^\d{ 4 }-\d{ 2 }-\d{ 2 } \d{ 2 }:\d{ 2 }:\d{ 2 }$/. test (since)) { since = '0000-00-00 00:00:00' ; } new mysql. Database (...). connect ( function (err) { if (err) { return r (err); } ... }); }); app. listen (port); console. log ( 'Server running at http://localhost:' + port);
  • 31.
    Node.js code: expressAvoids the typical: XMLHttpRequest cannot load URL. Origin URL is not allowed by Access-Control-Allow-Origin var r = function (err, data) { if (err) { console. log ( 'ERROR: ' + err); res. writeHead ( 503 ); return res. end (); } res.charset = 'UTF-8' ; res. contentType ( 'application/json' ); res. header ( 'Access-Control-Allow-Origin' , '*' ); res. send (data); };
  • 32.
    Node.js code: node-dbdb. query (). select ({ 'Message_id' : 'Message.id' , 'Message_message' : 'Message.message' , 'FromUser_id' : 'FromUser.id' , 'FromUser_name' : 'FromUser.name' , 'ToUser_name' : 'ToUser.name' }). from ({ 'Message' : 'messages' }). join ({ type: 'INNER' , table: 'users' , alias: 'FromUser' , conditions: 'FromUser.id = Message.from_user_id' }). join ({ type: 'INNER' , table: 'users' , alias: 'ToUser' , conditions: 'ToUser.id = Message.to_user_id' }). where ( 'Message.to_user_id = ?' , [ userId ]). and ( 'Message.created >= ?' , [ since ]). order ({ 'Message.created' : 'asc' }). execute ( function (err, rows) { ... });
  • 33.
    Node.js code: node-dbfunction (err, rows) { db. disconnect (); if (err) { return r (err); } for ( var i= 0 , limiti=rows.length; i < limiti; i++) { var row = {}; for ( var key in rows [i] ) { var p = key. indexOf ( '_' ), model = key. substring ( 0 , p), field = key. substring (p+ 1 ); if (!row [model] ) { row [model] = {}; } row [model][field] = rows [i][key] ; } rows [i] = row; } r ( null , rows); }
  • 34.
    Long polling ReduceHTTP requests Open one request and wait for response function fetch () { $. ajax ({ url: ..., async: true , cache: false , timeout: 60 * 1000 , success: function (data) { ... setTimeout ( fetch (), 1000 ); }, error: ... }); }
  • 35.
  • 36.
  • 37.
    Pooling connections https://github.com/coopernurse/node-poolvar mysql = require ( 'db-mysql' ), generic_pool = require ( 'generic-pool' ); var pool = generic_pool. Pool ({ name: 'mysql' , max: 30 , create: function (callback) { new mysql. Database ({ ... }). connect ( function (err) { callback (err, this ); }); }, destroy: function (db) { db. disconnect (); } }); pool. acquire ( function (err, db) { if (err) { return r (err); } ... pool. release (db); });
  • 38.
  • 39.
    Clustering express http://learnboost.github.com/clustervar cluster = require ( 'cluster' ), port = 1337 ; cluster ( 'app' ). on ( 'start' , function () { console. log ( 'Server running at http://localhost:' + port); }). on ( 'worker' , function (worker) { console. log ( 'Worker #' + worker.id + ' started' ); }). listen (port); var express = require ( 'express' ), generic_pool = require ( 'generic-pool' ); var pool = generic_pool. Pool ({ ... }); module.exports = express. createServer (); module.exports. get ( '/messages/incoming/:id' , function (req, res) { pool. acquire ( function (err, db) { ... }); });
  • 40.
  • 41.
    #3 Dealing withparallel tasks
  • 42.
    Dealing with paralleltasks Asynchronous code can get complex to manage Async offers utilities for collections Control flow series (tasks, [callback]) parallel (tasks, [callback]) waterfall (tasks, [callback]) https://github.com/caolan/async
  • 43.
    Dealing with paralleltasks var async = require ( 'async' ); async. waterfall ([ function (callback) { callback ( null , 4 ); }, function (id, callback) { callback ( null , { id: id, name: 'Jane Doe' }); }, function (user, callback) { console. log ( 'USER: ' , user); callback ( null ); } ]); $ node app.js USER: { id: 4, name: 'Jane Doe' }
  • 44.
  • 45.
    Unit testing Exporttests from a module Uses node's assert module: ok (value) equal (value, expected) notEqual (value, expected) throws (block, error) doesNotThrow (block, error) The expect () and done () functions https://github.com/caolan/nodeunit
  • 46.
    Unit testing var nodeunit = require (' nodeunit' ); exports[ 'group1' ] = nodeunit. testCase ({ setUp: function (cb) { cb (); }, tearDown: function (cb) { cb (); }, test1: function (test) { test. equals ( 1 + 1 , 2 ); test. done (); }, test2: function (test) { test. expect ( 1 ); ( function () { test. equals ( 'a' , 'a' ); })(); test. done (); } }); $ nodeunit tests.js nodeunit.js ✔ group1 – test1 ✔ group1 – test2
  • 47.
  • 48.
    Thanks! You rock!@mgiglesias http://marianoiglesias.com.ar

Editor's Notes

  • #5 ** NORMALLY DOING: It&apos;s waiting. You get a request, query the DB, and wait for results. You write to a file, and wait for it to be written. You encode a video, and wait for it to complete. ** CACHING: File caching, Memcached ** WORKERS: Use robot plugin, or Gearman, or ActiveMQ / RabbitMQ ** FASTER DB: NoSQL (MongoDB, CouchDB)
  • #6 You can&apos;t simply spawn a new process or thread for each connection NGINX is event based, so memory usage is low APACHE spawns a thread per request (or process depending on configuration) THREADS spend a lot of time being blocked waiting for I/O operations EVENTS are better when more time is spent waiting on external resources
  • #7 ** V8: Without DOM parts
  • #8 ** PROPERTIES: Object properties can be changed at runtime in JS: property dictionaries Instance variables have fixed offsetts set by compiler. V8 creates hidden classes, one for each property added and class transitions show changes to object properties (only first time) ** MACHINE CODE: when first executed JS is compiled into machine code, no intermediate byte code ** GARBAGE: when running GC, program stops. Each GC cycle only process part of the heap.
  • #9 ** LIBEIO: by Marc Lehmann. Can be used with any event library, or in POLLING mode. Relies only on POSIX threads. Uses threads internally, and used by Node.JS for file/network access, integrated in node to LIBEV ** LIBEV: also by Marc Lehmann. asynchronous notification of file descriptor changes through ev_io() ** ReadFile(): lots of JS code, on top of fs.open(), which is a wrapper around C++ Open(), which uses libeio
  • #10 Node on Windows is not loosing significant performance compared to Linux
  • #11 ** BUFFER: Outside of V8 heap. Conversion to JS through encoding (binary, utf8, ascii, etc.) ** C-ARES: gethostbyname(), custom DNS queries ** CHILD_PROCESSES: fork() is in JS land, special case of spawn(), with IPC messaging capabilities ** CRYPTO: create hashes, ciphers with different algorithms, signs with private keys, and verify signatures ** HTTP_PARSER: NO allocation, NO buffer ** TIMER: setTimeout() is one time, setInterval() is repeated
  • #14 ** SINGLE THREAD: Since node runs in a single thread, anything blocking that thread blocks node from processing other requests ** YOUR code: if your code uses I/O, that I/O is NON blocking ** I/O calls are the point at which Node.js switches between different requests. So Node.js expects requests to be done very fast. Intensive CPU should be done in another process
  • #15 Since Node.JS runs in a single process, its bound to a single CPU ** OS approach: the kernel load balances connections across processes ** CLUSTER: uses several workers (in different CPUs), we will cover this later
  • #16 ** PACKAGES can be installed locally or globally
  • #17 ** COMMONJS: defines JS APIs for any purpose, such as server side JS. Amongst the things it defines, are modules ** The EXPORTS object is created by the module system. If you want your module to be a class, assign the export object to MODULE.EXPORTS
  • #18 ** ENVIRONMENTS: Can specify different middleware for each environment, and change settings ** MIDDLEWARE: for manipulating all requests, or for specific routes ** ROUTING: Including support for HTTP methods ** VIEW RENDERING: with different template engines, and including partials (sort of like CakePHP elements) ** SESSION SUPPORT: part of Connect, can be configured even with Redis
  • #19 ** MIDDLEWARES: order is important. ** BodyParser() allows the parsing of forms (including JSON posted data as part of body) ** Other middlewares include: cookieParser(), session()
  • #20 With route middleware, it&apos;s easy to add authentication
  • #21 ** JADE is one of the view engines available: $ npm install jade
  • #25 ** CakePHP APP: including the use Auth with Form authentication ** JSON endpoint first in CakePHP, then in Node.js An alternative to the JSON endpoint is to use WEB SOCKETS, which offers a bi-directional communication channel over TCP. Its API is still being standarized by the W3C. The Node.JS module SOCKET.IO has great support for web sockets, and its client code works on almost every browser
  • #26 ** SIEGE: 25 concurrent users, 10 repetitions, 1 random secs delay ** NODE.JS: Using EXPRESS
  • #27 ** Even if you don&apos;t add CONSTRAINTS, always REMEMBER to add a KEY for each field that is to be used as foreign keys
  • #32 ** writeHead(statusCode, headersObject) ** ACCESS CONTROL: a wildcard, or an URI ** res.send(): automatically converts object to JSON string
  • #35 ** We are now doing a whole bunch of HTTP requests, which is not optimal, we need to reduce them to the bare minimum ** LONG polling: when a response is obtained, we schedule a new request ** CODE: when error, ALSO schedule a new request
  • #38 ** Allows prioritization in the queue ** Can handle idle timeouts (time a resource can go unused before it is destroyed)
  • #40 ** CLUSTER: can be extended via plugins. Spawns one worker per CPU. Handles signals and different types of shutdown. Can resucitate workers.
  • #43 ** COLLECTIONS: forEach, map, filter, and others ** SERIES(): runs in order, stops if one fails (error first arg). Calls final callback once they are all done (or ERRORED), with an array of all arguments sent to each task ** PARALLEL(): similar to SERIES(), does not stop on error ** WATERFALL(): like SERIES(), but each task pass its result as an argument to the next task
  • #46 ** EXPORT: create a module and define the test groups as elements of the module ** EXPECT(): specify how many assertions are expected to run in a test ** DONE(): ALL TESTS should call this. Marks the test as done