Going crazy with Node.JS and CakePHP

Going crazy with Node.js and CakePHP
CakeFest 2011
Manchester, UK
Mariano Iglesias @mgiglesias

Hello world!
Hailing from Miramar, Argentina
CakePHP developer since 2006
Worked in countless projects
Contact me if you are looking for work gigs!
A FOSS supporter, and contributor
CakePHP 1.3 book recently published
Survived Node Knockout 2011

Node.js... that's not CakePHP!
If there's something I'd like you to learn it'd be...
There are different solutions to different problems!
CakePHP
Python
Node.js
C++
NGINx / Lighttpd

What's the problem?
What's an app normally doing?
What can I do then?
Add caching
Add workers
Faster DB
Vertical scale: add more resources
Horizontal scale: add more servers
Still can't get n10K concurrent users?

Threads vs events
http://blog.wefaction.com/a-little-holiday-present

What is Node.js?
In a nutshell, it's JavaScript on the server
V8 JavaScript engine Evented I/O + =

V8 Engine
Property access through hidden classes
Machine code
Garbage collection
Performance is king http://code.google.com/apis/v8/design.html

Evented I/O
libeio : async I/O
libev : event loop
libuv : wrapper for libev and IOCP
db. query (). select ( '*' ). from ( 'users' ). execute ( function () { fs. readFile ( 'settings.json' , function () { // ... }); });

Libuv == Node.exe http_simple (/bytes/1024) over 1-gbit network, with 700 concurrent connections: windows-0.5.4 : 3869 r/s windows-latest : 4990 r/s linux-latest-legacy : 5215 r/s linux-latest-uv : 4970 r/s

More stuff
buffer : large portions of data
c-ares : async DNS
child_process : spawn(), exec(), fork() (0.5.x)
crypto : OpenSSL
http_parser : high performance HTTP parser
timer : setTimeout(), setInterval()

Should I throw away CakePHP?
Remember...
There are different solutions to different problems!

First node.js server var http = require ( 'http' ); http. createServer ( function (req, res) { res. writeHead ( 200 , { 'Content-type' : 'text/plain' }); res. end ( 'Hello world!' ); }). listen ( 1337 ); console. log ( 'Server running at http://localhost:1337' );

Understanding the event loop
There is a single thread running in Node.js
No parallel execution... for YOUR code
var http = require ( 'http' ); http. createServer ( function (req, res) { console. log ( 'New request' ); // Block for five seconds var now = new Date (). getTime (); while ( new Date (). getTime () < now + 5000 ) ; // Response res. writeHead ( 200 , { 'Content-type' : 'text/plain' }); res. end ( 'Hello world!' ); }). listen ( 1337 ); console. log ( 'Server running at http://localhost:1337' );

What about multiple cores?
The load balancer approach
:1337 :1338 :1339 The OS approach var http = require ( 'http' ), cluster = ...; var server = http. createServer ( function (req, res) { res. writeHead ( 200 , { 'Content-type' : 'text/plain' }); res. end ( 'Hello world!' ); }); cluster (server). listen ( 1337 );

Packaged modules $ curl http://npmjs.org/install.sh | sh $ npm install db-mysql There are more than 3350 packages, and more than 14 are added each day

Packaged modules var m = require ( './module' ); m. sum ( 1 , 3 , function (err, res) { if (err) { return console. log ( 'ERROR: ' + err); } console. log ( 'RESULT IS: ' + res); }); exports.sum = function (a, b, callback) { if ( isNaN (a) || isNaN (b)) { return callback ( new Error ( 'Invalid parameter' )); } callback ( null , a+b); };

Frameworks are everywhere
Multiple environments
Middleware
Routing
View rendering
Session support
http://expressjs.com

Multiple environments var express = require ( 'express' ); var app = express. createServer (); app. get ( '/' , function (req, res) { res. send ( 'Hello world!' ); }); app. listen ( 3000 ); console. log ( 'Server listening in http://localhost:3000' ); app. configure ( function () { app. use (express. bodyParser ()); }); app. configure ( 'dev' , function () { app. use (express. logger ()); }); $ NODE_ENV=dev node app.js

Middleware function getUser (req, res, next) { if (!req.params.id) { return next (); } else if (!users[req.params.id]) { return next ( new Error ( 'Invalid user' )); } req.user = users[req.params.id]; next (); } app. get ( '/users/:id?' , getUser, function (req, res, next) { if (!req.user) { return next (); } res. send (req.user); });

View rendering
views/index.jade
app. configure ( function () { app. set ( 'views' , __dirname + '/views' ); app. set ( 'view engine' , 'jade' ); }); app. get ( '/users/:id?' , function (req, res, next) { if (!req.params.id) { return next (); } if (!users[req.params.id]) { return next ( new Error ( 'Invalid user' )); } res. send (users[req.params.id]); }); app. get ( '/users' , function (req, res) { res. render ( 'index' , { layout: false , locals: { users: users } }); }); html body h1 Node.js ROCKS ul - each user, id in users li a(href='/users/#{id}') #{user.name}

node-db
What's the point?
Supported databases
Queries
Manual
API
JSON types
Buffer
http://nodejsdb.org

node-db var mysql = require ( 'db-mysql' ); new mysql. Database ({ hostname: 'localhost' , user: 'root' , password: 'password' , database: 'db' }). connect ( function (err) { if (err) { return console. log ( 'CONNECT error: ' , err); } this . query (). select ([ 'id' , 'email' ]). from ( 'users' ). where ( 'approved = ? AND role IN ?' , [ true , [ 'user' , 'admin' ] ]). execute ( function (err, rows, cols) { if (err) { return console. log ( 'QUERY error: ' , err); } console. log (rows, cols); }); });

Let's get to work

Sample application
Basic CakePHP 2.0 app
JSON endpoint for latest messages

Why are we doing this?
CakePHP: 442.90 trans/sec
Node.js: 610.09 trans/sec
Node.js & Pool: 727.19 trans/sec
Node.js & Pool & Cluster: 846.61 trans/sec
$ siege -d1 -r10 -c25

Sample application CREATE TABLE ùsers`( ìd` char(36) NOT NULL, èmail` varchar(255) NOT NULL, `password` text NOT NULL, `name` varchar(255) NOT NULL, PRIMARY KEY (ìd`), UNIQUE KEY èmail` (èmail`) ); CREATE TABLE `messages` ( ìd` char(36) NOT NULL, `from_user_id` char(36) NOT NULL, `to_user_id` char(36) NOT NULL, `message` text NOT NULL, `created` datetime NOT NULL, PRIMARY KEY (ìd`), KEY `from_user_id` (`from_user_id`), KEY `to_user_id` (`to_user_id`), CONSTRAINT `messages_from_user` FOREIGN KEY (`from_user_id`) REFERENCES ùsers` (ìd`), CONSTRAINT `messages_to_user` FOREIGN KEY (`to_user_id`) REFERENCES ùsers` (ìd`) );

Sample application
http://cakefest3.loc/messages/incoming/4e4c2155-e030-477e-985d-18b94c2971a2
[ { "Message": { "id":"4e4d8cf1-15e0-4b87-a3fc-62aa4c2971a2", "message":"Hello Mariano!" }, "FromUser": { "id":"4e4c2996-f964-4192-a084-19dc4c2971a2", "name":"Jane Doe" }, "ToUser": {"name":"Mariano Iglesias"} }, { "Message": { "id":"4e4d8cf5-9534-49b9-8cba-62bf4c2971a2", "message":"How are you?" }, "FromUser": { "id":"4e4c2996-f964-4192-a084-19dc4c2971a2", "name":"Jane Doe" }, "ToUser": {"name":"Mariano Iglesias"} } ]

CakePHP code class MessagesController extends AppController { public function incoming ( $userId ) { $since = ! empty ( $this ->request->query[ 'since' ]) ? urldecode ( $this ->request->query[ 'since' ]) : null; if ( empty ( $since ) || ! preg_match ( '/^d{4}-d{2}-d{2} d{2}:d{2}:d{2}$/' , $since ) ) { $since = '0000-00-00 00:00:00' ; } $messages = ... $this ->autoRender = false; $this ->response-> type ( 'json' ); $this ->response-> body ( json_encode ( $messages )); $this ->response-> send (); $this -> _stop (); } }

CakePHP code $messages = $this ->Message-> find ( 'all' , array ( 'fields' => array ( 'Message.id' , 'Message.message' , 'FromUser.id' , 'FromUser.name' , 'ToUser.name' ), 'joins' => array ( array ( 'type' => 'INNER' , 'table' => 'users' , 'alias' => 'FromUser' , 'conditions' => array ( 'FromUser.id = Message.from_user_id' ) ), array ( 'type' => 'INNER' , 'table' => 'users' , 'alias' => 'ToUser' , 'conditions' => array ( 'ToUser.id = Message.to_user_id' ) ), ), 'conditions' => array ( 'Message.to_user_id' => $userId , 'Message.created >=' => $since ), 'order' => array ( 'Message.created' => 'asc' ), 'recursive' => - 1 ));

Node.js code: express var express = require ( 'express' ), mysql = require ( 'db-mysql' ), port = 1337 ; var app = express. createServer (); app. get ( '/messages/incoming/:id' , function (req, res){ var r = ... var userId = req.params.id; if (!userId) { return r ( new Error ( 'No user ID provided' )); } var since = req.query.since ? req.query.since : false ; if (!since || !/^d{ 4 }-d{ 2 }-d{ 2 } d{ 2 }:d{ 2 }:d{ 2 }$/. test (since)) { since = '0000-00-00 00:00:00' ; } new mysql. Database (...). connect ( function (err) { if (err) { return r (err); } ... }); }); app. listen (port); console. log ( 'Server running at http://localhost:' + port);

Node.js code: express
Avoids the typical:
XMLHttpRequest cannot load URL. Origin URL is not allowed by Access-Control-Allow-Origin
var r = function (err, data) { if (err) { console. log ( 'ERROR: ' + err); res. writeHead ( 503 ); return res. end (); } res.charset = 'UTF-8' ; res. contentType ( 'application/json' ); res. header ( 'Access-Control-Allow-Origin' , '*' ); res. send (data); };

Node.js code: node-db db. query (). select ({ 'Message_id' : 'Message.id' , 'Message_message' : 'Message.message' , 'FromUser_id' : 'FromUser.id' , 'FromUser_name' : 'FromUser.name' , 'ToUser_name' : 'ToUser.name' }). from ({ 'Message' : 'messages' }). join ({ type: 'INNER' , table: 'users' , alias: 'FromUser' , conditions: 'FromUser.id = Message.from_user_id' }). join ({ type: 'INNER' , table: 'users' , alias: 'ToUser' , conditions: 'ToUser.id = Message.to_user_id' }). where ( 'Message.to_user_id = ?' , [ userId ]). and ( 'Message.created >= ?' , [ since ]). order ({ 'Message.created' : 'asc' }). execute ( function (err, rows) { ... });

Node.js code: node-db function (err, rows) { db. disconnect (); if (err) { return r (err); } for ( var i= 0 , limiti=rows.length; i < limiti; i++) { var row = {}; for ( var key in rows [i] ) { var p = key. indexOf ( '_' ), model = key. substring ( 0 , p), field = key. substring (p+ 1 ); if (!row [model] ) { row [model] = {}; } row [model][field] = rows [i][key] ; } rows [i] = row; } r ( null , rows); }

Long polling
Reduce HTTP requests
Open one request and wait for response
function fetch () { $. ajax ({ url: ..., async: true , cache: false , timeout: 60 * 1000 , success: function (data) { ... setTimeout ( fetch (), 1000 ); }, error: ... }); }

Bonus tracks

#1 Pooling connections

Pooling connections
https://github.com/coopernurse/node-pool
var mysql = require ( 'db-mysql' ), generic_pool = require ( 'generic-pool' ); var pool = generic_pool. Pool ({ name: 'mysql' , max: 30 , create: function (callback) { new mysql. Database ({ ... }). connect ( function (err) { callback (err, this ); }); }, destroy: function (db) { db. disconnect (); } }); pool. acquire ( function (err, db) { if (err) { return r (err); } ... pool. release (db); });

#2 Clustering express

Clustering express
http://learnboost.github.com/cluster
var cluster = require ( 'cluster' ), port = 1337 ; cluster ( 'app' ). on ( 'start' , function () { console. log ( 'Server running at http://localhost:' + port); }). on ( 'worker' , function (worker) { console. log ( 'Worker #' + worker.id + ' started' ); }). listen (port); var express = require ( 'express' ), generic_pool = require ( 'generic-pool' ); var pool = generic_pool. Pool ({ ... }); module.exports = express. createServer (); module.exports. get ( '/messages/incoming/:id' , function (req, res) { pool. acquire ( function (err, db) { ... }); });

Clustering express

#3 Dealing with parallel tasks

Dealing with parallel tasks
Asynchronous code can get complex to manage
Async offers utilities for collections
Control flow
series (tasks, [callback])
parallel (tasks, [callback])
waterfall (tasks, [callback])
https://github.com/caolan/async

Dealing with parallel tasks var async = require ( 'async' ); async. waterfall ([ function (callback) { callback ( null , 4 ); }, function (id, callback) { callback ( null , { id: id, name: 'Jane Doe' }); }, function (user, callback) { console. log ( 'USER: ' , user); callback ( null ); } ]); $ node app.js USER: { id: 4, name: 'Jane Doe' }

#4 Unit testing

Unit testing
Export tests from a module
Uses node's assert module:
ok (value)
equal (value, expected)
notEqual (value, expected)
throws (block, error)
doesNotThrow (block, error)
The expect () and done () functions
https://github.com/caolan/nodeunit

Unit testing var nodeunit = require (' nodeunit' ); exports[ 'group1' ] = nodeunit. testCase ({ setUp: function (cb) { cb (); }, tearDown: function (cb) { cb (); }, test1: function (test) { test. equals ( 1 + 1 , 2 ); test. done (); }, test2: function (test) { test. expect ( 1 ); ( function () { test. equals ( 'a' , 'a' ); })(); test. done (); } }); $ nodeunit tests.js nodeunit.js ✔ group1 – test1 ✔ group1 – test2

Questions?

Thanks!
You rock!
@mgiglesias http://marianoiglesias.com.ar

Going crazy with Node.JS and CakePHP

by Mariano Iglesias on Sep 03, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 8

Statistics

Going crazy with Node.JS and CakePHP — Presentation Transcript

http://paper.li	6
http://twitter.com	1
http://localhost	1