POP
Behind the sense
About me
Ben Lin
Co-founder at WOOMOO INC.
A full time entrepreneur & JavaScript lover
Fall in love with node.js on Jun 2011




                                                      dreamerslab.com
                                               ben@dreamerslab.com
                                              twitter.com/dreamerslab
                                              github.com/dreamerslab
About me

   COKE - Full stack node.js MVC framework
   Vodka - Functional testing framework
   Thunder - The lightning fast template engine


   And a lot more on
   https://github.com/dreamerslab
Agenda

         1. Software architecture
         2. Frameworks & Libraries
         3. Application stack
         4. Packages
         5. Scaling node.js
         6. Deployment
         7. Q & A
Software architecture
Software architecture


     Home brewed node.js MVC framework - COKE


              Why node.js?
             It’s fast and easy to scale horizontally
Software architecture
Restful router for api & web server



          module.exports = function ( map ){
            map.get( 'api', 'api/home#index' );
            map.namespace( 'api', function ( api ){
              api.resources( 'users', {
                only : [ 'create', 'show', 'update', 'destroy' ]
              }, function ( users ){
                users.get( 'projects/:id/full', 'user_projects#full' );
                users.resources( 'user_projects', {
                  path : 'projects',
                  only : [ 'create', 'index', 'show', 'update', 'destroy' ]
                });

                 // more routes ......
               });
          };
Software architecture
Validation   make sure all the arguments are correct before get into controllers



                  var form = require( 'express-form2' );
                  var field = form.field;
                  var r     = require( '../regex' );

                  module.exports = {

                       validate_create : form(
                          field( 'client_id' ).required().is( r.id, '01' ),
                          field( 'mockup_id' ).required().is( r.id, '01' ),
                          field( 'format' ).required().is( r.img, '014' )
                       ),

                       validate_index : form(
                          field( 'mockup_id' ).required().is( r.id, '01' )
                       ),

                       validate_destroy : form(
                         field( 'mockup_id' ).required().is( r.id, '01' ),
                         field( 'id' ).required().is( r.id, '01' )
                       )
                  };
Software architecture
Controller Only do data exchange & flow control

                       module.exports = Application.extend({

                         init : function ( before, after ){
                            before( this.is_authenticated );
                            before( this.validate_create );
                            before( this.has_file );
                            before( this.is_validate );
                         },

                         create : function ( req, res, next ){
                           var self = this;

                             ProjectImage.insert( req.form, next,
                               function (){
                                  self.forbidden( res );
                               },
                               function (){
                                  self.frozen( res );
                               },
                               function ( img ){
                                  self.created( res, img );
                               });
                         }
                       });
Software architecture
Model
Handel business logic and model relations, we add a wrapper on mongoose the ODM.

     module.exports = {
        virtuals : [ 'image' ],
        hooks : {
           pre : {
              save   : [ common.mark_new ],
              remove : [ common.backup_to( 'BakMockupImage' ), hooks.remove_from_mockup ]
           },
           post : {
              save   : [ hooks.add_to_mockup, common.add_to_auth( 'images' )],
              remove : [ hooks.remove_from_s3 ]
           }
        },
        statics : {
           insert : function ( args, next, forbidden, frozen, created ){ ... },
           index    : function ( args, next, no_content, forbidden, ok ){ .. },
           destroy : function ( args, next, no_content, forbidden, frozen, deleted ){ ... }
        },
        methods : {
           is_owner : function ( user_id ){ ... },
           created : function (){ ... },
           gen_url : function ( format ){ ... }
        }
     };
Software architecture
View
We use the fastest template engine thunder ( home brewed )



            <!DOCTYPE html>
            <html>
              <head>
                <meta charset="utf-8" />
                <title><?= it.title ?></title>
                <meta name="keywords" content="<?= it.keywords || '' ?>" />
                <meta name="description" content="<?= it.description || '' ?>" />
                <!--[if lt IE 9]>
                <meta http-equiv="refresh" content="0; url=/no-ie">
                <![endif]-->
                <?= it.css( it.styles ) ?>
              </head>
              <body>
                <? it.reset_asset_host(); ?>
                <?= it.body ?>
                <?= it.js( it.scripts ) ?>
              </body>
            </html>
Software architecture
Libraries
Extract reusable codes that is not relevant to model
Scaling node.js
Scaling node.js
Start small with built-in static server, node.js app & database all on the same server.
Scaling node.js
Use nginx as static server for better performance.
Scaling node.js
Use nginx as as proxy server as well to load balance requests.
The number of node.js app instance depends on how many CPU cores on the machine.
Scaling node.js
Split static files to different server for easier maintenance.
Scaling node.js
Use aws S3 for easier setup and maintenance.
Scaling node.js
Split database to another server. Make the node.js app server an unit.
Scaling node.js
Add a load balancer, add more app unit as the site scales up.
Scaling node.js
Add replica set if the database hits its limit.
Scaling node.js
Add CDN for static files for cross reign performance.
Scaling node.js
Split app to difference services as it scales up. Previous scaling steps apply to those services too.
Deployment
Deployment
With ssh, git; The server must stop during deployment.
Deployment
With ssh, git; 0 downtime deployment is possible since we have more than 1 instance( repo ).
Deployment
With ssh, git; Split static file makes it easier to deploy with multi instance( repo ) app.
Deployment
Deploying with multi machine it’s better to use image files on aws ec2.
THE END
 Thanks
QUESTIONS?

Heroku pop-behind-the-sense

  • 1.
  • 2.
    About me Ben Lin Co-founderat WOOMOO INC. A full time entrepreneur & JavaScript lover Fall in love with node.js on Jun 2011 dreamerslab.com ben@dreamerslab.com twitter.com/dreamerslab github.com/dreamerslab
  • 3.
    About me COKE - Full stack node.js MVC framework Vodka - Functional testing framework Thunder - The lightning fast template engine And a lot more on https://github.com/dreamerslab
  • 4.
    Agenda 1. Software architecture 2. Frameworks & Libraries 3. Application stack 4. Packages 5. Scaling node.js 6. Deployment 7. Q & A
  • 5.
  • 6.
    Software architecture Home brewed node.js MVC framework - COKE Why node.js? It’s fast and easy to scale horizontally
  • 7.
    Software architecture Restful routerfor api & web server module.exports = function ( map ){ map.get( 'api', 'api/home#index' ); map.namespace( 'api', function ( api ){ api.resources( 'users', { only : [ 'create', 'show', 'update', 'destroy' ] }, function ( users ){ users.get( 'projects/:id/full', 'user_projects#full' ); users.resources( 'user_projects', { path : 'projects', only : [ 'create', 'index', 'show', 'update', 'destroy' ] }); // more routes ...... }); };
  • 8.
    Software architecture Validation make sure all the arguments are correct before get into controllers var form = require( 'express-form2' ); var field = form.field; var r = require( '../regex' ); module.exports = { validate_create : form( field( 'client_id' ).required().is( r.id, '01' ), field( 'mockup_id' ).required().is( r.id, '01' ), field( 'format' ).required().is( r.img, '014' ) ), validate_index : form( field( 'mockup_id' ).required().is( r.id, '01' ) ), validate_destroy : form( field( 'mockup_id' ).required().is( r.id, '01' ), field( 'id' ).required().is( r.id, '01' ) ) };
  • 9.
    Software architecture Controller Onlydo data exchange & flow control module.exports = Application.extend({ init : function ( before, after ){ before( this.is_authenticated ); before( this.validate_create ); before( this.has_file ); before( this.is_validate ); }, create : function ( req, res, next ){ var self = this; ProjectImage.insert( req.form, next, function (){ self.forbidden( res ); }, function (){ self.frozen( res ); }, function ( img ){ self.created( res, img ); }); } });
  • 10.
    Software architecture Model Handel businesslogic and model relations, we add a wrapper on mongoose the ODM. module.exports = { virtuals : [ 'image' ], hooks : { pre : { save : [ common.mark_new ], remove : [ common.backup_to( 'BakMockupImage' ), hooks.remove_from_mockup ] }, post : { save : [ hooks.add_to_mockup, common.add_to_auth( 'images' )], remove : [ hooks.remove_from_s3 ] } }, statics : { insert : function ( args, next, forbidden, frozen, created ){ ... }, index : function ( args, next, no_content, forbidden, ok ){ .. }, destroy : function ( args, next, no_content, forbidden, frozen, deleted ){ ... } }, methods : { is_owner : function ( user_id ){ ... }, created : function (){ ... }, gen_url : function ( format ){ ... } } };
  • 11.
    Software architecture View We usethe fastest template engine thunder ( home brewed ) <!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title><?= it.title ?></title> <meta name="keywords" content="<?= it.keywords || '' ?>" /> <meta name="description" content="<?= it.description || '' ?>" /> <!--[if lt IE 9]> <meta http-equiv="refresh" content="0; url=/no-ie"> <![endif]--> <?= it.css( it.styles ) ?> </head> <body> <? it.reset_asset_host(); ?> <?= it.body ?> <?= it.js( it.scripts ) ?> </body> </html>
  • 12.
    Software architecture Libraries Extract reusablecodes that is not relevant to model
  • 13.
  • 14.
    Scaling node.js Start smallwith built-in static server, node.js app & database all on the same server.
  • 15.
    Scaling node.js Use nginxas static server for better performance.
  • 16.
    Scaling node.js Use nginxas as proxy server as well to load balance requests. The number of node.js app instance depends on how many CPU cores on the machine.
  • 17.
    Scaling node.js Split staticfiles to different server for easier maintenance.
  • 18.
    Scaling node.js Use awsS3 for easier setup and maintenance.
  • 19.
    Scaling node.js Split databaseto another server. Make the node.js app server an unit.
  • 20.
    Scaling node.js Add aload balancer, add more app unit as the site scales up.
  • 21.
    Scaling node.js Add replicaset if the database hits its limit.
  • 22.
    Scaling node.js Add CDNfor static files for cross reign performance.
  • 23.
    Scaling node.js Split appto difference services as it scales up. Previous scaling steps apply to those services too.
  • 24.
  • 25.
    Deployment With ssh, git;The server must stop during deployment.
  • 26.
    Deployment With ssh, git;0 downtime deployment is possible since we have more than 1 instance( repo ).
  • 27.
    Deployment With ssh, git;Split static file makes it easier to deploy with multi instance( repo ) app.
  • 28.
    Deployment Deploying with multimachine it’s better to use image files on aws ec2.
  • 29.
  • 30.