Building assets on the fly using
      node (no watcher)
            JS-Montreal




          Martin Drapeau
           2012-08-14
About Acquisio

• Automate web advertising


• The leading platform to buy, track,
 manage, optimize and report on                             Buy
 media across all major Search,
 Social and Display networks.
                                          Report                             Track
                                                          SEARCH
• Benefits                                                SOCIAL
   • Increase Efficiency                                  DISPLAY
   • Uncover New Business Opportunities
   • Serve Clients Better
                                               Optimize             Manage
   • Serve More Clients
   • Mitigate Risk



Visit www.acquisio.com for more info.
Current Front-End Technology Stack

• Flex (ActionScript)
   • EOL by Adobe


• Front-End Heavy
   • Client is +10MB (to be downloaded
     every time by customers)
   • Client contains a lot of business
     logic – heavy processing. A lot
     should be moved to the Back-End.


Need to change architecture!
Next Gen Front-End Technology Stack

                                          Javascript                                          Front-End
         -    MVC architecture                         -   Backbone JS & Underscore JS
         -    Presentation logic                       -   Twitter Bootstrap + Foundation     Efficient Development
                                                       -   Stylus for CSS (preprocessor)      - Zero compilation
                                                       -   Kendo UI for data-grid and other   - Dev in the browser
                                                           widgets                            - Re-use what’s out
                                                                                                   there



                          Apache/PHP                                 Node JS                  Serving
Private API




                                                           -   Build and deployment           Technologies
                -   Serve JS/CSS assets
                                                           -   JS dependency management
                -   Dynamic generation of HTML
                                                           -   Compilation of JS and CSS
                -   Harness for different
                                                           -   Test harness (unit,
                    environments (i.e. dev vs. prod)
                                                               functional and integration)

                                                               No communication
                                                               between PHP and Java


                                              Java                                            Service Layer
                           API & Business logic & Interface to Back-End                       “Middleware”
                                JBoss, JAX-RS (Jersey), JAXB (JSON)
Development in the browser

• Objective
  • Make it easy to develop in the browser
  • F5 must be the ONLY way to build


• How to do it
  • Harness PHP to manage assets on the fly


• Technology tools
   • Node.js – Driver our build scripts with JS!
   • Snockets – Manage JS dependencies, concat and minify
   • Stylus – CSS preprocessor
Managing JavaScript Assets

Snockets
• A JavaScript/CoffeeScript concatenation tool for Node.js
  inspired from Sprockets .
• We use it to manage dependencies and compile (concat and
  minify) production JS assets.

How to use (script-side)
• Declare dependencies directly in your JS files
       //= require jquery.js
or
       //= require_tree common
Managing JavaScript Assets

How to use (Node-side)

• Construct dependency chain
  var file = ‘my_file.js’;
  var dependencies = [];
  snockets.scan(file, {sync:1}, function(err, depGraph) {
    dependencies = depGraph.getChain(file);
  });


• Concatenate and minify (compilation)
  var file = ‘my_file.js’;
  snockets.getConcatenation(file, {minify:1, sync:1},
    function (err, js) {
      fs.writeFileSync(‘my_file.min.js’, js, 'utf8');});
    });
Managing CSS Assets

Stylus
• CSS preprocessor Expressive, dynamic, robust CSS

How to compile
• Use the Node.js stylus module.

  var file = ‘my_file.styl’;
  styl = fs.readFileSync(file, 'utf8').replace(/r/g, '');
  style = stylus(styl)
          .set('compress', true)
          .set('include css', true);
  style.render(function (err, css) {
    fs.writeFileSync('.css', css, 'utf8');
  });
PHP – Load JS assets

• Either load single minified JS file, or multiple JS files running
 Node.js to retrieve dependency chain

    function js($module) {
      if (USE_COMPILED_JS) {
        $this->view('script_include', array(
            'url' => WEB_ROOT."/javascripts/$module.min.js",
            'content_type' => 'text/javascript',
            'version' => VERSION
          ));
      } else {
        $assets = $this->getAssetsForJsModule($module);
        foreach ($assets as $asset)
          $this->view('script_include', $asset);
      }
    }
PHP – Load JS assets

• Run Node.js

  $chain_script = realpath(FCPATH.'../bin/js_chain.js');
  $cmd = 'node "'.$chain_script.'" '.$module;
  exec($cmd, $result);
  $chains = json_decode($result[0], TRUE);


• Create SCRIPT tags

  <script type="<?=$type?>" src="<?=$url?>"></script>
PHP – Load CSS assets

• If CSS files do not exist, or are stale – run Node.js to create
 them from Stylus files

  $stylus_script = realpath(FCPATH.'../bin/css_compile.js');
  $cmd = 'node "'.$stylus_script.'" --compress=true '.$module;
  exec($cmd, $result);
Performance in Development Environment

• JS Assets
   • 0.5 seconds to chain 70 dependencies (608KB raw JS)*


• CSS Assets
   • 1.7 seconds to compile 133 Stylus files (2.8MB) into 2 CSS files


• Time to Load page in Chrome

   With CSS Compilation
   92 requests ❘ 5.24MB transferred ❘ 5.75s (onload: 4.46s, DOMContentLoaded: 4.44s)

   With no CSS Compilation
   92 requests ❘ 5.24MB transferred ❘ 3.93s (onload: 2.57s, DOMContentLoaded: 2.55s)


 *Exculding 3rd party libraries jQuery and KendoUI
Other work in progress

• Automate builds with JavaScript thanks to Node.js!


• Deployment to different environments
  • QA, Staging, Pre-production and Production


• Automate running QA tests suites
   • Run QA unit, functional and integration tests
   • Use Zombie browser
Questions?

Building assets on the fly with Node.js

  • 1.
    Building assets onthe fly using node (no watcher) JS-Montreal Martin Drapeau 2012-08-14
  • 2.
    About Acquisio • Automateweb advertising • The leading platform to buy, track, manage, optimize and report on Buy media across all major Search, Social and Display networks. Report Track SEARCH • Benefits SOCIAL • Increase Efficiency DISPLAY • Uncover New Business Opportunities • Serve Clients Better Optimize Manage • Serve More Clients • Mitigate Risk Visit www.acquisio.com for more info.
  • 3.
    Current Front-End TechnologyStack • Flex (ActionScript) • EOL by Adobe • Front-End Heavy • Client is +10MB (to be downloaded every time by customers) • Client contains a lot of business logic – heavy processing. A lot should be moved to the Back-End. Need to change architecture!
  • 4.
    Next Gen Front-EndTechnology Stack Javascript Front-End - MVC architecture - Backbone JS & Underscore JS - Presentation logic - Twitter Bootstrap + Foundation Efficient Development - Stylus for CSS (preprocessor) - Zero compilation - Kendo UI for data-grid and other - Dev in the browser widgets - Re-use what’s out there Apache/PHP Node JS Serving Private API - Build and deployment Technologies - Serve JS/CSS assets - JS dependency management - Dynamic generation of HTML - Compilation of JS and CSS - Harness for different - Test harness (unit, environments (i.e. dev vs. prod) functional and integration) No communication between PHP and Java Java Service Layer API & Business logic & Interface to Back-End “Middleware” JBoss, JAX-RS (Jersey), JAXB (JSON)
  • 5.
    Development in thebrowser • Objective • Make it easy to develop in the browser • F5 must be the ONLY way to build • How to do it • Harness PHP to manage assets on the fly • Technology tools • Node.js – Driver our build scripts with JS! • Snockets – Manage JS dependencies, concat and minify • Stylus – CSS preprocessor
  • 6.
    Managing JavaScript Assets Snockets •A JavaScript/CoffeeScript concatenation tool for Node.js inspired from Sprockets . • We use it to manage dependencies and compile (concat and minify) production JS assets. How to use (script-side) • Declare dependencies directly in your JS files //= require jquery.js or //= require_tree common
  • 7.
    Managing JavaScript Assets Howto use (Node-side) • Construct dependency chain var file = ‘my_file.js’; var dependencies = []; snockets.scan(file, {sync:1}, function(err, depGraph) { dependencies = depGraph.getChain(file); }); • Concatenate and minify (compilation) var file = ‘my_file.js’; snockets.getConcatenation(file, {minify:1, sync:1}, function (err, js) { fs.writeFileSync(‘my_file.min.js’, js, 'utf8');}); });
  • 8.
    Managing CSS Assets Stylus •CSS preprocessor Expressive, dynamic, robust CSS How to compile • Use the Node.js stylus module. var file = ‘my_file.styl’; styl = fs.readFileSync(file, 'utf8').replace(/r/g, ''); style = stylus(styl) .set('compress', true) .set('include css', true); style.render(function (err, css) { fs.writeFileSync('.css', css, 'utf8'); });
  • 9.
    PHP – LoadJS assets • Either load single minified JS file, or multiple JS files running Node.js to retrieve dependency chain function js($module) { if (USE_COMPILED_JS) { $this->view('script_include', array( 'url' => WEB_ROOT."/javascripts/$module.min.js", 'content_type' => 'text/javascript', 'version' => VERSION )); } else { $assets = $this->getAssetsForJsModule($module); foreach ($assets as $asset) $this->view('script_include', $asset); } }
  • 10.
    PHP – LoadJS assets • Run Node.js $chain_script = realpath(FCPATH.'../bin/js_chain.js'); $cmd = 'node "'.$chain_script.'" '.$module; exec($cmd, $result); $chains = json_decode($result[0], TRUE); • Create SCRIPT tags <script type="<?=$type?>" src="<?=$url?>"></script>
  • 11.
    PHP – LoadCSS assets • If CSS files do not exist, or are stale – run Node.js to create them from Stylus files $stylus_script = realpath(FCPATH.'../bin/css_compile.js'); $cmd = 'node "'.$stylus_script.'" --compress=true '.$module; exec($cmd, $result);
  • 12.
    Performance in DevelopmentEnvironment • JS Assets • 0.5 seconds to chain 70 dependencies (608KB raw JS)* • CSS Assets • 1.7 seconds to compile 133 Stylus files (2.8MB) into 2 CSS files • Time to Load page in Chrome With CSS Compilation 92 requests ❘ 5.24MB transferred ❘ 5.75s (onload: 4.46s, DOMContentLoaded: 4.44s) With no CSS Compilation 92 requests ❘ 5.24MB transferred ❘ 3.93s (onload: 2.57s, DOMContentLoaded: 2.55s) *Exculding 3rd party libraries jQuery and KendoUI
  • 13.
    Other work inprogress • Automate builds with JavaScript thanks to Node.js! • Deployment to different environments • QA, Staging, Pre-production and Production • Automate running QA tests suites • Run QA unit, functional and integration tests • Use Zombie browser
  • 14.