An Open, End-to-End
                              JavaScript Stack

       Dylan Schiemann
       August 20, 2010



       © SitePen, Inc. All Rights Reserved

Thursday, August 12, 2010
© SitePen, Inc. All Rights Reserved

Thursday, August 12, 2010
© SitePen, Inc. All Rights Reserved

Thursday, August 12, 2010
Open Source

                   OSI license (BSD, AFL, Apache, MIT, etc.)
                   Governance of the code
                          Many companies, open participation, committer policies,
                          decisions
                   Reference implementation under OSI license
                   IP clean
                   Free Distribution, no restrictions on derived works
                   Not tied to another product
                   No restrictions on other software



    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Open Modules and Protocols

                   Vendor lock-in occurs, even with open source
                          The cost to leave
                                "When Free Isn’t Cheap Enough" - Alex Russell

                   Open Protocols
                          Without overspecifying
                   Modular Design




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
End-to-End JavaScript

                   Client-side
                   Server-side
                   Desktop
                   Mobile
                   Data Formats


                   The only end-to-end language
                   Flexible, easy to learn the basics
                   Performant


    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Proliferation: Ajax and GitHub Effect

                   Hundreds of toolkits
                   Hundreds of thousands of microtoolkits, plug-ins, modules
                   Easy to grab, modify, and use




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Rapid Application Development

                   Connect UI to Arbitrary Data
                          Start with reasonable building blocks
                   Quickly modify and iterate
                   Keep easy things easy, make difficult things less difficult
                   Evolve into production-ready results




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
"So you want to build a large app with jQuery? Awesome. Here are some
       solutions to consider...

       The Widget factory, Resig's simple inheritance, pub/sub, mustache templates,
       RequireJS, sammy - choose your application components a la carte.

       JavascriptMVC - add a more defined architecture style with plenty of tools and
       library code to support that

       Dojo with Plugd - Start off in Dojo with a jQuery-ish API and the ability to scale
       up to the whole framework.

       fuck it all and go use sencha, cappuccino, sproutcore, etc."

                                               - Paul Irish, jQuery Team (August 9, 2010)


       © SitePen, Inc. All Rights Reserved

Thursday, August 12, 2010
"I feel like the guys from dojo - slowly watching everyone in jQuery
            implement everything that JavaScriptMVC has already done."

                                             - Justin Meyer, JavaScriptMVC (August 9, 2010)




       © SitePen, Inc. All Rights Reserved

Thursday, August 12, 2010
"My personal preference, of late, has been to use Dojo, not because it
       purports to solve every problem or prescribes how to solve them, but
       because it gives me so many tools to use to solve a given problem. Time
       and again I find that “Dojo already did that” — they already wrote the
       tool I’m wishing I had...

       But let’s be very clear: no one’s forcing me to use that component! I
       can write my own if I want, or use someone else’s if I want, or
       change it a bit if I want! For example, on a current project I pulled in
       mustache.js because it’s pleasant and because a lot of templates for the
       project had already been written using it. The brilliant thing, though,
       was that integrating mustache.js into dijit._Templated instead of
       the standard templating system was trivial. That component, and so
       many others, are architected with exactly the flexibility you talk about
       in mind, while providing a rock-solid base for application development."

                                             - Rebecca Murphey (August 10, 2010)

       © SitePen, Inc. All Rights Reserved

Thursday, August 12, 2010
Markup and Style

                   HTML4 & 5, CSS 2.1 & 3
                          Some CSS toolkits for grid layouts
                          CSS Preprocessors
                          HTML templates
                                Client-side or Server-side
                                Django Templating Language, Mustache, etc.

                   SVG and Canvas
                          Drawing, Charts, Effects




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Client-Side JavaScript

                   ECMAScript 3.1 & 5
                   Toolkits and microtoolkits
                          Selectors (Sizzle, Acme, Slick)
                          Events (Desktop web vs. mobile)
                          DOM, HTTP/XHR/Comet
                          Effects
                          UI Widgets & Data
                          Modules (CommonJS, Dojo/RequireJS or LabJS loader alternative)
                          Patterns (kernel, MVC, etc.)
                          ...


    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Data Format, Transit, & API

                   Formats
                          XML, HTML
                          JSON
                                JSON-P, JSON Schema, JSON Referencing, RQL

                   Wrappers
                          Data Stores & YQL
                          Bayeux, CSP, PubSub
                   Transit: HTTP, Web Socket
                   API: REST, RPC



    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Data Storage

                   SQL
                          MySQL, etc.
                   NoSQL
                          Make the database just about data storage, move everything else
                          to the application layer
                          MongoDB
                          CouchDB
                          JSON
                          Redis (advanced key-value store)




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
App Server Engines

                   Node.js
                          V8 + C++
                   Narwhal + Jack
                          Java
                          Narwhal: JavaScript standard library, package manager, and more.
                          Jack: port of Rack for JSGI web server interface




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Middleware & Business Logic

                   Modules
                   Security
                          Object capability model and facets
                   Cross-domain capabilities
                   Compression and caching
                   Sessions
                   Content negotiation
                   Transactions
                   REST handling


    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Comet

                   Node.js (V8 + C++)
                          Faye
                          Persevere (Tunguska)
                   Narwhal + Jack (Java)
                          Persevere (Tunguska)
                   Or non-JS Comet servers off-board with JS API:
                          Hookbox (Python)
                          cometD (Jetty, Java)
                          ...



    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Persevere 1.0

                   Secure RESTful JSON interface for data interaction and
                   storage of dynamic data, JSONQuery/JSONPath querying,
                   Comet-based real-time data notification through Rest
                   Channels and Bayeux support, class-based based object-
                   oriented business logic with data integrity and validation
                   through JSON Schema in its server-side JavaScript
                   environment.




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Persevere 2.0

                   Breaks the monolithic model of Persevere 1
                   General middleware modules:
                          perstore: CommonJS persistence/object storage based on W3C's
                          object store API.
                          pintura: JSGI-based RESTful JSON/JavaScript server written in JS
                          rql: Resource Query Language: querying in URIs with object style
                          data structures
                          tunguska: Comet-based distributed pubsub hub for server-side
                          JavaScript




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
More Persevere 2.0

                   Persevere 2.0
                          transporter: JSGI app serves modules to browser, dependencies
                          included and packaged in CommonJS module transport format
                          promised-io: Promise-based IO for JavaScript
                          commonjs-utils: Utility modules for CommonJS
                          persevere-template: Application templating
                          patr: Promise-based asynchronous test runner for JavaScript
                   Security through facets, object capability model




         http://www.sitepen.com/blog/series/ssjs/
    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Persevere and Node

                   Persevere's Node-specific modules
                          nodules: Asynchronous URL-based CommonJS module loader for
                          Node with automated dependency resolution and module
                          reloading
                          multi-node: launching multiple NodeJS processes for TCP/HTTP
                          serving
                          node-promise: Promise utilities for Node
                          jsgi-node: JSGI Adapter for Node.js
                   Any of these can be used with Node in isolation.




    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Dojo + Persevere Strengths

                   Same language to learn and use
                   Share modules on Client-side & Server-side (e.g. validation)
                   Same communication (JSON, object literals)
                   Same paradigms (events, callbacks, promises)
                   Coherent data usage
                          Widgets
                          Dojo data and data stores
                          JSON + REST
                          Persevere stack



    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Recap: Anatomy of an App

                   Markup
                   Style
                   Script
                   Data
                   APIs
                   Business Logic
                   Data Storage
                   Security
                   Performance


    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010
Putting it all together?




       © SitePen, Inc. All Rights Reserved

Thursday, August 12, 2010
A world with too many good choices and options
                          “Everything should be made as simple as possible, but not
                          simpler.” — Einstein
                   How do I pick and choose?
                   Or choose for me, but let me change my mind later
                          “Simplicity does not precede complexity, but follows it. “ — Alan
                          Perlis


                   Late August or September 2010, from SitePen
                          An early beta



    © SitePen, Inc. All Rights Reserved
Thursday, August 12, 2010

Open End To End Js Stack

  • 1.
    An Open, End-to-End JavaScript Stack Dylan Schiemann August 20, 2010 © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 2.
    © SitePen, Inc.All Rights Reserved Thursday, August 12, 2010
  • 3.
    © SitePen, Inc.All Rights Reserved Thursday, August 12, 2010
  • 4.
    Open Source OSI license (BSD, AFL, Apache, MIT, etc.) Governance of the code Many companies, open participation, committer policies, decisions Reference implementation under OSI license IP clean Free Distribution, no restrictions on derived works Not tied to another product No restrictions on other software © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 5.
    Open Modules andProtocols Vendor lock-in occurs, even with open source The cost to leave "When Free Isn’t Cheap Enough" - Alex Russell Open Protocols Without overspecifying Modular Design © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 6.
    End-to-End JavaScript Client-side Server-side Desktop Mobile Data Formats The only end-to-end language Flexible, easy to learn the basics Performant © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 7.
    Proliferation: Ajax andGitHub Effect Hundreds of toolkits Hundreds of thousands of microtoolkits, plug-ins, modules Easy to grab, modify, and use © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 8.
    Rapid Application Development Connect UI to Arbitrary Data Start with reasonable building blocks Quickly modify and iterate Keep easy things easy, make difficult things less difficult Evolve into production-ready results © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 9.
    "So you wantto build a large app with jQuery? Awesome. Here are some solutions to consider... The Widget factory, Resig's simple inheritance, pub/sub, mustache templates, RequireJS, sammy - choose your application components a la carte. JavascriptMVC - add a more defined architecture style with plenty of tools and library code to support that Dojo with Plugd - Start off in Dojo with a jQuery-ish API and the ability to scale up to the whole framework. fuck it all and go use sencha, cappuccino, sproutcore, etc." - Paul Irish, jQuery Team (August 9, 2010) © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 10.
    "I feel likethe guys from dojo - slowly watching everyone in jQuery implement everything that JavaScriptMVC has already done." - Justin Meyer, JavaScriptMVC (August 9, 2010) © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 11.
    "My personal preference,of late, has been to use Dojo, not because it purports to solve every problem or prescribes how to solve them, but because it gives me so many tools to use to solve a given problem. Time and again I find that “Dojo already did that” — they already wrote the tool I’m wishing I had... But let’s be very clear: no one’s forcing me to use that component! I can write my own if I want, or use someone else’s if I want, or change it a bit if I want! For example, on a current project I pulled in mustache.js because it’s pleasant and because a lot of templates for the project had already been written using it. The brilliant thing, though, was that integrating mustache.js into dijit._Templated instead of the standard templating system was trivial. That component, and so many others, are architected with exactly the flexibility you talk about in mind, while providing a rock-solid base for application development." - Rebecca Murphey (August 10, 2010) © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 12.
    Markup and Style HTML4 & 5, CSS 2.1 & 3 Some CSS toolkits for grid layouts CSS Preprocessors HTML templates Client-side or Server-side Django Templating Language, Mustache, etc. SVG and Canvas Drawing, Charts, Effects © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 13.
    Client-Side JavaScript ECMAScript 3.1 & 5 Toolkits and microtoolkits Selectors (Sizzle, Acme, Slick) Events (Desktop web vs. mobile) DOM, HTTP/XHR/Comet Effects UI Widgets & Data Modules (CommonJS, Dojo/RequireJS or LabJS loader alternative) Patterns (kernel, MVC, etc.) ... © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 14.
    Data Format, Transit,& API Formats XML, HTML JSON JSON-P, JSON Schema, JSON Referencing, RQL Wrappers Data Stores & YQL Bayeux, CSP, PubSub Transit: HTTP, Web Socket API: REST, RPC © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 15.
    Data Storage SQL MySQL, etc. NoSQL Make the database just about data storage, move everything else to the application layer MongoDB CouchDB JSON Redis (advanced key-value store) © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 16.
    App Server Engines Node.js V8 + C++ Narwhal + Jack Java Narwhal: JavaScript standard library, package manager, and more. Jack: port of Rack for JSGI web server interface © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 17.
    Middleware & BusinessLogic Modules Security Object capability model and facets Cross-domain capabilities Compression and caching Sessions Content negotiation Transactions REST handling © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 18.
    Comet Node.js (V8 + C++) Faye Persevere (Tunguska) Narwhal + Jack (Java) Persevere (Tunguska) Or non-JS Comet servers off-board with JS API: Hookbox (Python) cometD (Jetty, Java) ... © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 19.
    Persevere 1.0 Secure RESTful JSON interface for data interaction and storage of dynamic data, JSONQuery/JSONPath querying, Comet-based real-time data notification through Rest Channels and Bayeux support, class-based based object- oriented business logic with data integrity and validation through JSON Schema in its server-side JavaScript environment. © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 20.
    Persevere 2.0 Breaks the monolithic model of Persevere 1 General middleware modules: perstore: CommonJS persistence/object storage based on W3C's object store API. pintura: JSGI-based RESTful JSON/JavaScript server written in JS rql: Resource Query Language: querying in URIs with object style data structures tunguska: Comet-based distributed pubsub hub for server-side JavaScript © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 21.
    More Persevere 2.0 Persevere 2.0 transporter: JSGI app serves modules to browser, dependencies included and packaged in CommonJS module transport format promised-io: Promise-based IO for JavaScript commonjs-utils: Utility modules for CommonJS persevere-template: Application templating patr: Promise-based asynchronous test runner for JavaScript Security through facets, object capability model http://www.sitepen.com/blog/series/ssjs/ © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 22.
    Persevere and Node Persevere's Node-specific modules nodules: Asynchronous URL-based CommonJS module loader for Node with automated dependency resolution and module reloading multi-node: launching multiple NodeJS processes for TCP/HTTP serving node-promise: Promise utilities for Node jsgi-node: JSGI Adapter for Node.js Any of these can be used with Node in isolation. © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 23.
    Dojo + PersevereStrengths Same language to learn and use Share modules on Client-side & Server-side (e.g. validation) Same communication (JSON, object literals) Same paradigms (events, callbacks, promises) Coherent data usage Widgets Dojo data and data stores JSON + REST Persevere stack © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 24.
    Recap: Anatomy ofan App Markup Style Script Data APIs Business Logic Data Storage Security Performance © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 25.
    Putting it alltogether? © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010
  • 26.
    A world withtoo many good choices and options “Everything should be made as simple as possible, but not simpler.” — Einstein How do I pick and choose? Or choose for me, but let me change my mind later “Simplicity does not precede complexity, but follows it. “ — Alan Perlis Late August or September 2010, from SitePen An early beta © SitePen, Inc. All Rights Reserved Thursday, August 12, 2010