JavaScript & NoSQL
    a Love Story!

   by Alexandre Morgaut
    NoSQL matters - Barcelona 2012
Presentation
W3C AC member

Web Architect

JS Expert

REST Lover

NoSQL Fanboy


  @amorgaut
NoSQL facts
NoSQL Facts

Mostly Schemaless
Often with REST / JSON API
Many store JSON
Many embed a JavaScript engine
  Map / Reduce
  events
Many propose a JavaScript Shell
JavaScript facts
JavaScript Facts

Created in 1995
Running in Netscape Server in 1996, and then in IIS
Mostly Event-Driven
Recommended in the REST definition in 2000
Integrated in CouchDB in 2007
HTML5 Datastores appeared in 2009
JavaScript & Databases
JavaScript & Databases


Netscape Enterprise Server
Microsoft: JScript, JScript.NET, ActiveX
Mozilla Rhino
JSDB
APE Project
W3C Web SQL
Netscape Enterprise Server


   SQLTable()                      pool = new DbPool("ORACLE", addr, user, pwd, "", 5, true);
                                   connection = pool.connection("A connection");

                                   cursor = connection.cursor("select name from customer");
   DbPool                          if ( cursor && (connection.majorErrorCode() == 0) ) {
                                      // Get the first row
                                      cursor.next();

   start/home.html
                                      // Display the values
                                      write("<B>Customer Name:</B> " + cursor.name + "<BR>");
                                      //Close the cursor
                                      cursor.close();
                                   }

   Shared Connections


Note: Netscape Enterprise Server merged with Javagator to become iPlanet
MS JScript on the server


                                    conn = Server.CreateObject("ADODB.Connection");

Active Server Page                  conn.Open(dsn, login, password);

                                    reccordset = conn.Execute(sqlQuery);
                                    result = [];

 runat=”server”                     while (!reccordset.EOF) {

                                        result.push({

Windows Script Hosting
                                            field1: reccordset("field1"),
                                            field2: reccordset("field2")
                                        });
                                        rsAuthor2.MoveNext;


.NET                                }

                                    conn.Close();
                                    conn = null;




    Note: JScript is running on Microsoft IIS since 1997
MS JScript in the Browser


                var connection = new ActiveXObject("ADODB.Connection");
                  
                connection.Open(dsn, login, password);

ActiveXObject
                var reccordset = new ActiveXObject("ADODB.Recordset");
                 
                reccordset.Open(sqlQuery, connection);
                reccordset.MoveFirst;

 only IE        while(!reccordset.eof) {
                   document.write(reccordset.fields(1));
                   reccordset.movenext;
                }
                 
                reccordset.close;
                connection.close;
Mozilla Rhino


                                importPackage(java.sql);
                                java.lang.Class.forName("org.sqlite.JDBC");

Java environment                conn = DriverManager.getConnection("jdbc:sqlite:test.db");
                                stat = conn.createStatement();
                                resultSet = stat.executeQuery("select * from people;");


Access to JDBC                  while (resultSet.next()){
                                    print(
                                        resultSet.getString("name") +
                                        " - " +

ex: RingoJS
                                        resultSet.getString("occupation")
                                    );
                                }

                                resultSet.close();
                                stat.close();
                                conn.close();




              https://developer.mozilla.org/en-US/docs/Rhino
JSDB


                              var db = new ODBC(dsn);
                              var result = db.query("select * from mytable");


SpiderMonkey                  var searcher = new Index;
                              for (var i=1; i <= result.count; i++)
                              {
                                searcher.add(result.get('NAME'));

Shell
                              }

                              var i = searcher.find('Mr. Smith');
                              var r = result.getRow(i + 1);


JS Server                     writeln('Data for Mr. Smith');
                              writeln(r.toString());

                              db.close();




               http://www.jsdb.org/
APE Project

                            var sql = new Ape.MySQL(ip + ":3306", user, password, db);

                            Ape.registerCmd('foo', true, function(params, cmd) {
Real Time Web                     cmd.user.sendRaw(
                                      'bar', {
                                          hello: 'world',

 Comet                            );
                                      }
                                          echo: params.ping



                                  sql.query(

 Web Sockets                          'INSERT INTO table(log) VALUES("' +
                                      Ape.MySQL.escape(params.ping) +
                                      '")',
                                      function(res, errorNo) {

MySQL
                                          Ape.log('Inserted ' + this.getInsertId());
                                      }
                                  );
                            });




         Note: APE means “Ajax Push Engine”
                   http://www.ape-project.org/
W3C Web SQL


       HTML5                var db = openDatabase('mydb', '1.0', 'my first db', 2 * 1024 * 1024);




          Safari
                         db.transaction(function (tx) {
                           tx.executeSql('CREATE TABLE IF NOT EXISTS foo (id unique, text)');
                           tx.executeSql('INSERT INTO foo (id, text) VALUES (1, "synergies")');
                         });


          Chrome                  tx.executeSql('SELECT * FROM foo', [], function (tx, results) {
                                    for (var i = 0, len = results.rows.length; i < len; i++) {
                                      alert(results.rows.item(i).text);

          Opera                     }
                                  });




       Sync / Async
                                         result = x.executeSqlSync('SELECT * FROM foo');
                                         for (var i = 0, len = results.rows.length; i < len; i++) {
                                           alert(results.rows.item(i).text);
                                         }



Note: standard paused because current implementations are only based on SQLite
                         http://html5doctor.com/introducing-web-sql-databases/
Hiding SQL
Hiding SQL


GData




OData
Touching the DB heart
Touching the DB Heart


Key-Value: Web Storage, Riak

Document: CouchDB, MongoDB, IndexedDB

Object: JavaScriptDB, WakandaDB

Multi-Model: OrientDB & ArangoDB
Key - Value
Web Storage


   W3C / WHATWG                             // set or get items by methods
                                            localStorage.setItem("storedItem", "value");
                                            var value = localStorage.getItem("storedItem");



      HTML5                                        // set or get items using the store as a map
                                                   localStorage.storedItem = value;
                                                   var value = localStorage.storedItem;


   local                                           // accessible only for this session
                                                   var foo = sessionStorage.bar;

   session
                                                   sessionStorage.bar = foo;



                                    // sync interface when data change, even from other window
                                    window.addEventListener("storage", handle_storage, false);
   events

Note: Firefox also implement “globalStorage”, and Wakanda “user.storage”
                          http://www.w3.org/TR/webstorage/
Riak
                                        {"inputs": "goog",

Buckets                                   "query": [
                                            {"map": {"language": "javascript",
                                              "source": "function(value, keyData, arg){...}"
                                            }},

REST / JSON
                                            {"reduce": {"language": "javascript",
                                              "source": "function(values, arg){...}",
                                              "keep": true
                                            }}
                                          ]

Map / Reduce                            }


                                      // Map: compute the daily variance, key it by the month

Erlang alternative:
                                      function(value, keyData, arg){
                                        var data = Riak.mapValuesJson(value)[0];
                                        var month = value.key.split('-').slice(0,2).join('-');
                                        var obj = {};
                                        obj[month] = data.High - data.Low;

  SpiderMonkey                        }
                                        return [ obj ];



          // Reduce: find the maximum variance per month
          function(values, arg) {
            return [values.reduce(function(acc, item){
              for (var month in item) {
                 acc[month] = acc[month] ? Math.max(item[month], acc[month]) : item[month];
              }
              return acc;
            })];
          }


               http://wiki.basho.com/MapReduce-Implementation.html
Document
IndexedDB

                                        var request = indexedDB.open("MyTestDatabase", 3);
W3C / WHATWG
                                              request.onerror =    function(event) {

 HTML5
                                                 // Do something   with request.errorCode!
                                              };
                                              request.onsuccess    = function(event) {
                                                 // Do something   with request.result!
                                              };

Sync / Async                                  request.onupgradeneeded = function(event) {
                                                 // Update object stores and indices ....

Indexes
                                              }



                  var objectStore = db.createObjectStore("customers", { keyPath: "ssn" });

Transactions      objectStore.createIndex("name", "name", { unique: false });
                  objectStore.add({ ssn: "444-44-4444", name: "Bill", age: 35});



Cursors        var transaction = db.transaction(["customers"], IDBTransaction.READ_WRITE);




                     http://www.w3.org/TR/IndexedDB/
CouchDB
                     function(doc) {
                       if (doc.Type == "customer") {
                         emit(doc.LastName, {FirstName: doc.FirstName, Addr: doc.Address});
                         emit(doc.FirstName, {LastName: doc.LastName, Addr: doc.Address});

Views                }
                       }




JSON*                                {
                                         "total_rows":2,
                                         "offset":0,
                                         "rows":

Map / Reduce                             [
                                           {
                                             "id":"64ACF01B05F53ACFEC48C062A5D01D89",
                                             "key":"Katz",
                                             "value":{"FirstName":"Damien", "Addr":"Charlotte NC"}

Erlang alternative:                        },
                                           {
                                             "id":"64ACF01B05F53ACFEC48C062A5D01D89",
                                             "key":"Damien",

  Spidermonkey
                                             "value":{"LastName":"Katz", "Addr":"Charlotte NC"}
                                           },
                                         ]
                                     }



  Note: CouchDB moved from XML to JSON & JS in 2007*
                      http://wiki.apache.org/couchdb/HTTP_view_API
          * http://jan.prima.de/~jan/plok/archives/89-CouchDb-updates.html
CouchBase


CouchDB alternative
 using V8
 adding memcache
 meant to be more scalable


                http://www.couchbase.com/
MongoDB
                                      { text: "lmao! great article!",

Spidermonkey                          }
                                        author: 'kbanker',
                                        votes: 2




BSON                                        var map = function() {
                                               emit(this.author, {votes: this.votes});
                                            };

Map / Reduce
                                               // Add up all the votes for each key.

REST / JSON
                                               var reduce = function(key, values) {
                                                  var sum = 0;
                                                  values.forEach(function(doc) {
                                                    sum += doc.votes;
                                                  });

REST / JS                                      };
                                                  return {votes: sum};




Shell           var op = db.comments.mapReduce(map, reduce, {out: "mr_results"});




            http://www.mongodb.org/display/DOCS/MapReduce
Object
Persevere JavaScriptDB
                                                      var d = new Data();
                                                      d.test = 12345;




Rhino
                                                                {"id":"generated.js",
                                                                "sources":[
                                                                  {
                                                                    "name":"Data",
                                                                    "extends":"Object",
REST / JSON                                                         "schema":{
                                                                       "extends":{"$ref":"../Class/Object"},
                                                                       hello : function() {
                                                                          console.log("hello hi");
JSON Schema                                                            },
                                                                       "prototype":{
                                                                       }
                                                                     }
JSON Path                                                       }
                                                                  }]




JSON Query                                               curl localhost:8080/Data/1

                                                         ({"id":"1","test":12345})


                                                      Data.hello(); // INFO: hi there


   http://www.sitepen.com/blog/2009/04/20/javascriptdb-perseveres-new-high-performance-storage-engine/
WakandaDB

Webkit JavaScriptCore
REST / JSON
Data Classes
 auto-updatable
 accessors
               john = ds.Person.fid("fistName eq John");

 events        conferences = john.allConferences;

               JohnJSConferences = conferences.filter("title eq :1", "*JavaScript*");

 methods       JSAttendeesJohnMet = JSConferences.allPeople;


                      http://wakanda.org/
Multi-Model
ArangoDB


V8                   actions.defineHttp({
                       url : "world",
                       context : "api",

  events                callback : function (req, res) {
                          var collection = db._collection(req.parameter.collection);

                           if (collection == null) {

  filters                   }
                             ...

                           else {
                             ...
                           }

  transactions         }
                           actions.resultOk(req, res, actions.HTTP_OK, result);

                     });

Shell


                  http://www.ape-project.org/
OrientDB


rhino
                                                          // create function getAllCustomer
REST / JSON                                               return db.query("select from V")




transactions
                                                               db.begin();
                                                               try {

hooks (events)                                                 "
                                                               "
                                                                   // some code
                                                                   db.commit()
                                                               } catch (e) {
                                                               "   db.rollback();"
                                                               }

stored procedures
new com.orientechnologies.orient.core.record.impl.ODocument('Profile').field('name', 'Luca').save()



                                    http://www.orientdb.org/
reaching the DB
Reaching the DB

mongoose                      arango.client

mongodb-rhino                 WAF (Wakanda)

riak control                  Ext.data.proxy.Wakanda

backbone-couchdb              voltdb-client-nodejs

dojox.data.couchDBRestStore   redis-node-client

node-cassandra                orientdb-api.js
Thank you!



San Jose, CA - Oct. 26-27th     Paris, France - Nov. 16-17th

                         Join us at
                   JS.everywhere(2012)

NoSQL and JavaScript: a Love Story

  • 1.
    JavaScript & NoSQL a Love Story! by Alexandre Morgaut NoSQL matters - Barcelona 2012
  • 2.
    Presentation W3C AC member WebArchitect JS Expert REST Lover NoSQL Fanboy @amorgaut
  • 3.
  • 4.
    NoSQL Facts Mostly Schemaless Oftenwith REST / JSON API Many store JSON Many embed a JavaScript engine Map / Reduce events Many propose a JavaScript Shell
  • 5.
  • 6.
    JavaScript Facts Created in1995 Running in Netscape Server in 1996, and then in IIS Mostly Event-Driven Recommended in the REST definition in 2000 Integrated in CouchDB in 2007 HTML5 Datastores appeared in 2009
  • 7.
  • 8.
    JavaScript & Databases NetscapeEnterprise Server Microsoft: JScript, JScript.NET, ActiveX Mozilla Rhino JSDB APE Project W3C Web SQL
  • 9.
    Netscape Enterprise Server SQLTable() pool = new DbPool("ORACLE", addr, user, pwd, "", 5, true); connection = pool.connection("A connection"); cursor = connection.cursor("select name from customer"); DbPool if ( cursor && (connection.majorErrorCode() == 0) ) { // Get the first row cursor.next(); start/home.html // Display the values write("<B>Customer Name:</B> " + cursor.name + "<BR>"); //Close the cursor cursor.close(); } Shared Connections Note: Netscape Enterprise Server merged with Javagator to become iPlanet
  • 10.
    MS JScript onthe server conn = Server.CreateObject("ADODB.Connection"); Active Server Page conn.Open(dsn, login, password); reccordset = conn.Execute(sqlQuery); result = []; runat=”server” while (!reccordset.EOF) { result.push({ Windows Script Hosting field1: reccordset("field1"), field2: reccordset("field2") }); rsAuthor2.MoveNext; .NET } conn.Close(); conn = null; Note: JScript is running on Microsoft IIS since 1997
  • 11.
    MS JScript inthe Browser var connection = new ActiveXObject("ADODB.Connection");    connection.Open(dsn, login, password); ActiveXObject var reccordset = new ActiveXObject("ADODB.Recordset");   reccordset.Open(sqlQuery, connection); reccordset.MoveFirst; only IE while(!reccordset.eof) { document.write(reccordset.fields(1)); reccordset.movenext; }   reccordset.close; connection.close;
  • 12.
    Mozilla Rhino importPackage(java.sql); java.lang.Class.forName("org.sqlite.JDBC"); Java environment conn = DriverManager.getConnection("jdbc:sqlite:test.db"); stat = conn.createStatement(); resultSet = stat.executeQuery("select * from people;"); Access to JDBC while (resultSet.next()){ print( resultSet.getString("name") + " - " + ex: RingoJS resultSet.getString("occupation") ); } resultSet.close(); stat.close(); conn.close(); https://developer.mozilla.org/en-US/docs/Rhino
  • 13.
    JSDB var db = new ODBC(dsn); var result = db.query("select * from mytable"); SpiderMonkey var searcher = new Index; for (var i=1; i <= result.count; i++) { searcher.add(result.get('NAME')); Shell } var i = searcher.find('Mr. Smith'); var r = result.getRow(i + 1); JS Server writeln('Data for Mr. Smith'); writeln(r.toString()); db.close(); http://www.jsdb.org/
  • 14.
    APE Project var sql = new Ape.MySQL(ip + ":3306", user, password, db); Ape.registerCmd('foo', true, function(params, cmd) { Real Time Web cmd.user.sendRaw( 'bar', { hello: 'world', Comet ); } echo: params.ping sql.query( Web Sockets 'INSERT INTO table(log) VALUES("' + Ape.MySQL.escape(params.ping) + '")', function(res, errorNo) { MySQL Ape.log('Inserted ' + this.getInsertId()); } ); }); Note: APE means “Ajax Push Engine” http://www.ape-project.org/
  • 15.
    W3C Web SQL HTML5 var db = openDatabase('mydb', '1.0', 'my first db', 2 * 1024 * 1024); Safari db.transaction(function (tx) { tx.executeSql('CREATE TABLE IF NOT EXISTS foo (id unique, text)'); tx.executeSql('INSERT INTO foo (id, text) VALUES (1, "synergies")'); }); Chrome tx.executeSql('SELECT * FROM foo', [], function (tx, results) { for (var i = 0, len = results.rows.length; i < len; i++) { alert(results.rows.item(i).text); Opera } }); Sync / Async result = x.executeSqlSync('SELECT * FROM foo'); for (var i = 0, len = results.rows.length; i < len; i++) { alert(results.rows.item(i).text); } Note: standard paused because current implementations are only based on SQLite http://html5doctor.com/introducing-web-sql-databases/
  • 16.
  • 17.
  • 18.
  • 19.
    Touching the DBHeart Key-Value: Web Storage, Riak Document: CouchDB, MongoDB, IndexedDB Object: JavaScriptDB, WakandaDB Multi-Model: OrientDB & ArangoDB
  • 20.
  • 21.
    Web Storage W3C / WHATWG // set or get items by methods localStorage.setItem("storedItem", "value"); var value = localStorage.getItem("storedItem"); HTML5 // set or get items using the store as a map localStorage.storedItem = value; var value = localStorage.storedItem; local // accessible only for this session var foo = sessionStorage.bar; session sessionStorage.bar = foo; // sync interface when data change, even from other window window.addEventListener("storage", handle_storage, false); events Note: Firefox also implement “globalStorage”, and Wakanda “user.storage” http://www.w3.org/TR/webstorage/
  • 22.
    Riak {"inputs": "goog", Buckets "query": [ {"map": {"language": "javascript", "source": "function(value, keyData, arg){...}" }}, REST / JSON {"reduce": {"language": "javascript", "source": "function(values, arg){...}", "keep": true }} ] Map / Reduce } // Map: compute the daily variance, key it by the month Erlang alternative: function(value, keyData, arg){ var data = Riak.mapValuesJson(value)[0]; var month = value.key.split('-').slice(0,2).join('-'); var obj = {}; obj[month] = data.High - data.Low; SpiderMonkey } return [ obj ]; // Reduce: find the maximum variance per month function(values, arg) { return [values.reduce(function(acc, item){ for (var month in item) { acc[month] = acc[month] ? Math.max(item[month], acc[month]) : item[month]; } return acc; })]; } http://wiki.basho.com/MapReduce-Implementation.html
  • 23.
  • 24.
    IndexedDB var request = indexedDB.open("MyTestDatabase", 3); W3C / WHATWG request.onerror = function(event) { HTML5 // Do something with request.errorCode! }; request.onsuccess = function(event) { // Do something with request.result! }; Sync / Async request.onupgradeneeded = function(event) { // Update object stores and indices .... Indexes } var objectStore = db.createObjectStore("customers", { keyPath: "ssn" }); Transactions objectStore.createIndex("name", "name", { unique: false }); objectStore.add({ ssn: "444-44-4444", name: "Bill", age: 35}); Cursors var transaction = db.transaction(["customers"], IDBTransaction.READ_WRITE); http://www.w3.org/TR/IndexedDB/
  • 25.
    CouchDB function(doc) { if (doc.Type == "customer") { emit(doc.LastName, {FirstName: doc.FirstName, Addr: doc.Address}); emit(doc.FirstName, {LastName: doc.LastName, Addr: doc.Address}); Views } } JSON* { "total_rows":2, "offset":0, "rows": Map / Reduce [ { "id":"64ACF01B05F53ACFEC48C062A5D01D89", "key":"Katz", "value":{"FirstName":"Damien", "Addr":"Charlotte NC"} Erlang alternative: }, { "id":"64ACF01B05F53ACFEC48C062A5D01D89", "key":"Damien", Spidermonkey "value":{"LastName":"Katz", "Addr":"Charlotte NC"} }, ] } Note: CouchDB moved from XML to JSON & JS in 2007* http://wiki.apache.org/couchdb/HTTP_view_API * http://jan.prima.de/~jan/plok/archives/89-CouchDb-updates.html
  • 26.
    CouchBase CouchDB alternative usingV8 adding memcache meant to be more scalable http://www.couchbase.com/
  • 27.
    MongoDB { text: "lmao! great article!", Spidermonkey } author: 'kbanker', votes: 2 BSON var map = function() { emit(this.author, {votes: this.votes}); }; Map / Reduce // Add up all the votes for each key. REST / JSON var reduce = function(key, values) { var sum = 0; values.forEach(function(doc) { sum += doc.votes; }); REST / JS }; return {votes: sum}; Shell var op = db.comments.mapReduce(map, reduce, {out: "mr_results"}); http://www.mongodb.org/display/DOCS/MapReduce
  • 28.
  • 29.
    Persevere JavaScriptDB var d = new Data(); d.test = 12345; Rhino {"id":"generated.js", "sources":[ { "name":"Data", "extends":"Object", REST / JSON "schema":{ "extends":{"$ref":"../Class/Object"}, hello : function() { console.log("hello hi"); JSON Schema }, "prototype":{ } } JSON Path } }] JSON Query curl localhost:8080/Data/1 ({"id":"1","test":12345}) Data.hello(); // INFO: hi there http://www.sitepen.com/blog/2009/04/20/javascriptdb-perseveres-new-high-performance-storage-engine/
  • 30.
    WakandaDB Webkit JavaScriptCore REST /JSON Data Classes auto-updatable accessors john = ds.Person.fid("fistName eq John"); events conferences = john.allConferences; JohnJSConferences = conferences.filter("title eq :1", "*JavaScript*"); methods JSAttendeesJohnMet = JSConferences.allPeople; http://wakanda.org/
  • 31.
  • 32.
    ArangoDB V8 actions.defineHttp({ url : "world", context : "api", events callback : function (req, res) { var collection = db._collection(req.parameter.collection); if (collection == null) { filters } ... else { ... } transactions } actions.resultOk(req, res, actions.HTTP_OK, result); }); Shell http://www.ape-project.org/
  • 33.
    OrientDB rhino // create function getAllCustomer REST / JSON return db.query("select from V") transactions db.begin(); try { hooks (events) " " // some code db.commit() } catch (e) { " db.rollback();" } stored procedures new com.orientechnologies.orient.core.record.impl.ODocument('Profile').field('name', 'Luca').save() http://www.orientdb.org/
  • 34.
  • 35.
    Reaching the DB mongoose arango.client mongodb-rhino WAF (Wakanda) riak control Ext.data.proxy.Wakanda backbone-couchdb voltdb-client-nodejs dojox.data.couchDBRestStore redis-node-client node-cassandra orientdb-api.js
  • 36.
    Thank you! San Jose,CA - Oct. 26-27th Paris, France - Nov. 16-17th Join us at JS.everywhere(2012)