Developing for Node.JS with MySQL and NoSQL

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Insert Picture Here
1
Developing for Node.JS with
MySQL and NoSQL
 J.D. Duncan, john.duncan@oracle.com
 Craig Russell, craig.russell@oracle.com

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
MySQL Cluster Architecture
MySQL Cluster Data Nodes
Data Layer
Clients
Application Layer
Management

Data Layer
Clients
Application Layer
Management
Management

Data Layer
Application Layer
Management
Management
Clients

mysql-js: a MySQL connector for Node.JS
 Included in MySQL Cluster 7.3
 in storage/ndb/nodejs
 Also at github:
 https://github.com/mysql/mysql-js
 One JavaScript API with two back-end adapters
 The MySQL back end uses node-mysql (Felix Geisendorfer's all-
JavaScript MySQL connector)
 The NDB (MySQL Cluster) back end uses native NDBAPI
 Twitter-like demo application in samples/tweet/

What is Node.JS
 A server-side web application platform
 Single-threaded* and event-driven
 Application code is written in JavaScript
 JavaScript code runs in a very fast VM
 Node.JS includes a useful set of standard libraries
* One JavaScript thread;
a pool of background worker threads perform blocking I/O.

Why Node.JS
 Servers like lighttpd, nginx, and tornado can solve the C10K problem
for static or cached content
 Node.JS can solve the C10K problem for a dynamic web application
– ... but what about 10,000 back-end database connections?
 mysql-js with NDB aims to solve the C10K problem for a dynamic and
database-driven web application
The "C10K" problem: how to handle 10,000+ client connections

Sample Code
Node HTTP Server
function handleRequest(request, response) {
var page = getContent(request.url);
response.statusCode = page ? 200 : 404;
response.write(page);
response.end();
}
function startWebServer() {
http.createServer(handleRequest).listen(8080);
console.log("Server started");
}

Sample Code with database access
Node HTTP Server and mysql-js
function dbConnected(connectionError, sessionFactory) {
function onDbResponse(err, data) {
var page = renderPage(data);
response.end();
}
function onDbSession(err, session) {
runDbOperation(session, request, onDbResponse);
}
sessionFactory.openSession(null, onDbSession);
}
}
nosql.connect(... , dbConnected);

User's View
10

Twitter-like Demo Application
11

Twitter-like Demo Application
12
 Create user
 Find user by name
 Delete user by name
(cascading)
 Create a tweet
 Find tweet by id
 Delete tweet by id (cascading)
 Make User A a follower of
User B
 List followers of user
 List who user is following
 Get the most recent tweets
 Get the 20 latest tweets by a
user
 Get the 20 latest tweets
@mentioning a user
 Get the 20 latest tweets
containing a #hashtag

JavaScript Constructor
function Tweet(author, message) {
if(author !== undefined) {
this.date_created = new Date();
this.author = author;
this.message = message;
}
}
Creates an instance of a Domain Object

SQL Table
CREATE TABLE tweet (
id bigint unsigned auto_increment not null,
author varchar(20),
message varchar(140),
date_created timestamp(2),
...
)

Mapping
var nosql = require("../mysql-js");
// Map SQL Tables to JS Constructors
var mappings = [];
mappings.push(
new nosql.TableMapping('tweet').
applyToClass(Tweet)
);

Explicit Mappings are not required
 All operations take either a constructor or table name
 If table name is used, default mapping is created
 Default mapping maps all columns to JS properties, where the JS type
of each property is the default for its corresponding SQL Data Type

connect function
 connect is asynchronous
 First parameter is object with connection properties
 Second parameter is array of table names or constructors
– Metadata for these will be resolved before returning session factory
 Third parameter is callback (err, sessionFactory)
 err will be falsy if no error

connect Example
/* Connection Properties */
dbProperties = {
adapter : "ndb",
ndb_connectstring : "localhost:1186"
};
// Map SQL Tables to JS Constructors
mappings = [];
mappings.push(new nosql.TableMapping('tweet').
applyToClass(Tweet));
// Connect
nosql.connect(dbProperties, mappings,
runCmdlineOperation, operation);

The connect callback
function dbConnected(connectionError, sessionFactory) {
function onDbResponse(err, data) {
var page = renderPage(data);
response.end();
}
function onDbSession(err, session) {
runDbOperation(session, request, onDbResponse);
}
sessionFactory.openSession(null, onDbSession);
}
}
nosql.connect(dbProperties, mappings, dbConnected);

find Example
function findAuthor(name, callback) {
session.find(Author, name, function(err, result) {
if (err) {
callback(err);
} else {
// result is the row in the author table
callback(null, result);
});
}

find function on Session
 find is asynchronous
 First parameter is table name or constructor (mapped class)
 Second parameter is key to use locating the record
– Primary key
– Unique key
 Third parameter is callback (err, result)
– err will be falsy if no error
– result will be new object with mapped properties

persist Example
function persistAuthor(name, full, callback) {
var author = new Author(name, full);
session.persist(author, function(err) {
if (err) {
callback(err);
} else {
callback(null, author);
}
});
}

persist function on Session
 persist is asynchronous
 First parameter is instance of mapped class
 Second parameter callback (err)
 Variants allow persist into default mapped table (no constructor)

update Example
function updateAuthor(author, new_full_name,
callback) {
author.full_name = new_full_name;
session.update(author, callback);
}

update function on Session
 update is asynchronous
 First parameter is instance of mapped class
– Modified properties are written to database
 Second parameter callback (err)
 Variants allow update of default mapped table (no constructor)

remove Example
function removeByAuthorName(name, callback) {
session.remove(Author, name, function(err, result)
{
if (err) {
callback(err);
} else {
// row has been removed
callback(null);
});
}

remove function on Session
 remove is asynchronous
– First parameter is table name or constructor (mapped class)
– Second parameter is key to use locating the record
 Primary key
 Unique key
– Third parameter is callback (err)
 err will be falsy if no error
 Alternately
– First parameter is instance of constructor
– Second parameter is callback (err)

Transactions
/* Insert a tweet.
- Start a transaction.
- Persist the tweet.
- Create & persist #hashtag & @user records
- Increment the author's tweet count.
- Then commit the transaction.
*/
function InsertTweetOperation(params, data) {
session.currentTransaction().begin();
...
session.currentTransaction().commit();
}

Query Example
// Last 20 tweets @user
TweetsAtUserOperation.run = function(tag) {
if(tag.charAt(0) == "@") tag = tag.substring(1);
this.session.createQuery(Mention,
function(error, query) {
var queryParams = { "order" : "desc",
"limit" : 20 ,
"tag" : tag};
query.where(query.at_user.eq(query.param("tag")));
query.execute(queryParams, fetchTweetsInBatch);
});
};

query function
 Session is Query factory using asynchronous api
 query has a filter based on constructor (mapped class) properties
 Filter has comparators and boolean operations
– eq, ne, gt, ge, le, lt, between,isNull,isNotNull comparators
– and, or, not,andNot,orNot operations
 Query execution is asynchronous
– Filter determines query strategy
 Primary/unique key lookup; index scan; table scan
– Properties govern query execution
– Results are given in callback

Query Comparators
 Comparators compare properties to parameters
 Query Domain Type property names correspond to Constructor field
names (properties)
 Parameters are created by name
– qdt.param('date_low')
 Properties are referenced by field name in Constructor
– qdt.date_created
 Comparators are properties of qdt properties
– qdt.date_created.gt(qdt.param('date_low'));
 Comparators return predicates

Query Operators
 Predicates are used as query filters via where function
 Predicates are results of comparators or operators
 Operators combine predicates using and, or, andNot, orNot, not
– var predicate1 =
qdt.date_created.gt(qdt.param('date_low'));
– var predicate2 =
qdt.date_created.lt(qdt.param('date_high'));
– var predicate = predicate1.and(predicate2);
– var predicate = predicate3.andNot(predicate4);
– qdt.where(predicate)

Batching
function fetchTweetsInBatch(error, mentions,
finalCallback) {
var resultData = [];
function addTweetToResults(err, tweet) {
if(tweet && ! err) resultData.push(tweet);
}
var batch = session.createBatch();
while(var mention = mentions.pop());
batch.find(Tweet, mention.id, addTweetToResults);
batch.execute(finalCallback, resultData);
}

Batching
 Batching allows parallel database operations
– Any combination of find, persist, update, remove
 Batch is created using session.createBatch()
 Operations (with their own callbacks) are added to the batch
 Batch is executed with its own callback
– Operation callbacks are run first
– Then the batch callback

Get mysql-js & run the Tweet demo
### First install MySQL Cluster 7.3
### Then get the latest code from github:
% git clone https://github.com/mysql/mysql-js.git
### build it
% node configure.js
% node-gyp configure build
### Go to the demo directory
% cd samples/tweet

Get mysql-js & run the Tweet demo
### create the schema
% mysql -u root < tweet.sql
### Run with the mysql adapter
% node tweet.js -a mysql put user mr_jdd
'John David Duncan'
### Run with the ndb adapter
% node tweet.js -a ndb get user mr_jdd
### Look at some more examples
% cat test_tweet.sh

Developing for Node.JS with MySQL and NoSQL

More Related Content

What's hot

Similar to Developing for Node.JS with MySQL and NoSQL

Recently uploaded

Developing for Node.JS with MySQL and NoSQL

Editor's Notes