My presentation on MongoDB and node.js from MongoDC 2011

1. Using MongoDB with node.js
Jonathan Altman
@async_io
http://async.io/
http://github.com/jonathana
MongoDC 2011

2. what is node.js?
• A non-browser Javascript toolkit/framework
started by Ryan Dahl
• Available for *nix-based systems: Linux, OS X, OpenSolaris, and now Windows
• Built on top of Google’s V8 Javascript engine
• Compatible with many common Javascript libraries out of the box via CommonJS
support (http://www.commonjs.org/)
• A batteries-included framework: HTTP and socket support baked into the core of the
framework
• A framework that is easy to build tooling on top of
• Most importantly: asynchronous from the ground up
2

3. what is node.js?
• A non-browser Javascript toolkit/framework started by Ryan Dahl
• Available for *nix-based systems: Linux, OS X,
OpenSolaris, and now Windows
• Built on top of Google’s V8 Javascript engine
• Compatible with many common Javascript libraries out of the box via CommonJS
support (http://www.commonjs.org/)
• A batteries-included framework: HTTP and socket support baked into the core of the
framework
• A framework that is easy to build tooling on top of
• Most importantly: asynchronous from the ground up
3

4. what is node.js?
• A non-browser Javascript toolkit/framework started by Ryan Dahl
• Available for *nix-based systems: Linux, OS X, OpenSolaris, and now Windows
• Built on top of Google’s V8 Javascript engine
• Compatible with many common Javascript libraries out of the box via
CommonJS support (http://www.commonjs.org/)
• A batteries-included framework: HTTP and socket support baked into the
core of the framework
• A framework that is easy to build tooling on top of
• Most importantly: asynchronous from the ground up
4

5. what is node.js?
• A non-browser Javascript toolkit/framework started by Ryan Dahl
• Available for *nix-based systems: Linux, OS X, OpenSolaris, and now Windows
• Built on top of Google’s V8 Javascript engine
• Compatible with many common Javascript libraries out of
the box via CommonJS support (http://www.commonjs.org/)
• A batteries-included framework: HTTP and socket support baked into the
core of the framework
• A framework that is easy to build tooling on top of
• Most importantly: asynchronous from the ground up
5

6. what is node.js?
• A non-browser Javascript toolkit/framework started by Ryan Dahl
• Available for *nix-based systems: Linux, OS X, OpenSolaris, and now Windows
• Built on top of Google’s V8 Javascript engine
• Compatible with many common Javascript libraries out of the box via CommonJS
support (http://www.commonjs.org/)
• A batteries-included framework: HTTP and socket
support baked into the core of the framework
• A framework that is easy to build tooling on top of
• Most importantly: asynchronous from the ground up
6

7. what is node.js?
• A non-browser Javascript toolkit/framework started by Ryan Dahl
• Available for *nix-based systems: Linux, OS X, OpenSolaris, and now Windows
• Built on top of Google’s V8 Javascript engine
• Compatible with many common Javascript libraries out of the box via CommonJS
support (http://www.commonjs.org/)
• A batteries-included framework: HTTP and socket support baked into the core of
the framework
• A framework that is easy to build tooling on top of
• Most importantly: asynchronous from the ground up
7

8. what is node.js?
• A non-browser Javascript toolkit/framework started by Ryan Dahl
• Available for *nix-based systems: Linux, OS X, OpenSolaris, and now Windows
• Built on top of Google’s V8 Javascript engine
• Compatible with many common Javascript libraries out of the box via CommonJS
support (http://www.commonjs.org/)
• A batteries-included framework: HTTP and socket support baked into the core of the
framework
• A framework that is easy to build tooling on top of
• Most importantly: asynchronous from the ground up
8

9. getting started with node
• Longer than we have time for today
• I have some node.js resources pulled together at the
end of the presentation
• But (shameless plug) may I recommend http://
www.slideshare.net/async_io/dcjq-nodejs-presentation
as a good starting point?

10. why node with MongoDB?
• Both toolkits heavily leverage Javascript for their
capabilities
• Commonality of programming language
• MongoDB’s speed/programming model is highly
compatible with node’s asynchronous model
• Mostly compatible data types, but more on that later

11. using MongoDB in node.js
• connect-mongodb: web framework middleware for
a MongoDB-backed session
• node-mongodb-native: native node driver for
MongoDB
• mongoose: Javascript<->MongoDB object mapper
• mongolia: “non-magic” layer on the mongodb native
driver

12. installing the packages
• All packages are available through npm, the node package
manager:
npm install connect-mongodb
npm install mongodb #(node-mongodb-
native driver)
npm install mongoose
npm install mongolia

13. connect-mongodb
• Provides MongoDB-backed session storage for the
connect/express web development stack
• connect: Middleware layer for node.js (think python WSGI
or ruby’s rack for node)
• Express: “Sinatra inspired web development framework for
node.js -- insanely fast, flexible, and sexy”
• We are going to examine use with Express
• npm install express

14. wire connect-mongodb
sessions into express
var MongoDBSessionStore = require('connect-mongodb');
var app = module.exports = express.createServer(
express.bodyParser(),
express.methodOverride(),
express.cookieParser(),
// You *NEED* to use a better secret than this, and store it
in a better way...
express.session({store: new MongoDBSessionStore({ }), secret:
'foobar'})
);

15. get/set values from session
req.session.pageRenders = req.session.pageRenders || 0;
req.session.pageRenders++;
// increment some view counts
if ( !req.session.viewCount)
{ req.session.viewCount = {};}
if ( !req.session.viewCount.hasOwnProperty(calledPage) )
{ req.session.viewCount[calledPage] = 0; }
req.session.viewCount[calledPage] += 1;

16. example app: heatNode

17. how much magic do you
want?
node MongoDB database drivers

18. node-mongodb-native driver
• Exposes the MongoDB API to node
• Fairly light wrapper
• Pushes the need to write library/utility/wrapper
functionality onto the developer
• However, it has no preconceived vision of how to
interact with MongoDB, and very little with node

19. node-mongodb code snippet
var mongo = require('mongodb');
var Db= mongo.Db,
ObjectID= mongo.BSONPure.BSON.ObjectID,
Server= mongo.Server;
HeatmapProvider = function(host, port) {
this.db= new Db('heatNode', new Server(host, port, {auto_reconnect: true}, {}));
this.db.open(function(){});
};
HeatmapProvider.prototype.getCollection= function(callback) {
this.db.collection('heatevents', function(error, heatevents_collection) {
if( error ) callback(error);
else callback(null, heatevents_collection);
});
// Most of the *useful* code removed

20. mongoose: object mapping
and persistence
• Declarative description of Javascript objects that can be
persisted, retrieved, etc. with MongoDB
• Can provide defaults, constraints (validation), virtual
(calculated) fields
• Downside: reduces the plasticity of MongoDB
document collections through its Schema
• Has several useful plugins built on top of it:
authentication/authorization for example

21. using mongoose
• Define a schema:
var mongoose = require('mongoose'),
Schema = mongoose.Schema;
var HeatEvent = new Schema({
type : { type: String, enum: ['click']}
, eventStamp : { type: Date, default: Date.now }
, payload : {
clickTarget : String
, pageUrl : { type: String, index: true }
, clickPoint : {
X : Number
, Y : Number
}
}
});
HeatEvent.index({ 'type': 1, 'payload.pageUrl': 1});
var db = mongoose.connect('mongodb://localhost/heatNode');
module.exports = db.model('heatEvent', HeatEvent);

22. what did that schema buy us?
• CRUD and various other operations pre-built
• Constraints/Validation: built-in validation like this:
: type : { type: String, enum: ['click']}
or define your own callbacks
• Defaults: , eventStamp : { type: Date,
default: Date.now }
• Synthetic fields: map non-persisted values into/out of other
persisted fields in the schema

23. mongolia: no-magic object
mapping
• Layer built on top of the native mongodb driver
• Exposes the collection operations of the driver
• Provides a way to build type mapping
• Provides an event-hook system where you can build
validations, defaulting, other features
• Not magic because it provides facilities for you to roll
your own magic

24. tradeoffs in the drivers
• native mongodb driver: least overhead, but you
will end up needing to customize on top
• mongolia: tools to build mapping, data type/casting
support, event hooks. Some overhead, even for unused
capabilities
• mongoose: full field-level declarative mapper, events;
plugin system. “Full stack” object mapper, full overhead
• More magic == more overhead

25. more info on node.js and
resources

26. sample web development
stack
• node-inspector: debugging
• express (and connect): web framework, middleware, routing, controllers, views
• spark2: nice front-end for controlling node servers
• ejs templates: embedded javascript, for views
• connect-mongodb: mongoDB-backed sessions
• mongoose: mongoDB-based object mapper for models
• test: yeah, you should pick some and use them
• jsdom: manipulate html DOMs server-side
• jquery and/or YUI3: do cool stuff server side with the DOM
• backbone: nifty client and server controller framework
• socket.io: client/server Comet toolkit
26

27. more from me on node.js
and MongoDB:
• A longer presentation I did on what node.js is, and
getting it up and running: http://www.slideshare.net/
async_io/dcjq-nodejs-presentation
• Heatmapper app I built with node and MongoDB:
https://github.com/jonathana/heatNode

28. more info on the packages:
• connect-mongodb: https://github.com/masylum/
connect-mongodb
• node-mongodb-native: https://github.com/
christkv/node-mongodb-native
• mongolia: https://github.com/masylum/mongolia
• mongoose: http://mongoosejs.com/ and https://
github.com/learnboost/mongoose/

29. appendix: Resources
• Ryan Dahl: (http://tinyclouds.org/, https://github.com/ry)
• Ryan’s jsconf 2009 presentation: http://s3.amazonaws.com/four.livejournal/20091117/
jsconf.pdf
• Simon Willison’s blog post re: node.js: http://simonwillison.net/2009/Nov/23/node/
• node.js home: http://nodejs.org/, git repo: https://github.com/joyent/node/
• node modules: https://github.com/joyent/node/wiki/modules
• Isaac Schlueter: (npm and nave) https://github.com/isaacs/, http://blog.izs.me/
• Dav Glass’ mind-bending demonstration of using YUI server-side: http://
developer.yahoo.com/yui/theater/video.php?v=glass-node
• Nice list of some apps built using node.js + express: http://expressjs.com/
applications.html
29