Round Pegs
and
Square Holes
Django and MongoDB
by Daniel Greenfeld and Audrey Roy

Audrey / Danny
• Principals at Cartwheel
Web
Photo credit: Christopher Neugebauer
• Co-founders of
consumer.io
• Met at PyCon 2010
• Now affianced
@pydanny / @audreyr cartwheelweb.com

What is MongoDB?
• NoSQL
• Fast, Scalable, Indexable, Shardable
• Schema-less
@pydanny / @audreyr cartwheelweb.com

What is MongoDB?
• Written in C++
• Stores data in BSON (kinda like
JSON)
• Uses JavaScript internally for
scripting
• Has Python, Ruby, PHP, C, other
drivers
• Huge community
@pydanny / @audreyr cartwheelweb.com

MongoDB: SQL Analogy
• Collections are like tables
• Documents are like records (rows)
@pydanny / @audreyr cartwheelweb.com

What is a MongoDB queryset?
As served by pymongo
Minimalist view
@pydanny / @audreyr cartwheelweb.com

What is a MongoDB queryset?
As served by pymongo
A list
of dictionaries
Minimalist view
@pydanny / @audreyr cartwheelweb.com

What it looks like
list Minimalist view
collection = []
document = {
'_objectId': ObjectId('4f844e916c97c1000c000003'),
'username': 'pydanny',
'fiancee': {
'username': 'audreyr',
'location': 'Zurich'
}
}
collection = [document, ]
@pydanny / @audreyr cartwheelweb.com

What it looks like
list Minimalist view
dictiona
collection = []
document = {
'_objectId': ObjectId('4f844e916c97c1000c000003'),
'username': 'pydanny',
'fiancee': {
'username': 'audreyr',
'location': 'Zurich'
}
}
collection = [document, ]
@pydanny / @audreyr cartwheelweb.com

What it looks like
list Minimalist view
dictiona
collection = []
document = {
'_objectId': ObjectId('4f844e916c97c1000c000003'),
'username': 'pydanny',
'fiancee': {
'username': 'audreyr',
'location': 'Zurich'
}
}
collection = [document, ]
list of dictionaries!
@pydanny / @audreyr cartwheelweb.com

Tools for using
MongoDB with Django
(actually, most of these are not Django-

“MongoDB connectors”
• pymongo
• mongoengine
• mongokit
• Django-nonrel
• the only one here requiring
Django
@pydanny / @audreyr cartwheelweb.com

Option 1
pymongo

pymongo
http://api.mongodb.org/python/current/
Official Python binding for MongoDB
>>> from pymongo import Connection
>>> connection = Connection()
>>> my_data = {'rating': 3, 'title': 'I like ice cream'}
>>> connection.reviews.insert(my_data)
>>> your_data = {'rating': 3, 'subject': 'You like ice cream'}
>>> connection.reviews.insert(your_data)
@pydanny / @audreyr cartwheelweb.com

pymongo
http://api.mongodb.org/python/current/
Official Python binding for MongoDB
my_data with title
>>> from pymongo import Connection
>>> connection = Connection()
>>> my_data = {'rating': 3, 'title': 'I like ice cream'}
>>> connection.reviews.insert(my_data)
>>> your_data = {'rating': 3, 'subject': 'You like ice cream'}
>>> connection.reviews.insert(your_data)
@pydanny / @audreyr cartwheelweb.com

pymongo
http://api.mongodb.org/python/current/
Official Python binding for MongoDB
my_data with title
>>> from pymongo import Connection
>>> connection = Connection()
>>> my_data = {'rating': 3, 'title': 'I like ice cream'}
>>> connection.reviews.insert(my_data)
>>> your_data = {'rating': 3, 'subject': 'You like ice cream'}
>>> connection.reviews.insert(your_data)
your_data with subject
@pydanny / @audreyr cartwheelweb.com

pymongo
[
{'rating': 3,'title': 'I like ice cream'},
{'rating': 3, 'subject': 'You like ice cream'}
]
>>> connection = pymongo.Connection()
>>> db = connection.db
>>> for review in db.reviews.find({'rating': 3}):
... review['title']
>>> for review in db.reviews.find(
... {"title": {"$regex": "ice cream"} }
... ):
... review['title']
@pydanny / @audreyr cartwheelweb.com

pymongo
[
{'rating': 3,'title': 'I like ice cream'},
{'rating': 3, 'subject': 'You like ice cream'}
]
>>> connection = pymongo.Connection()
>>> db = connection.db
>>> for review in db.reviews.find({'rating': 3}):
... review['title']
>>> for review in db.reviews.find(
... {"title": {"$regex": "ice cream"} }
... ):
... review['title']
Finds all reviews
with a rating of 3
@pydanny / @audreyr cartwheelweb.com

pymongo
[
{'rating': 3,'title': 'I like ice cream'},
{'rating': 3, 'subject': 'You like ice cream'}
]
>>> connection = pymongo.Connection()
>>> db = connection.db
>>> for review in db.reviews.find({'rating': 3}):
... review['title']
>>> for review in db.reviews.find(
... {"title": {"$regex": "ice cream"} }
... ):
... review['title']
@pydanny / @audreyr cartwheelweb.com

pymongo
[
{'rating': 3,'title': 'I like ice cream'},
{'rating': 3, 'subject': 'You like ice cream'}
]
>>> connection = pymongo.Connection()
>>> db = connection.db
>>> for review in db.reviews.find({'rating': 3}):
... review['title']
>>> for review in db.reviews.find(
... {"title": {"$regex": "ice cream"} }
... ):
... review['title']
Only finds the document
with ‘title’ in it.
@pydanny / @audreyr cartwheelweb.com

pymongo
Pros
• Really fast, bare metal
• Lets you go schema-crazy
• Supported directly by 10gen
• They say “PyMongo to work with
recommended way
is the
MongoDB from Python.”
@pydanny / @audreyr cartwheelweb.com

pymongo
Cons
• “PyMongo introspection” is an
oxymoron
• Very low-level
• Lose out on ModelForms, Auth,
Admin, etc
• Syntax mapperclean as with an
object
not as
@pydanny / @audreyr cartwheelweb.com

Option 2
MongoEngine

MongoEngine
http://mongoengine.org/
Doesn’t this look
import mongoengine as me
like the Django
class Review(me.Document):
ORM?
title = me.StringField()
body = me.StringField()
author = me.StringField()
created = me.DateTimeField(default=datetime.utcnow)
rating = me.IntField()
@pydanny / @audreyr cartwheelweb.com

MongoEngine
Doesn’t this look
like a Django
query?
>>> from reviews.models import Review
>>> for review in Review.objects.all():
... review.title
@pydanny / @audreyr cartwheelweb.com

MongoEngine
Pros
• Looks similar to Django ORM code
• You can develop SUPER-QUICKLY
• Can use with django-mongonaut for
introspection
• Light schema, unenforced by the db
• Supports some inter-document
connections
@pydanny / @audreyr cartwheelweb.com

MongoEngine
Cons
• Some feel there’s too much structure
• Validation messages sometimes
unclear
• Using it with Django,introspection*on
the Django Admin’s
you lose out
and ModelForms
* django-mongonaut addresses this.
@pydanny / @audreyr cartwheelweb.com

mongoengine and
django-mongonaut
http://bit.ly/django-mongonaut
@pydanny / @audreyr cartwheelweb.com

Option 3
MongoKit

MongoKit
http://namlook.github.com/mongokit/
from mongokit import Document, Connection
connection = Connection()
@connection.register
class Review(Document):
structure = {
'title':unicode,
'body':unicode,
'author':unicode,
'created':datetime.datetime,
'rating':int
}
required_fields = ['title', 'author', 'created']
default_values = {'rating': 0,
'created': datetime.utcnow}
@pydanny / @audreyr cartwheelweb.com

MongoKit
http://namlook.github.com/mongokit/
from mongokit import Document, Connection
connection = Connection()
@connection.register
class Review(Document): Review model
structure = { representing
'title':unicode, MongoDB
'body':unicode,
'author':unicode,
collection
'created':datetime.datetime,
'rating':int
}
required_fields = ['title', 'author', 'created']
default_values = {'rating': 0,
'created': datetime.utcnow}
@pydanny / @audreyr cartwheelweb.com

MongoKit
http://namlook.github.com/mongokit/
from mongokit import Document, Connection
connection = Connection()
@connection.register
class Review(Document): Review model
structure = { representing
'title':unicode, MongoDB
'body':unicode,
Expected 'author':unicode,
collection
fields 'created':datetime.datetime,
'rating':int
}
required_fields = ['title', 'author', 'created']
default_values = {'rating': 0,
'created': datetime.utcnow}
@pydanny / @audreyr cartwheelweb.com

MongoKit
http://namlook.github.com/mongokit/
Connect
to
from mongokit import Document, Connection reviews
collectio
connection = Connection()
n
@connection.register
class Review(Document): Review model
structure = { representing
'title':unicode, MongoDB
'body':unicode,
Expected 'author':unicode,
collection
fields 'created':datetime.datetime,
'rating':int
}
required_fields = ['title', 'author', 'created']
default_values = {'rating': 0,
'created': datetime.utcnow}
@pydanny / @audreyr cartwheelweb.com

MongoKit
http://namlook.github.com/mongokit/
>>> from mongokit import Connection
>>> connection = Connection()
>>> for review in connection.Review.find({'rating': 3}):
... review['title']
Identical to pymongo queries
@pydanny / @audreyr cartwheelweb.com

MongoKit
Pros
• Light schema, unenforced by the db
• Or can go all out schemaless!
• Speed
• Types are a mix of Python &
MongoDB
• Uses pymongo-style queries
@pydanny / @audreyr cartwheelweb.com

MongoKit
Cons
• Using itDjango Admin’s lose out
on the
with Django, you
introspection, ModelForms, auth,
etc
• Introspection is hard
@pydanny / @audreyr cartwheelweb.com

Option 4
Django-nonrel
+
mongodb-engine

Django-nonrel
+ mongodb-engine
http://docs.django-nonrel.org
• A patch to Django thatORM NoSQL
support to the Django
adds
• Works with GAE, MongoDB, even SQL
DBs
@pydanny / @audreyr cartwheelweb.com

Django-nonrel
+ mongodb-engine
Pros
• Can use Django as you normally
would
• Mirrors the ORM functionality
• Introspection via djangotoolbox
@pydanny / @audreyr cartwheelweb.com

Django-nonrel
+ mongodb-engine
Cons
• Fork of the whole Django project.
• Dependent Django core (still on
parity with
on others to maintain
Django 1.3).
• Multi-db usage is confusing
@pydanny / @audreyr cartwheelweb.com

Summary
• pymongo is low-level and well-supported
by 10gen.
• MongoEngine is like schemaless Django
models.
• MongoKit is like pymongo with extra
structure.
• Django-nonrel is a fork of Django 1.3.
@pydanny / @audreyr cartwheelweb.com

The Take Away?

Danny’s Thoughts
Can we build a “simple” bridge?
What about a single third-party app that lets you
combine critical Django apps and MongoDB?
• django.contrib.auth
• django.forms
• django-social-auth /
registration
• others...
@pydanny / @audreyr cartwheelweb.com

Danny’s Thoughts
I wonder, why add schemas to schema-less
when:
Relational Databases
South
High level Caching
tools*
allow you to develop fast moving
datastores with transactions and built-
in Djangoorsupport?
* cache-machine johnny-cache
@pydanny / @audreyr cartwheelweb.com

Danny’s Thoughts
Introspection Tool Idea
Treat introspection like MongoDB
Queries
Immediate introspection tool
(no ‘title’ then don’t show title)
@pydanny / @audreyr cartwheelweb.com

Audrey’s Thoughts
• Schemaless dbs promise
performance advantages
• Especially for distributed systems
• Tradeoff: ACID compliance
http://stackoverflow.com/questions/3856222/whats-the-attraction-of-schemaless-
database-systems
@pydanny / @audreyr cartwheelweb.com

Audrey’s Thoughts
“Schemaless database”
==
ACID-noncompliant database
OK when performance is more important than
being consistent 100% of the time.
@pydanny / @audreyr cartwheelweb.com

Audrey’s Thoughts
Schemaless Python models
!=
Schemaless MongoDB collections
I think MongoEngine is best unless your use case
calls for schema anarchy.
@pydanny / @audreyr cartwheelweb.com

Going Forward

Using Django With
MongoDB
• Big hurdles: ORM, Admin, ModelForms,
Auth
• Were built for relational data
• But the situation is improving rapidly
@pydanny / @audreyr cartwheelweb.com

What needs to be
done
• New introspection tools (working on it)
• django.forms bridge.
• django.contrib.admin bridge.
• Drop-in 1.5?)
(Django
replacement for Django Auth
• Creation of best practices document for
use of MongoDB with Django.
@pydanny / @audreyr cartwheelweb.com

Django Mongonaut
• Introspection tool for MongoEngine
• Works well so far
• Integrate graphing tools
• Make independent from mongoengine
• Contributors wanted:
• https://github.com/pydanny/django-
mongonaut
@pydanny / @audreyr cartwheelweb.com

Django Mongonaut
Based off of immediate
introspection - not
definition
@pydanny / @audreyr
Mockup cartwheelweb.com

Django Mongonaut
Based off of immediate
introspection - not
definition
SVG w/links
for examining
nested
structures
@pydanny / @audreyr
Mockup cartwheelweb.com

Final Summary
{
pymongo
mongoengine
Python/ mongokit
MongoDB tools django-nonrel
to consider django-mongodb
django-
mongonaut
@pydanny / @audreyr cartwheelweb.com