Round pegs and square holes

Round Pegs
and
Square Holes
Django and MongoDB

by Daniel Greenfeld and Audrey Roy

Audrey / Danny

• Principals at Cartwheel
Web
Photo credit: Christopher Neugebauer
• Co-founders of
consumer.io
• Met at PyCon 2010
• Now affianced

@pydanny / @audreyr cartwheelweb.com

What is MongoDB?

• NoSQL
• Fast, Scalable, Indexable, Shardable
• Schema-less


What is MongoDB?
• Written in C++
• Stores data in BSON (kinda like
JSON)
• Uses JavaScript internally for
scripting
• Has Python, Ruby, PHP, C, other
drivers
• Huge community

MongoDB: SQL Analogy

• Collections are like tables
• Documents are like records (rows)


What is a MongoDB queryset?
As served by pymongo

Minimalist view


What is a MongoDB queryset?
As served by pymongo

A list
of dictionaries
Minimalist view


What it looks like
list Minimalist view

collection = []
document = {
'_objectId': ObjectId('4f844e916c97c1000c000003'),
'username': 'pydanny',
'fiancee': {
'username': 'audreyr',
'location': 'Zurich'
}
}
collection = [document, ]


What it looks like

dictiona
collection = []
document = {
'fiancee': {
}
}


What it looks like

dictiona
collection = []
document = {
'fiancee': {
}
}

list of dictionaries!

Tools for using
MongoDB with Django
(actually, most of these are not Django-

“MongoDB connectors”

• pymongo
• mongoengine
• mongokit
• Django-nonrel
• the only one here requiring
Django


pymongo
http://api.mongodb.org/python/current/
Ofﬁcial Python binding for MongoDB
>>> from pymongo import Connection

>>> connection = Connection()

>>> my_data = {'rating': 3, 'title': 'I like ice cream'}

>>> connection.reviews.insert(my_data)

>>> your_data = {'rating': 3, 'subject': 'You like ice cream'}

>>> connection.reviews.insert(your_data)


pymongo
my_data with title







pymongo
my_data with title






your_data with subject

pymongo
[
{'rating': 3,'title': 'I like ice cream'},
{'rating': 3, 'subject': 'You like ice cream'}
]

>>> connection = pymongo.Connection()
>>> db = connection.db
>>> for review in db.reviews.find({'rating': 3}):
... review['title']
>>> for review in db.reviews.find(
... {"title": {"$regex": "ice cream"} }
... ):
... review['title']


pymongo
[
]

... review['title']
... ):
... review['title']

Finds all reviews
with a rating of 3

pymongo
[
]

... review['title']
... ):
... review['title']

Only ﬁnds the document
with ‘title’ in it.

pymongo
Pros

• Really fast, bare metal
• Lets you go schema-crazy
• Supported directly by 10gen
• They say “PyMongo to work with
recommended way
is the
MongoDB from Python.”


pymongo
Cons

• “PyMongo introspection” is an
oxymoron
• Very low-level
• Lose out on ModelForms, Auth,
Admin, etc
• Syntax mapperclean as with an
object
not as


MongoEngine
http://mongoengine.org/

Doesn’t this look
import mongoengine as me
like the Django
class Review(me.Document):
ORM?
title = me.StringField()
body = me.StringField()
author = me.StringField()
created = me.DateTimeField(default=datetime.utcnow)
rating = me.IntField()


MongoEngine

Doesn’t this look
like a Django
query?
>>> from reviews.models import Review
>>> for review in Review.objects.all():
... review.title


MongoEngine
Pros

• Looks similar to Django ORM code
• You can develop SUPER-QUICKLY
• Can use with django-mongonaut for
introspection
• Light schema, unenforced by the db
• Supports some inter-document
connections


MongoEngine
Cons

• Some feel there’s too much structure
• Validation messages sometimes
unclear
• Using it with Django,introspection*on
the Django Admin’s
you lose out
and ModelForms

* django-mongonaut addresses this.


mongoengine and
django-mongonaut
http://bit.ly/django-mongonaut


MongoKit
http://namlook.github.com/mongokit/

from mongokit import Document, Connection

connection = Connection()

@connection.register
class Review(Document):
structure = {
'title':unicode,
'body':unicode,
'author':unicode,
'created':datetime.datetime,
'rating':int
}
required_fields = ['title', 'author', 'created']
default_values = {'rating': 0,
'created': datetime.utcnow}


MongoKit



class Review(Document): Review model
structure = { representing
'title':unicode, MongoDB
'body':unicode,
'author':unicode,
collection
'created':datetime.datetime,
'rating':int
}


MongoKit



'body':unicode,
Expected 'author':unicode,
collection
ﬁelds 'created':datetime.datetime,
'rating':int
}


MongoKit
Connect
to
from mongokit import Document, Connection reviews
collectio
n
'body':unicode,
Expected 'author':unicode,
collection
ﬁelds 'created':datetime.datetime,
'rating':int
}


MongoKit

>>> from mongokit import Connection
>>> for review in connection.Review.find({'rating': 3}):
... review['title']

Identical to pymongo queries


MongoKit
Pros

• Light schema, unenforced by the db
• Or can go all out schemaless!
• Speed
• Types are a mix of Python &
MongoDB
• Uses pymongo-style queries

MongoKit
Cons

• Using itDjango Admin’s lose out
on the
with Django, you
introspection, ModelForms, auth,
etc
• Introspection is hard


Option 4
Django-nonrel
+
mongodb-engine

Django-nonrel
+ mongodb-engine
http://docs.django-nonrel.org

• A patch to Django thatORM NoSQL
support to the Django
adds

• Works with GAE, MongoDB, even SQL
DBs


Django-nonrel
+ mongodb-engine
Pros

• Can use Django as you normally
would
• Mirrors the ORM functionality
• Introspection via djangotoolbox


Django-nonrel
+ mongodb-engine
Cons

• Fork of the whole Django project.
• Dependent Django core (still on
parity with
on others to maintain
Django 1.3).
• Multi-db usage is confusing


Summary

• pymongo is low-level and well-supported
by 10gen.
• MongoEngine is like schemaless Django
models.
• MongoKit is like pymongo with extra
structure.
• Django-nonrel is a fork of Django 1.3.

Danny’s Thoughts
Can we build a “simple” bridge?

What about a single third-party app that lets you
combine critical Django apps and MongoDB?

• django.contrib.auth
• django.forms
• django-social-auth /
registration
• others...

Danny’s Thoughts
I wonder, why add schemas to schema-less
when:
Relational Databases
South
High level Caching
tools*
allow you to develop fast moving
datastores with transactions and built-
in Djangoorsupport?
* cache-machine johnny-cache


Danny’s Thoughts

Introspection Tool Idea

Treat introspection like MongoDB
Queries

Immediate introspection tool
(no ‘title’ then don’t show title)


Audrey’s Thoughts

• Schemaless dbs promise
performance advantages
• Especially for distributed systems
• Tradeoff: ACID compliance
http://stackoverﬂow.com/questions/3856222/whats-the-attraction-of-schemaless-
database-systems


Audrey’s Thoughts
“Schemaless database”
==
ACID-noncompliant database

OK when performance is more important than
being consistent 100% of the time.


Audrey’s Thoughts
Schemaless Python models
!=
Schemaless MongoDB collections

I think MongoEngine is best unless your use case
calls for schema anarchy.


Using Django With
MongoDB

• Big hurdles: ORM, Admin, ModelForms,
Auth
• Were built for relational data
• But the situation is improving rapidly


What needs to be
done
• New introspection tools (working on it)
• django.forms bridge.
• django.contrib.admin bridge.
• Drop-in 1.5?)
(Django
replacement for Django Auth

• Creation of best practices document for
use of MongoDB with Django.


Django Mongonaut

• Introspection tool for MongoEngine
• Works well so far
• Integrate graphing tools
• Make independent from mongoengine
• Contributors wanted:
• https://github.com/pydanny/django-
mongonaut


Django Mongonaut
Based off of immediate
introspection - not
deﬁnition

@pydanny / @audreyr
Mockup cartwheelweb.com

Django Mongonaut
Based off of immediate
introspection - not
deﬁnition
SVG w/links
for examining
nested
structures

@pydanny / @audreyr
Mockup cartwheelweb.com

Final Summary

{
pymongo
mongoengine
Python/ mongokit
MongoDB tools django-nonrel
to consider django-mongodb
django-
mongonaut


Round pegs and square holes

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Round pegs and square holes

Similar to Round pegs and square holes (20)

More from Daniel Greenfeld

More from Daniel Greenfeld (13)

Recently uploaded

Recently uploaded (20)

Round pegs and square holes

Editor's Notes