This intermediate-level talk will teach you techniques using the popular NoSQL database MongoDB and the Python library Ming to write maintainable, high-performance, and scalable applications. We will cover everything you need to become an effective Ming/MongoDB developer from basic PyMongo queries to high-level object-document mapping setups in Ming.

1. Rick Copeland @rick446
Arborian Consulting, LLC

2. http://www.flickr.com/photos/fazen/9079179/

3. - Get started with PyMongo
- Sprinkle in some Ming schemas
- ODM: When a dict just won’t do

4. >>>importpymongo
>>>conn = pymongo.Connection()
>>>conn
Connection('localhost', 27017)
>>>conn.test
Database(Connection('localhost', 27017), u'test')
>>>conn.test.foo
Collection(Database(Connection('localhost', 27017), u'test'), u'foo')
>>>conn['test-db']
Database(Connection('localhost', 27017), u'test-db')
>>>conn['test-db']['foo-collection']
Collection(Database(Connection('localhost', 27017), u'test-db'), u'foo-
collection')
>>>conn.test.foo.bar.baz
Collection(Database(Connection('localhost', 27017), u'test'), u'foo.bar.baz')

5. >>>db = conn.test
>>>id = db.foo.insert({'bar':1, 'baz':[ 1, 2, {'k':5} ] })
>>>id
ObjectId('4e712e21eb033009fa000000') Auto-Generated _id
>>>db.foo.find()
<pymongo.cursor.Cursor object at 0x29c7d50>
>>>list(db.foo.find()) Cursors are python
[{u'bar': 1, u'_id': ObjectId('4e712e21eb033009fa000000'), u'baz': [1, 2, {'k': 5}]}]
generators
>>>db.foo.update({'_id':id}, {'$set': { 'bar':2}})
>>>db.foo.find().next()
{u'bar': 2, u'_id': ObjectId('4e712e21eb033009fa000000'), u'baz': [1, 2, {'k': 5}]}
>>>db.foo.remove({'_id':id})
>>>list(db.foo.find())
[]
Remove uses same
query language as find()

6. >>>db.foo.insert([ dict(x=x) forxinrange(10) ])
[ObjectId('4e71313aeb033009fa00000b'), … ]
>>>list(db.foo.find({ 'x': {'$gt': 3} })) Range Query
[{u'x': 4, u'_id': ObjectId('4e71313aeb033009fa00000f')}, {u'x': 5, u'_id':
ObjectId('4e71313aeb033009fa000010')}, {u'x': 6, u'_id':
ObjectId('4e71313aeb033009fa000011')}, …]
>>>list(db.foo.find({ 'x': {'$gt': 3} }, { '_id':0 } ))
[{u'x': 4}, {u'x': 5}, {u'x': 6}, {u'x': 7}, {u'x': 8}, {u'x': 9}]
>>>list(db.foo.find({ 'x': {'$gt': 3} }, { '_id':0 } )
....skip(1).limit(2))
[{u'x': 5}, {u'x': 6}]
>>>db.foo.ensure_index([
Partial Retrieval
...('x’,pymongo.ASCENDING),('y’,pymongo.DESCENDING)])
u'x_1_y_-1’
Compound Indexes

7. One Rule (for now): Avoid Javascript
http://www.flickr.com/photos/lizjones/295567490/

8.  You gotta write Javascript (for now)
 It’s pretty slow (single-threaded JS engine) 
MongoDB 2.2 with New
 Javascript is used by Aggregation Framework
 $where in a query Coming Real Soon Now ™
 .group(key, condition, initial, reduce, finalize=None)
 .map_reduce(map, reduce, out, finalize=None, …)
 Sharding gives some parallelism with .map_reduce() (and
possibly ‘$where’). Otherwise you’re single threaded.

9. >>>importgridfs
>>>fs = gridfs.GridFS(db) Python context
>>>withfs.new_file() asfp: manager
... fp.write('The file')
...
>>>fp
<gridfs.grid_file.GridIn object at 0x2cae910>
>>>fp._id
ObjectId('4e727f64eb03300c0b000003') Retrieve file by _id
>>>fs.get(fp._id).read()
'The file'
 Arbitrary data can be stored in the ‘fp’ object – it’s
just a Document (but please put it in ‘fp.metadata’)
 Mime type, links to other docs, etc.

10. >>>file_id = fs.put('Moar data!', filename='foo.txt')
>>>fs.get_last_version('foo.txt').read()
'Moar data!’
Create file by
filename
>>>file_id = fs.put('Evenmoar data!', filename='foo.txt')
>>>fs.get_last_version('foo.txt').read()
'Even moar data!’
>>>fs.get_version('foo.txt', -2).read()
'Moar data!’
>>>fs.list()
“2nd from the last”
[u'foo.txt']
>>>fs.delete(fs.get_last_version('foo.txt')._id)
>>>fs.list()
[u'foo.txt']
>>>fs.delete(fs.get_last_version('foo.txt')._id)
>>>fs.list()
[]

11. - Get started with PyMongo
- Sprinkle in some Ming schemas
- ODM: When a dict just won’t do

12.  Your data has a schema
 Your database can define and enforce it
 It can live in your application (as with MongoDB)
 Nice to have the schema defined in one place in the code
 Sometimes you need a “migration”
 Changing the structure/meaning of fields
 Adding indexes, particularly unique indexes
 Sometimes lazy, sometimes eager
 “Unit of work:” Queuing up all your updates can be handy

13. Model Datastore
(schema) (database)
Session

14. >>>importming.datastore
>>>ds = ming.datastore.DataStore('mongodb://localhost:27017', database='test')
>>>ds.db Connection +
Database(Connection('localhost', 27017), u'test') Database
>>>session = ming.Session(ds)
>>>session.db
Database(Connection('localhost', 27017), u'test')
>>>ming.configure(**{ Optimized for config
... 'ming.main.master':'mongodb://localhost:27017', files
... 'ming.main.database':'test'})
>>>Session.by_name('main').db
Database(Connection(u'localhost', 27017), u'test')

15. http://www.flickr.com/photos/pictureclara/5333266789/

16. frommingimport schema, Field
WikiDoc= collection(‘wiki_page', session,
Field('_id', schema.ObjectId()),
Index created on
Field('title', str, index=True),
import
Field('text', str))
CommentDoc= collection(‘comment', session,
Field('_id', schema.ObjectId()),
Field('page_id', schema.ObjectId(), index=True),
Field('text', str))
Shorthand for
schema.String

17. frommingimport Document, Session, Field
classWikiDoc(Document):
class__mongometa__:
session=Session.by_name(’main')
name='wiki_page’
indexes=[ ('title') ]
title = Field(str)
text = Field(str)
 Old declarative syntax continues to exist and be
supported, but it’s not being actively improved
 Sometimes nice when you want additional
methods/attrs on your document class

18. >>>doc = WikiDoc(dict(title='Cats', text='I can hazcheezburger?'))
>>>doc.m.save()
>>>WikiDoc.m.find()
<ming.base.Cursor object at 0x2c2cd90>
>>>WikiDoc.m.find().all() Documents are dict
subclasses
[{'text': u'I can hazcheezburger?', '_id': ObjectId('4e727163eb03300c0b000001'),
'title': u'Cats'}]
>>>WikiDoc.m.find().one().text
u'Ican hazcheezburger?’
>>>doc = WikiDoc(dict(tietul='LOL', text='Invisible bicycle'))
>>>doc.m.save()
Traceback(most recent call last):
File "<stdin>", line 1,
… Exception pinpoints
ming.schema.Invalid: <class 'ming.metadata.Document<wiki_page>'>:
problem
Extra keys: set(['tietul'])

19. >>>ming.datastore.DataStore('mim://', database='test').db
mim.Database(test)
 MongoDB is (generally) fast
 … except when creating databases
 … particularly when you preallocate
 Unit tests like things to be isolated
 MIM gives you isolation at the expense of speed & scaling

20. - Get started with PyMongo
- Sprinkle in some Ming schemas
- ODM: When a dict just won’t do

21. frommingimport schema, Field
fromming.odmimport (mapper, Mapper, RelationProperty,
ForeignIdProperty)
Plain Old Python
WikiDoc= collection('wiki_page',session, … ) Classes
CommentDoc= collection(’comment’, session, … )
classWikiPage(object): pass Map classes to
classComment(object): pass collection + session
odmsession.mapper(WikiPage, WikiDoc, properties=dict(
comments=RelationProperty('WikiComment')))
odmsession.mapper(Comment, CommentDoc, properties=dict(
page_id=ForeignIdProperty('WikiPage'),
page=RelationProperty('WikiPage')))
“Relations”

22. classWikiPage(MappedClass):
class__mongometa__:
session = main_odm_session
name='wiki_page’
indexes = [ 'title']
_id = FieldProperty(S.ObjectId)
title = FieldProperty(str)
text = FieldProperty(str)
comments = RelationProperty(’Comment’)

23.  Session ODMSession
 My_collection.m… My_mapped_class.query…
 ODMSessionactually does stuff
 Track object identity
 Track object modifications
 Unit of work flushing all changes at once
>>>pg = WikiPage(title='MyPage', text='is here')
>>>session.db.wiki_page.count()
0
>>>main_orm_session.flush()
>>>session.db.wiki_page.count()
1

24. http://www.flickr.com/photos/39747297@N05/5229733647/

25.  Various plug points in the session
 before_flush
 after_flush
 Some uses
 Logging changes to sensitive data or for analytics
 Full-text search indexing
 “last modified” fields
 Performance instrumentation

26.  Various plug points in the mapper
 before_/after_:
 Insert
 Update
 Delete
 Remove
 Some uses
 Collection/model-specific logging (user creation, etc.)
 Anything you might want a SessionExtension for but
would rather do per-model

27. PyMongo
http://api.mongodb.org/python
Apache License
Ming
http://sf.net/projects/merciless/
MIT License

28. Rick Copeland @rick446
Arborian Consulting, LLC
Feedback? http://www.surveymonkey.com/s/5DLCYKN
http://www.flickr.com/photos/f-oxymoron/5005673112/