Lessons learnt building                @RossC0                http://github.com/rozza
WHAT IS MONGODB? A document database   { _id : ObjectId("..."),                         author : "Ross", Highly scalable  ...
WHAT IS MONGODB?{ "_id" : ObjectId("..."),             { _id : ObjectId("..."),  "author" : "Ross",                     au...
Want to know more?http://education.10gen.com
WHY DO YOU EVEN NEED AN ODM?http://www.flickr.com/photos/51838104@N02/5841690990
SCHEMA LESS != CHAOSMongoDB a good fitDocuments schema in codeEnforces schemaData validationSpeeds up developmentBuild tool...
Inspired by Djangos ORMSupports Python 2.5 - Python 3.3Originally authored by Harry Marr 2010I took over development in Ma...
INTRODUCING                 MONGOENGINEclass Post(Document):    title = StringField(max_length=120, required=True)    auth...
CREATING A MODELclass Post(Document):    title = StringField(max_length=120, required=True)    author = ReferenceField(Use...
INSERTING DATA# Pass data into the constructoruser = User(email="ross@10gen.com", name="Ross").save()# Create instance and...
QUERYING DATA# An `objects` manager is added to every `Document` classusers = User.objects(email=ross@10gen.com)# Pass kwa...
6 LESSONS LEARNT
LESSON 1: DIVE IN!http://www.flickr.com/photos/jackace/565857899/
PROJECT STALLED     In May 2011>200 forks>100 issues~50 pull requests      I needed itVolunteered to helpStarted reviewing...
L E S S O N 2 : M E TA C L A S S E Shttp://www.flickr.com/photos/ubique/135848053
WHATS NEEDED TO MAKE      AN ORM?Instance methods validation data manipulate data convert data to and from mongodbQueryset...
METACLASSESclass Document(object):    __metaclass__ = TopLevelDocumentMetaclass    ...class EmbeddedDocument(object):    _...
METACLASSES 101         IN               TopLevelDocumentcls, name, bases, attrs                             Document     ...
METACLASSES              IN                      TopLevelDocument Creates default meta data inheritance rules, id_field, in...
METACLASSES          IN                    TopLevelDocumentMerges all fields from parentsAdds in own field definitions       ...
METACLASSES             OUT                                 TopLevelDocumentAdds in handling for delete rules             ...
METACLASSES         OUT                   TopLevelDocumentBuilds index specifications        DocumentInjects queryset manag...
LESSONS LEARNTSpend time learning what is being done and whyDont continually patch:Rewrote the metaclasses in 0.7
L E S S O N 3 : S T R AY I N G F R O M T H E PAT Hhttp://www.flickr.com/photos/51838104@N02/5841690990
REWRITING THE QUERY             LANGUAGE# In pymongo you pass dictionaries to queryuk_pages = db.page.find({"published": T...
REWRITING THE QUERY             LANGUAGE#Somethings are nicer - regular expresion searchdb.post.find({title: re.compile(Mo...
LESSON 4: NOT ALL IDEAS ARE GOODhttp://www.flickr.com/photos/abiding_silence/6951229015
CHANGING SAVE# In pymongo save replaces the whole documentdb.post.save({_id: my_id, title: MongoDB,              published...
CHANGING SAVEAny field changes are notedHow to monitor lists and dicts?  Custom List and Dict classes  Observes changes and...
HOW IT WORKSclass Post(Document):    title = StringField(max_length=120, required=True)    author = ReferenceField(User)  ...
HOW IT WORKSpost = Post.objects.first()      Postpost.comments[1].name = Fredpost.save()                                 -...
HOW IT WORKSpost.comments[1].name = Fred   Post1.Convert the                                 - comments  comments         ...
HOW IT WORKSpost.comments[1].name = Fred   Post2. Convert the                                 - comments   comment        ...
HOW IT WORKSpost.comments[1].name = Fred   Post3.Change name                                 - comments  to "Fred"        ...
HOW IT WORKSpost.save()                       Post5.On save()                                  - commentsIterate all the c...
A GOOD IDEA?+ Makes it easier to use+ save acts how people think it should- Its expensive- Doesnt help people understand M...
LESSON 5: MANAGING A COMMUNIT Yhttp://kingscross.the-hub.net/
CODERS JUST WANT TO CODE         *       Github effect >10 django mongoengine projects None active on pypi Little cross pro...
REACH OUT  Flask-mongoengine on  pypi   There were 2 different  projects   Now has extra maintainers   from the flask-mongor...
THE COMMUNITYNot all ideas are good!Vocal people dont always have great ideasTravis is great** but you still have to read ...
LESSON 6: DON T BE AFR AID TO ASKhttp://www.flickr.com/photos/kandyjaxx/2012468692
I NEED HELP ;) Website Documentation Tutorials Framework support Core mongoenginehttp://twitter.com/RossC0          http:/...
QUESTIONS?http://www.flickr.com/photos/9550033@N04/5020799468
Upcoming SlideShare
Loading in …5
×

Pyconie 2012

434 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
434
On SlideShare
0
From Embeds
0
Number of Embeds
63
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Pyconie 2012

  1. 1. Lessons learnt building @RossC0 http://github.com/rozza
  2. 2. WHAT IS MONGODB? A document database { _id : ObjectId("..."), author : "Ross", Highly scalable date : ISODate("2012-07-05..."), text : "About MongoDB...", tags : [ "tech", "databases" ], Developer friendly comments : [{ author : "Tim", date : ISODate("2012-07-05..."), text : "Best Post Ever!" }], comment_count : 1 }http://mongodb.org In BSON
  3. 3. WHAT IS MONGODB?{ "_id" : ObjectId("..."), { _id : ObjectId("..."), "author" : "Ross", author : "Ross", "date" : datetime(2012,7,5,10,0), date : ISODate("2012-07-05..."), "text" : "About MongoDB...", text : "About MongoDB...", "tags" : ["tech", "databases"], tags : [ "tech", "databases" ], "comments" : [{ comments : [{ "author" : "Tim", author : "Tim", "date" : datetime(2012,7,5,11,35), date : ISODate("2012-07-05..."), "text" : "Best Post Ever!" text : "Best Post Ever!" }], }], "comment_count" : 1 comment_count : 1} } In Python In BSON
  4. 4. Want to know more?http://education.10gen.com
  5. 5. WHY DO YOU EVEN NEED AN ODM?http://www.flickr.com/photos/51838104@N02/5841690990
  6. 6. SCHEMA LESS != CHAOSMongoDB a good fitDocuments schema in codeEnforces schemaData validationSpeeds up developmentBuild tooling off itCan DRY up code...
  7. 7. Inspired by Djangos ORMSupports Python 2.5 - Python 3.3Originally authored by Harry Marr 2010I took over development in May 2011Current release 0.7.5http://github.com/MongoEngine/mongoengine
  8. 8. INTRODUCING MONGOENGINEclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment))class Comment(EmbeddedDocument): content = StringField() name = StringField(max_length=120)class User(Document): email = StringField(required=True) first_name = StringField(max_length=50) last_name = StringField(max_length=50)
  9. 9. CREATING A MODELclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment)) Define a class inheriting from Document Map a field to a defined data type strings, ints, binary, files, lists etc.. By default all declared fields arent required Pass keyword arguments to apply constraints eg set if unique, max_length, default values.
  10. 10. INSERTING DATA# Pass data into the constructoruser = User(email="ross@10gen.com", name="Ross").save()# Create instance and edit / update in placepost = Post()post.title = "mongoengine"post.author = userpost.tags = [odm, mongodb, python]post.save() Create instance of the object Update its attributes Call save, insert, update to persist the data
  11. 11. QUERYING DATA# An `objects` manager is added to every `Document` classusers = User.objects(email=ross@10gen.com)# Pass kwargs to commands are lazy and be extended as neededusers.filter(auth=True)# Iterating evaluates the querysetprint [u for u in users] Documents have a queryset manager (objects) for querying You can continually extend it Queryset evaluated on iteration
  12. 12. 6 LESSONS LEARNT
  13. 13. LESSON 1: DIVE IN!http://www.flickr.com/photos/jackace/565857899/
  14. 14. PROJECT STALLED In May 2011>200 forks>100 issues~50 pull requests I needed itVolunteered to helpStarted reviewing issuesSupported Harry andcommunity
  15. 15. L E S S O N 2 : M E TA C L A S S E Shttp://www.flickr.com/photos/ubique/135848053
  16. 16. WHATS NEEDED TO MAKE AN ORM?Instance methods validation data manipulate data convert data to and from mongodbQueryset methods Finding data Bulk changes
  17. 17. METACLASSESclass Document(object): __metaclass__ = TopLevelDocumentMetaclass ...class EmbeddedDocument(object): __metaclass__ = DocumentMetaclass ... Needed for: 1. inspect the object inheritance 2. inject functionality to the class Its surprisingly simple - all we need is: __new__
  18. 18. METACLASSES 101 IN TopLevelDocumentcls, name, bases, attrs Document Out pythons type new class
  19. 19. METACLASSES IN TopLevelDocument Creates default meta data inheritance rules, id_field, index Document information, default ordering. Merges in parents meta Validation pythons type abstract flag on an inherited class collection set on a subclassManipulates the attrs going in.
  20. 20. METACLASSES IN TopLevelDocumentMerges all fields from parentsAdds in own field definitions DocumentCreates lookups_db_field_map_reverse_db_field_map pythons typeDetermine superclasses(for model inheritance)
  21. 21. METACLASSES OUT TopLevelDocumentAdds in handling for delete rules DocumentSo we can handle deleted ReferencesAdds class to the registrySo we can load the data into the correct class pythons type
  22. 22. METACLASSES OUT TopLevelDocumentBuilds index specifications DocumentInjects queryset managerSets primary key (if needed) pythons type
  23. 23. LESSONS LEARNTSpend time learning what is being done and whyDont continually patch:Rewrote the metaclasses in 0.7
  24. 24. L E S S O N 3 : S T R AY I N G F R O M T H E PAT Hhttp://www.flickr.com/photos/51838104@N02/5841690990
  25. 25. REWRITING THE QUERY LANGUAGE# In pymongo you pass dictionaries to queryuk_pages = db.page.find({"published": True})# In mongoengineuk_pages = Page.objects(published=True)# pymongo dot syntax to query sub documentsuk_pages = db.page.find({"author.country": "uk"})# In mongoengine we use dunder insteaduk_pages = Page.objects(author__country=uk)
  26. 26. REWRITING THE QUERY LANGUAGE#Somethings are nicer - regular expresion searchdb.post.find({title: re.compile(MongoDB, re.IGNORECASE)})Post.objects(title__icontains=MongoDB) # In mongoengine# Chaining is nicerdb.post.update({"published": False}, {"$set": {"published": True}}, multi=True)Post.objects(published=False).update(set__published=True)
  27. 27. LESSON 4: NOT ALL IDEAS ARE GOODhttp://www.flickr.com/photos/abiding_silence/6951229015
  28. 28. CHANGING SAVE# In pymongo save replaces the whole documentdb.post.save({_id: my_id, title: MongoDB, published: True})# In mongoengine we track changespost = Post.objects(_id=my_id).first()post.published = Truepost.save()# Results in:db.post.update({_id: my_id}, {$set: {published: True}})
  29. 29. CHANGING SAVEAny field changes are notedHow to monitor lists and dicts? Custom List and Dict classes Observes changes and marks as dirty
  30. 30. HOW IT WORKSclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment))class User(Document): email = StringField(required=True) first_name = StringField(max_length=50) last_name = StringField(max_length=50)class Comment(EmbeddedDocument): content = StringField() name = StringField(max_length=120)
  31. 31. HOW IT WORKSpost = Post.objects.first() Postpost.comments[1].name = Fredpost.save() - comments comment comment comment
  32. 32. HOW IT WORKSpost.comments[1].name = Fred Post1.Convert the - comments comments comment data to a BaseList commentBaseList Stores commentthe instance andname / location
  33. 33. HOW IT WORKSpost.comments[1].name = Fred Post2. Convert the - comments comment comment data to BaseDict commentsets name as: commentcomments.1
  34. 34. HOW IT WORKSpost.comments[1].name = Fred Post3.Change name - comments to "Fred" comment4. Tell Post comment comments.1.name has changed comment
  35. 35. HOW IT WORKSpost.save() Post5.On save() - commentsIterate all the changeson post and run comment$set / $unset queries commentdb.post.update( {_id: my_id}, {$set: { comment comments.1.name: Fred}})
  36. 36. A GOOD IDEA?+ Makes it easier to use+ save acts how people think it should- Its expensive- Doesnt help people understand MongoDB
  37. 37. LESSON 5: MANAGING A COMMUNIT Yhttp://kingscross.the-hub.net/
  38. 38. CODERS JUST WANT TO CODE * Github effect >10 django mongoengine projects None active on pypi Little cross project communication* Side effect of being stalled?
  39. 39. REACH OUT Flask-mongoengine on pypi There were 2 different projects Now has extra maintainers from the flask-mongorest Django-mongoengine* Spoke to authors of 7 projects and merged their work together to form a single library* Coming soon!
  40. 40. THE COMMUNITYNot all ideas are good!Vocal people dont always have great ideasTravis is great** but you still have to read the pull requestCommunities have to be managedIve only just started to learn how to herding cats
  41. 41. LESSON 6: DON T BE AFR AID TO ASKhttp://www.flickr.com/photos/kandyjaxx/2012468692
  42. 42. I NEED HELP ;) Website Documentation Tutorials Framework support Core mongoenginehttp://twitter.com/RossC0 http://github.com/MongoEngine
  43. 43. QUESTIONS?http://www.flickr.com/photos/9550033@N04/5020799468

×