Your SlideShare is downloading. ×
  • Like
Pyconie 2012
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply
Published

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
225
On SlideShare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
7
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Lessons learnt building @RossC0 http://github.com/rozza
  • 2. WHAT IS MONGODB? A document database { _id : ObjectId("..."), author : "Ross", Highly scalable date : ISODate("2012-07-05..."), text : "About MongoDB...", tags : [ "tech", "databases" ], Developer friendly comments : [{ author : "Tim", date : ISODate("2012-07-05..."), text : "Best Post Ever!" }], comment_count : 1 }http://mongodb.org In BSON
  • 3. WHAT IS MONGODB?{ "_id" : ObjectId("..."), { _id : ObjectId("..."), "author" : "Ross", author : "Ross", "date" : datetime(2012,7,5,10,0), date : ISODate("2012-07-05..."), "text" : "About MongoDB...", text : "About MongoDB...", "tags" : ["tech", "databases"], tags : [ "tech", "databases" ], "comments" : [{ comments : [{ "author" : "Tim", author : "Tim", "date" : datetime(2012,7,5,11,35), date : ISODate("2012-07-05..."), "text" : "Best Post Ever!" text : "Best Post Ever!" }], }], "comment_count" : 1 comment_count : 1} } In Python In BSON
  • 4. Want to know more?http://education.10gen.com
  • 5. WHY DO YOU EVEN NEED AN ODM?http://www.flickr.com/photos/51838104@N02/5841690990
  • 6. SCHEMA LESS != CHAOSMongoDB a good fitDocuments schema in codeEnforces schemaData validationSpeeds up developmentBuild tooling off itCan DRY up code...
  • 7. Inspired by Djangos ORMSupports Python 2.5 - Python 3.3Originally authored by Harry Marr 2010I took over development in May 2011Current release 0.7.5http://github.com/MongoEngine/mongoengine
  • 8. INTRODUCING MONGOENGINEclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment))class Comment(EmbeddedDocument): content = StringField() name = StringField(max_length=120)class User(Document): email = StringField(required=True) first_name = StringField(max_length=50) last_name = StringField(max_length=50)
  • 9. CREATING A MODELclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment)) Define a class inheriting from Document Map a field to a defined data type strings, ints, binary, files, lists etc.. By default all declared fields arent required Pass keyword arguments to apply constraints eg set if unique, max_length, default values.
  • 10. INSERTING DATA# Pass data into the constructoruser = User(email="ross@10gen.com", name="Ross").save()# Create instance and edit / update in placepost = Post()post.title = "mongoengine"post.author = userpost.tags = [odm, mongodb, python]post.save() Create instance of the object Update its attributes Call save, insert, update to persist the data
  • 11. QUERYING DATA# An `objects` manager is added to every `Document` classusers = User.objects(email=ross@10gen.com)# Pass kwargs to commands are lazy and be extended as neededusers.filter(auth=True)# Iterating evaluates the querysetprint [u for u in users] Documents have a queryset manager (objects) for querying You can continually extend it Queryset evaluated on iteration
  • 12. 6 LESSONS LEARNT
  • 13. LESSON 1: DIVE IN!http://www.flickr.com/photos/jackace/565857899/
  • 14. PROJECT STALLED In May 2011>200 forks>100 issues~50 pull requests I needed itVolunteered to helpStarted reviewing issuesSupported Harry andcommunity
  • 15. L E S S O N 2 : M E TA C L A S S E Shttp://www.flickr.com/photos/ubique/135848053
  • 16. WHATS NEEDED TO MAKE AN ORM?Instance methods validation data manipulate data convert data to and from mongodbQueryset methods Finding data Bulk changes
  • 17. METACLASSESclass Document(object): __metaclass__ = TopLevelDocumentMetaclass ...class EmbeddedDocument(object): __metaclass__ = DocumentMetaclass ... Needed for: 1. inspect the object inheritance 2. inject functionality to the class Its surprisingly simple - all we need is: __new__
  • 18. METACLASSES 101 IN TopLevelDocumentcls, name, bases, attrs Document Out pythons type new class
  • 19. METACLASSES IN TopLevelDocument Creates default meta data inheritance rules, id_field, index Document information, default ordering. Merges in parents meta Validation pythons type abstract flag on an inherited class collection set on a subclassManipulates the attrs going in.
  • 20. METACLASSES IN TopLevelDocumentMerges all fields from parentsAdds in own field definitions DocumentCreates lookups_db_field_map_reverse_db_field_map pythons typeDetermine superclasses(for model inheritance)
  • 21. METACLASSES OUT TopLevelDocumentAdds in handling for delete rules DocumentSo we can handle deleted ReferencesAdds class to the registrySo we can load the data into the correct class pythons type
  • 22. METACLASSES OUT TopLevelDocumentBuilds index specifications DocumentInjects queryset managerSets primary key (if needed) pythons type
  • 23. LESSONS LEARNTSpend time learning what is being done and whyDont continually patch:Rewrote the metaclasses in 0.7
  • 24. L E S S O N 3 : S T R AY I N G F R O M T H E PAT Hhttp://www.flickr.com/photos/51838104@N02/5841690990
  • 25. REWRITING THE QUERY LANGUAGE# In pymongo you pass dictionaries to queryuk_pages = db.page.find({"published": True})# In mongoengineuk_pages = Page.objects(published=True)# pymongo dot syntax to query sub documentsuk_pages = db.page.find({"author.country": "uk"})# In mongoengine we use dunder insteaduk_pages = Page.objects(author__country=uk)
  • 26. REWRITING THE QUERY LANGUAGE#Somethings are nicer - regular expresion searchdb.post.find({title: re.compile(MongoDB, re.IGNORECASE)})Post.objects(title__icontains=MongoDB) # In mongoengine# Chaining is nicerdb.post.update({"published": False}, {"$set": {"published": True}}, multi=True)Post.objects(published=False).update(set__published=True)
  • 27. LESSON 4: NOT ALL IDEAS ARE GOODhttp://www.flickr.com/photos/abiding_silence/6951229015
  • 28. CHANGING SAVE# In pymongo save replaces the whole documentdb.post.save({_id: my_id, title: MongoDB, published: True})# In mongoengine we track changespost = Post.objects(_id=my_id).first()post.published = Truepost.save()# Results in:db.post.update({_id: my_id}, {$set: {published: True}})
  • 29. CHANGING SAVEAny field changes are notedHow to monitor lists and dicts? Custom List and Dict classes Observes changes and marks as dirty
  • 30. HOW IT WORKSclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment))class User(Document): email = StringField(required=True) first_name = StringField(max_length=50) last_name = StringField(max_length=50)class Comment(EmbeddedDocument): content = StringField() name = StringField(max_length=120)
  • 31. HOW IT WORKSpost = Post.objects.first() Postpost.comments[1].name = Fredpost.save() - comments comment comment comment
  • 32. HOW IT WORKSpost.comments[1].name = Fred Post1.Convert the - comments comments comment data to a BaseList commentBaseList Stores commentthe instance andname / location
  • 33. HOW IT WORKSpost.comments[1].name = Fred Post2. Convert the - comments comment comment data to BaseDict commentsets name as: commentcomments.1
  • 34. HOW IT WORKSpost.comments[1].name = Fred Post3.Change name - comments to "Fred" comment4. Tell Post comment comments.1.name has changed comment
  • 35. HOW IT WORKSpost.save() Post5.On save() - commentsIterate all the changeson post and run comment$set / $unset queries commentdb.post.update( {_id: my_id}, {$set: { comment comments.1.name: Fred}})
  • 36. A GOOD IDEA?+ Makes it easier to use+ save acts how people think it should- Its expensive- Doesnt help people understand MongoDB
  • 37. LESSON 5: MANAGING A COMMUNIT Yhttp://kingscross.the-hub.net/
  • 38. CODERS JUST WANT TO CODE * Github effect >10 django mongoengine projects None active on pypi Little cross project communication* Side effect of being stalled?
  • 39. REACH OUT Flask-mongoengine on pypi There were 2 different projects Now has extra maintainers from the flask-mongorest Django-mongoengine* Spoke to authors of 7 projects and merged their work together to form a single library* Coming soon!
  • 40. THE COMMUNITYNot all ideas are good!Vocal people dont always have great ideasTravis is great** but you still have to read the pull requestCommunities have to be managedIve only just started to learn how to herding cats
  • 41. LESSON 6: DON T BE AFR AID TO ASKhttp://www.flickr.com/photos/kandyjaxx/2012468692
  • 42. I NEED HELP ;) Website Documentation Tutorials Framework support Core mongoenginehttp://twitter.com/RossC0 http://github.com/MongoEngine
  • 43. QUESTIONS?http://www.flickr.com/photos/9550033@N04/5020799468