• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Pyconie 2012

Pyconie 2012






Total Views
Views on SlideShare
Embed Views



4 Embeds 50

http://g33ktalk.com 43
http://www.hakkalabs.co 4
https://hakka.herokuapp.com 2
http://feeds.feedburner.com 1



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Pyconie 2012 Pyconie 2012 Presentation Transcript

    • Lessons learnt building @RossC0 http://github.com/rozza
    • WHAT IS MONGODB? A document database { _id : ObjectId("..."), author : "Ross", Highly scalable date : ISODate("2012-07-05..."), text : "About MongoDB...", tags : [ "tech", "databases" ], Developer friendly comments : [{ author : "Tim", date : ISODate("2012-07-05..."), text : "Best Post Ever!" }], comment_count : 1 }http://mongodb.org In BSON
    • WHAT IS MONGODB?{ "_id" : ObjectId("..."), { _id : ObjectId("..."), "author" : "Ross", author : "Ross", "date" : datetime(2012,7,5,10,0), date : ISODate("2012-07-05..."), "text" : "About MongoDB...", text : "About MongoDB...", "tags" : ["tech", "databases"], tags : [ "tech", "databases" ], "comments" : [{ comments : [{ "author" : "Tim", author : "Tim", "date" : datetime(2012,7,5,11,35), date : ISODate("2012-07-05..."), "text" : "Best Post Ever!" text : "Best Post Ever!" }], }], "comment_count" : 1 comment_count : 1} } In Python In BSON
    • Want to know more?http://education.10gen.com
    • WHY DO YOU EVEN NEED AN ODM?http://www.flickr.com/photos/51838104@N02/5841690990
    • SCHEMA LESS != CHAOSMongoDB a good fitDocuments schema in codeEnforces schemaData validationSpeeds up developmentBuild tooling off itCan DRY up code...
    • Inspired by Djangos ORMSupports Python 2.5 - Python 3.3Originally authored by Harry Marr 2010I took over development in May 2011Current release 0.7.5http://github.com/MongoEngine/mongoengine
    • INTRODUCING MONGOENGINEclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment))class Comment(EmbeddedDocument): content = StringField() name = StringField(max_length=120)class User(Document): email = StringField(required=True) first_name = StringField(max_length=50) last_name = StringField(max_length=50)
    • CREATING A MODELclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment)) Define a class inheriting from Document Map a field to a defined data type strings, ints, binary, files, lists etc.. By default all declared fields arent required Pass keyword arguments to apply constraints eg set if unique, max_length, default values.
    • INSERTING DATA# Pass data into the constructoruser = User(email="ross@10gen.com", name="Ross").save()# Create instance and edit / update in placepost = Post()post.title = "mongoengine"post.author = userpost.tags = [odm, mongodb, python]post.save() Create instance of the object Update its attributes Call save, insert, update to persist the data
    • QUERYING DATA# An `objects` manager is added to every `Document` classusers = User.objects(email=ross@10gen.com)# Pass kwargs to commands are lazy and be extended as neededusers.filter(auth=True)# Iterating evaluates the querysetprint [u for u in users] Documents have a queryset manager (objects) for querying You can continually extend it Queryset evaluated on iteration
    • LESSON 1: DIVE IN!http://www.flickr.com/photos/jackace/565857899/
    • PROJECT STALLED In May 2011>200 forks>100 issues~50 pull requests I needed itVolunteered to helpStarted reviewing issuesSupported Harry andcommunity
    • L E S S O N 2 : M E TA C L A S S E Shttp://www.flickr.com/photos/ubique/135848053
    • WHATS NEEDED TO MAKE AN ORM?Instance methods validation data manipulate data convert data to and from mongodbQueryset methods Finding data Bulk changes
    • METACLASSESclass Document(object): __metaclass__ = TopLevelDocumentMetaclass ...class EmbeddedDocument(object): __metaclass__ = DocumentMetaclass ... Needed for: 1. inspect the object inheritance 2. inject functionality to the class Its surprisingly simple - all we need is: __new__
    • METACLASSES 101 IN TopLevelDocumentcls, name, bases, attrs Document Out pythons type new class
    • METACLASSES IN TopLevelDocument Creates default meta data inheritance rules, id_field, index Document information, default ordering. Merges in parents meta Validation pythons type abstract flag on an inherited class collection set on a subclassManipulates the attrs going in.
    • METACLASSES IN TopLevelDocumentMerges all fields from parentsAdds in own field definitions DocumentCreates lookups_db_field_map_reverse_db_field_map pythons typeDetermine superclasses(for model inheritance)
    • METACLASSES OUT TopLevelDocumentAdds in handling for delete rules DocumentSo we can handle deleted ReferencesAdds class to the registrySo we can load the data into the correct class pythons type
    • METACLASSES OUT TopLevelDocumentBuilds index specifications DocumentInjects queryset managerSets primary key (if needed) pythons type
    • LESSONS LEARNTSpend time learning what is being done and whyDont continually patch:Rewrote the metaclasses in 0.7
    • L E S S O N 3 : S T R AY I N G F R O M T H E PAT Hhttp://www.flickr.com/photos/51838104@N02/5841690990
    • REWRITING THE QUERY LANGUAGE# In pymongo you pass dictionaries to queryuk_pages = db.page.find({"published": True})# In mongoengineuk_pages = Page.objects(published=True)# pymongo dot syntax to query sub documentsuk_pages = db.page.find({"author.country": "uk"})# In mongoengine we use dunder insteaduk_pages = Page.objects(author__country=uk)
    • REWRITING THE QUERY LANGUAGE#Somethings are nicer - regular expresion searchdb.post.find({title: re.compile(MongoDB, re.IGNORECASE)})Post.objects(title__icontains=MongoDB) # In mongoengine# Chaining is nicerdb.post.update({"published": False}, {"$set": {"published": True}}, multi=True)Post.objects(published=False).update(set__published=True)
    • LESSON 4: NOT ALL IDEAS ARE GOODhttp://www.flickr.com/photos/abiding_silence/6951229015
    • CHANGING SAVE# In pymongo save replaces the whole documentdb.post.save({_id: my_id, title: MongoDB, published: True})# In mongoengine we track changespost = Post.objects(_id=my_id).first()post.published = Truepost.save()# Results in:db.post.update({_id: my_id}, {$set: {published: True}})
    • CHANGING SAVEAny field changes are notedHow to monitor lists and dicts? Custom List and Dict classes Observes changes and marks as dirty
    • HOW IT WORKSclass Post(Document): title = StringField(max_length=120, required=True) author = ReferenceField(User) tags = ListField(StringField(max_length=30)) comments = ListField(EmbeddedDocumentField(Comment))class User(Document): email = StringField(required=True) first_name = StringField(max_length=50) last_name = StringField(max_length=50)class Comment(EmbeddedDocument): content = StringField() name = StringField(max_length=120)
    • HOW IT WORKSpost = Post.objects.first() Postpost.comments[1].name = Fredpost.save() - comments comment comment comment
    • HOW IT WORKSpost.comments[1].name = Fred Post1.Convert the - comments comments comment data to a BaseList commentBaseList Stores commentthe instance andname / location
    • HOW IT WORKSpost.comments[1].name = Fred Post2. Convert the - comments comment comment data to BaseDict commentsets name as: commentcomments.1
    • HOW IT WORKSpost.comments[1].name = Fred Post3.Change name - comments to "Fred" comment4. Tell Post comment comments.1.name has changed comment
    • HOW IT WORKSpost.save() Post5.On save() - commentsIterate all the changeson post and run comment$set / $unset queries commentdb.post.update( {_id: my_id}, {$set: { comment comments.1.name: Fred}})
    • A GOOD IDEA?+ Makes it easier to use+ save acts how people think it should- Its expensive- Doesnt help people understand MongoDB
    • LESSON 5: MANAGING A COMMUNIT Yhttp://kingscross.the-hub.net/
    • CODERS JUST WANT TO CODE * Github effect >10 django mongoengine projects None active on pypi Little cross project communication* Side effect of being stalled?
    • REACH OUT Flask-mongoengine on pypi There were 2 different projects Now has extra maintainers from the flask-mongorest Django-mongoengine* Spoke to authors of 7 projects and merged their work together to form a single library* Coming soon!
    • THE COMMUNITYNot all ideas are good!Vocal people dont always have great ideasTravis is great** but you still have to read the pull requestCommunities have to be managedIve only just started to learn how to herding cats
    • LESSON 6: DON T BE AFR AID TO ASKhttp://www.flickr.com/photos/kandyjaxx/2012468692
    • I NEED HELP ;) Website Documentation Tutorials Framework support Core mongoenginehttp://twitter.com/RossC0 http://github.com/MongoEngine
    • QUESTIONS?http://www.flickr.com/photos/9550033@N04/5020799468