MongoDB and Python
•
10 likes•3,134 views
MongoDB is one of the most popular databases these days and there are a few reasons for such popularity. One of these reasons is the excellent integration with different programming languages and development frameworks. In the case of Python we take it a few notches up (native use of dictionaries, integration with asynchronous libraries (twisted, gevent), good support for web frameworks like django, flask, bottle ... (mongoengine anyone?). This talk is about the several different projects that we support, the way to effectively use Python and MongoDB together and a few other improvements and announcements.
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (19)
Similar to MongoDB and Python
Similar to MongoDB and Python (20)
More from Norberto Leite
More from Norberto Leite (20)
Recently uploaded
Recently uploaded (20)
MongoDB and Python
- 2. MongoDB + Python Norberto Leite Technical Evangelist norberto@mongodb.com
- 4. Ola, I'm Norberto Norberto Leite Technical Evangelist ! Madrid, Spain @nleite norberto@mongodb.com http://www.mongodb.com/norberto
- 5. MongoDB
- 6. MongoDB GENERAL PURPOSE DOCUMENT DATABASE OPEN-SOURCE
- 8. MongoDB Features JSON Document Model with Dynamic Schemas Auto-Sharding for Horizontal Scalability Text Search Aggregation Framework and MapReduce Full, Flexible Index Support and Rich Queries Built-In Replication for High Availability Advanced Security Large Media Storage with GridFS
- 9. MongoDB Inc. 400+ employees 2,000+ customers Over $311 million in funding13 offices around the world
- 10. THE LARGEST ECOSYSTEM 9,000,000+ MongoDB Downloads 250,000+ Online Education Registrants 35,000+ MongoDB User Group Members 35,000+ MongoDB Management Service (MMS) Users 750+ Technology and Services Partners 2,000+ Customers Across All Industries
- 11. pymongo
- 12. pymongo • MongoDB Python official driver • Rockstart developer team • Jesse Jiryu Davis, Bernie Hackett • One of oldest and better maintained drivers • Python and MongoDB are a natural fit • BSON is very similar to dictionaries • (everyone likes dictionaries) • http://api.mongodb.org/python/current/ • https://github.com/mongodb/mongo-python-driver
- 13. pymongo 2.8 • Support for upcoming MongoDB 3.0 • New get collections and get indexes commands for Wired Tiger • Backward compatible w/ 2.6 ! • Future releases of pymongo (3.0) • Server discovery spec • Monitoring spec • Faster client startup when connecting to Replica Set • Faster failover • More robust replica set connections • API clean up
- 14. Connecting
- 15. Connecting #!/bin/python from pymongo import MongoClient ! mc = MongoClient() client instance
- 16. Connecting #!/bin/python from pymongo import MongoClient ! uri = 'mongodb://127.0.0.1' mc = MongoClient(uri)
- 17. Connecting #!/bin/python from pymongo import MongoClient ! uri = 'mongodb://127.0.0.1' mc = MongoClient(host=uri, max_pool_size=10)
- 18. Connecting to Replica Set #!/bin/python from pymongo import MongoClient ! uri = ‘mongodb://127.0.0.1?replicaSet=MYREPLICA' mc = MongoClient(uri)
- 19. Connecting to Replica Set #!/bin/python from pymongo import MongoClient ! uri = ‘mongodb://127.0.0.1' mc = MongoClient(host=uri, replicaSet='MYREPLICA')
- 20. Database Instance #!/bin/python from pymongo import MongoClient mc = MongoClient() ! db = mc['zurich_pug'] ! #or ! db = mc.zurich_pug database instance
- 21. Collection Instance #!/bin/python from pymongo import MongoClient mc = MongoClient() ! coll = mc[‘zurich_pug']['testcollection'] ! #or ! coll = mc.zurich_pug.testcollection collection instance
- 23. Operations • Insert • Remove • Update • Query • Aggregate • Create Indexes • …
- 24. CRUD • Insert • Remove • Update • Query • Aggregate • Create Indexes • …
- 25. Insert #!/bin/python from pymongo import MongoClient mc = MongoClient() ! coll = mc['zurich_pug']['testcollection'] ! ! coll.insert( {'field_one': 'some value'})
- 26. Find #!/bin/python from pymongo import MongoClient mc = MongoClient() ! coll = mc['zurich_pug']['testcollection'] ! ! cur = coll.find( {'field_one': 'some value'}) ! for d in cur: print d
- 27. Update #!/bin/python from pymongo import MongoClient mc = MongoClient() ! coll = mc['zurich_pug']['testcollection'] ! ! result = coll.update( {'field_one': 'some value'}, {"$set": {'field_one': 'new_value'}} ) ! print(result) !
- 28. Remove #!/bin/python from pymongo import MongoClient mc = MongoClient() ! coll = mc['zurich_pug']['testcollection'] ! ! result = coll.remove( {'field_one': 'some value'}) ! print(result) !
- 30. Aggregation Framework • Analytical workload solution • Pipeline processing • Several Stages • $match • $group • $project • $unwind • $sort • $limit • $skip • $out ! • http://docs.mongodb.org/manual/aggregation/
- 31. Aggregation Framework #!/bin/python from pymongo import MongoClient mc = MongoClient() ! coll = mc['zurich_pug']['testcollection'] ! ! cur = coll.aggregate( [ {"$match": {'field_one': {"$exists": True }}} , {"$project": { "new_label": "$field_one" }} ] ) ! for d in cur: print(d)
- 33. GridFS • MongoDB has a 16MB document size limit • So how can we store data bigger than 16MB? • Media files (images, pdf’s, long binary files …) • GridFS • Convention more than a feature • All drivers implement this convention • pymongo is no different • Very flexible approach • Handy out-of-the-box solution
- 34. GridFS #!/bin/python from pymongo import MongoClient import gridfs ! ! mc = MongoClient() database = mc.grid_example ! ! gfs = gridfs.GridFS( database) ! read_file = open( '/tmp/somefile', 'r') ! gfs.put(read_file, author='Norberto', tags=['awesome', 'zurich', 'pug']) call grids lib w/ database
- 35. GridFS #!/bin/python from pymongo import MongoClient import gridfs ! ! mc = MongoClient() database = mc.grid_example ! ! gfs = gridfs.GridFS( database) ! read_file = open( '/tmp/somefile', 'r') ! gfs.put(read_file, author='Norberto', tags=['awesome', 'zurich', 'pug']) open file for reading
- 36. GridFS #!/bin/python from pymongo import MongoClient import gridfs ! ! mc = MongoClient() database = mc.grid_example ! ! gfs = gridfs.GridFS( database) ! read_file = open( '/tmp/somefile', 'r') ! gfs.put(read_file, author='Norberto', tags=['awesome', 'zurich', 'pug']) call put to store file and metadata
- 37. GridFS mongo nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show dbs grid_sample 0.246GB local 0.000GB nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show collections fs.chunks 258.995MB / 252.070MB fs.files 0.000MB / 0.016MB database created
- 38. GridFS mongo nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show dbs grid_sample 0.246GB local 0.000GB nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show collections fs.chunks 258.995MB / 252.070MB fs.files 0.000MB / 0.016MB 2 collections
- 39. GridFS mongo nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show dbs grid_sample 0.246GB local 0.000GB nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show collections fs.chunks 258.995MB / 252.070MB fs.files 0.000MB / 0.016MB chunks collection holds binary data files holds metada data
- 40. Indexes
- 41. Indexes • Single Field • Compound • Multikey • Geospatial • 2d • 2dSphere - GeoJSON • Full Text • Hash Based • TTL indexes • Unique • Sparse
- 42. Single Field Index from pymongo import ASCENDING, MongoClient mc = MongoClient() ! coll = mc.zurich_pug.testcollection ! coll.ensure_index( 'some_single_field', ASCENDING ) indexed field indexing order
- 43. Compound Field Index from pymongo import ASCENDING, DESCENDING, MongoClient mc = MongoClient() ! coll = mc.zurich_pug.testcollection ! coll.ensure_index( [('field_ascending', ASCENDING), ('field_descending', DESCENDING)] ) indexed fields indexing order
- 44. Multikey Field Index mc = MongoClient() ! coll = mc.zurich_pug.testcollection ! ! coll.insert( {'array_field': [1, 2, 54, 89]}) ! coll.ensure_index( 'array_field') indexed field
- 45. Geospatial Field Index from pymongo import GEOSPHERE import geojson ! ! p = geojson.Point( [-73.9858603477478, 40.75929362758241]) ! coll.insert( {'point', p) ! coll.ensure_index( [( 'point', GEOSPHERE )]) index type
- 46. ODM and others
- 47. Friends • mongoengine • http://mongoengine.org/ • Motor • http://motor.readthedocs.org/en/stable/ • async driver • Tornado • Greenlets • ming • http://sourceforge.net/projects/merciless/
- 48. Let's recap
- 49. Recap • MongoDB is Awesome • Specially to work with Python • pymongo • super well supported • fully in sync with MongoDB server
- 50. MongoDB 3.0 is coming