MongoDB dessi-codemotion


Published on

Codemotion 2010 MongoDB slides, examples with scala casbah salat

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

MongoDB dessi-codemotion

  1. 2. Speaker Software architect/developer Pronetics/Sourcesense Founder Spring Italian User Group Chairman JugSardegna Committer/Contributor OpenNMS - MongoDB Author Spring 2.5 Aspect Oriented Programming
  2. 3. MongoDB Philosophy
  3. 4. Some MongoDB production deployments
  4. 5. Main Features <ul><li>Document Oriented
  5. 6. - Documents (objects) map nicely to programming language data types
  6. 7. - Dynamically-typed (schemaless) for easy schema evolution </li></ul>- No joins and no transactions for high performance and easy scalability <ul><li>High Performance
  7. 8. - No joins and no transactions makes reads and writes fast
  8. 9. - Indexes with indexing into embedded documents and arrays
  9. 10. - Optional asynchronous writes
  10. 11. Rich Query Language </li></ul><ul><li>Easy scalability
  11. 12. - &quot;slaveOK&quot; reads are distributed over replicated servers
  12. 13. - Automatic sharding (auto-partitioning of data across servers)
  13. 14. - Reads and writes are distributed over shards </li></ul>- No joins and no transactions make distributed queries easy and fast <ul><li>High Availability
  14. 15. - Replicated servers with automatic master failover
  15. 16. Indexing
  16. 17. Stored JavaScript
  17. 18. Fixed-size collection
  18. 19. File storage
  19. 20. MapReduce </li></ul>
  20. 21. No Sql Injection Mongo is invulnerable to injection attacks, no code execution
  21. 22. Document as Basic unit of Data in BSON (Binary JSON) format <ul>{ &quot;name&quot; : &quot;MongoDB&quot;, </ul> &quot;info&quot; : { &quot;storage&quot; : &quot;Binary JSON (BSON)&quot;, &quot;full index&quot; : &quot;true&quot;, &quot;scale&quot; : &quot;Autosharding&quot;, &quot;query&quot; : &quot;Rich document-base queries&quot;, &quot;replication&quot; : &quot;Replica sets&quot;, &quot;atomic modifiers&quot; : &quot;Fast in place update&quot;, &quot;binary content&quot; : &quot;GridFS&quot;, &quot;batch operation&quot; : &quot;Map/Reduce”, &quot; js server side&quot; : ”true” } &quot;greeting&quot;: {&quot;international&quot; : &quot;Hello, world!&quot;, &quot;italy&quot; :&quot;Ciao Mondo !&quot; } }, &quot;_id” : &quot;024x6f279578a64bb0666945&quot; Document: an Ordered set of keys with associated values
  22. 23. Grouping SQL Table contains Rows MONGO Collection and subcollections contains Documents * Document Limit: Larger than 4 Mb, the entire text of War and Peace is 3.14Mb Collection are are created dynamically and automatically grow in size to fit additional data
  23. 24. A single instance of MongoDB can host multiple independent databases, each of which can have its own collections and permissions. Photo from
  24. 25. No Join's cost <ul><li>SQL </li></ul>SELECT * FROM posts INNER JOIN posts_tags ON = posts_tags.post_id INNER JOIN tags ON posts_tags.tag_id == WHERE tags.text = 'politics' AND posts.vote_count > 10; <ul><li>MONGO </li></ul>db.posts.find({'tags': 'politics', 'vote_count': {'$gt': 10}});
  25. 26. Collections Schema free Documents within a single collection can have any number of different &quot;shapes” In theory, each document in a collection can have a completely different structure; in practice, a collection's documents will be relatively uniform.
  26. 27. Driver C, C#, C++, Clojure, D, Delphi, Erlang, Factor, Fantom, F#, Go, Groovy, Haskell, Java, Javascript, Lua, Nodejs, ObjectiveC, Perl, PHP, Python, R, Ruby, Scala, Scheme (PLT), Smalltalk
  27. 28. Common operations Fastest : Fire and Forget Command with response : getLastError Examples with: Server side Javascript via mongo shell Java via MongoDB Official 10gen Driver Scala via Casbah Official 10gen scala driver + Salat serializer
  28. 29. Inserting //mongo shell db.dcComicsCollection.insert({ &quot; name &quot; : &quot; bruce &quot;, &quot;sur name&quot; : &quot; wayne &quot;, &quot;alias&quot; : &quot; batman &quot; }) //Java Map fields = batman.toMap() dcComicsCollection.insert(BasicDBObjectBuilder.start(fields).get()) //java driver //scala dcComicsCollection.insert(grater[BlogPost].asDBObject(post))
  29. 30. Removing //mongo shell db.dcComicsCollection.delete({ &quot; alias &quot; : &quot; superman &quot; }) //java Map query = batman.getMap() DBObject obj = BasicDBObjectBuilder.start(fields).get() dcComicsColl .remove(obj); //scala dcComicsCollection .remove(grater[BlogPost].asDBObject(post))
  30. 31. Updating ( The schema can be changed dinamically ) //mongo shell var hero = db.dcComicsColl.findOne({ &quot; alias &quot; : &quot; batman &quot; }); hero.gadget = { &quot; car &quot; : &quot; batmobile ” }; db.dcComicsColl.update({ ” alias ” : ” batman &quot; }, hero, true ); //java DbObject query = BasicDBObjectBuilder.start(). add(&quot;surname&quot;,”wayne”).get(); DbObject hero = BasicDBObjectBuilder.start().add(&quot;gadget&quot;,”batmobile”).get(); dcComicsCollection.update(query, hero, false, true ); //scala val query = MongoDBObject( &quot;name&quot; -> &quot;bruce&quot; ) val hero = MongoDBObject( &quot;gadget&quot; -> &quot;batmobile &quot; ) dcComicsCollection.update(query, hero, true, false) * The blue value it's the upsert, update or insert if not present
  31. 32. Querying //mongo shell db.dcComicsColl.find({ &quot; alias &quot; : &quot; batman &quot; }) // all field of the doc db.dcComicsColl.find( {&quot;alias&quot;:&quot;batman&quot;} ,{”surname&quot;:1}) //surname desc //java DBObject query = BasicDBObjectBuilder.start().add(&quot;alias&quot;, &quot;batman&quot;).get() ; DBObject out = BasicDBObjectBuilder.start().add(&quot;surname&quot;, &quot;1&quot;).get() DBCursor cursor = dcComicsColl.find(query, out ); //scala val query = MongoDBObject( &quot;alias&quot; -> &quot;batman&quot; ) val obj = dcComicsColl.findOne(query)
  32. 33. Modifier Partial updates $set (set a key, or add if not present) $inc (with numbers) $push (add to the end of an array) $ne $addToSet $each $pop (remove from the end) $pull (remove element that match criteria)
  33. 34. Query Criteria Conditionals $lt $lte $gt $gte start = new Date(&quot;01/01/20111&quot;) db.mycollection.find({&quot;registered&quot; : {&quot;$lt&quot; : start}}) db.dcComicsColl.find({ &quot;surname&quot;:{&quot;$ne&quot;:&quot;parker&quot;}, &quot;name&quot;:{&quot;$ne&quot;:&quot;peter&quot;} })
  34. 35. Query Criteria Conditionals $in $or, $nin, $not <, <=, >, >=, $ne, $mod, $all, $size, $exists db.attendees.find({”external_num&quot; : {&quot;$nin&quot; : [1356, 525, 874]}}) db.attendees.find({&quot;$or&quot; : [{&quot;ticket_no&quot; : 513}, {&quot;name&quot; : foo}]})
  35. 36. Query Regex Perl Compatible Regular Expression //mongo shell db.mycollection.find({&quot;name&quot; : /foo?/i}) //java String pattern = new StringBuilder().append(character).append( &quot;*&quot; ).toString(); DbObject query =BasicDBObjectBuilder.start() .add( &quot;surname&quot; , java.util.regex.Pattern.compile(pattern)).get() List<DBObject> objects = coll .find(query ).toArray(); //scala coll.find(MongoDBObject(&quot;surname&quot; -> &quot;.*yne$&quot;.r))) {...
  36. 37. Query on array and grouping Array inside a document $all $size $slice Grouping count distinct group finalize $key
  37. 38. Javascript as part of a query db.mycollection.find({ &quot;$where&quot; : function (){ for (var current in this) { … . } } })
  38. 39. Indexing db.mycollection.ensureIndex({&quot;name&quot; : 1, &quot;info&quot; : 1 } Geospatial indexing db.mycollection.ensureIndex({&quot;gps&quot; : &quot;2d&quot;}) {&quot;light-years&quot; : &quot;2d&quot;}, {&quot;min”:-10, &quot;max”:10}) Yes it's a Foursquare feature :)
  39. 40. GridFS GridFS is a specification for storing large files such video, photos, blob in MongoDB . GridFS uses two collections to store data: <ul><li>files contains the object metadata
  40. 41. chunks contains the binary chunks with some additional accounting information </li></ul>
  41. 42. Map Reduce map = function() { for (var key in this) { emit(key, {count : 1}); }; } reduce = function(key, emits) { total = 0; for (var i in emits) { total += emits[i].count; } return {&quot;count&quot; : total}; }
  42. 43. Scaling
  43. 44. Scaling
  44. 45. Shard Sharding = break up collections into smaller chunks Splitting data and storing different portions of the data on different machines, also know as partitioning The chunks can be distributed across shards so that each shard is responsible for a subset of the total data set. A shard is a container that holds a subset of a collection’s data. A shard is either a single mongod server (for development/testing) or a replica set (for production). Thus, even if there are many servers in a shard, there is only one master, and all of the servers contain the same data.
  45. 46. Mongos The client don't know what shard has what data, or even that data is broken up on different shards. In front of the shard run routing process called mongos. This router know when the data are located and the client can see a normal mongod, like a noshard environment. This is the router process and comes with all MongoDB distributions. It basically just routes requests and aggregates responses. It doesn’t store any data or config- uration information. (Although it does cache information from the config servers.)
  46. 47. Config server Config servers store the configuration of the cluster: which data is on which shard. Because mongos doesn’t store anything permanently, it needs somewhere to get the shard configuration. It syncs this data from the config servers.
  47. 48. Replication Replica set, clustering with automatic failover Master is elected by the cluster and may change to another node if the current master goes down. This election process will be initiated by any node that cannot reach the primary. The highest-priority most-up-to-date server will become the new primary. The replication is asynchronous
  48. 49. Scala Webapp Demo
  49. 50. Admin Interface
  50. 51. Mongo Shell
  51. 52. Mac Client MongoHub
  52. 53. Books
  53. 54. References <ul><li>MongoDB :
  54. 55. Casbah:
  55. 56. Salat:
  56. 57. Slides scala code example: </li></ul>
  57. 58. Thanks ! Massimiliano Dessì