MongoDB Basic Concepts                                  Norberto Leite                          Senior Solutions Architect...
Agenda       •Overview       •Replication       •Scalability       •Consistency & Durability       •Flexibility, Developer...
Your data needs started here...                                  http://bit.ly/OT71M4Sunday, 21 October 12
...but soon you had to be here                                  http://bit.ly/OxcsisSunday, 21 October 12
Basic Concepts                           Application    Document                                          Oriented        ...
Tradeoff: Scale vs Functionality                                    • memcached        scalability & performance          ...
ReplicationSunday, 21 October 12
Why do we need replication       •Failover       •Backups       •Secondary batch jobs       •High availabilitySunday, 21 O...
Replica Sets       Data Availability across nodes       • Data Protection         • Multiple copies of the data         • ...
Replica Sets                App     Write                                 Primary                                         ...
Replica Sets                App     Write                                 Primary                        Read             ...
Replica Sets                App                                 Primary                        Write                      ...
Replica Sets                App                                Recovering                        Write                New ...
Replica Sets                App                                Secondary                        Read                      ...
ScalabilitySunday, 21 October 12
Horizontal ScalabilitySunday, 21 October 12
Sharding       Data Distribution across nodes       • Data location transparent to your code       • Data distribution is ...
Sharding - Range distribution                        sh.shardCollection("test.tweets", {_id: 1} , false)                  ...
Sharding - Range distribution                        shard01   shard02   shard03                         a-i        j-r   ...
Sharding - Splits                        shard01   shard02   shard03                         a-i      ja-jz      s-z      ...
Sharding - Splits                        shard01   shard02   shard03                         a-i       ja-ji     s-z      ...
Sharding - Auto Balancing                        shard01   shard02   shard03                         a-i       ja-ji     s...
Sharding - Auto Balancing                        shard01   shard02   shard03                         a-i       ja-ji     n...
Sharding - Routed Query                                            find({_id: "norberto"})                        shard01  ...
Sharding - Routed Query                                            find({_id: "norberto"})                        shard01  ...
Sharding - Scatter Gather                                       find({email: "norberto@10gen.com"})                        ...
Sharding - Scatter Gather                                       find({email: "norberto@10gen.com"})                        ...
Sharding - Caching                         96 GB Mem                        3:1 Data/Mem                        shard01   ...
Aggregate Horizontal Resources                         96 GB Mem      96 GB Mem      96 GB Mem                        1:1 ...
Consistency & DurabilitySunday, 21 October 12
Two choices for consistency       •Eventual consistency                •Allow updates when a system has been partitioned  ...
Durability       •For how long is my data available?       •When do I now that my data is safe?       •Where?       •Mongo...
Fire and Forget               Driver   PrimarySunday, 21 October 12
Fire and Forget               Driver           Primary                        writeSunday, 21 October 12
Fire and Forget               Driver           Primary                        write                                       ...
Get Last Error               Driver   PrimarySunday, 21 October 12
Get Last Error               Driver           Primary                        write                   getLastErrorSunday, 2...
Get Last Error               Driver           Primary                        write                   getLastError         ...
Get Last Error               Driver           Primary                        write                   getLastError         ...
Journal Sync               Driver   PrimarySunday, 21 October 12
Journal Sync               Driver           Primary                        write                   getLastError           ...
Journal Sync               Driver           Primary                        write                   getLastError           ...
Journal Sync               Driver           Primary                        write                   getLastError           ...
Replicas Safe               Driver   Primary   SecondarySunday, 21 October 12
Replicas Safe               Driver           Primary                     Secondary                        write           ...
Replicas Safe               Driver           Primary                     Secondary                        write           ...
Replicas Safe               Driver           Primary                     Secondary                        write           ...
FlexibilitySunday, 21 October 12
What MongoDB solves                                  • Applications store complex data that is easier to                  ...
Challenges for Databases       ✓ Build a database for scaleout                • Run on clusters of 100s of commodity      ...
Data Model       • Why JSON?                • Provides a simple, well understood                encapsulation of data     ...
Json        place1 = {         name : "10gen HQ",         address : "578 Broadway 7th Floor",         city : "New York",  ...
Schema Design       Relational DatabaseSunday, 21 October 12
Schema Design       MongoDB                    embedding                        linkingSunday, 21 October 12
Schemas in MongoDB     Design documents that simply map to     your application     post = {author: "Hergé",          date...
Embedding       > db.blogs.find( { author: "Hergé"} )         { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"),           auth...
JSON & Scaleout       • Embedding removes need for                • Distributed Joins                • Two Phase commit   ...
http://bit.ly/UmUnsUSunday, 21 October 12
http://bit.ly/cnP77LSunday, 21 October 12
http://bit.ly/ODoMhhSunday, 21 October 12
http://bit.ly/uW2nkSunday, 21 October 12
Sunday, 21 October 12
Upcoming SlideShare
Loading in …5
×

A Morning with MongoDB Barcelona: MongoDB Basic Concepts

2,845 views

Published on

http://www.10gen.com/events/MongoDB-Morning-Barcelona

  • Be the first to comment

A Morning with MongoDB Barcelona: MongoDB Basic Concepts

  1. 1. MongoDB Basic Concepts Norberto Leite Senior Solutions Architect, EMEA norberto@10gen.com @nleiteSunday, 21 October 12
  2. 2. Agenda •Overview •Replication •Scalability •Consistency & Durability •Flexibility, Developer ExperiencSunday, 21 October 12
  3. 3. Your data needs started here... http://bit.ly/OT71M4Sunday, 21 October 12
  4. 4. ...but soon you had to be here http://bit.ly/OxcsisSunday, 21 October 12
  5. 5. Basic Concepts Application Document Oriented High { author : “steve”, date : new Date(), text : “About MongoDB...”, Performance tags : [“tech”, “database”]} Fully Consistent Horizontally ScalableSunday, 21 October 12
  6. 6. Tradeoff: Scale vs Functionality • memcached scalability & performance •key/value • RDBMS depth of functionalitySunday, 21 October 12
  7. 7. ReplicationSunday, 21 October 12
  8. 8. Why do we need replication •Failover •Backups •Secondary batch jobs •High availabilitySunday, 21 October 12
  9. 9. Replica Sets Data Availability across nodes • Data Protection • Multiple copies of the data • Spread across Data Centers, AZs • High Availability • Automated Failover • Automated RecoverySunday, 21 October 12
  10. 10. Replica Sets App Write Primary Asynchronous Read Replication Secondary Read Secondary ReadSunday, 21 October 12
  11. 11. Replica Sets App Write Primary Read Secondary Read Secondary ReadSunday, 21 October 12
  12. 12. Replica Sets App Primary Write Primary Automatic Election of new Primary Read Secondary ReadSunday, 21 October 12
  13. 13. Replica Sets App Recovering Write New primary serves Primary data Read Secondary ReadSunday, 21 October 12
  14. 14. Replica Sets App Secondary Read Write Primary Read Secondary ReadSunday, 21 October 12
  15. 15. ScalabilitySunday, 21 October 12
  16. 16. Horizontal ScalabilitySunday, 21 October 12
  17. 17. Sharding Data Distribution across nodes • Data location transparent to your code • Data distribution is automatic • Data re-distribution is automatic • Aggregate system resources horizontally • No code changesSunday, 21 October 12
  18. 18. Sharding - Range distribution sh.shardCollection("test.tweets", {_id: 1} , false) shard01 shard02 shard03Sunday, 21 October 12
  19. 19. Sharding - Range distribution shard01 shard02 shard03 a-i j-r s-zSunday, 21 October 12
  20. 20. Sharding - Splits shard01 shard02 shard03 a-i ja-jz s-z k-rSunday, 21 October 12
  21. 21. Sharding - Splits shard01 shard02 shard03 a-i ja-ji s-z ji-js js-jw jz-rSunday, 21 October 12
  22. 22. Sharding - Auto Balancing shard01 shard02 shard03 a-i ja-ji s-z ji-js js-jw js-jw jz-r jz-rSunday, 21 October 12
  23. 23. Sharding - Auto Balancing shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-rSunday, 21 October 12
  24. 24. Sharding - Routed Query find({_id: "norberto"}) shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-rSunday, 21 October 12
  25. 25. Sharding - Routed Query find({_id: "norberto"}) shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-rSunday, 21 October 12
  26. 26. Sharding - Scatter Gather find({email: "norberto@10gen.com"}) shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-rSunday, 21 October 12
  27. 27. Sharding - Scatter Gather find({email: "norberto@10gen.com"}) shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-rSunday, 21 October 12
  28. 28. Sharding - Caching 96 GB Mem 3:1 Data/Mem shard01 a-i 300 GB Data j-r n-z 300 GBSunday, 21 October 12
  29. 29. Aggregate Horizontal Resources 96 GB Mem 96 GB Mem 96 GB Mem 1:1 Data/Mem 1:1 Data/Mem 1:1 Data/Mem shard01 shard02 shard03 a-i j-r n-z 300 GB Data 100 GB 100 GB 100 GBSunday, 21 October 12
  30. 30. Consistency & DurabilitySunday, 21 October 12
  31. 31. Two choices for consistency •Eventual consistency •Allow updates when a system has been partitioned •Resolve conflicts later •Example: CouchDB, Cassandra •Immediate consistency •Limit the application of updates to a single master node for a given slice of data •Another node can take over after a failure is detected •Avoids the possibility of conflicts •Example: MongoDBSunday, 21 October 12
  32. 32. Durability •For how long is my data available? •When do I now that my data is safe? •Where? •Mongodb style •Fire and Forget •Get Last Error •Journal Sync •Replica SafeSunday, 21 October 12
  33. 33. Fire and Forget Driver PrimarySunday, 21 October 12
  34. 34. Fire and Forget Driver Primary writeSunday, 21 October 12
  35. 35. Fire and Forget Driver Primary write apply in memorySunday, 21 October 12
  36. 36. Get Last Error Driver PrimarySunday, 21 October 12
  37. 37. Get Last Error Driver Primary write getLastErrorSunday, 21 October 12
  38. 38. Get Last Error Driver Primary write getLastError apply in memorySunday, 21 October 12
  39. 39. Get Last Error Driver Primary write getLastError apply in memorySunday, 21 October 12
  40. 40. Journal Sync Driver PrimarySunday, 21 October 12
  41. 41. Journal Sync Driver Primary write getLastError apply in memory j:trueSunday, 21 October 12
  42. 42. Journal Sync Driver Primary write getLastError apply in memory j:true write to journalSunday, 21 October 12
  43. 43. Journal Sync Driver Primary write getLastError apply in memory j:true write to journalSunday, 21 October 12
  44. 44. Replicas Safe Driver Primary SecondarySunday, 21 October 12
  45. 45. Replicas Safe Driver Primary Secondary write getLastError w:2 apply in memorySunday, 21 October 12
  46. 46. Replicas Safe Driver Primary Secondary write getLastError w:2 apply in memory replicateSunday, 21 October 12
  47. 47. Replicas Safe Driver Primary Secondary write getLastError w:2 apply in memory replicateSunday, 21 October 12
  48. 48. FlexibilitySunday, 21 October 12
  49. 49. What MongoDB solves • Applications store complex data that is easier to Agility • model as documents Schemaless DB enables faster development cycles • Relaxed transactional semantics enable easy scale Flexibility • out Auto Sharding for scale down and scale up • Cost effective operationalize abundant data Cost (clickstreams, logs, tweets, ...)Sunday, 21 October 12
  50. 50. Challenges for Databases ✓ Build a database for scaleout • Run on clusters of 100s of commodity machines •… that enables agile development •… and is usable for a broad variety of applicationsSunday, 21 October 12
  51. 51. Data Model • Why JSON? • Provides a simple, well understood encapsulation of data • Maps simply to the object in your OO language • Linking & Embedding to describe relationshipsSunday, 21 October 12
  52. 52. Json place1 = { name : "10gen HQ", address : "578 Broadway 7th Floor", city : "New York", zip : "10011", tags : [ "business", "tech" ] }Sunday, 21 October 12
  53. 53. Schema Design Relational DatabaseSunday, 21 October 12
  54. 54. Schema Design MongoDB embedding linkingSunday, 21 October 12
  55. 55. Schemas in MongoDB Design documents that simply map to your application post = {author: "Hergé", date: new Date(), text: "Destination Moon", tags: ["comic", "adventure"]} > db.posts.save(post)Sunday, 21 October 12
  56. 56. Embedding > db.blogs.find( { author: "Hergé"} ) { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "Hergé", date : ISODate("2011-09-18T09:56:06.298Z"), text : "Destination Moon", tags : [ "comic", "adventure" ], comments : [ ! { ! ! author : "Kyle", ! ! date : ISODate("2011-09-19T09:56:06.298Z"), ! ! text : "great book" ! } ] }Sunday, 21 October 12
  57. 57. JSON & Scaleout • Embedding removes need for • Distributed Joins • Two Phase commit • Enables data to be distributed across many nodes without penaltySunday, 21 October 12
  58. 58. http://bit.ly/UmUnsUSunday, 21 October 12
  59. 59. http://bit.ly/cnP77LSunday, 21 October 12
  60. 60. http://bit.ly/ODoMhhSunday, 21 October 12
  61. 61. http://bit.ly/uW2nkSunday, 21 October 12
  62. 62. Sunday, 21 October 12

×