What is MongoDB?
          What Makes Mongo Special?
                         MapReduce




                           Mon...
What is MongoDB?
              What Makes Mongo Special?
                             MapReduce




    The resulting [Mon...
What is MongoDB?
              What Makes Mongo Special?
                             MapReduce




    Compared to hadoop...
What is MongoDB?
              What Makes Mongo Special?
                             MapReduce



1   What is MongoDB?
  ...
What is MongoDB?     Document Oriented
              What Makes Mongo Special?    JavaScript Enabled
                     ...
What is MongoDB?     Document Oriented
            What Makes Mongo Special?    JavaScript Enabled
                       ...
What is MongoDB?     Document Oriented
            What Makes Mongo Special?    JavaScript Enabled
                       ...
What is MongoDB?     Document Oriented
                 What Makes Mongo Special?    JavaScript Enabled
                  ...
What is MongoDB?     Document Oriented
What Makes Mongo Special?    JavaScript Enabled
               MapReduce     Fast, ...
Native Language Integration
                      What is MongoDB?
                                           Rich Data Ty...
Native Language Integration
                     What is MongoDB?
                                          Rich Data Type...
Native Language Integration
                     What is MongoDB?
                                          Rich Data Type...
Native Language Integration
                     What is MongoDB?
                                          Rich Data Type...
Native Language Integration
                     What is MongoDB?
                                          Rich Data Type...
Native Language Integration
                     What is MongoDB?
                                          Rich Data Type...
Native Language Integration
                     What is MongoDB?
                                          Rich Data Type...
Native Language Integration
                      What is MongoDB?
                                           Rich Data Ty...
Native Language Integration
                      What is MongoDB?
                                           Rich Data Ty...
Native Language Integration
                         What is MongoDB?
                                              Rich D...
Native Language Integration
                      What is MongoDB?
                                           Rich Data Ty...
Native Language Integration
                    What is MongoDB?
                                         Rich Data Types
...
Native Language Integration
                        What is MongoDB?
                                             Rich Dat...
What is MongoDB?     Built-In MapReduce
              What Makes Mongo Special?    Easy Hadoop-Mongo Integration
         ...
What is MongoDB?     Built-In MapReduce
           What Makes Mongo Special?    Easy Hadoop-Mongo Integration
            ...
What is MongoDB?     Built-In MapReduce
               What Makes Mongo Special?    Easy Hadoop-Mongo Integration
        ...
What is MongoDB?     Built-In MapReduce
               What Makes Mongo Special?    Easy Hadoop-Mongo Integration
        ...
What is MongoDB?
              What Makes Mongo Special?
                             MapReduce


Upcoming events




    ...
What is MongoDB?
                What Makes Mongo Special?
                               MapReduce


Links




        ht...
Upcoming SlideShare
Loading in...5
×

10 Gen 20100217 Hadoop Bay Area

7,273

Published on

0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,273
On Slideshare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
0
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

Transcript of "10 Gen 20100217 Hadoop Bay Area"

  1. 1. What is MongoDB? What Makes Mongo Special? MapReduce MongoDB Or how I learned to stop worrying and love the database Mathias Stearn 10gen Bay Area Hadoop Meetup – February 17, 2010 Mathias Stearn MongoDB
  2. 2. What is MongoDB? What Makes Mongo Special? MapReduce The resulting [MongoDB] application has literally changed the way the pharma company conducts business. Whereas in the past, patient queries could take minutes to hours, results are now essentially real-time. http://news.cnet.com/8301-13846_3-10451248-62.html Why bother [with] memcached for caching HTTP sessions when you have an authoritative MongoDB? The performance is there. Carl Byström, @cgbystrom on twitter Mathias Stearn MongoDB
  3. 3. What is MongoDB? What Makes Mongo Special? MapReduce Compared to hadoop, Mongo’s speed and startup time make developing new queries much easier; what took us two weeks to get working on hadoop was done in two days on mongo. Emmett Shear, CTO (and developer) at Justin.tv It took me half a day to go from not touching MongoDB to writing some fairly good functionality against it. It makes setting up, configuring, and interfacing with MySQL look archaic – ridiculously archaic. http://geekaustin.org/2010/01/31/ mongodb-day-geek-austin-data-series Mathias Stearn MongoDB
  4. 4. What is MongoDB? What Makes Mongo Special? MapReduce 1 What is MongoDB? Document Oriented JavaScript Enabled Fast, Scalable, Available, and Reliable 2 What Makes Mongo Special? Native Language Integration Rich Data Types Atomic Modifiers Dynamic Queries 3 MapReduce Built-In MapReduce Easy Hadoop-Mongo Integration Better Hadoop-Mongo Integration Mathias Stearn MongoDB
  5. 5. What is MongoDB? Document Oriented What Makes Mongo Special? JavaScript Enabled MapReduce Fast, Scalable, Available, and Reliable 1 What is MongoDB? Document Oriented JavaScript Enabled Fast, Scalable, Available, and Reliable 2 What Makes Mongo Special? Native Language Integration Rich Data Types Atomic Modifiers Dynamic Queries 3 MapReduce Built-In MapReduce Easy Hadoop-Mongo Integration Better Hadoop-Mongo Integration Mathias Stearn MongoDB
  6. 6. What is MongoDB? Document Oriented What Makes Mongo Special? JavaScript Enabled MapReduce Fast, Scalable, Available, and Reliable Document Oriented Organized into Databases and Collections (like Tables) JSON-like (BSON) Schemaless Dynamic, Strong Typing Database can “reach into” objects db.people.insert({ _id: "mstearn", name: "Mathias Stearn", karma: 42, active: true, birthdate: new Date(517896000000), interests: ["MongoDB", "Python", "Üñíçø¯˘"], de subobject: {foo: "bar"} }); Mathias Stearn MongoDB
  7. 7. What is MongoDB? Document Oriented What Makes Mongo Special? JavaScript Enabled MapReduce Fast, Scalable, Available, and Reliable JavaScript used for: Shell and Documentation (Very) Advanced Queries “Group By” Queries MapReduce db.users.find({$where: "this.a + this.b >= 42"}); db.posts.group( { key: "user" , initial: {count:0, comments:0} , reduce: function(doc,out){ out.count++; out.comments += doc.comments.length; } , finalize: function(out){ out.avg = out.comments / out.count; } }); Mathias Stearn MongoDB
  8. 8. What is MongoDB? Document Oriented What Makes Mongo Special? JavaScript Enabled MapReduce Fast, Scalable, Available, and Reliable Fast, Scalable, Available, and Reliable Master-Slave replication for Availability and Reliability Replica-Pairs support auto-negotiation for master Auto-Sharding for Horizontal Scalability Distributes based on specified field Currently alpha MMAP database files to automatically use available RAM Asynchronous modifications Mathias Stearn MongoDB
  9. 9. What is MongoDB? Document Oriented What Makes Mongo Special? JavaScript Enabled MapReduce Fast, Scalable, Available, and Reliable Mathias Stearn MongoDB
  10. 10. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries 1 What is MongoDB? Document Oriented JavaScript Enabled Fast, Scalable, Available, and Reliable 2 What Makes Mongo Special? Native Language Integration Rich Data Types Atomic Modifiers Dynamic Queries 3 MapReduce Built-In MapReduce Easy Hadoop-Mongo Integration Better Hadoop-Mongo Integration Mathias Stearn MongoDB
  11. 11. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Official Java/JVM Python Ruby C/C++ Perl PHP Community Supported Closure, Scala, C#, Haskell, Erlang, and More Mathias Stearn MongoDB
  12. 12. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries JSON String (UTF8) Double Object (hash/map/dict) Array Bool Null / Undefined Extras Date Int32 / Int64 ObjectID (12 bytes: timestamp + host + pid + counter) Binary (with type byte) Mathias Stearn MongoDB
  13. 13. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries $set $inc $multiply (soon) $push / $pushAll $pull / $pullAll db.posts.update({_id:SOMEID}, {$push:{tags:"mongodb"}}) db.tags.update({_id:"mongodb"}, {$inc:{count:1}}, {upsert:true}}) Mathias Stearn MongoDB
  14. 14. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Simple db.posts.findOne({ user: "mstearn" }); var cursor = db.posts.find({ user: "mstearn" }); cursor.forEach(function(){ doSomething(this.text); }); Mathias Stearn MongoDB
  15. 15. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Sorted db.posts.find( { user: "mstearn" } ).sort({timestamp:-1}) Mathias Stearn MongoDB
  16. 16. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Paginated db.posts.find( { user: "mstearn" } ).sort({timestamp:-1}).skip(10).limit(10); Mathias Stearn MongoDB
  17. 17. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Simple Tag Search db.posts.find( { user: "mstearn" , tags: "mongo" } ).sort({timestamp:-1}).skip(10).limit(10); Mathias Stearn MongoDB
  18. 18. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Complex Tag Search db.posts.find( { user: "mstearn" , tags: {$in: ["mongo", "mongodb"]} } ).sort({timestamp:-1}).skip(10).limit(10); Mathias Stearn MongoDB
  19. 19. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Nested Objects db.posts.find( { user: "mstearn" , tags: {$in: ["mongo", "mongodb"]} , comments.user: "mdirolf" } ).sort({timestamp:-1}).skip(10).limit(10); Mathias Stearn MongoDB
  20. 20. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Regular Expressions db.posts.find( { user: "mstearn" , tags: {$in: ["mongo", "mongodb"]} , comments.user: "mdirolf" , text: /windows/i } ).sort({timestamp:-1}).skip(10).limit(10); Mathias Stearn MongoDB
  21. 21. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Ranges db.posts.find( { user: "mstearn" , tags: {$in: ["mongo", "mongodb"]} , comments.user: "mdirolf" , text: /windows/i , points: {$gt: 10, $lt: 100} } ).sort({timestamp:-1}).skip(10).limit(10); Mathias Stearn MongoDB
  22. 22. Native Language Integration What is MongoDB? Rich Data Types What Makes Mongo Special? Atomic Modifiers MapReduce Dynamic Queries Arbitrary JavaScript db.posts.find( { user: "mstearn" , tags: {$in: ["mongo", "mongodb"]} , comments.user: "mdirolf" , text: /windows/i , points: {$gt: 10, $lt 100} , $where: "this.a + this.b >= 42" } ).sort({timestamp:-1}).skip(10).limit(10); Mathias Stearn MongoDB
  23. 23. What is MongoDB? Built-In MapReduce What Makes Mongo Special? Easy Hadoop-Mongo Integration MapReduce Better Hadoop-Mongo Integration 1 What is MongoDB? Document Oriented JavaScript Enabled Fast, Scalable, Available, and Reliable 2 What Makes Mongo Special? Native Language Integration Rich Data Types Atomic Modifiers Dynamic Queries 3 MapReduce Built-In MapReduce Easy Hadoop-Mongo Integration Better Hadoop-Mongo Integration Mathias Stearn MongoDB
  24. 24. What is MongoDB? Built-In MapReduce What Makes Mongo Special? Easy Hadoop-Mongo Integration MapReduce Better Hadoop-Mongo Integration db.posts.mapReduce( function() { this.comments.forEach(c){ emit(c.user, {count:1, words:c.text.split().length; } } , function(key, values){ for (var i=1; i<values.length; i++){ values[0].count += values[i].count; values[0].words += values[i].words; } return values[0]; } , { finalize: function(out){ out.avg = out.words / out.count; return out; } , query: {posted: {$gt: new Date(2010,0,1)}} , out: ’posts.comment_stats’ } }); Mathias Stearn MongoDB
  25. 25. What is MongoDB? Built-In MapReduce What Makes Mongo Special? Easy Hadoop-Mongo Integration MapReduce Better Hadoop-Mongo Integration Easy Hadoop-Mongo Integration mongoexport can export to JSON/CSV/TSV Can also easily use a custom script Process in Hadoop Use mongoimport to get data back into MongoDB Mathias Stearn MongoDB
  26. 26. What is MongoDB? Built-In MapReduce What Makes Mongo Special? Easy Hadoop-Mongo Integration MapReduce Better Hadoop-Mongo Integration Better Hadoop-Mongo Integration mongodump writes a stream of BSON to a file Write an InputFilter and RecordReader to read BSON Write a BSONWriter class to directly use the data Just added two methods to driver to make this easier Process the data with the Java/Scala/Closure driver Write a custom RecordWriter to either: Dump to a file and use mongorestore Dump the output directly to MongoDB Optional: use renameCollection to mimic our MapReduce Mathias Stearn MongoDB
  27. 27. What is MongoDB? What Makes Mongo Special? MapReduce Upcoming events NoSQL Live! from Boston (March 11) MongoDB Training in San Francisco (March 25) San Fransisco MySQL Meetup (April 12) Mathias Stearn MongoDB
  28. 28. What is MongoDB? What Makes Mongo Special? MapReduce Links http://mongo.kylebanker.com (Try mongo in your browser) http://www.mongodb.org #mongodb on irc.freenode.net mongodb-user on google groups mathias@10gen.com @mathias_mongo on twitter Mathias Stearn MongoDB

×