Zero to Mongo in 60
                               Hours
                               Ryan Angilly
                     ...
I’m a pretty awesome dude




Wednesday, April 28, 2010
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education




Wed...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education

      ...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education

      ...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education

      ...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education

      ...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education
       ...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education
       ...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education
       ...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education
       ...
I’m a pretty awesome dude
                    •       Electrical Engineer by
                            education
       ...
MyPunchbowl is a pretty awesome company




Wednesday, April 28, 2010
MyPunchbowl is a pretty awesome company

              •      Leader in start to finish party planning




Wednesday, April...
MyPunchbowl is a pretty awesome company

              •      Leader in start to finish party planning

                   ...
MyPunchbowl is a pretty awesome company

              •      Leader in start to finish party planning

                   ...
MyPunchbowl is a pretty awesome company

              •      Leader in start to finish party planning

                   ...
MyPunchbowl is a pretty awesome company

              •      Leader in start to finish party planning

                   ...
MyPunchbowl is a pretty awesome company

              •      Leader in start to finish party planning

                   ...
MyPunchbowl is a pretty awesome company

              •      Leader in start to finish party planning

                   ...
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Our engineers play with everything




Wednesday, April 28, 2010
Wednesday, April 28, 2010
Search by category and
                                    location




Wednesday, April 28, 2010
Search by business name   Search by category and
                        and location              location




Wednesday,...
Wednesday, April 28, 2010
Track searches




Wednesday, April 28, 2010
Track searches




                            Track vendor impressions



Wednesday, April 28, 2010
MongoDB +
                            MyPunchbowl
                             A tale in conversations




Wednesday, Apri...
Tracking requirement a good excuse to
                              finally use MongoDB




Wednesday, April 28, 2010
Tracking requirement a good excuse to
                              finally use MongoDB

                        Me: Hey Bl...
Tracking requirement a good excuse to
                              finally use MongoDB

                        Me: Hey Bl...
Tracking requirement a good excuse to
                              finally use MongoDB

                        Me: Hey Bl...
Tracking requirement a good excuse to
                              finally use MongoDB

                        Me: Hey Bl...
Tracking requirement a good excuse to
                              finally use MongoDB

                        Me: Hey Bl...
6 reasons to use MongoDB
                    • Easy to get running (~5 minutes on OSX)
                    • Open Source
 ...
MongoDB feels right




Wednesday, April 28, 2010
MongoDB feels right
                        Me: MongoDB gives me the warm fuzzies
                        that Rails did.
...
MongoDB feels right
                        Me: MongoDB gives me the warm fuzzies
                        that Rails did.
...
MongoDB feels right
                        Me: MongoDB gives me the warm fuzzies
                        that Rails did.
...
MongoDB feels right
                        Me: MongoDB gives me the warm fuzzies
                        that Rails did.
...
Support is INSANE




                             11:35pm. Wednesday. Founder.
                                60 second ...
2.5 days? Really?




Wednesday, April 28, 2010
2.5 days? Really?


                                   Yes


Wednesday, April 28, 2010
2.5 days? Really?


                                   Yes*




Wednesday, April 28, 2010
2.5 days to MongoDB




        0 Days                  1 Day             2 Days   200+ Days
                             ...
2.5 days to MongoDB
              Decision
               to use
              MongoDB




        0 Days                 ...
2.5 days to MongoDB
              Decision
               to use
              MongoDB




        0 Days                 ...
2.5 days to MongoDB
              Decision
               to use
              MongoDB


                                 ...
2.5 days to MongoDB
              Decision
               to use
              MongoDB


                                 ...
2.5 days to MongoDB
              Decision                                       Build test rig
               to use
    ...
2.5 days to MongoDB
              Decision                                       Build test rig
               to use
    ...
2.5 days to MongoDB
              Decision                                       Build test rig
               to use
    ...
2.5 days to MongoDB
              Decision                                       Build test rig
               to use
    ...
A stroll through Mongo’s OSS ecosystem




Wednesday, April 28, 2010
A stroll through Mongo’s OSS ecosystem

                    • mongodb (C++)




Wednesday, April 28, 2010
A stroll through Mongo’s OSS ecosystem

                    • mongodb (C++)
                    • mongo shell (JS via Spid...
A stroll through Mongo’s OSS ecosystem

                    • mongodb (C++)
                    • mongo shell (JS via Spid...
A stroll through Mongo’s OSS ecosystem

                    • mongodb (C++)
                    • mongo shell (JS via Spid...
A stroll through Mongo’s OSS ecosystem

                    • mongodb (C++)
                    • mongo shell (JS via Spid...
A stroll through Mongo’s OSS ecosystem

                    • mongodb (C++)
                    • mongo shell (JS via Spid...
A stroll through Mongo’s OSS ecosystem

                    • mongodb (C++)
                    • mongo shell (JS via Spid...
Document ‘schema’




Wednesday, April 28, 2010
Document ‘schema’
         • What do documents look like?




Wednesday, April 28, 2010
Document ‘schema’
         • What do documents look like?
         •     Example Document
               {
               ...
Document ‘schema’
         • What do documents look like?   • How do you update the documents?
         •     Example Docu...
Document ‘schema’
         • What do documents look like?   • How do you update the documents?
         •     Example Docu...
Document ‘schema’
         • What do documents look like?   • How do you update the documents?
         •     Example Docu...
Document ‘schema’
         • What do documents look like?   • How do you update the documents?
         •     Example Docu...
Document ‘schema’
         • What do documents look like?   • How do you update the documents?
         •     Example Docu...
Cuhhraazy Indexes




Wednesday, April 28, 2010
Cuhhraazy Indexes
       •     Original Document
             {
               city: “Boston”,
               state: “MA”,...
Cuhhraazy Indexes
       •     Original Document
             {
               city: “Boston”,
               state: “MA”,...
Cuhhraazy Indexes
       •     Original Document
             {
               city: “Boston”,
               state: “MA”,...
Cuhhraazy Indexes
       •     Original Document
             {
               city: “Boston”,
               state: “MA”,...
Cuhhraazy Indexes
       •     Original Document
             {
               city: “Boston”,
               state: “MA”,...
Cuhhraazy Indexes
       •     Original Document                  •   More Complex Document w/ Embedded Document
         ...
Cuhhraazy Indexes
       •     Original Document                  •   More Complex Document w/ Embedded Document
         ...
Cuhhraazy Indexes
       •     Original Document                  •   More Complex Document w/ Embedded Document
         ...
Cuhhraazy Indexes
       •     Original Document                  •   More Complex Document w/ Embedded Document
         ...
Cuhhraazy Indexes
       •     Original Document                  •   More Complex Document w/ Embedded Document
         ...
Let’s write some Ruby




Wednesday, April 28, 2010
Let’s write some Ruby
                class SearchStats
                  cattr_accessor :db, :collection

               ...
Let’s write some Ruby
                class SearchStats
                  cattr_accessor :db, :collection

               ...
Let’s write some Ruby
                class SearchStats
                  cattr_accessor :db, :collection

               ...
TESTING




Wednesday, April 28, 2010
TESTING
                    • Pretty much the same as anything else




Wednesday, April 28, 2010
TESTING
                    • Pretty much the same as anything else
                     • test/unit/search_stats_test.rb
...
TESTING
                    • Pretty much the same as anything else
                     • test/unit/search_stats_test.rb
...
TESTING
                    • Pretty much the same as anything else
                     • test/unit/search_stats_test.rb
...
TESTING
                    • Pretty much the same as anything else
                     • test/unit/search_stats_test.rb
...
TESTING
                    • Pretty much the same as anything else
                     • test/unit/search_stats_test.rb
...
TESTING
                    • Pretty much the same as anything else
                     • test/unit/search_stats_test.rb
...
TESTING
                    • Pretty much the same as anything else
                     • test/unit/search_stats_test.rb
...
Other developers gotta
                       use it too




Wednesday, April 28, 2010
Other developers gotta
                       use it too
                    • Rails’ database.yml takes care of MySQL



...
Other developers gotta
                       use it too
                    • Rails’ database.yml takes care of MySQL
   ...
Other developers gotta
                       use it too
                    • Rails’ database.yml takes care of MySQL
   ...
Other developers gotta
                       use it too
                    • Rails’ database.yml takes care of MySQL
   ...
Other developers gotta
                       use it too
                    • Rails’ database.yml takes care of MySQL
   ...
Making production
                                  ready




Wednesday, April 28, 2010
Making production
                                  ready
                    • Replication was iffy in 1.0.0 (solid now)
...
Making production
                                  ready
                    • Replication was iffy in 1.0.0 (solid now)
...
Making production
                                  ready
                    • Replication was iffy in 1.0.0 (solid now)
...
Making production
                                  ready
                    • Replication was iffy in 1.0.0 (solid now)
...
Deploy




Wednesday, April 28, 2010
Deploy
                    • Original deployment uneventful




Wednesday, April 28, 2010
Deploy
                    • Original deployment uneventful
                     • Just worked




Wednesday, April 28, 20...
Deploy
                    • Original deployment uneventful
                     • Just worked
                    • Futur...
Deploy
                    • Original deployment uneventful
                     • Just worked
                    • Futur...
Day 200




Wednesday, April 28, 2010
Day 200
                    • One production process




Wednesday, April 28, 2010
Day 200
                    • One production process
                    • One database




Wednesday, April 28, 2010
Day 200
                    • One production process
                    • One database
                    • Daily dumps ...
Day 200
                    • One production process
                    • One database
                    • Daily dumps ...
Day 200
                    • One production process
                    • One database
                    • Daily dumps ...
Day 200
                    • One production process
                    • One database
                    • Daily dumps ...
Day 200
                    • One production process
                    • One database
                    • Daily dumps ...
What tripped me up?



Wednesday, April 28, 2010
Simple Test
                     should "get impressions_by_date" do
                       record1 = {:date => 100, :impr...
Crazy Failure
                            1) Failure:
                            test: Mongo::Vendor should get
         ...
Wednesday, April 28, 2010
Mongo’s BSON OrderedHash doesn’t behave
              like ActiveSupport::OrderedHash




Wednesday, April 28, 2010
Mongo’s BSON OrderedHash doesn’t behave
              like ActiveSupport::OrderedHash

   irb(main):001:0> require 'active...
Mongo’s BSON OrderedHash doesn’t behave
              like ActiveSupport::OrderedHash

   irb(main):001:0> require 'active...
Mongo’s BSON OrderedHash doesn’t behave
              like ActiveSupport::OrderedHash

   irb(main):001:0> require 'active...
Wednesday, April 28, 2010
And it turns out...




Wednesday, April 28, 2010
And it turns out...

                            MongoDB will not add indexes to your
                                    ...
And it turns out...

                            MongoDB will not add indexes to your
                                    ...
add indexes

                              add indexes

                               add indexes

                      ...
But it’ll help pick up the pieces

                 Before index
           > db.vendors.findOne({'date': {'$gt': 127189520...
But it’ll help pick up the pieces

                 Before index
           > db.vendors.findOne({'date': {'$gt': 127189520...
But it’ll help pick up the pieces

                 Before index
           > db.vendors.findOne({'date': {'$gt': 127189520...
But it’ll help pick up the pieces

                 Before index
           > db.vendors.findOne({'date': {'$gt': 127189520...
But it’ll help pick up the pieces

                 Before index
           > db.vendors.findOne({'date': {'$gt': 127189520...
Up next: Top Secret Project




Wednesday, April 28, 2010
Up next: Top Secret Project
        •      Project involving




Wednesday, April 28, 2010
Up next: Top Secret Project
        •      Project involving
              •  SCREAMS for a document-based solution




We...
Up next: Top Secret Project
        •      Project involving
              •  SCREAMS for a document-based solution
      ...
Up next: Top Secret Project
        •      Project involving
              •  SCREAMS for a document-based solution
      ...
Up next: Top Secret Project
        •      Project involving
              •  SCREAMS for a document-based solution
      ...
Up next: Top Secret Project
        •      Project involving
              •  SCREAMS for a document-based solution
      ...
Up next: Top Secret Project
        •      Project involving
              •  SCREAMS for a document-based solution
      ...
The end

                            Questions?


Wednesday, April 28, 2010
Contact Me

                                 ryan@angilly.com
                            http://www.mypunchbowl.com
     ...
Links

               Slide 3/4 logos, wget’d from all over the Internets.

               Dunce Cap http://steynian.files....
Upcoming SlideShare
Loading in...5
×

Zero to Mongo in 60 Hours

9,584

Published on

Published in: Technology
1 Comment
49 Likes
Statistics
Notes
  • = ENTREPRENEURS WANTED =

    We Are a Global Community of Entrepreneurs Looking for New Leaders to Increase the Synergy of our Qualified Team.

    We are Entrepreneurs Working Together from the Comfort of Home. Come Join Us, Let’s Do this Together!!

    Review: http://bit.ly/db01RT
    Website: http://myentrepreneuronlinebusiness.com
    <br /><object type="application/x-shockwave-flash" data="http://www.youtube.com/v/DKzEHqygElA&amp;hl=es_MX&amp;fs=1?rel=0" width="350" height="288"><param name="movie" value="http://www.youtube.com/v/DKzEHqygElA&amp;hl=es_MX&amp;fs=1?rel=0"></param><embed src="http://www.youtube.com/v/DKzEHqygElA&amp;hl=es_MX&amp;fs=1?rel=0" width="350" height="288" type="application/x-shockwave-flash"></embed></object>
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
9,584
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
488
Comments
1
Likes
49
Embeds 0
No embeds

No notes for slide

Zero to Mongo in 60 Hours

  1. 1. Zero to Mongo in 60 Hours Ryan Angilly MyPunchbowl.com @angilly Wednesday, April 28, 2010
  2. 2. I’m a pretty awesome dude Wednesday, April 28, 2010
  3. 3. I’m a pretty awesome dude • Electrical Engineer by education Wednesday, April 28, 2010
  4. 4. I’m a pretty awesome dude • Electrical Engineer by education • Ex-EMC’er Wednesday, April 28, 2010
  5. 5. I’m a pretty awesome dude • Electrical Engineer by education • Ex-EMC’er • Founded MessageSling.com Wednesday, April 28, 2010
  6. 6. I’m a pretty awesome dude • Electrical Engineer by education • Ex-EMC’er • Founded MessageSling.com • Entered the deadpool in January Wednesday, April 28, 2010
  7. 7. I’m a pretty awesome dude • Electrical Engineer by education • Ex-EMC’er • Founded MessageSling.com • Entered the deadpool in January • Senior developer @ MyPunchbowl.com Wednesday, April 28, 2010
  8. 8. I’m a pretty awesome dude • Electrical Engineer by education • Built several web apps • Ex-EMC’er • Founded MessageSling.com • Entered the deadpool in January • Senior developer @ MyPunchbowl.com Wednesday, April 28, 2010
  9. 9. I’m a pretty awesome dude • Electrical Engineer by education • Built several web apps • Ex-EMC’er • All Ruby. Mostly Rails • Founded MessageSling.com • Entered the deadpool in January • Senior developer @ MyPunchbowl.com Wednesday, April 28, 2010
  10. 10. I’m a pretty awesome dude • Electrical Engineer by education • Built several web apps • Ex-EMC’er • All Ruby. Mostly Rails • Founded • All SQL MessageSling.com • Entered the deadpool in January • Senior developer @ MyPunchbowl.com Wednesday, April 28, 2010
  11. 11. I’m a pretty awesome dude • Electrical Engineer by education • Built several web apps • Ex-EMC’er • All Ruby. Mostly Rails • Founded • All SQL MessageSling.com • No experience w/ NoSQL/Mongo as of • Entered the deadpool in January September ’09 • Senior developer @ MyPunchbowl.com Wednesday, April 28, 2010
  12. 12. I’m a pretty awesome dude • Electrical Engineer by education • Built several web apps • Ex-EMC’er • All Ruby. Mostly Rails • Founded • All SQL MessageSling.com • No experience w/ NoSQL/Mongo as of • Entered the deadpool in January September ’09 • Senior developer @ • Want to provide a beginner’s perspective MyPunchbowl.com Wednesday, April 28, 2010
  13. 13. MyPunchbowl is a pretty awesome company Wednesday, April 28, 2010
  14. 14. MyPunchbowl is a pretty awesome company • Leader in start to finish party planning Wednesday, April 28, 2010
  15. 15. MyPunchbowl is a pretty awesome company • Leader in start to finish party planning • Connecting party planners to party vendors -- subject of this talk Wednesday, April 28, 2010
  16. 16. MyPunchbowl is a pretty awesome company • Leader in start to finish party planning • Connecting party planners to party vendors -- subject of this talk • Currently employs 11 people Wednesday, April 28, 2010
  17. 17. MyPunchbowl is a pretty awesome company • Leader in start to finish party planning • Connecting party planners to party vendors -- subject of this talk • Currently employs 11 people • Funded over 2 rounds by Intel & Contour Wednesday, April 28, 2010
  18. 18. MyPunchbowl is a pretty awesome company • Leader in start to finish party planning • Connecting party planners to party vendors -- subject of this talk • Currently employs 11 people • Funded over 2 rounds by Intel & Contour • Strategic partnership w/ OTC among others Wednesday, April 28, 2010
  19. 19. MyPunchbowl is a pretty awesome company • Leader in start to finish party planning • Connecting party planners to party vendors -- subject of this talk • Currently employs 11 people • Funded over 2 rounds by Intel & Contour • Strategic partnership w/ OTC among others • Amount of traffic, users, events, invites, vendors, etc... make our data sets large enough to be interesting :) Wednesday, April 28, 2010
  20. 20. MyPunchbowl is a pretty awesome company • Leader in start to finish party planning • Connecting party planners to party vendors -- subject of this talk • Currently employs 11 people • Funded over 2 rounds by Intel & Contour • Strategic partnership w/ OTC among others • Amount of traffic, users, events, invites, vendors, etc... make our data sets large enough to be interesting :) • We’re talking many millions of ‘things’ Wednesday, April 28, 2010
  21. 21. Our engineers play with everything Wednesday, April 28, 2010
  22. 22. Our engineers play with everything Wednesday, April 28, 2010
  23. 23. Our engineers play with everything Wednesday, April 28, 2010
  24. 24. Our engineers play with everything Wednesday, April 28, 2010
  25. 25. Our engineers play with everything Wednesday, April 28, 2010
  26. 26. Our engineers play with everything Wednesday, April 28, 2010
  27. 27. Our engineers play with everything Wednesday, April 28, 2010
  28. 28. Our engineers play with everything Wednesday, April 28, 2010
  29. 29. Our engineers play with everything Wednesday, April 28, 2010
  30. 30. Our engineers play with everything Wednesday, April 28, 2010
  31. 31. Our engineers play with everything Wednesday, April 28, 2010
  32. 32. Our engineers play with everything Wednesday, April 28, 2010
  33. 33. Our engineers play with everything Wednesday, April 28, 2010
  34. 34. Our engineers play with everything Wednesday, April 28, 2010
  35. 35. Wednesday, April 28, 2010
  36. 36. Search by category and location Wednesday, April 28, 2010
  37. 37. Search by business name Search by category and and location location Wednesday, April 28, 2010
  38. 38. Wednesday, April 28, 2010
  39. 39. Track searches Wednesday, April 28, 2010
  40. 40. Track searches Track vendor impressions Wednesday, April 28, 2010
  41. 41. MongoDB + MyPunchbowl A tale in conversations Wednesday, April 28, 2010
  42. 42. Tracking requirement a good excuse to finally use MongoDB Wednesday, April 28, 2010
  43. 43. Tracking requirement a good excuse to finally use MongoDB Me: Hey Blake I’m gonna use MongoDB to keep track of all this search stuff. Wednesday, April 28, 2010
  44. 44. Tracking requirement a good excuse to finally use MongoDB Me: Hey Blake I’m gonna use MongoDB to keep track of all this search stuff. Blake: Cool. Wednesday, April 28, 2010
  45. 45. Tracking requirement a good excuse to finally use MongoDB Me: Hey Blake I’m gonna use MongoDB to keep track of all this search stuff. Blake: Cool. Matt (CEO): What’s that? Wednesday, April 28, 2010
  46. 46. Tracking requirement a good excuse to finally use MongoDB Me: Hey Blake I’m gonna use MongoDB to keep track of all this search stuff. Blake: Cool. Matt (CEO): What’s that? Blake & Me: Something cool. Wednesday, April 28, 2010
  47. 47. Tracking requirement a good excuse to finally use MongoDB Me: Hey Blake I’m gonna use MongoDB to keep track of all this search stuff. Blake: Cool. Matt (CEO): What’s that? Blake & Me: Something cool. Matt (CEO): K Wednesday, April 28, 2010
  48. 48. 6 reasons to use MongoDB • Easy to get running (~5 minutes on OSX) • Open Source • Support in multiple (computer) languages • Prototype in Ruby, move to Java if needed • Very active development • Full featured • Great ecosystem Wednesday, April 28, 2010
  49. 49. MongoDB feels right Wednesday, April 28, 2010
  50. 50. MongoDB feels right Me: MongoDB gives me the warm fuzzies that Rails did. Wednesday, April 28, 2010
  51. 51. MongoDB feels right Me: MongoDB gives me the warm fuzzies that Rails did. Darren: Just like that Nunemaker post. Wednesday, April 28, 2010
  52. 52. MongoDB feels right Me: MongoDB gives me the warm fuzzies that Rails did. Darren: Just like that Nunemaker post. Me: ? *runs to google* Wednesday, April 28, 2010
  53. 53. MongoDB feels right Me: MongoDB gives me the warm fuzzies that Rails did. Darren: Just like that Nunemaker post. Me: ? *runs to google* Wednesday, April 28, 2010
  54. 54. Support is INSANE 11:35pm. Wednesday. Founder. 60 second response time. Wednesday, April 28, 2010
  55. 55. 2.5 days? Really? Wednesday, April 28, 2010
  56. 56. 2.5 days? Really? Yes Wednesday, April 28, 2010
  57. 57. 2.5 days? Really? Yes* Wednesday, April 28, 2010
  58. 58. 2.5 days to MongoDB 0 Days 1 Day 2 Days 200+ Days and on... Wednesday, April 28, 2010
  59. 59. 2.5 days to MongoDB Decision to use MongoDB 0 Days 1 Day 2 Days 200+ Days and on... Wednesday, April 28, 2010
  60. 60. 2.5 days to MongoDB Decision to use MongoDB 0 Days 1 Day 2 Days 200+ Days and on... Play around with available OSS Wednesday, April 28, 2010
  61. 61. 2.5 days to MongoDB Decision to use MongoDB Figure out document ‘schema’ 0 Days 1 Day 2 Days 200+ Days and on... Play around with available OSS Wednesday, April 28, 2010
  62. 62. 2.5 days to MongoDB Decision to use MongoDB Figure out document ‘schema’ 0 Days 1 Day 2 Days 200+ Days and on... Play around with available OSS Write some stuff on top of mongo-ruby-driver Wednesday, April 28, 2010
  63. 63. 2.5 days to MongoDB Decision Build test rig to use MongoDB Figure out document ‘schema’ 0 Days 1 Day 2 Days 200+ Days and on... Play around with available OSS Write some stuff on top of mongo-ruby-driver Wednesday, April 28, 2010
  64. 64. 2.5 days to MongoDB Decision Build test rig to use MongoDB Figure out document ‘schema’ 0 Days 1 Day 2 Days 200+ Days and on... Play around with Setup Configuration available OSS Management Write some stuff on top of mongo-ruby-driver Wednesday, April 28, 2010
  65. 65. 2.5 days to MongoDB Decision Build test rig to use MongoDB Figure out document Get production ready ‘schema’ 0 Days 1 Day 2 Days 200+ Days and on... Play around with Setup Configuration available OSS Management Write some stuff on top of mongo-ruby-driver Wednesday, April 28, 2010
  66. 66. 2.5 days to MongoDB Decision Build test rig to use MongoDB Figure out document Get production ready ‘schema’ 0 Days 1 Day 2 Days 200+ Days and on... Play around with Setup Configuration available OSS Management Write some stuff on top Deploy of mongo-ruby-driver Wednesday, April 28, 2010
  67. 67. A stroll through Mongo’s OSS ecosystem Wednesday, April 28, 2010
  68. 68. A stroll through Mongo’s OSS ecosystem • mongodb (C++) Wednesday, April 28, 2010
  69. 69. A stroll through Mongo’s OSS ecosystem • mongodb (C++) • mongo shell (JS via SpiderMonkey) Wednesday, April 28, 2010
  70. 70. A stroll through Mongo’s OSS ecosystem • mongodb (C++) • mongo shell (JS via SpiderMonkey) • mongo-ruby-driver Wednesday, April 28, 2010
  71. 71. A stroll through Mongo’s OSS ecosystem • mongodb (C++) • mongo shell (JS via SpiderMonkey) • mongo-ruby-driver • mongo-java-driver Wednesday, April 28, 2010
  72. 72. A stroll through Mongo’s OSS ecosystem • mongodb (C++) • mongo shell (JS via SpiderMonkey) • mongo-ruby-driver • mongo-java-driver • mongo_record (Ruby) Wednesday, April 28, 2010
  73. 73. A stroll through Mongo’s OSS ecosystem • mongodb (C++) • mongo shell (JS via SpiderMonkey) • mongo-ruby-driver • mongo-java-driver • mongo_record (Ruby) • mongo_mapper (Ruby) Wednesday, April 28, 2010
  74. 74. A stroll through Mongo’s OSS ecosystem • mongodb (C++) • mongo shell (JS via SpiderMonkey) • mongo-ruby-driver • mongo-java-driver • mongo_record (Ruby) • mongo_mapper (Ruby) • Integration w/ Rails Wednesday, April 28, 2010
  75. 75. Document ‘schema’ Wednesday, April 28, 2010
  76. 76. Document ‘schema’ • What do documents look like? Wednesday, April 28, 2010
  77. 77. Document ‘schema’ • What do documents look like? • Example Document { city: “Boston”, state: “MA”, date: 1272153600, occurrences: 1, category_id: 1 } Wednesday, April 28, 2010
  78. 78. Document ‘schema’ • What do documents look like? • How do you update the documents? • Example Document { city: “Boston”, state: “MA”, date: 1272153600, occurrences: 1, category_id: 1 } Wednesday, April 28, 2010
  79. 79. Document ‘schema’ • What do documents look like? • How do you update the documents? • Example Document • Mongo shell command { coll.update({‘city’: ‘Boston’, city: “Boston”, ‘state’: ‘MA’, state: “MA”, ‘date’: 1272153600, date: 1272153600, ‘category_id’: 1}, occurrences: 1, {‘$inc’: {‘occurrences’: 1}}, category_id: 1 true) } Wednesday, April 28, 2010
  80. 80. Document ‘schema’ • What do documents look like? • How do you update the documents? • Example Document • Mongo shell command { coll.update({‘city’: ‘Boston’, city: “Boston”, ‘state’: ‘MA’, state: “MA”, ‘date’: 1272153600, date: 1272153600, ‘category_id’: 1}, occurrences: 1, {‘$inc’: {‘occurrences’: 1}}, category_id: 1 true) } Document to match Wednesday, April 28, 2010
  81. 81. Document ‘schema’ • What do documents look like? • How do you update the documents? • Example Document • Mongo shell command { coll.update({‘city’: ‘Boston’, city: “Boston”, ‘state’: ‘MA’, state: “MA”, ‘date’: 1272153600, date: 1272153600, ‘category_id’: 1}, occurrences: 1, {‘$inc’: {‘occurrences’: 1}}, category_id: 1 true) } Document to match Operation to perform Wednesday, April 28, 2010
  82. 82. Document ‘schema’ • What do documents look like? • How do you update the documents? • Example Document • Mongo shell command { coll.update({‘city’: ‘Boston’, city: “Boston”, ‘state’: ‘MA’, state: “MA”, ‘date’: 1272153600, date: 1272153600, ‘category_id’: 1}, occurrences: 1, {‘$inc’: {‘occurrences’: 1}}, category_id: 1 true) } Document to match Operation to perform ‘upsert’: update if it’s there insert if it’s not Wednesday, April 28, 2010
  83. 83. Cuhhraazy Indexes Wednesday, April 28, 2010
  84. 84. Cuhhraazy Indexes • Original Document { city: “Boston”, state: “MA”, date: 1272153600, occurrences: 1, category_id: 1 } Wednesday, April 28, 2010
  85. 85. Cuhhraazy Indexes • Original Document { city: “Boston”, state: “MA”, date: 1272153600, occurrences: 1, category_id: 1 } • 40k cities, 200 days, 60 categories Wednesday, April 28, 2010
  86. 86. Cuhhraazy Indexes • Original Document { city: “Boston”, state: “MA”, date: 1272153600, occurrences: 1, category_id: 1 } • 40k cities, 200 days, 60 categories • 480M potential documents Wednesday, April 28, 2010
  87. 87. Cuhhraazy Indexes • Original Document { city: “Boston”, state: “MA”, date: 1272153600, occurrences: 1, category_id: 1 } • 40k cities, 200 days, 60 categories • 480M potential documents • Composite indexes • state_1_city_1 • state_1_date_1_city_1 • category_id_1_date_1 • date_1 Wednesday, April 28, 2010
  88. 88. Cuhhraazy Indexes • Original Document { city: “Boston”, state: “MA”, date: 1272153600, occurrences: 1, category_id: 1 } • 40k cities, 200 days, 60 categories • 480M potential documents • Composite indexes • state_1_city_1 • state_1_date_1_city_1 • category_id_1_date_1 • date_1 • Flexibility & performance in querying and aggregating Wednesday, April 28, 2010
  89. 89. Cuhhraazy Indexes • Original Document • More Complex Document w/ Embedded Document { { city: “Boston”, city: “Boston”, state: “MA”, state: “MA”, date: 1272153600, date: 1272153600, occurrences: 1, occurrences: 9581555, category_id: 1 category_id: 1, } mobile_sources: { • 40k cities, 200 days, 60 categories browsers: { windows_mobile: 1, • 480M potential documents palm_os: 0, • Composite indexes }, iphone_4g: 9481554 • state_1_city_1 zip_codes: [“01518”] • state_1_date_1_city_1 } • category_id_1_date_1 } • date_1 • Flexibility & performance in querying and aggregating Wednesday, April 28, 2010
  90. 90. Cuhhraazy Indexes • Original Document • More Complex Document w/ Embedded Document { { city: “Boston”, city: “Boston”, state: “MA”, state: “MA”, date: 1272153600, date: 1272153600, occurrences: 1, occurrences: 9581555, category_id: 1 category_id: 1, } mobile_sources: { • 40k cities, 200 days, 60 categories browsers: { windows_mobile: 1, • 480M potential documents palm_os: 0, • Composite indexes }, iphone_4g: 9481554 • state_1_city_1 zip_codes: [“01518”] • state_1_date_1_city_1 } • category_id_1_date_1 } • date_1 • Additional, complex data down the road? • Flexibility & performance in querying and aggregating Wednesday, April 28, 2010
  91. 91. Cuhhraazy Indexes • Original Document • More Complex Document w/ Embedded Document { { city: “Boston”, city: “Boston”, state: “MA”, state: “MA”, date: 1272153600, date: 1272153600, occurrences: 1, occurrences: 9581555, category_id: 1 category_id: 1, } mobile_sources: { • 40k cities, 200 days, 60 categories browsers: { windows_mobile: 1, • 480M potential documents palm_os: 0, • Composite indexes }, iphone_4g: 9481554 • state_1_city_1 zip_codes: [“01518”] • state_1_date_1_city_1 } • category_id_1_date_1 } • date_1 • Additional, complex data down the road? • Flexibility & performance in • Add deep embedded indexes! querying and aggregating Wednesday, April 28, 2010
  92. 92. Cuhhraazy Indexes • Original Document • More Complex Document w/ Embedded Document { { city: “Boston”, city: “Boston”, state: “MA”, state: “MA”, date: 1272153600, date: 1272153600, occurrences: 1, occurrences: 9581555, category_id: 1 category_id: 1, } mobile_sources: { • 40k cities, 200 days, 60 categories browsers: { windows_mobile: 1, • 480M potential documents palm_os: 0, • Composite indexes }, iphone_4g: 9481554 • state_1_city_1 zip_codes: [“01518”] • state_1_date_1_city_1 } • category_id_1_date_1 } • date_1 • Additional, complex data down the road? • Flexibility & performance in • Add deep embedded indexes! querying and aggregating • ‘mobile_sources.zip_codes’ Wednesday, April 28, 2010
  93. 93. Cuhhraazy Indexes • Original Document • More Complex Document w/ Embedded Document { { city: “Boston”, city: “Boston”, state: “MA”, state: “MA”, date: 1272153600, date: 1272153600, occurrences: 1, occurrences: 9581555, category_id: 1 category_id: 1, } mobile_sources: { • 40k cities, 200 days, 60 categories browsers: { windows_mobile: 1, • 480M potential documents palm_os: 0, • Composite indexes }, iphone_4g: 9481554 • state_1_city_1 zip_codes: [“01518”] • state_1_date_1_city_1 } • category_id_1_date_1 } • date_1 • Additional, complex data down the road? • Flexibility & performance in • Add deep embedded indexes! querying and aggregating • ‘mobile_sources.zip_codes’ • ‘mobile_sources.browsers.iphone_4g’ Wednesday, April 28, 2010
  94. 94. Let’s write some Ruby Wednesday, April 28, 2010
  95. 95. Let’s write some Ruby class SearchStats cattr_accessor :db, :collection self.collection = $mongo_db.collection ‘searches‘ def self.record_search(city, state, category_id, vendor_id) collection.update({:vendor_id => vendor_id, :city => city, :state => state, :category_id => category_id, :date => Time.now.utc.beginning_of_day.to_i}, {'$inc' => {:occurrence => 1}}, {:upsert => true}) end end Wednesday, April 28, 2010
  96. 96. Let’s write some Ruby class SearchStats cattr_accessor :db, :collection self.collection = $mongo_db.collection ‘searches‘ def self.record_search(city, state, category_id, vendor_id) collection.update({:vendor_id => vendor_id, :city => city, :state => state, :category_id => category_id, :date => Time.now.utc.beginning_of_day.to_i}, {'$inc' => {:occurrence => 1}}, {:upsert => true}) end end irb> SearchStats.record_search(‘Boston’, ‘MA’, 9, 42) Wednesday, April 28, 2010
  97. 97. Let’s write some Ruby class SearchStats cattr_accessor :db, :collection self.collection = $mongo_db.collection ‘searches‘ def self.record_search(city, state, category_id, vendor_id) collection.update({:vendor_id => vendor_id, :city => city, :state => state, :category_id => category_id, :date => Time.now.utc.beginning_of_day.to_i}, {'$inc' => {:occurrence => 1}}, {:upsert => true}) end end irb> SearchStats.record_search(‘Boston’, ‘MA’, 9, 42) mongo> db.searches.findOne() { "city" : "Boston", "date" : 1272326400, "impression" : 1, "category_id" : 9, "state" : "MA", "vendor_id" : 42 } Wednesday, April 28, 2010
  98. 98. TESTING Wednesday, April 28, 2010
  99. 99. TESTING • Pretty much the same as anything else Wednesday, April 28, 2010
  100. 100. TESTING • Pretty much the same as anything else • test/unit/search_stats_test.rb Wednesday, April 28, 2010
  101. 101. TESTING • Pretty much the same as anything else • test/unit/search_stats_test.rb • config/environments/test.rb Wednesday, April 28, 2010
  102. 102. TESTING • Pretty much the same as anything else • test/unit/search_stats_test.rb • config/environments/test.rb • $mongo_db = Mongo::Connection.new.db ‘mongo-sf-db’ Wednesday, April 28, 2010
  103. 103. TESTING • Pretty much the same as anything else • test/unit/search_stats_test.rb • config/environments/test.rb • $mongo_db = Mongo::Connection.new.db ‘mongo-sf-db’ • SearchStatsTest#setup & #teardown Wednesday, April 28, 2010
  104. 104. TESTING • Pretty much the same as anything else • test/unit/search_stats_test.rb • config/environments/test.rb • $mongo_db = Mongo::Connection.new.db ‘mongo-sf-db’ • SearchStatsTest#setup & #teardown • No transactions Wednesday, April 28, 2010
  105. 105. TESTING • Pretty much the same as anything else • test/unit/search_stats_test.rb • config/environments/test.rb • $mongo_db = Mongo::Connection.new.db ‘mongo-sf-db’ • SearchStatsTest#setup & #teardown • No transactions • ActiveSupport spoils us Wednesday, April 28, 2010
  106. 106. TESTING • Pretty much the same as anything else • test/unit/search_stats_test.rb • config/environments/test.rb • $mongo_db = Mongo::Connection.new.db ‘mongo-sf-db’ • SearchStatsTest#setup & #teardown • No transactions • ActiveSupport spoils us • Gotta clean up after yourself Wednesday, April 28, 2010
  107. 107. Other developers gotta use it too Wednesday, April 28, 2010
  108. 108. Other developers gotta use it too • Rails’ database.yml takes care of MySQL Wednesday, April 28, 2010
  109. 109. Other developers gotta use it too • Rails’ database.yml takes care of MySQL • Gotta switch up MongoDBs on your own Wednesday, April 28, 2010
  110. 110. Other developers gotta use it too • Rails’ database.yml takes care of MySQL • Gotta switch up MongoDBs on your own • Simple config system file inspired by another Nunemaker post. Wednesday, April 28, 2010
  111. 111. Other developers gotta use it too • Rails’ database.yml takes care of MySQL • Gotta switch up MongoDBs on your own • Simple config system file inspired by another Nunemaker post. • No database migrations! Wednesday, April 28, 2010
  112. 112. Other developers gotta use it too • Rails’ database.yml takes care of MySQL • Gotta switch up MongoDBs on your own • Simple config system file inspired by another Nunemaker post. • No database migrations! • Must write rake tasks to create indexes unless your ORM takes care of it for you Wednesday, April 28, 2010
  113. 113. Making production ready Wednesday, April 28, 2010
  114. 114. Making production ready • Replication was iffy in 1.0.0 (solid now) Wednesday, April 28, 2010
  115. 115. Making production ready • Replication was iffy in 1.0.0 (solid now) • Nightly dumps w/ mongoexport Wednesday, April 28, 2010
  116. 116. Making production ready • Replication was iffy in 1.0.0 (solid now) • Nightly dumps w/ mongoexport • Upload to S3 Wednesday, April 28, 2010
  117. 117. Making production ready • Replication was iffy in 1.0.0 (solid now) • Nightly dumps w/ mongoexport • Upload to S3 • Monitor process w/ monit Wednesday, April 28, 2010
  118. 118. Deploy Wednesday, April 28, 2010
  119. 119. Deploy • Original deployment uneventful Wednesday, April 28, 2010
  120. 120. Deploy • Original deployment uneventful • Just worked Wednesday, April 28, 2010
  121. 121. Deploy • Original deployment uneventful • Just worked • Future deployments that required a database upgrade required downtime (bummer) Wednesday, April 28, 2010
  122. 122. Deploy • Original deployment uneventful • Just worked • Future deployments that required a database upgrade required downtime (bummer) • We wanted to play with the geospatial search Wednesday, April 28, 2010
  123. 123. Day 200 Wednesday, April 28, 2010
  124. 124. Day 200 • One production process Wednesday, April 28, 2010
  125. 125. Day 200 • One production process • One database Wednesday, April 28, 2010
  126. 126. Day 200 • One production process • One database • Daily dumps using mongoexport + upload to S3 Wednesday, April 28, 2010
  127. 127. Day 200 • One production process • One database • Daily dumps using mongoexport + upload to S3 • Replication not running Wednesday, April 28, 2010
  128. 128. Day 200 • One production process • One database • Daily dumps using mongoexport + upload to S3 • Replication not running • Several collections Wednesday, April 28, 2010
  129. 129. Day 200 • One production process • One database • Daily dumps using mongoexport + upload to S3 • Replication not running • Several collections • 100k to >10M documents Wednesday, April 28, 2010
  130. 130. Day 200 • One production process • One database • Daily dumps using mongoexport + upload to S3 • Replication not running • Several collections • 100k to >10M documents • Scores of deep, composite indexes Wednesday, April 28, 2010
  131. 131. What tripped me up? Wednesday, April 28, 2010
  132. 132. Simple Test should "get impressions_by_date" do record1 = {:date => 100, :impression => 10} record2 = {:date => 90, :impression => 8} Mongo::Vendor.collection.insert record1 Mongo::Vendor.collection.insert record2 assert_same_elements [record1, record2], Mongo::Vendor.impressions_by_date end Wednesday, April 28, 2010
  133. 133. Crazy Failure 1) Failure: test: Mongo::Vendor should get impressions_by_date. (Mongo::VendorTest) ... -<{{"date"=>100.0, "csum"=>10.0}=>1, {"date"=>90.0, "csum"=>8.0}=>1}> +<{{"date"=>90.0, "csum"=>8.0}=>1, {"date"=>100.0, "csum"=>10.0}=>1}> Wednesday, April 28, 2010
  134. 134. Wednesday, April 28, 2010
  135. 135. Mongo’s BSON OrderedHash doesn’t behave like ActiveSupport::OrderedHash Wednesday, April 28, 2010
  136. 136. Mongo’s BSON OrderedHash doesn’t behave like ActiveSupport::OrderedHash irb(main):001:0> require 'active_support' irb(main):002:0> include ActiveSupport irb(main):003:0> oh = OrderedHash.new irb(main):004:0> oh[:b] = 2 irb(main):005:0> oh[:c] = 3 irb(main):006:0> oh[:a] = 1 irb(main):007:0> a = {:a => 1, :b => 2, :c => 3} irb(main):008:0> a == oh => true irb(main):009:0> oh == a => true Wednesday, April 28, 2010
  137. 137. Mongo’s BSON OrderedHash doesn’t behave like ActiveSupport::OrderedHash irb(main):001:0> require 'active_support' irb(main):001:0> require 'mongo' irb(main):002:0> include ActiveSupport irb(main):002:0> oh = OrderedHash.new irb(main):003:0> oh = OrderedHash.new irb(main):003:0> oh[:b] = 2 irb(main):004:0> oh[:b] = 2 irb(main):004:0> oh[:c] = 3 irb(main):005:0> oh[:c] = 3 irb(main):005:0> oh[:a] = 1 irb(main):006:0> oh[:a] = 1 irb(main):006:0> a = {:a => 1, :b => 2, :c irb(main):007:0> a = {:a => 1, :b => 2, :c => 3} => 3} irb(main):008:0> a == oh irb(main):007:0> a == oh => true => true irb(main):009:0> oh == a => true Wednesday, April 28, 2010
  138. 138. Mongo’s BSON OrderedHash doesn’t behave like ActiveSupport::OrderedHash irb(main):001:0> require 'active_support' irb(main):001:0> require 'mongo' irb(main):002:0> include ActiveSupport irb(main):002:0> oh = OrderedHash.new irb(main):003:0> oh = OrderedHash.new irb(main):003:0> oh[:b] = 2 irb(main):004:0> oh[:b] = 2 irb(main):004:0> oh[:c] = 3 irb(main):005:0> oh[:c] = 3 irb(main):005:0> oh[:a] = 1 irb(main):006:0> oh[:a] = 1 irb(main):006:0> a = {:a => 1, :b => 2, :c irb(main):007:0> a = {:a => 1, :b => 2, :c => 3} => 3} irb(main):008:0> a == oh irb(main):007:0> a == oh => true => true irb(main):009:0> oh == a irb(main):008:0> oh == a => true => false Wednesday, April 28, 2010
  139. 139. Wednesday, April 28, 2010
  140. 140. And it turns out... Wednesday, April 28, 2010
  141. 141. And it turns out... MongoDB will not add indexes to your collections. Wednesday, April 28, 2010
  142. 142. And it turns out... MongoDB will not add indexes to your collections. Unless you tell it to. Wednesday, April 28, 2010
  143. 143. add indexes add indexes add indexes add indexes add indexes add indexes add indexes add indexes add indexes add indexes add indexes add indexes add indexes Wednesday, April 28, 2010
  144. 144. But it’ll help pick up the pieces Before index > db.vendors.findOne({'date': {'$gt': 1271895200}}) { ... } Sun Apr 25 01:19:51 query v_production.vendors ntoreturn:1 reslen:114 nscanned:~10M { date: { $gt: 1271895200.0 } } nreturned:1 3492ms During index > db.vendors.ensureIndex({'date': 1}) Sun Apr 25 01:20:17 building new index on { date: 1.0 } for v_production.vendors Sun Apr 25 01:20:17 Buildindex v_production.vendors idxNo:2 { _id: ObjId(4bd398d1dbf523027368d566), ns: "v_production.vendors", key: { date: 1.0 }, name: "date_1" } ! ! ... Sun Apr 25 01:21:35 ! done building bottom layer, going to commit Sun Apr 25 01:21:35 done for ~10M records 78.56secs Sun Apr 25 01:21:35 insert v_production.system.indexes 78560ms After Index > db.vendors.find({'date': 1272153600}).explain() { ! ... ! "nscanned" : 11220, ! "millis" : 12, ! ... } Wednesday, April 28, 2010
  145. 145. But it’ll help pick up the pieces Before index > db.vendors.findOne({'date': {'$gt': 1271895200}}) { ... } Sun Apr 25 01:19:51 query v_production.vendors ntoreturn:1 reslen:114 nscanned:~10M { date: { $gt: 1271895200.0 } } nreturned:1 3492ms Full collection scan ~3.5s During index > db.vendors.ensureIndex({'date': 1}) Sun Apr 25 01:20:17 building new index on { date: 1.0 } for v_production.vendors Sun Apr 25 01:20:17 Buildindex v_production.vendors idxNo:2 { _id: ObjId(4bd398d1dbf523027368d566), ns: "v_production.vendors", key: { date: 1.0 }, name: "date_1" } ! ! ... Sun Apr 25 01:21:35 ! done building bottom layer, going to commit Sun Apr 25 01:21:35 done for ~10M records 78.56secs Sun Apr 25 01:21:35 insert v_production.system.indexes 78560ms After Index > db.vendors.find({'date': 1272153600}).explain() { ! ... ! "nscanned" : 11220, ! "millis" : 12, ! ... } Wednesday, April 28, 2010
  146. 146. But it’ll help pick up the pieces Before index > db.vendors.findOne({'date': {'$gt': 1271895200}}) { ... } Sun Apr 25 01:19:51 query v_production.vendors ntoreturn:1 reslen:114 nscanned:~10M { date: { $gt: 1271895200.0 } } nreturned:1 3492ms Full collection scan ~3.5s During index > db.vendors.ensureIndex({'date': 1}) Sun Apr 25 01:20:17 building new index on { date: 1.0 } for v_production.vendors Sun Apr 25 01:20:17 Buildindex v_production.vendors idxNo:2 { _id: ObjId(4bd398d1dbf523027368d566), ns: "v_production.vendors", key: { date: 1.0 }, name: "date_1" } ! ! ... Sun Apr 25 01:21:35 ! done building bottom layer, going to commit Sun Apr 25 01:21:35 done for ~10M records 78.56secs Sun Apr 25 01:21:35 insert v_production.system.indexes 78560ms Index added ~79s No locks. After Index > db.vendors.find({'date': 1272153600}).explain() { ! ... ! "nscanned" : 11220, ! "millis" : 12, ! ... } Wednesday, April 28, 2010
  147. 147. But it’ll help pick up the pieces Before index > db.vendors.findOne({'date': {'$gt': 1271895200}}) { ... } Sun Apr 25 01:19:51 query v_production.vendors ntoreturn:1 reslen:114 nscanned:~10M { date: { $gt: 1271895200.0 } } nreturned:1 3492ms Full collection scan ~3.5s During index > db.vendors.ensureIndex({'date': 1}) Sun Apr 25 01:20:17 building new index on { date: 1.0 } for v_production.vendors Sun Apr 25 01:20:17 Buildindex v_production.vendors idxNo:2 { _id: ObjId(4bd398d1dbf523027368d566), ns: "v_production.vendors", key: { date: 1.0 }, name: "date_1" } ! ! ... Sun Apr 25 01:21:35 ! done building bottom layer, going to commit Sun Apr 25 01:21:35 done for ~10M records 78.56secs Sun Apr 25 01:21:35 insert v_production.system.indexes 78560ms Index added ~79s No locks. After Index > db.vendors.find({'date': 1272153600}).explain() { ! ... ! "nscanned" : 11220, ! "millis" : 12, 12ms! ! ... } Wednesday, April 28, 2010
  148. 148. But it’ll help pick up the pieces Before index > db.vendors.findOne({'date': {'$gt': 1271895200}}) { ... } Sun Apr 25 01:19:51 query v_production.vendors ntoreturn:1 reslen:114 nscanned:~10M { date: { $gt: 1271895200.0 } } nreturned:1 3492ms Full collection scan ~3.5s During index > db.vendors.ensureIndex({'date': 1}) Sun Apr 25 01:20:17 building new index on { date: 1.0 } for v_production.vendors Sun Apr 25 01:20:17 Buildindex v_production.vendors idxNo:2 { _id: ObjId(4bd398d1dbf523027368d566), ns: "v_production.vendors", key: { date: 1.0 }, name: "date_1" } ! ! ... Sun Apr 25 01:21:35 ! done building bottom layer, going to commit Sun Apr 25 01:21:35 done for ~10M records 78.56secs Sun Apr 25 01:21:35 insert v_production.system.indexes 78560ms Index added ~79s No locks. After Index > db.vendors.find({'date': 1272153600}).explain() During index creation, system completely { ! ... CRUD responsive. No impact to ! ! "nscanned" : 11220, "millis" : 12, 12ms! MyPunchbowl.com ! ... } Wednesday, April 28, 2010
  149. 149. Up next: Top Secret Project Wednesday, April 28, 2010
  150. 150. Up next: Top Secret Project • Project involving Wednesday, April 28, 2010
  151. 151. Up next: Top Secret Project • Project involving • SCREAMS for a document-based solution Wednesday, April 28, 2010
  152. 152. Up next: Top Secret Project • Project involving • SCREAMS for a document-based solution • mongosphinx • MongoDB’s ad-hoc full text search ok, lacks infix matching and bulk index building • Geo-spatial search not yet ellipsoidal (or even spherical) Wednesday, April 28, 2010
  153. 153. Up next: Top Secret Project • Project involving • SCREAMS for a document-based solution • mongosphinx • MongoDB’s ad-hoc full text search ok, lacks infix matching and bulk index building • Geo-spatial search not yet ellipsoidal (or even spherical) • paperclip+mongomapper Wednesday, April 28, 2010
  154. 154. Up next: Top Secret Project • Project involving • SCREAMS for a document-based solution • mongosphinx • MongoDB’s ad-hoc full text search ok, lacks infix matching and bulk index building • Geo-spatial search not yet ellipsoidal (or even spherical) • paperclip+mongomapper • MyPunchbowl user/event/invite data mining/visualization Wednesday, April 28, 2010
  155. 155. Up next: Top Secret Project • Project involving • SCREAMS for a document-based solution • mongosphinx • MongoDB’s ad-hoc full text search ok, lacks infix matching and bulk index building • Geo-spatial search not yet ellipsoidal (or even spherical) • paperclip+mongomapper • MyPunchbowl user/event/invite data mining/visualization • Charding + MapReduce = Nerdstorm Wednesday, April 28, 2010
  156. 156. Up next: Top Secret Project • Project involving • SCREAMS for a document-based solution • mongosphinx • MongoDB’s ad-hoc full text search ok, lacks infix matching and bulk index building • Geo-spatial search not yet ellipsoidal (or even spherical) • paperclip+mongomapper • MyPunchbowl user/event/invite data mining/visualization • Charding + MapReduce = Nerdstorm • Building tools to speed up ETL out of MySQL and into MongoDB Wednesday, April 28, 2010
  157. 157. The end Questions? Wednesday, April 28, 2010
  158. 158. Contact Me ryan@angilly.com http://www.mypunchbowl.com http://ryanangilly.com @angilly Wednesday, April 28, 2010
  159. 159. Links Slide 3/4 logos, wget’d from all over the Internets. Dunce Cap http://steynian.files.wordpress.com/2009/09/dunce-cap.jpg http://railstips.org/blog/archives/2009/12/18/why-i-think-mongo-is-to-databases-what-rails-was-to-frameworks/ http://railstips.org/blog/archives/2009/11/10/config-so-simple-your-mama-could-use-it/ Vendors configuration http://gist.github.com/380213 Wednesday, April 28, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×