DrupalCon Chicago Practical MongoDB and Drupal

  • 7,438 views
Uploaded on

DrupalCon Chicago 2011 talk by douggreen and chx on MongoDB and Drupal, see also http://chicago2011.drupal.org/sessions/practical-mongodb-and-drupal

DrupalCon Chicago 2011 talk by douggreen and chx on MongoDB and Drupal, see also http://chicago2011.drupal.org/sessions/practical-mongodb-and-drupal

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
7,438
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
123
Comments
2
Likes
13

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1.
      Practical MongoDB and Drupal
      Doug Green (douggreen)
    • Karoly Negyesi (chx)
    • 2. #drupalchi, #drupalchimongo for questions
    • 3. http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal
  • 4.
      Design for Speed and Scalability
  • 5.
      Examiner.com
    “ I work at Examiner.com, a Quantified top 100 North American website, probably the highest traffic Drupal 7 website in the world. Most minutes on most days we deliver customized non-cached content to over a thousand people, every page built to the individual based on location or preference. In the time it takes you to count the number of people near you, we'll probably touch more people. “ … well maybe not in this room today, but still you'll have to count fast!
  • 6.
      Examiner.com
    “ ... our daily traffic has increased and our monthly availability has gone from 99% uptime to 99.97% uptime – hours of downtime are now measured in minutes. Just today we hit another record day, breaking the record from just yesterday, breaking the record from last week. That's all significant!“ - Doug Green, Drupal Watchdog, March 2011
  • 7. Design for Speed and Scalability
    • Understand databases, queries, and indexes, and know the pros and cons and tradeoffs
    • Cache, memcache entities, Drupal render cache, mongodb block cache
    • 8. Never call a web page from your web page
    • 9. CSS and JS aggregation
    • 10. CDN's
  • 11.
    • Change your frame
    RDBMS – traditional SQL solutions NoSQL solutions … but let's change the name ... Document Object DB NO sql
  • 12.
    • What do we call this?
    Document-oriented Database ? DODB ? Distributed Key-Value Pair Graph Database ? We really can't agree, I think NoSQL is the wrong Frame, let's just call this a “Document-oriented Database Management System” or DODBMS
  • 13.
    • RDBMS verses DODBMS
      Bleeding Edge?
      Leading Edge?
      or
  • 14.
      RDBMS verses DODBMS
      or
      Disk is cheap !?
  • 15.
      RDBMS verses DODBMS
      SELECT * FROM foo;
      db.foo.find();
      A query is just a query in a different language
      Parle vous BSON?
  • 16.
      MongoDB is DODBMS
      Open Source? Community? Support?
  • 17.
      DODBMS built for 2010 not 1980
      3 computer years?
      25 computer years?
  • 18.
      DODBMS+MongoDB Summary
      Pros
      Cons
      • Fast, document structure is perfect for Drupal entities
      • 19. Takes best advantage of 2010 Operating Systems that manage memory well
      • 20. Takes best advantage of 2010 Hardware the recognizes disk and memory is cheap
      • 21. Open source code can be read and modified (40-64 index example)
      • Bleeding edge, software is new and not as well battle tested
      • 22. Lack of GUI based tools
      • 23. Small user community and new language that must be learned
      • 24. 10gen support is excellent, free and paid (but as with all companies they are seeking ways of funding their efforts, so this is changing)
  • 25.
      MySQL to MongoDB primer
      SELECT * FROM foo WHERE bar = 1;
      db.foo.find({bar:1});
      This is the session on “Practical MongoDB” so let's do a few more...
  • 26.
      MySQL to MongoDB primer
      SELECT some FROM foo WHERE bar = 1;
      db.foo.find({bar:1}, {some:1});
  • 27.
      MySQL to MongoDB primer
      INSERT INTO foo (bar) VALUES (1);
      db.foo.insert({bar:1});
      Inserts are simple
  • 28.
      MySQL to MongoDB primer
      CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1);
      db.foo.insert({bar:1});
      Inserts are simple, but to be fair...
  • 29.
      MySQL to MongoDB primer
      CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1);
      db.foo.insert({bar:1}); db.foo.insert({bar:”1”});
      Inserts are simple, but be more fair...
  • 30.
      MySQL to MongoDB primer
      db.foo.insert({bar:1}); db.foo.find(); { “_id”: ObjectId(“4d...”), “bar” : 1 }
      Inserts are simple, but let's be accurate, mongo auto-creates unique keys
  • 31.
      MySQL to MongoDB primer
      db.foo.insert({bar:1, _id:123456}); db.foo.find(); { “id”: 123456 “bar” : 1 }
      Or you can add your own unique key
  • 32.
      MySQL to MongoDB primer
      INSERT foo set some = 1 WHERE bar = 2;
      db.foo.update({bar:2}, {some:2}); db.foo.update({bar:2}, {$set:{some:2}});
      Updates aren't that hard either
      or
  • 33.
      MySQL to MongoDB primer
      db.foo.update( {_id:2}, {$set:{bar:1}}, true);
      But then there's upserts!
  • 34.
      Drupal 7 Enhancements
      DBTNG
      Field API
      But, we use Drupal, not the database command prompt, so...
    chx
  • 35.
      Drupal 7 Enhancements
      DBTNG
      Field API
      FieldAPI is perfect for MongoDB!
      … well, not yet
  • 36.
      Drupal Modules for Mongo
      • Fields - no table joins
      • 37. Blocks – conditions in DB instead of code
      • 38. Queues – mostly to avoid another daemon and it's fast enough
      • 39. Watchdog – capped collections, never run out of space, counts and groups by id
      • 40. Cache – not used on Examiner.com
      • 41. Session – perfect fit for D7
      http://drupal.org/project/mongodb
  • 42.
      MongoDB Watchdog
      t() replaceable arguments i.e., “page not found: %q”
  • 43.
      MongoDB Watchdog Details
      Arguments filled in on details
  • 44.
      Drupal Field API
      $query = new EntityFieldQuery; $results ->entityCondition('entity_type', 'node') ->propertyCondition('uid', (int) $account->uid) ->fieldCondition('bar', 'value', 1) ->propertyCondition('status', 1) ->execute(); $nodes = empty($results['node']) ? array() : node_load_multiple(array_keys($result['node']));
  • 45.
      Php http://php.net/mongo
      $query = array( 'uid' => (int) $account->uid, 'bar.value' => 1, 'status' => 1 ); $fields = array( '_id' => TRUE, ); $results = mongodb_collection('fields_current', 'node') ->find($query, $fields); foreach ($results as $result) { }
  • 46.
      Indexes Hints
      Indexes Hints
      Indexes Hints
      Always remember the tree
      You can only traverse one branch at a time
  • 47.
      Indexes Hints
      Indexes Hints
      Indexes Hints
      1. The sort column must be the last column used in the index.
    • find({a:1}).sort({a:1}); find({a:1}).sort({b:1}) find({a:1,b:2}).sort({c:1});
    • find({a:1}).sort({c:1});
      Good
      Bad
      Assume an index on a, b, c on all examples to follow
  • 48.
      Indexes Hints
      Indexes Hints
      Indexes Hints
      2. The range query must also be the last column in an index, this is an corollary of 1 above.
    • find({a:1, b:{$gt:2}); find({a:{$gt:1, $lt:10}}) find({a:{$gt:1}).sort({a:1});
    • find({a:{$gt:1}, b:2});
      Good
      Bad
  • 49.
      Indexes Hints
      Indexes Hints
      Indexes Hints
      3. Only use a range query or a sort on one column.
    • find(a=1,b=2).sort(c) find(a=1,b>2) find(a=1,b>2,b<4) find(a=1,b>2).sort(b)
    • find(a>1,b>2) find(a=1,b>2).sort(c) find(a=1,b in [1, 2]).sort(c)
      Good
      Bad
      Note, I'm switching to psuedo code shorthand, not BSON syntax
  • 50.
      Indexes Hints
      Indexes Hints
      Indexes Hints
      4. Conserve indexes by re-ordering columns used in straight = queries If you have the following two queries, then you only need a single index a, b, d, c find(a=1, b=1, d=1) find(a=1, b=1, c=1, d=1)
  • 51.
      Summary
      • Examiner.com is mostly happy with MongoDB solution.
      • 52. We miss relational nature of MySQL with a GUI query reporting tool, all reports have to be programmed.
      • 53. Site uptime is much higher than before, but still have some instability related to an immature product
      • 54. Drupal 7, Field API, DBTNG, MongoDB, and Community are awesome
  • 55.
      Future
      • DBTNG, will require hooks for modules to provide translations of relational SQL to Document object store, we could provide this for core, but each contrib module would then also need to provide this … not ideal, but it is a solution
      • 56. I think MongoDB is a perfect fit for search, and I'm the right guy to do it, but every time I mention this, people say, but we have SOLR, why would you do this?
  • 57.
      What did you think?
      Locate this session on the DCC website:
      http://chicago2011.drupal.org/sessions
    • http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal
    • Click the “Take the Survey” link.
      Thanks!