DrupalCon Chicago Practical MongoDB and Drupal
Upcoming SlideShare
Loading in...5
×
 

DrupalCon Chicago Practical MongoDB and Drupal

on

  • 4,957 views

DrupalCon Chicago 2011 talk by douggreen and chx on MongoDB and Drupal, see also http://chicago2011.drupal.org/sessions/practical-mongodb-and-drupal

DrupalCon Chicago 2011 talk by douggreen and chx on MongoDB and Drupal, see also http://chicago2011.drupal.org/sessions/practical-mongodb-and-drupal

Statistics

Views

Total Views
4,957
Views on SlideShare
4,953
Embed Views
4

Actions

Likes
13
Downloads
122
Comments
2

2 Embeds 4

http://www.techgig.com 3
https://si0.twimg.com 1

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    DrupalCon Chicago Practical MongoDB and Drupal DrupalCon Chicago Practical MongoDB and Drupal Presentation Transcript

      • Practical MongoDB and Drupal
        Doug Green (douggreen)
      • Karoly Negyesi (chx)
      • #drupalchi, #drupalchimongo for questions
      • http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal
      • Design for Speed and Scalability
      • Examiner.com
      “ I work at Examiner.com, a Quantified top 100 North American website, probably the highest traffic Drupal 7 website in the world. Most minutes on most days we deliver customized non-cached content to over a thousand people, every page built to the individual based on location or preference. In the time it takes you to count the number of people near you, we'll probably touch more people. “ … well maybe not in this room today, but still you'll have to count fast!
      • Examiner.com
      “ ... our daily traffic has increased and our monthly availability has gone from 99% uptime to 99.97% uptime – hours of downtime are now measured in minutes. Just today we hit another record day, breaking the record from just yesterday, breaking the record from last week. That's all significant!“ - Doug Green, Drupal Watchdog, March 2011
    • Design for Speed and Scalability
      • Understand databases, queries, and indexes, and know the pros and cons and tradeoffs
      • Cache, memcache entities, Drupal render cache, mongodb block cache
      • Never call a web page from your web page
      • CSS and JS aggregation
      • CDN's
      • Change your frame
      RDBMS – traditional SQL solutions NoSQL solutions … but let's change the name ... Document Object DB NO sql
      • What do we call this?
      Document-oriented Database ? DODB ? Distributed Key-Value Pair Graph Database ? We really can't agree, I think NoSQL is the wrong Frame, let's just call this a “Document-oriented Database Management System” or DODBMS
      • RDBMS verses DODBMS
        Bleeding Edge?
        Leading Edge?
        or
      • RDBMS verses DODBMS
        or
        Disk is cheap !?
      • RDBMS verses DODBMS
        SELECT * FROM foo;
        db.foo.find();
        A query is just a query in a different language
        Parle vous BSON?
      • MongoDB is DODBMS
        Open Source? Community? Support?
      • DODBMS built for 2010 not 1980
        3 computer years?
        25 computer years?
      • DODBMS+MongoDB Summary
        Pros
        Cons
        • Fast, document structure is perfect for Drupal entities
        • Takes best advantage of 2010 Operating Systems that manage memory well
        • Takes best advantage of 2010 Hardware the recognizes disk and memory is cheap
        • Open source code can be read and modified (40-64 index example)
        • Bleeding edge, software is new and not as well battle tested
        • Lack of GUI based tools
        • Small user community and new language that must be learned
        • 10gen support is excellent, free and paid (but as with all companies they are seeking ways of funding their efforts, so this is changing)
      • MySQL to MongoDB primer
        SELECT * FROM foo WHERE bar = 1;
        db.foo.find({bar:1});
        This is the session on “Practical MongoDB” so let's do a few more...
      • MySQL to MongoDB primer
        SELECT some FROM foo WHERE bar = 1;
        db.foo.find({bar:1}, {some:1});
      • MySQL to MongoDB primer
        INSERT INTO foo (bar) VALUES (1);
        db.foo.insert({bar:1});
        Inserts are simple
      • MySQL to MongoDB primer
        CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1);
        db.foo.insert({bar:1});
        Inserts are simple, but to be fair...
      • MySQL to MongoDB primer
        CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1);
        db.foo.insert({bar:1}); db.foo.insert({bar:”1”});
        Inserts are simple, but be more fair...
      • MySQL to MongoDB primer
        db.foo.insert({bar:1}); db.foo.find(); { “_id”: ObjectId(“4d...”), “bar” : 1 }
        Inserts are simple, but let's be accurate, mongo auto-creates unique keys
      • MySQL to MongoDB primer
        db.foo.insert({bar:1, _id:123456}); db.foo.find(); { “id”: 123456 “bar” : 1 }
        Or you can add your own unique key
      • MySQL to MongoDB primer
        INSERT foo set some = 1 WHERE bar = 2;
        db.foo.update({bar:2}, {some:2}); db.foo.update({bar:2}, {$set:{some:2}});
        Updates aren't that hard either
        or
      • MySQL to MongoDB primer
        db.foo.update( {_id:2}, {$set:{bar:1}}, true);
        But then there's upserts!
      • Drupal 7 Enhancements
        DBTNG
        Field API
        But, we use Drupal, not the database command prompt, so...
      chx
      • Drupal 7 Enhancements
        DBTNG
        Field API
        FieldAPI is perfect for MongoDB!
        … well, not yet
      • Drupal Modules for Mongo
        • Fields - no table joins
        • Blocks – conditions in DB instead of code
        • Queues – mostly to avoid another daemon and it's fast enough
        • Watchdog – capped collections, never run out of space, counts and groups by id
        • Cache – not used on Examiner.com
        • Session – perfect fit for D7
        http://drupal.org/project/mongodb
      • MongoDB Watchdog
        t() replaceable arguments i.e., “page not found: %q”
      • MongoDB Watchdog Details
        Arguments filled in on details
      • Drupal Field API
        $query = new EntityFieldQuery; $results ->entityCondition('entity_type', 'node') ->propertyCondition('uid', (int) $account->uid) ->fieldCondition('bar', 'value', 1) ->propertyCondition('status', 1) ->execute(); $nodes = empty($results['node']) ? array() : node_load_multiple(array_keys($result['node']));
      • Php http://php.net/mongo
        $query = array( 'uid' => (int) $account->uid, 'bar.value' => 1, 'status' => 1 ); $fields = array( '_id' => TRUE, ); $results = mongodb_collection('fields_current', 'node') ->find($query, $fields); foreach ($results as $result) { }
      • Indexes Hints
        Indexes Hints
        Indexes Hints
        Always remember the tree
        You can only traverse one branch at a time
      • Indexes Hints
        Indexes Hints
        Indexes Hints
        1. The sort column must be the last column used in the index.
      • find({a:1}).sort({a:1}); find({a:1}).sort({b:1}) find({a:1,b:2}).sort({c:1});
      • find({a:1}).sort({c:1});
        Good
        Bad
        Assume an index on a, b, c on all examples to follow
      • Indexes Hints
        Indexes Hints
        Indexes Hints
        2. The range query must also be the last column in an index, this is an corollary of 1 above.
      • find({a:1, b:{$gt:2}); find({a:{$gt:1, $lt:10}}) find({a:{$gt:1}).sort({a:1});
      • find({a:{$gt:1}, b:2});
        Good
        Bad
      • Indexes Hints
        Indexes Hints
        Indexes Hints
        3. Only use a range query or a sort on one column.
      • find(a=1,b=2).sort(c) find(a=1,b>2) find(a=1,b>2,b<4) find(a=1,b>2).sort(b)
      • find(a>1,b>2) find(a=1,b>2).sort(c) find(a=1,b in [1, 2]).sort(c)
        Good
        Bad
        Note, I'm switching to psuedo code shorthand, not BSON syntax
      • Indexes Hints
        Indexes Hints
        Indexes Hints
        4. Conserve indexes by re-ordering columns used in straight = queries If you have the following two queries, then you only need a single index a, b, d, c find(a=1, b=1, d=1) find(a=1, b=1, c=1, d=1)
      • Summary
        • Examiner.com is mostly happy with MongoDB solution.
        • We miss relational nature of MySQL with a GUI query reporting tool, all reports have to be programmed.
        • Site uptime is much higher than before, but still have some instability related to an immature product
        • Drupal 7, Field API, DBTNG, MongoDB, and Community are awesome
      • Future
        • DBTNG, will require hooks for modules to provide translations of relational SQL to Document object store, we could provide this for core, but each contrib module would then also need to provide this … not ideal, but it is a solution
        • I think MongoDB is a perfect fit for search, and I'm the right guy to do it, but every time I mention this, people say, but we have SOLR, why would you do this?
      • What did you think?
        Locate this session on the DCC website:
        http://chicago2011.drupal.org/sessions
      • http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal
      • Click the “Take the Survey” link.
        Thanks!