<ul>Practical MongoDB and Drupal </ul><ul>Doug Green (douggreen) <li>Karoly Negyesi (chx)
#drupalchi, #drupalchimongo for questions
http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal </li></ul>
<ul>Design for Speed and Scalability </ul>
<ul>Examiner.com </ul>“ I work at Examiner.com, a  Quantified top 100 North American website, probably the highest traffic...
<ul>Examiner.com </ul>“ ... our daily traffic has increased and our monthly availability has gone from 99% uptime to 99.97...
Design for Speed and Scalability <ul><li>Understand databases, queries, and indexes, and know the pros and cons and tradeo...
Never call a web page from your web page
CSS and JS aggregation
CDN's </li></ul>
<ul><li>Change your frame </li></ul>RDBMS – traditional SQL solutions NoSQL solutions … but let's change the name ... Docu...
<ul><li>What do we call this? </li></ul>Document-oriented Database ? DODB ? Distributed Key-Value Pair Graph Database ? We...
<ul><li>RDBMS verses DODBMS </li></ul><ul>Bleeding Edge? </ul><ul>Leading Edge? </ul><ul>or </ul>
<ul>RDBMS verses DODBMS </ul><ul>or </ul><ul>Disk is cheap !? </ul>
<ul>RDBMS verses DODBMS </ul><ul>SELECT * FROM foo; </ul><ul>db.foo.find(); </ul><ul>A query is just a query in a differen...
<ul>MongoDB is DODBMS </ul><ul>Open Source?  Community? Support? </ul>
<ul>DODBMS built for 2010 not 1980 </ul><ul>3 computer years? </ul><ul>25 computer years? </ul>
<ul>DODBMS+MongoDB Summary </ul><ul>Pros </ul><ul>Cons </ul><ul><ul><li>Fast, document structure is perfect for Drupal ent...
Takes best advantage of 2010 Operating Systems that manage memory well
Takes best advantage of 2010 Hardware the recognizes disk and memory is cheap
Open source code can be read and modified (40-64 index example) </li></ul></ul><ul><ul><li>Bleeding edge, software is new ...
Lack of GUI based tools
Small user community and new language that must be learned
10gen support is excellent, free and paid (but as with all companies they are seeking ways of funding their efforts, so th...
<ul>MySQL to MongoDB primer </ul><ul>SELECT * FROM foo WHERE bar = 1; </ul><ul>db.foo.find({bar:1}); </ul><ul>This is the ...
<ul>MySQL to MongoDB primer </ul><ul>SELECT some FROM foo WHERE bar = 1; </ul><ul>db.foo.find({bar:1}, {some:1}); </ul>
<ul>MySQL to MongoDB primer </ul><ul>INSERT INTO foo (bar) VALUES (1); </ul><ul>db.foo.insert({bar:1}); </ul><ul>Inserts a...
<ul>MySQL to MongoDB primer </ul><ul>CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1); </ul><ul>db.foo....
<ul>MySQL to MongoDB primer </ul><ul>CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1); </ul><ul>db.foo....
<ul>MySQL to MongoDB primer </ul><ul>db.foo.insert({bar:1}); db.foo.find(); { “_id”: ObjectId(“4d...”), “bar” : 1 } </ul><...
<ul>MySQL to MongoDB primer </ul><ul>db.foo.insert({bar:1, _id:123456}); db.foo.find(); { “id”: 123456 “bar” : 1 } </ul><u...
<ul>MySQL to MongoDB primer </ul><ul>INSERT foo set some  = 1 WHERE bar = 2; </ul><ul>db.foo.update({bar:2}, {some:2}); db...
<ul>MySQL to MongoDB primer </ul><ul>db.foo.update(  {_id:2},   {$set:{bar:1}},  true); </ul><ul>But then there's upserts!...
<ul>Drupal 7 Enhancements </ul><ul>DBTNG </ul><ul>Field API </ul><ul>But, we use Drupal, not the database command prompt, ...
<ul>Drupal 7 Enhancements </ul><ul>DBTNG </ul><ul>Field API </ul><ul>FieldAPI is perfect for MongoDB! </ul><ul>…  well, no...
<ul>Drupal Modules for Mongo </ul><ul><ul><li>Fields - no table joins
Blocks – conditions in DB instead of code
Upcoming SlideShare
Loading in...5
×

DrupalCon Chicago Practical MongoDB and Drupal

7,636

Published on

DrupalCon Chicago 2011 talk by douggreen and chx on MongoDB and Drupal, see also http://chicago2011.drupal.org/sessions/practical-mongodb-and-drupal

Published in: Technology
2 Comments
13 Likes
Statistics
Notes
No Downloads
Views
Total Views
7,636
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
124
Comments
2
Likes
13
Embeds 0
No embeds

No notes for slide

DrupalCon Chicago Practical MongoDB and Drupal

  1. 1. <ul>Practical MongoDB and Drupal </ul><ul>Doug Green (douggreen) <li>Karoly Negyesi (chx)
  2. 2. #drupalchi, #drupalchimongo for questions
  3. 3. http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal </li></ul>
  4. 4. <ul>Design for Speed and Scalability </ul>
  5. 5. <ul>Examiner.com </ul>“ I work at Examiner.com, a Quantified top 100 North American website, probably the highest traffic Drupal 7 website in the world. Most minutes on most days we deliver customized non-cached content to over a thousand people, every page built to the individual based on location or preference. In the time it takes you to count the number of people near you, we'll probably touch more people. “ … well maybe not in this room today, but still you'll have to count fast!
  6. 6. <ul>Examiner.com </ul>“ ... our daily traffic has increased and our monthly availability has gone from 99% uptime to 99.97% uptime – hours of downtime are now measured in minutes. Just today we hit another record day, breaking the record from just yesterday, breaking the record from last week. That's all significant!“ - Doug Green, Drupal Watchdog, March 2011
  7. 7. Design for Speed and Scalability <ul><li>Understand databases, queries, and indexes, and know the pros and cons and tradeoffs </li></ul><ul><li>Cache, memcache entities, Drupal render cache, mongodb block cache
  8. 8. Never call a web page from your web page
  9. 9. CSS and JS aggregation
  10. 10. CDN's </li></ul>
  11. 11. <ul><li>Change your frame </li></ul>RDBMS – traditional SQL solutions NoSQL solutions … but let's change the name ... Document Object DB NO sql
  12. 12. <ul><li>What do we call this? </li></ul>Document-oriented Database ? DODB ? Distributed Key-Value Pair Graph Database ? We really can't agree, I think NoSQL is the wrong Frame, let's just call this a “Document-oriented Database Management System” or DODBMS
  13. 13. <ul><li>RDBMS verses DODBMS </li></ul><ul>Bleeding Edge? </ul><ul>Leading Edge? </ul><ul>or </ul>
  14. 14. <ul>RDBMS verses DODBMS </ul><ul>or </ul><ul>Disk is cheap !? </ul>
  15. 15. <ul>RDBMS verses DODBMS </ul><ul>SELECT * FROM foo; </ul><ul>db.foo.find(); </ul><ul>A query is just a query in a different language </ul><ul>Parle vous BSON? </ul>
  16. 16. <ul>MongoDB is DODBMS </ul><ul>Open Source? Community? Support? </ul>
  17. 17. <ul>DODBMS built for 2010 not 1980 </ul><ul>3 computer years? </ul><ul>25 computer years? </ul>
  18. 18. <ul>DODBMS+MongoDB Summary </ul><ul>Pros </ul><ul>Cons </ul><ul><ul><li>Fast, document structure is perfect for Drupal entities
  19. 19. Takes best advantage of 2010 Operating Systems that manage memory well
  20. 20. Takes best advantage of 2010 Hardware the recognizes disk and memory is cheap
  21. 21. Open source code can be read and modified (40-64 index example) </li></ul></ul><ul><ul><li>Bleeding edge, software is new and not as well battle tested
  22. 22. Lack of GUI based tools
  23. 23. Small user community and new language that must be learned
  24. 24. 10gen support is excellent, free and paid (but as with all companies they are seeking ways of funding their efforts, so this is changing) </li></ul></ul>
  25. 25. <ul>MySQL to MongoDB primer </ul><ul>SELECT * FROM foo WHERE bar = 1; </ul><ul>db.foo.find({bar:1}); </ul><ul>This is the session on “Practical MongoDB” so let's do a few more... </ul>
  26. 26. <ul>MySQL to MongoDB primer </ul><ul>SELECT some FROM foo WHERE bar = 1; </ul><ul>db.foo.find({bar:1}, {some:1}); </ul>
  27. 27. <ul>MySQL to MongoDB primer </ul><ul>INSERT INTO foo (bar) VALUES (1); </ul><ul>db.foo.insert({bar:1}); </ul><ul>Inserts are simple </ul>
  28. 28. <ul>MySQL to MongoDB primer </ul><ul>CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1); </ul><ul>db.foo.insert({bar:1}); </ul><ul>Inserts are simple, but to be fair... </ul>
  29. 29. <ul>MySQL to MongoDB primer </ul><ul>CREATE TABLE (foo) (bar int (11)); INSERT INTO foo (bar) VALUES (1); </ul><ul>db.foo.insert({bar:1}); db.foo.insert({bar:”1”}); </ul><ul>Inserts are simple, but be more fair... </ul>
  30. 30. <ul>MySQL to MongoDB primer </ul><ul>db.foo.insert({bar:1}); db.foo.find(); { “_id”: ObjectId(“4d...”), “bar” : 1 } </ul><ul>Inserts are simple, but let's be accurate, mongo auto-creates unique keys </ul>
  31. 31. <ul>MySQL to MongoDB primer </ul><ul>db.foo.insert({bar:1, _id:123456}); db.foo.find(); { “id”: 123456 “bar” : 1 } </ul><ul>Or you can add your own unique key </ul>
  32. 32. <ul>MySQL to MongoDB primer </ul><ul>INSERT foo set some = 1 WHERE bar = 2; </ul><ul>db.foo.update({bar:2}, {some:2}); db.foo.update({bar:2}, {$set:{some:2}}); </ul><ul>Updates aren't that hard either </ul><ul>or </ul>
  33. 33. <ul>MySQL to MongoDB primer </ul><ul>db.foo.update( {_id:2}, {$set:{bar:1}}, true); </ul><ul>But then there's upserts! </ul>
  34. 34. <ul>Drupal 7 Enhancements </ul><ul>DBTNG </ul><ul>Field API </ul><ul>But, we use Drupal, not the database command prompt, so... </ul>chx
  35. 35. <ul>Drupal 7 Enhancements </ul><ul>DBTNG </ul><ul>Field API </ul><ul>FieldAPI is perfect for MongoDB! </ul><ul>… well, not yet </ul>
  36. 36. <ul>Drupal Modules for Mongo </ul><ul><ul><li>Fields - no table joins
  37. 37. Blocks – conditions in DB instead of code
  38. 38. Queues – mostly to avoid another daemon and it's fast enough
  39. 39. Watchdog – capped collections, never run out of space, counts and groups by id
  40. 40. Cache – not used on Examiner.com
  41. 41. Session – perfect fit for D7 </li></ul></ul><ul>http://drupal.org/project/mongodb </ul>
  42. 42. <ul>MongoDB Watchdog </ul><ul>t() replaceable arguments i.e., “page not found: %q” </ul>
  43. 43. <ul>MongoDB Watchdog Details </ul><ul>Arguments filled in on details </ul>
  44. 44. <ul>Drupal Field API </ul><ul>$query = new EntityFieldQuery; $results ->entityCondition('entity_type', 'node') ->propertyCondition('uid', (int) $account->uid) ->fieldCondition('bar', 'value', 1) ->propertyCondition('status', 1) ->execute(); $nodes = empty($results['node']) ? array() : node_load_multiple(array_keys($result['node'])); </ul>
  45. 45. <ul>Php http://php.net/mongo </ul><ul>$query = array( 'uid' => (int) $account->uid, 'bar.value' => 1, 'status' => 1 ); $fields = array( '_id' => TRUE, ); $results = mongodb_collection('fields_current', 'node') ->find($query, $fields); foreach ($results as $result) { } </ul>
  46. 46. <ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>Always remember the tree </ul><ul>You can only traverse one branch at a time </ul>
  47. 47. <ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>1. The sort column must be the last column used in the index. </ul><ul><li>find({a:1}).sort({a:1}); find({a:1}).sort({b:1}) find({a:1,b:2}).sort({c:1}); </li></ul><ul><li>find({a:1}).sort({c:1}); </li></ul><ul>Good </ul><ul>Bad </ul><ul>Assume an index on a, b, c on all examples to follow </ul>
  48. 48. <ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>2. The range query must also be the last column in an index, this is an corollary of 1 above. </ul><ul><li>find({a:1, b:{$gt:2}); find({a:{$gt:1, $lt:10}}) find({a:{$gt:1}).sort({a:1}); </li></ul><ul><li>find({a:{$gt:1}, b:2}); </li></ul><ul>Good </ul><ul>Bad </ul>
  49. 49. <ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>3. Only use a range query or a sort on one column. </ul><ul><li>find(a=1,b=2).sort(c) find(a=1,b>2) find(a=1,b>2,b<4) find(a=1,b>2).sort(b) </li></ul><ul><li>find(a>1,b>2) find(a=1,b>2).sort(c) find(a=1,b in [1, 2]).sort(c) </li></ul><ul>Good </ul><ul>Bad </ul><ul>Note, I'm switching to psuedo code shorthand, not BSON syntax </ul>
  50. 50. <ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>Indexes Hints </ul><ul>4. Conserve indexes by re-ordering columns used in straight = queries If you have the following two queries, then you only need a single index a, b, d, c find(a=1, b=1, d=1) find(a=1, b=1, c=1, d=1) </ul>
  51. 51. <ul>Summary </ul><ul><ul><li>Examiner.com is mostly happy with MongoDB solution.
  52. 52. We miss relational nature of MySQL with a GUI query reporting tool, all reports have to be programmed.
  53. 53. Site uptime is much higher than before, but still have some instability related to an immature product
  54. 54. Drupal 7, Field API, DBTNG, MongoDB, and Community are awesome </li></ul></ul>
  55. 55. <ul>Future </ul><ul><ul><li>DBTNG, will require hooks for modules to provide translations of relational SQL to Document object store, we could provide this for core, but each contrib module would then also need to provide this … not ideal, but it is a solution
  56. 56. I think MongoDB is a perfect fit for search, and I'm the right guy to do it, but every time I mention this, people say, but we have SOLR, why would you do this? </li></ul></ul>
  57. 57. <ul>What did you think? </ul><ul>Locate this session on the DCC website: </ul><ul>http://chicago2011.drupal.org/sessions <li>http://www.slideshare.net/douggreen/drupalcon-chicago-practical-mongodb-and-drupal </li></ul><ul><li>Click the “Take the Survey” link. </li></ul><ul>Thanks! </ul>
  1. ¿Le ha llamado la atención una diapositiva en particular?

    Recortar diapositivas es una manera útil de recopilar información importante para consultarla más tarde.

×