• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Mongo-Drupal
 

Mongo-Drupal

on

  • 11,639 views

Slides from a talk I gave at MongoNYC on using MongoDB with Drupal. I will most likely be doing this as a webcast and giving this presentation at Drupalcamp NYC 8 this July.

Slides from a talk I gave at MongoNYC on using MongoDB with Drupal. I will most likely be doing this as a webcast and giving this presentation at Drupalcamp NYC 8 this July.

Statistics

Views

Total Views
11,639
Views on SlideShare
11,396
Embed Views
243

Actions

Likes
18
Downloads
204
Comments
1

22 Embeds 243

http://tunephp.wordpress.com 103
http://www.slideshare.net 47
http://tunephp.blogspot.com 40
http://lanyrd.com 17
http://tunephp.blogspot.it 6
http://tunephp.blogspot.be 4
http://tunephp.blogspot.jp 3
http://tunephp.blogspot.de 3
http://tunephp.blogspot.co.il 3
http://tunephp.blogspot.co.uk 3
http://tunephp.blogspot.fr 2
http://tunephp.blogspot.in 2
http://tunephp.blogspot.com.au 1
http://localhost 1
http://lanyrd.dev 1
http://tunephp.blogspot.hk 1
http://213.8.145.174:82 1
http://tunephp.blogspot.tw 1
http://tunephp.blogspot.ca 1
http://tunephp.blogspot.gr 1
http://tunephp.blogspot.co.at 1
http://tunephp.blogspot.kr 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • not config? help me config?
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • However Schema-based RDBMS tend to be the worst offenders; SQL in particular has a number of issues: > MySQL / InnoDB has a very unfriendly method of doing schema changes: it rebuilds the table and blocks all writes until the new table is ready. > Big price – transactions. Worked so hard to get transactions in, wait, why are we working hard? > Rich feature set string functions you've never used > More abstractly, reality, or at least the reality of the Interwebs doesn't play nice with SQL as the former doesn't have a rigid schema that the later wants to enforce. A thousand blog posts bear witness to the list of reasons that essentially boil down to the big price you have to pay to support features of a system that reality doesn't map well to reality. And this is of course is one of the most important things these systems/machines/programs do: map reality. And while it may seem to be a fairly high level of abstraction to compare database models to reality ('itself') there is in fact a crucial dialogue in database programming between abstraction and actuality that is exemplified by what my signature quote In Theory, ...
  • I have seen the future and it is Mongo(DB)

Mongo-Drupal Mongo-Drupal Presentation Transcript

  • Drupal + Mongo: Bigger is Better?
    • by Forest Mars
  • Drupal (FTW!) Drupal: Aspect-oriented (modular) “social-publishing” framework, written in php (pdo) that allows easy creation and integration of multiple data-rich social networking sites and robust web applications & services. Used on large high-performance websites. Roadmap anticipates future web (rdf, ggg) & emerging technologies (MongoDB!)
  • The Problem with SQL* Webapps > Once you have related data, you need joins > Indexing on joins does not work that well ...So you: > Introduce denormalisation > Build extra tables *(RDBMS)
  • The Problem with Schema > Changes broadly lock data, requiring downtime or putting excessive, short-term load on the system. > The data integrity constraints don’t quite support application integrity constraints. For example, there’s no standard way in SQL to require a column to contain only valid URLs. > Coordinating schema changes with application code changes is difficult, largely because schema changes lack convenient coupling with code.
  • “ In theory, theory in practice are exactly the same. In practice, they're completely different.”
    • Practice
    • WiP (works in practice)
    • Scales well
    • Future-proof
    Theory Clean abstractions Strong semantics Smart-proof
  • A C I D / B A S E
    • Base
    • Basically Available
    • Scales well
    • Eventually consistant
    Acid Atomic Consistent Isolated Durable
  • ( Why Mongo ?) Highly Available Easily Scalable Partition Tolerant
  • ( Why Mongo ?) Tableless Queriless Schemaless Blazingly fast Faster Development times Nicer learning curves Code is trimmer Future-proof
  • Performance / Scaling Whitehouse.gov / direct engagement 15K/day contact requests 2M records in db 4GB db: replication risks MongoDB: 180M+ documents in 1 collection
  • Writing Performant Queries Sort column must be the last column used in the index. Range query must also be the last column in an index, Only use a range query or sort on one column. Conserve indexes by re-ordering columns used in straight = queries Never use Mongo's $ne or $nin operator's Never use Mongo's $exists operator http://jira.mongodb.org/browse/WEBSITE-12
  • Install Mongo public array authenticate (string $username, string $password ) public array command ( array $data ) __construct ( Mongo $conn , string $name ) public MongoCollection createCollection ( string $name [, bool $capped = FALSE [, int $size = 0 [, int $max = 0 ]]] ) public array createDBRef ( string $collection , mixed $a ) public array drop ( void ) public array dropCollection ( mixed $coll ) public array execute ( mixed $code [, array $args = array() ] ) public bool forceError ( void ) public MongoCollection __get ( string $name ) public array getDBRef ( array $ref ) public MongoGridFS getGridFS ([ string $prefix = "fs" ] ) public int getProfilingLevel ( void ) public array lastError ( void ) public array listCollections ( void ) public array prevError ( void ) public array repair ([ bool $preserve_cloned_files = FALSE [, bool $backup_original_files = FALSE ]] ) public array resetError ( void ) public MongoCollection selectCollection ( string $name ) public int setProfilingLevel ( int $level ) public string __toString ( void )
  • Real World Example list the nodes of a user ordered by comment count uid is stored in the node table and the comment count is in node_comment_statistics > thus query cannot be indexed (Comparison of dissimilar columns may prevent use of indexes if values cannot be compared directly without conversion.)
  • What's already in Drupal * mongodb: support library for the other modules (D7/D6) * mongodb_block: Store block information in mongodb. Very close to the core block API. * mongodb_cache: Store cache items in mongodb. * mongodb_session: Store sessions in mongodb. * mongodb_watchdog: Store watchdog messages in mongodb * mongodb_queue: DrupalQueueInterface implementation using mongodb. * mongodb_field_storage: Store the fields in mongodb.
  • Mongo Watchdog
  • mongodb_watchdog mongo> db.watchdog.drop(); mongo> db.createCollection("watchdog", {capped:true, size:1000000, max:10000} );
  • mongodb_cache $conf['page_cache_without_database'] = TRUE;
  • mongodb_sessions $conf['session_inc'] = 'sites/all/modules/mongodb/ mongodb_session/mongodb_session.inc';
  • mongodb_sessions function mongodb_session_user_update($edit, $account) { if (!module_exists('mongodb_field_storage')) { $roles = _mongodb_session_get_roles($account); $save = (array) $account + array( '_id' => (int) $account->uid, '@bundle' => 'user', '@fields' => array(), 'roles' => $roles, ); foreach (array('uid', 'created', 'access', 'login', 'status', 'picture') as $key) { $save[$key] = (int) $save[$key]; } mongodb_collection('fields_current', 'user')->save($save); } return $roles; }
  • mongodb_sessions * The user-level session storage handlers: * - _drupal_session_open() * - _drupal_session_close() * - _drupal_session_read() * - _drupal_session_write() * - _drupal_session_destroy() * - _drupal_session_garbage_collection() assigned by session_set_save_handler() in bootstrap.inc
  • mongodb_block function hook_block_view_alter(&$data, $block) { // Remove the contextual links on all blocks that provide them. if (is_array($data['content']) && isset($data['content']['#contextual_links'])) { unset($data['content']['#contextual_links']); } // Add a theme wrapper function defined by the current module to all blocks // provided by the "somemodule" module. if (is_array($data['content']) && $block->module == 'somemodule') { $data['content']['#theme_wrappers'][] = 'mymodule_special_block'; } }
  • Block rebuild Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ).
  • Render main content block function mongodb_block_theme() { 'block' => array( 'render element' => 'elements', 'template' => 'block', 'path' => drupal_get_path('module', 'block'), ), } function mongodb_block_mongodb_block_info_alter(&$blocks) { // Enable the main content block. $blocks['system_main']['region'] = 'content'; $blocks['system_main']['weight'] = 0; $blocks['system_main']['status'] = 1; } function mongodb_block_rehash($redirect = FALSE) { $collection = mongodb_collection('block'); $theme = variable_get('theme_default', 'garland');
  • mongodb_field_storage don't : variable_set('field_storage_default', 'mongodb_field_storage'); instead : $conf['field_storage_default'] = 'mongodb_field_storage'; in settings.php ESP. for session/caching backends
  • Drupal 7 Everything In MongoDB* *(some restrictions may apply)
  • Import all Nodes > MongoDB* (* in 14 l.o.c.) // Connect $mongo = new Mongo(); // Get the database (it is created automatically) $db = $mongo->testDatabase; // Get the collection for nodes (it is created automatically) $collection = $db->nodes; // Get a listing of all of the node IDs $r = db_query('SELECT nid FROM {node}'); // Loop through all of the nodes... while($row = db_fetch_object($r)) { print "Writing node $row->nid "; // Load each node and convert it to an array. $node = (array)node_load($row->nid); // Store the node in MongoDB $collection->save($node); }
  • Import all Nodes > MongoDB* (* in 14 l.o.c.) # drush script mongoimport.php # use testDatabase; # db.nodes.find( {title: /about/i} , {title: true}).limit(4);
  • Import all Nodes > MongoDB* (* in 14 l.o.c.) <?php // Connect $mongo = new Mongo(); // Write our search filter (same as shell example above) $filter = array( 'title' => new MongoRegex('/about/i'), ); // Run the query, getting only 5 results. $res = $mongo->quiddity->nodes->find($filter)->limit(5); // Loop through and print the title of each article. foreach ($res as $row) { print $row['title'] . PHP_EOL; } ?>
  • What's Next? Multiple DB servers – Data Persistance Query logging - Devel support Query builder – Views integration DBTNG – Full DB Abstraction MongoDB API
  • Query Logging Extend Mongo collection class Pass instance back from mongodb_collection Implement all collection methods
  • Drupal Mongo API $collection = mongodb_collection('myname'); $collection->find(array('key' => $value)); $collection->insert($object); $collection->remove(array('_id' => $item->id));
  • Full DBTNG Impementation DO NOT USE !!!
  • awesomesauce page callback => 'drupal_json' $items['node/%node/json'] = array('page callback' => 'drupal_json', 'page arguments' => array(1), 'type' => MENU_CALLBACK);
  • Where Mongo Won't Work Joining across Entities ex. return birthday from profile belonging to author of current node
  • Thanks! Comments & questions to: ForestMars @gmail.com ForestMars @googlewave.com Facebook, LinkedIn, etc. twitter: @elvetica (identica@forest)