Mongo-Drupal

Drupal + Mongo: Bigger is Better?
by Forest Mars

Drupal (FTW!) Drupal: Aspect-oriented (modular) “social-publishing” framework, written in php (pdo) that allows easy creation and integration of multiple data-rich social networking sites and robust web applications & services. Used on large high-performance websites. Roadmap anticipates future web (rdf, ggg) & emerging technologies (MongoDB!)

The Problem with SQL* Webapps > Once you have related data, you need joins > Indexing on joins does not work that well ...So you: > Introduce denormalisation > Build extra tables *(RDBMS)

The Problem with Schema > Changes broadly lock data, requiring downtime or putting excessive, short-term load on the system. > The data integrity constraints don’t quite support application integrity constraints. For example, there’s no standard way in SQL to require a column to contain only valid URLs. > Coordinating schema changes with application code changes is difficult, largely because schema changes lack convenient coupling with code.

“ In theory, theory in practice are exactly the same. In practice, they're completely different.”
Practice
WiP (works in practice)
Scales well
Future-proof
Theory Clean abstractions Strong semantics Smart-proof

A C I D / B A S E
Base
Basically Available
Scales well
Eventually consistant
Acid Atomic Consistent Isolated Durable

( Why Mongo ?) Highly Available Easily Scalable Partition Tolerant

( Why Mongo ?) Tableless Queriless Schemaless Blazingly fast Faster Development times Nicer learning curves Code is trimmer Future-proof

Performance / Scaling Whitehouse.gov / direct engagement 15K/day contact requests 2M records in db 4GB db: replication risks MongoDB: 180M+ documents in 1 collection

Writing Performant Queries Sort column must be the last column used in the index. Range query must also be the last column in an index, Only use a range query or sort on one column. Conserve indexes by re-ordering columns used in straight = queries Never use Mongo's $ne or $nin operator's Never use Mongo's $exists operator http://jira.mongodb.org/browse/WEBSITE-12

Install Mongo public array authenticate (string $username, string $password ) public array command ( array $data ) __construct ( Mongo $conn , string $name ) public MongoCollection createCollection ( string $name [, bool $capped = FALSE [, int $size = 0 [, int $max = 0 ]]] ) public array createDBRef ( string $collection , mixed $a ) public array drop ( void ) public array dropCollection ( mixed $coll ) public array execute ( mixed $code [, array $args = array() ] ) public bool forceError ( void ) public MongoCollection __get ( string $name ) public array getDBRef ( array $ref ) public MongoGridFS getGridFS ([ string $prefix = "fs" ] ) public int getProfilingLevel ( void ) public array lastError ( void ) public array listCollections ( void ) public array prevError ( void ) public array repair ([ bool $preserve_cloned_files = FALSE [, bool $backup_original_files = FALSE ]] ) public array resetError ( void ) public MongoCollection selectCollection ( string $name ) public int setProfilingLevel ( int $level ) public string __toString ( void )

Real World Example list the nodes of a user ordered by comment count uid is stored in the node table and the comment count is in node_comment_statistics > thus query cannot be indexed (Comparison of dissimilar columns may prevent use of indexes if values cannot be compared directly without conversion.)

What's already in Drupal * mongodb: support library for the other modules (D7/D6) * mongodb_block: Store block information in mongodb. Very close to the core block API. * mongodb_cache: Store cache items in mongodb. * mongodb_session: Store sessions in mongodb. * mongodb_watchdog: Store watchdog messages in mongodb * mongodb_queue: DrupalQueueInterface implementation using mongodb. * mongodb_field_storage: Store the fields in mongodb.

Mongo Watchdog

mongodb_watchdog mongo> db.watchdog.drop(); mongo> db.createCollection("watchdog", {capped:true, size:1000000, max:10000} );

mongodb_cache $conf['page_cache_without_database'] = TRUE;

mongodb_sessions $conf['session_inc'] = 'sites/all/modules/mongodb/ mongodb_session/mongodb_session.inc';

mongodb_sessions function mongodb_session_user_update($edit, $account) { if (!module_exists('mongodb_field_storage')) { $roles = _mongodb_session_get_roles($account); $save = (array) $account + array( '_id' => (int) $account->uid, '@bundle' => 'user', '@fields' => array(), 'roles' => $roles, ); foreach (array('uid', 'created', 'access', 'login', 'status', 'picture') as $key) { $save[$key] = (int) $save[$key]; } mongodb_collection('fields_current', 'user')->save($save); } return $roles; }

mongodb_sessions * The user-level session storage handlers: * - _drupal_session_open() * - _drupal_session_close() * - _drupal_session_read() * - _drupal_session_write() * - _drupal_session_destroy() * - _drupal_session_garbage_collection() assigned by session_set_save_handler() in bootstrap.inc

mongodb_block function hook_block_view_alter(&$data, $block) { // Remove the contextual links on all blocks that provide them. if (is_array($data['content']) && isset($data['content']['#contextual_links'])) { unset($data['content']['#contextual_links']); } // Add a theme wrapper function defined by the current module to all blocks // provided by the "somemodule" module. if (is_array($data['content']) && $block->module == 'somemodule') { $data['content']['#theme_wrappers'][] = 'mymodule_special_block'; } }

Block rebuild Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ). Notice : Undefined variable: block_html_id in include() (line 4 of /var/www/Drupal/drupal-7.0-alpha4/themes/garland/block.tpl.php ).

Render main content block function mongodb_block_theme() { 'block' => array( 'render element' => 'elements', 'template' => 'block', 'path' => drupal_get_path('module', 'block'), ), } function mongodb_block_mongodb_block_info_alter(&$blocks) { // Enable the main content block. $blocks['system_main']['region'] = 'content'; $blocks['system_main']['weight'] = 0; $blocks['system_main']['status'] = 1; } function mongodb_block_rehash($redirect = FALSE) { $collection = mongodb_collection('block'); $theme = variable_get('theme_default', 'garland');

mongodb_field_storage don't : variable_set('field_storage_default', 'mongodb_field_storage'); instead : $conf['field_storage_default'] = 'mongodb_field_storage'; in settings.php ESP. for session/caching backends

Drupal 7 Everything In MongoDB* *(some restrictions may apply)

Import all Nodes > MongoDB* (* in 14 l.o.c.) // Connect $mongo = new Mongo(); // Get the database (it is created automatically) $db = $mongo->testDatabase; // Get the collection for nodes (it is created automatically) $collection = $db->nodes; // Get a listing of all of the node IDs $r = db_query('SELECT nid FROM {node}'); // Loop through all of the nodes... while($row = db_fetch_object($r)) { print "Writing node $row->nid "; // Load each node and convert it to an array. $node = (array)node_load($row->nid); // Store the node in MongoDB $collection->save($node); }

Import all Nodes > MongoDB* (* in 14 l.o.c.) # drush script mongoimport.php # use testDatabase; # db.nodes.find( {title: /about/i} , {title: true}).limit(4);

Import all Nodes > MongoDB* (* in 14 l.o.c.) <?php // Connect $mongo = new Mongo(); // Write our search filter (same as shell example above) $filter = array( 'title' => new MongoRegex('/about/i'), ); // Run the query, getting only 5 results. $res = $mongo->quiddity->nodes->find($filter)->limit(5); // Loop through and print the title of each article. foreach ($res as $row) { print $row['title'] . PHP_EOL; } ?>

What's Next? Multiple DB servers – Data Persistance Query logging - Devel support Query builder – Views integration DBTNG – Full DB Abstraction MongoDB API

Query Logging Extend Mongo collection class Pass instance back from mongodb_collection Implement all collection methods

Drupal Mongo API $collection = mongodb_collection('myname'); $collection->find(array('key' => $value)); $collection->insert($object); $collection->remove(array('_id' => $item->id));

Full DBTNG Impementation DO NOT USE !!!

awesomesauce page callback => 'drupal_json' $items['node/%node/json'] = array('page callback' => 'drupal_json', 'page arguments' => array(1), 'type' => MENU_CALLBACK);

Where Mongo Won't Work Joining across Entities ex. return birthday from profile belonging to author of current node

Thanks! Comments & questions to: ForestMars @gmail.com ForestMars @googlewave.com Facebook, LinkedIn, etc. twitter: @elvetica (identica@forest)

http://tunephp.wordpress.com	83
http://www.slideshare.net	47
http://tunephp.blogspot.com	25
http://lanyrd.com	17
http://tunephp.blogspot.co.il	2
http://tunephp.blogspot.co.uk	2
http://tunephp.blogspot.ca	1
http://tunephp.blogspot.fr	1
http://tunephp.blogspot.in	1
http://tunephp.blogspot.be	1
http://tunephp.blogspot.gr	1
http://tunephp.blogspot.de	1
http://lanyrd.dev	1
http://tunephp.blogspot.tw	1
http://213.8.145.174:82	1
http://tunephp.blogspot.co.at	1
http://tunephp.blogspot.it	1

Mongo-Drupal

by Forest Mars

Accessibility

Upload Details

Usage Rights

17 Embeds 187

Statistics

Mongo-Drupal Presentation Transcript

Mongo-Drupal

by Forest Mars

Accessibility

Upload Details

Usage Rights

17 Embeds 187

Statistics

Mongo-Drupal Presentation Transcript

Let LinkedIn power your SlideShare experience

Let LinkedIn power your SlideShare experience

Customize SlideShare content based on your interests

Keep up to date when your LinkedIn contacts post on SlideShare