Migrating data into Drupal using the migrate module


Published on

Migrating data into Drupal using the migrate module presentation at Oxford Drupal Camp, 22 June 2012

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Migrating data into Drupal using the migrate module

  1. 1. Migrating data into Drupalusing the migrate module Oxford DrupalCamp Friday 22 June, 2012 Johan Gant
  2. 2. Whats involved? Migrate me an army of data! Uh, oh!!
  3. 3. Whats involved? Planning/analysisTech/dev work Acceptance Done!
  4. 4. Successful migrationsMove data from A to B in a planned, methodical manner.RepeatableMeasurableWork with realistic dataSafe Use the right tools for the task The complexity of your solution should be driven by the task at hand, not your tech
  5. 5. When stuff goes wrong...Un... Safe Reliable Planned DoneUpset... Processes Clients Developers RAGEBALLS FOR EVERYONE
  6. 6. Drupal migrate moduleProvides a great framework for moving content into DrupalEncourages good habitsDrush supportmigrate_ui adds a nice front endThe Drupal way – so other Drupal devs canpick up your work without too much bother.Not a one stop shop for migrationsUse the right tools, or combination oftools, to get the job done. Lets look at some code
  7. 7. <?phpclass DemoMigration extends Migration { public function __construct() { parent::construct(); $this->description = t(A sample migration module); // Define the map between your source and destination $this->map = new MigrateSQLMap( $this->machineName, // defaults to your migration class name array( id => array( type => int, .... ), http://drupal.org/node/1528934 ), MigrateDestinationNode::getKeySchema(); ); // Define the fields from your source $source_fields = array( id => t(description), field1 => t(Node title), ..... ); $query = db_select(source_table_name, sql-alias) ->fields(alias, array_keys($source_fields)) ->orderBy(id, ASC); $this->source = new MigrateSourceSQL($query); // Define what sort of destination you have // - options include Nodes, Terms, Users and Comments $this->destination = new MigrateDestinationNode(node_machine_name); // Define your field mappings - nice options include default values (can include PHP), // groupings, callbacks – see beer.inc example class for details $this->addFieldMapping(source_body, field_body) ->defaultValue(Wibble), ..... }}
  8. 8. /** * Useful function to help massage your awkward source data into shape */public function prepareRow($row) { // Convert ISO date to UNIX timestamp if ($row->created_at) { $row->created_at = strtotime($row->created_at); }}Handling taxonomy terms/** * Term syntax/format a bit awkward,depends on whether youre using tid or name. * Migrate module will match on tid or name and handle the rest for you :) */$this->addFieldMapping(source_col_name, terms) ->arguments(array(source_type => name)), // use tid if youve got term ids ->separator($$); // Used to split long string of term names in sourceHandling node reference fields/** * Not documented when I used migrate, but expects a nid */if ($row->some-identifier) { $row->reference_nid = my_own_function_to_lookup_a_nid($row->some_identifier);}
  9. 9. Migration!Rails/PostgreSQL > Drupal 6Export data into CSV, import into Drupal db via tablewizard modulePre-migration script to create image nodesRun migrationClient demo and deployment5 days from start to finish
  10. 10. Problems! Steep(ish) learning curve Documentation a bit sparse in places Awkward handling of taxonomy, node reference, and domain data Pre-migration fudges Performance
  11. 11. Should I use Drupal migrate?Yes / absolutely / do it now Probably not● Want to move a reasonable ● Low volume or low complexity volume of data INTO Drupal from MySQL/XML/JSON/CSV ● Not familiar with coding● Have complex data mappings that ● Already got something that works are best expressed well programmatically ● Trying to move data OUT of Drupal● Make best use of Drupal tools and to somewhere else existing code● Want to recycle code between projects
  12. 12. Drupal migrate - resources● Migrate module page - http://drupal.org/project/migrate● Economist migration (great summary) - http://drupal.org/node/915102● Torchbox Team Drupal blog - http://drupalblog.torchbox.com/