• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Drupal content-migration
 

Drupal content-migration

on

  • 5,919 views

Learn how to use the migrate module to migrate content from various sources into Drupal.

Learn how to use the migrate module to migrate content from various sources into Drupal.

Statistics

Views

Total Views
5,919
Views on SlideShare
5,787
Embed Views
132

Actions

Likes
1
Downloads
41
Comments
1

6 Embeds 132

http://btmash.com 72
http://stuartrunyan.tumblr.com 27
http://stuartrunyan.com 22
http://mswebmatrix.com 6
https://twitter.com 3
http://nextleveldrupal.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Hi! Thanks for this presentation, but for me as a blogger it's hard to comprehend. I've found a video by CMS2CMS which describes a way of migration to Drupal with no coding: http://www.youtube.com/watch?v=cCkW-vwzlEg

    Hope it helps someone.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Drupal content-migration Drupal content-migration Presentation Transcript

    • Ashok Modi (BTMash) – Drupal LA – March 2011
      Migrating content to drupal – Migrate Module
    • Agenda
      Different Methods
      Steps to Work with Migrate
      Hooks
      Build a Class
      Description
      Source Adding
      Mapping
      Additional Data
      drush commands
      Q & A
    • Disclaimer
      Code Heavy!
      You may fall asleep.
      New territory!
      Talk about one *small* aspect of migration (migrating nodes)
      Not talking about creating own DestinationHandlers
      Possibly not talking about creating own FieldHandlers (depends on time)
      Can walk through an example migration that I did if preferred.
      Ask questions?
      It will make the presentation less terrible 
    • Possible method – by hand
      Should be accurate
      Get all files
      Mapping / everything works
      Time consuming
      Not feasible if you have a lot of content.
      Good way to test interns / punish coworkers (?)
    • Possible methods – Node Export
      Node Export (http://drupal.org/project/node_export)
      Has 7.x branch
      But no way to update content from 6.x -> 7.x
      No way to go back
      *easy* to set up (requires exact setup between source and destination in field names, etc)
    • Possible Methods - Feeds
      Really good method
      Map fields from source to destination
      Can import RSS / Atom / Various types of feeds
      Also flat files such as CSV
      Well documented
      Other than flat files, requires a feed source
      Might run into issues if content is updated in source
      *might be tricky in another cms*
    • Method demonstrated - Migrate
      Already defined many possible import sources
      XML, JSON, CSV, Databases (any currently supported by Drupal!)
      Can import many different types of content
      Users, Nodes, Comments, Taxonomy, Files … all core entities
      Can define your own import handler (not covered in presentation)
      Can define own method for importing custom fields
      Current already defined for all core field types
      Has support for Media Module importing
      Patch underway for getting date import
      Can define your own field handler (possibly not covered in presentation)
      Drush integration
      Rollback, Status Updates, Limited import.
      Caveat – Confusing documentation
      Only a status UI – all mapping must be done in code.
    • Assumptions made for presentation
      Migrating from a database
      Files directory for source are on same machine as the destination site directory
    • Steps to work with Migrate
      Let Migrate know about your module (1 hook!)
      Build a Migration Class
      Give it a description
      Let Migrate know what you’re getting content from.
      Let Migrate know about the type of content.
      Map the fields the migrate class it going to fill.
      (Optional) Massage / Add any fields you couldn’t get in the initial mapping (query).
    • Step 1: Hook
      Implement one hook – hook_migrate_api
      Provide the api version number (currently at version 2)
      That’s it!
      function mymodule_migrate_api() {
      return array(
      ‘api’ => 2,
      );
      }
    • Step 2: Build a Class
      Implement classes
      Class defines type of content that will be imported
      class NodeContentTypeMigration extends Migration {
      public function __construct() {
      parent::__construct();

      }
      public function prepareRow($current_row) {

      }
      }
    • Step 2: Build a Class (functions inside)
      public function __construct() {…}
      Constructor for the class
      Allows migrate to know content type (user, node, tags)
      Where content is mapped from (db, csv, xml, etc)
      All the mappings coming in (fields)
      (optional)public function prepareRow($current_row) {…}
      Any extra data (things that cannot be pulled in a single query(?))
      Massage any of the data that was pulled in (clean up text, clean up links, etc)
    • Step 2a: Create a description
      Create a description
      Class Description
      Any additional source fields (not found in initial query)
      Initial source -> destination mapping (what is the key in the source db?)
      $this->description = t(“Import all nodes of type PAGE”);
      Define Source Fields
      Fields that may not be getting pulled in via your query or in the flat file data but will be getting migrated somehow
      $source_fields = array(
      'nid' => t('The node ID of the page'),
      ’my_files' => t(’The set of files in a field for this node'),
      );
    • Off course: query source database
      Set up query (if need be, switch DBs using Database::getConnection)
      $query = Database::getConnection('for_migration', 'default');
      Then write out rest of the query
      Alternatively, if source db is on same machine as destination db, use mysql db shortcut
      db_select(MY_MIGRATION_DATABASE_NAME .’.table_name’, ‘t’)
    • Step 2b: Call to grab data
      NOTE: This is only for migrations from databases
      Set up query (if need be, switch DBs using Database::getConnection)
      $query = db_select(MY_MIGRATION_DATABASE_NAME .'.node', 'n’)
      ->fields('n', array('nid', 'vid', 'type', 'language', 'title', 'uid', 'status', 'created', 'changed', 'comment', 'promote', 'moderate', 'sticky', 'tnid', 'translate'))
      ->condition('n.type', 'page', '=');
      $query->join(MY_MIGRATION_DATABASE_NAME .'.node_revisions', 'nr', 'n.vid = nr.vid');
      $query->addField('nr', 'body');
      $query->addField('nr', 'teaser');
      $query->join(MY_MIGRATION_DATABASE_NAME .'.users', 'u', 'n.uid = u.uid');
      $query->addField('u', 'name');
      $query->orderBy('n.changed');
    • Step 2b: Why the orderby?
      Migrate module has a feature called ‘highwater’
      It is a key to designate and figure out if a piece of content needs to be updated rather than inserted.
      Means content can be updated!
      $this->highwaterField = array(
      'name' => 'changed',
      'alias' => 'n’,
      );
    • Step 2c: Mappings
      Add a ‘mapping’ (this is for tracking relationships between the rows from the source db and the rows that will come in the destination site) – essentially key of source DB.
      $this->map = new MigrateSQLMap(
      $this->machineName,
      array(
      'nid' => array(
      'type' => 'int’,
      'unsigned' => TRUE,
      'not null' => TRUE,
      'description' => 'D6 Unique Node ID’,
      'alias' => 'n',
      )
      ),
      MigrateDestinationNode::getKeySchema());
    • Step 2c: Mappings (cont’d)
      Now let the migrate module know what kind of mapping is being performed.
      $this->source = new MigrateSourceSQL($query, $source_fields);
      Along with the type of content
      $this->destination = new MigrateDestinationNode('page');
      .
    • Step 2d: Map Fields
      Usually follows the form
      $this->addFieldMapping(‘destination_field_name’, ‘source_field_name’);
      $this->addFieldMapping('revision_uid', 'uid');
      Can provide default values
      $this->addFieldMapping('pathauto_perform_alias')->defaultValue('1');
      Can provide no value
      $this->addFieldMapping('path')->issueGroup(t('DNM'));
      Can provide arguments and separators for certain field types (body, file, etc require this methodology)
    • Step 3: Additional data / cleanup
      Optional
      public function prepareRow($current_row)
      Use it to add any additional data / cleanup any fields that were mapped in.
      // Get the correct uid based on username & setauthor id for node touid
      $user_query = db_select('users', 'u’)
      ->fields('u', array('uid'))
      ->condition('u.name', $username, '=');
      $results = $user_query->execute();
      foreach ($results as $row) {
      $current_row->uid = $current_row->revision_uid = $row->uid;
      break;
      }
    • DrushCommands (the important ones)
      drush ms – List various migration import classes
      drush mi <importclass> - Import content
      drushmr <importclass> - Rollback content
      Options
      --idlist=id1,id2,… - Import content with specific IDs
      --itemlimit=n – Only import up to ‘n’ items
      --feedback=“n seconds” – Show status report every ‘n’ seconds
      --feedback=“n items” – Show status report every ‘n’ items
    • Resources
      http://drupal.org/project/migrate
      http://drupal.org/node/415260
      Look at the example modules
      http://drupal.org/project/migrate_extras
      http://drupal.org/project/wordpress_migrate
      http://cyrve.com/import (drush documentation)
      http://goo.gl/3e1Jm (additional documentation to be added to core project documentation)
      http://goo.gl/2qDLh (another example module)
    • Questions?
      Thank you 