Talks about Migration framework in Drupal. How to migrate from a legacy DB to a new Drupal instance, how to run it form command-line using drush.

1. Migrating Sites to DRUPAL
Piyuesh kumar
QED42

2. AGENDA
√ Old school migration strategies.
√ Migrate Module
√ Additional features of Migrate
√ Walk through migration from rails 3 to
Drupal7
√ Q&A

3. Old School Migration Strategies
√ Writing Migration(php) scripts.
√ Accuracy required.
√ Time consuming.
√ Not good when migrating large content.
√ x_export (node_export, user_export et al):
√ Easy to setup but requires exact mapping
√ No way to rollback

4. Migrate Module
√ Rollback functionality provides for iterative
development.
√ Simple and easy to understand migration
classes.
√ Drush support.
√ Extremely elaborate UI showing all the statistics.
√ Support for importing custom fields data.

5. How To Migrate
√ Identify you module to drupal using
“hook_migrate_api”.
√ Extend migration classes as per the requirement.

6. Example
Function mymodule_migrate_api {
return array(
‘api’ => 2
);
}

7. Extending Migration Class
√ Public function __construct()
√ Constructor for the migrate class.
√ Tells migrate about entity_type(node,taxonomy,users)
√ Source of the content(Databases, csv, xml)
√ Public function prepareRow()
√ Handles the post-processing of data imported.
√ Can also be used to add extra data to entities being
imported.

8. Example
class DemoMigration extends Migration {
public function __construct() {
/**
* Override the basic Migration class here.
*/
}
}

9. Example
class DemoMigration extends Migration {
public function prepareRow($current_row) {
/**
* Alter the data for the row being migrated here.
*/
}
}

10. Metadata required in constructor
√ Description : Text to tell what are we migrating.
√ Source_fields: Primary keys and any field that is not found in
the initial query.
√ Query: Query to fetch the data from legacy database.
√ Source: MigrateSourceSql instance to define the source of
migration.
√ Destination: MigrateDestination<entity_type> instance to
define the destination.
√ Map: Mapping to track the relationship between the source
rows from the source database and their resulting Drupal
objects.
√ Create one-on-one mapping between legacy db and drupal
objects (Using addFieldMapping).

11. Query
√ Query legacy database to fetch data to be
migrated.
√ Legacy db on Local
− Use db_select
√ Legacy db on remote server
− Database::getConnection();

12. MigrateSourceSql & Destination
√ MigrateSourceSql: Takes the query object
and source_fields as an argument and
defines the souce of data.
√ MigrateDestionation<entitytype>: Defines the
Destination drupal object type.

13. Example
√ $this->source = new
MigrateSourceSQL($query, $sourceFields);
√ $this->destination = new
MigrateDestinationNode('posts');

14. MigrateSqlMap
√ Responsible for creating mapping (source_id
+ destination_id).
√ Useful while creating relations between
imported drupal objects whose primary key
value has changed after migration.

15. Example
##User_legacy.inc
$this->map = new MigrateSQLMap($this->machineName,
array(
'uid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => t('User id from old site'),
'alias' => 'u',
)
),
MigrateDestinationUser::getKeySchema()
);
##Content.inc
$this->addFieldMapping('uid', 'uid')
->sourceMigration(‘User_legacy')

16. Mapping fields
√ AddFieldMapping : Takes the source field
name and dest field name as arguments.
√ Options:
√ DefaultValue
√ SourceMigration
√ DNM
√ Arguments and seperators.

17. Example
√ $this->addFieldMapping('uid', 'user_id')
->sourceMigration('user')
->defaultValue(1);
√ $this->addFieldMapping('revision_uid')
->issueGroup(t('DNM'));
√ $this->addFieldMapping('body', 'body')
->arguments(MigrateTextFieldHandler::arguments(NULL,
'wysiwyg'));

18. Database Architecture
√ Migrate_map_<classname> : Stores the
mapping data.
√ Migrate_message_<classname> : Stores the
warnings or errors which come over while
migration.
√ Migrate_status: Stores the status for all
migration classes.
√ Migrate_log: Logs the migrations ran.

19. Processing data
√ Pre-processing: function prepareRow().
√ Post-processing: function complete()
e.g., D6 uses md5 encryption while d7 uses
sha512 for password. Logic for handling this
while migrating should go in here.

20. Example
public function complete($entity, stdClass $row) {
$pass = 'U'.$entity->pass;
$uid = $entity->uid;
db_update('users')
->fields(array(
'pass' => $pass,
))
->condition('uid', $uid)
->execute();
}

21. Additional features of migrate v2
√ Classes for varied import sources – Xml, JSON,
CSV, Databases
√ Allows to define own field handlers.
√ Allows migration of different entities: node,
taxonomy,users.
√ Drush integration makes it just awesome.
√ Rollback allows iterative migration easier i.e.,
you don't have to migrate all at once, migrate
one item, test, fix and repeat.
√ Highwater Mark

22. Highwater Mark
√ Allows to re-import newly added/modified
content in the legacy db.
√ How??
√ Field marked as highwater inside migration class
is saved by migrate when we import the data first
time.
√ Overtime more content gets added/modified to
the legacy db (for which this fields value is larger).
√ Next time we run import, migrate alters the source
query to fetch only those content which have the
value larger than highwater value saved by
migrate.

23. How to choose highwater field?
√ It should change everytime the content gets
modified.
√ It should have a greater value for a new
content added(compared to all the previous
nodes).
√ In case of drupal this can be the “changed”
property of nodes.
√ Make sure to order the data from legacy db
using this highwater field, so that last row
processed has the highest highwater value.

24. Example
√ Use of orderby clause in query.
$query->orderby(p.modified');
√ Define highwater field.
$this->highwaterField = array(
'name' => ’modified',
'alias' => ‘p'
);

25. Drush Commands
√ Drush ms: lists all migration classes
√ Drush mi <classname>: Import content
√ Drush mr <classname>: rollback content
√ Options:
√ Idlist: allows migration of content with specific ids.
√ limit: allows to migrate n no of items.
√ Feedback = n seconds/items :status of migration
after n items or seconds.

26. Modules based on Migrate
√ Ubercart_migrate
√ Migrate_d2d
√ Wordpress_migrate
√ TYPO3_migrate
√ phpbb2drupal

27. Putting it all together
Demo

28. Sources
√ https://github.com/qed42/rdm-drupal
√ https://github.com/piyuesh23/joomla_drupal
√ https://github.com/qed42/rdm-rails

29. Thank You!!
Questions?