A quick overview of Migration API in Drupal 8, migration path, plugin system and options to extend

1. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Migrate in Drupal 8
Moldcamp 2015 Alexei Gorobets (asgorobets) 31.05.2015

2. Alexei
Gorobets
drupal.org: asgorobets
email: asgorobets@gmail.com

3. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
What about you?

4. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Agenda.Today
01 Upgrade path overview
02 Migrate in Core, what’s there
03 Migrate API
04 Demo

6. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Drupal upgrade.
In 4 easy steps:
1. Get latest version of your current Drupal core.
2. Through next major version on top of your current
3. Run upgrade.php
4. PRAY

7. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Drupal upgrade.
Pros:
- No coding required
- Easy to run
- Basic Drupal core coverage
Cons:
- No support for contrib modules, not even CCK
- Not customizable
- No way to jump versions
- Clutter in database and a pleathora of bugs

8. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Drupal Migrate.
In 4 painful steps:
1. Develop a completely new website with same functionality from scratch
2. Create migration classes for all your entity types and bundles
3. Implement missing migrate support for contrib modules and contribute it
back to the community
4. Run migrations and PRAY

9. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Drupal Migrate.
Pros:
- Flexible and extendable Migrate API
- Available solutions for contrib modules
- No clutter in database, as database is new
- You can jump major versions
- Basic support for simplest mappings from UI
Cons:
- Considerable amount of coding is usually required
- No way to migrate configurations
- No warranties and no core and community support

10. Migrate module in D8

11. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Migrate in D8.
Migrate was completely rewritten and was merged in Drupal 8 core.
Migrate modules:
>migrate (core)
>migrate_drupal (core)
>migrate_update (contrib)
>migrate_plus (Migrate UI formerly - contrib)

12. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Migrate in D8.
Drupal to Drupal migrations allow:
>Content migration with predefined mappings
>Configurations migration is supported, thanks to CMI (Content Types, User
profile, fields with field settings, widget settings, formatter settings)
>Excellent support for i18n, both nodes and UI translations
>Migration of every variable from D6
>~100 migrations defined

13. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Migrate in D8.
What is in Drupal core now:
>Migrate API and Drupal 8 destination plugins
>Drupal 6 migrate path finalized, couple of bugs left
>Drupal 7 migration path started
>Drupal 8 migration path planned for future releases
>No interface for rollbacks, and idlist yet
Not yet there:

14. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Migration API.
Basically Drupal’s implementation of Extract-Transform-Load (ETL)
Phases:
Extract > source
Transform > process
Load > destination

15. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Plugins everywhere.
Migration API works with plugins. Drupal 8 has a lot of plugins and a common
plugin system. Think of it like Ctools plugins in D7 but sexier.
Source plugins - extract data. From DB, YAML, JSON, whatever you need
Process plugins - pipeline for data massaging. Can have unlimited number of
process plugins chained
Destination plugins - saves data in Drupal. Be it an entity, config or URL
alias

16. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Definition.
Migrations are defined as config entities.
id: d6_system_site
label: Drupal 6 site configuration
migration_groups:
- Drupal 6
source:
plugin: variable
variables:
- site_name
- site_mail
- site_slogan
- site_frontpage
- site_403
- site_404
- drupal_weight_select_max
- admin_compact_mode
process:
name: site_name
mail: site_mail
slogan: site_slogan
'page/front': site_frontpage
'page/403': site_403
'page/404': site_404
weight_select_max: drupal_weight_select_max
admin_compact_mode: admin_compact_mode
destination:
plugin: config
config_name: system.site

17. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Process pipeline
process:
format:
-
plugin: machine_name
source: name
-
plugin: dedupe_entity
entity_type: filter_format
field: format
length: 32
name: name
cache: cache
…
filters:
plugin: iterator
source: filters
key: @id
process:
id:
plugin: static_map
default_value: filter_null
source:
- module
- delta
map:
filter:
- filter_html
- filter_autop
- filter_url
- filter_htmlcorrector
- filter_html_escape
php:
- php_code
settings: settings
status:
plugin: default_value
default_value: true

18. Running migrations

19. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
With Drush and manifest.yml
File manifest.yml
# nodes
- d6_node
- d6_node_revision
- d6_node_type
- d6_view_modes
- d6_filter_format
- d6_field_instance_per_form_display
- d6_field_instance_widget_settings
- d6_field_formatter_settings
- d6_field_instance
- d6_field
- d6_field_settings
- d6_node_settings
…
- d6_cck_field_values:*
- d6_cck_field_revision:*
- d6_term_node_revision
- d6_term_node
- d6_vocabulary_entity_display
- d6_vocabulary_entity_form_display
- d6_vocabulary_field_instance
- d6_vocabulary_field
- d6_user
- d6_user_role
- d6_taxonomy_vocabulary
> drush migrate-manifest manifest.yml --legacy-db-url=mysql://user:password@host/dbname

20. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
With Migrate Upgrade

21. What if your data needs
some massaging?

22. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Simple massaging
function hook_migrate_prepare_row(Row $row, MigrateSourceInterface $source, MigrationInterface $migration) {
if ($migration->id() == 'd6_filter_formats') {
$value = $source->getDatabase()->query('SELECT value FROM {variable} WHERE name = :name’,
array(':name' => 'mymodule_filter_foo_' . $row->getSourceProperty('format')))->fetchField();
if ($value) {
$row->setSourceProperty('settings:mymodule:foo', unserialize($value));
}
}
}

23. Advanced topics

24. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Plugin definition
/**
* Drupal 6 menu source from database.
*
* @MigrateSource(
* id = "d6_menu",
* source_provider = "menu"
* )
*/
class Menu extends DrupalSqlBase {
public function query() {
$query = $this->select('menu_custom', 'm')
->fields('m', array('menu_name', 'title', 'description'));
return $query;
}
public function fields() {
return array(
'menu_name' => $this->t('The menu name. Primary key.'),
'title' => $this->t('The human-readable name of the menu.'),
'description' => $this->t('A description of the menu'),
);
}
public function getIds() {
$ids['menu_name']['type'] = 'string';
return $ids;
}
}

25. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Load plugins
id: d6_cck_field_values
label: Drupal 6 field values
migration_groups:
- Drupal 6
load:
plugin: drupal_entity
bundle_migration: d6_node_type
source:
plugin: d6_cck_field_values
process:
nid:
plugin: migration
migration: d6_node
source: nid
destination:
plugin: entity:node
migration_dependencies:
required:
- d6_node
- d6_field_formatter_settings
- d6_field_instance_widget_settings

26. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Cckfield plugins/**
* @PluginID("openlayers_wkt")
*/
class OpenlayersCckField extends CckFieldPluginBase {
public function processField(MigrationInterface $migration) {
// The field would be geofield rather than link if it existed.
$process[0]['map'][$this->pluginId]['openlayers_wkt_widget'] = 'geofield';
$migration->mergeProcessOfProperty('type', $process);
}
public function processFieldWidget(MigrationInterface $migration) {
// The widget would be geofield rather than link if it existed.
$process['type']['map']['openlayers_wkt_widget'] = 'geofield_default';
$migration->mergeProcessOfProperty('options/type', $process);
}
public function getFieldFormatterMap() {
return [
'default' => 'geofield_default',
'openlayers_wkt' => 'geofield_default',
'openlayers_map_default' => 'geofield_default',
'hidden' => 'hidden',
];
}
public function processCckFieldValues(MigrationInterface $migration, $field_name, $data) {
$process = [
'plugin' => 'get',
'value' => 'openlayers_wkt',
];
$migration->mergeProcessOfProperty($field_name, $process);
}
}

27. Demo time!

28. Formerly known as Bysted, Propeople, Blink Reaction, Chainbizz and Geekpolis
Further study
https://www.drupal.org/project/examples
https://www.drupal.org/developing/api/8
https://api.drupal.org/api/drupal/8
https://www.drupal.org/list-changes
https://drupalize.me/blog/201409/unravelling-drupal-8-plugin-system
https://drupalize.me/blog/201408/preparing-drupal-8-psr-4-autoloading
https://www.drupal.org/documentation/administer/config

29. … and happy migrations!
Thank you