Migrating from drupal to plone with transmogrifier

  • 5,985 views
Uploaded on

Transmogrifier is a migration framework that can help you easily migrate from one platform to another. It has been written in a way that allows re-use of migration code through blueprints. In this …

Transmogrifier is a migration framework that can help you easily migrate from one platform to another. It has been written in a way that allows re-use of migration code through blueprints. In this talk we will walk through the steps necessary to migrate from Drupal, a popular CMS written in PHP, into Plone. We will see how to use the various blueprints available to build a pipeline that prepares and imports the content into Plone

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
5,985
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
28
Comments
2
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Migrating From Drupal to Plone with Transmogrifier Clayton Parker, Senior Developer PLONE SYMPOSIUM EAST 2011
  • 2. Who Am I? PLONE SYMPOSIUM EAST 2011• claytron• Python dev since 2003• Plone Core Committer• Foundation Member
  • 3. What will we learn? PLONE SYMPOSIUM EAST 2011• Transmogrifier basics• Migration Planning Process• Creating a pipeline
  • 4. Migrations PLONE SYMPOSIUM EAST 2011• One off scripts• In multiple places• Little to no re-usability
  • 5. Transmogrifier PLONE SYMPOSIUM EAST 2011• A framework for migrations• Re-usable parts
  • 6. Basics PLONE SYMPOSIUM EAST 2011• Pipeline• Blueprints• Sources• Section
  • 7. Sources PLONE SYMPOSIUM EAST 2011• A blueprint• First item in your pipeline
  • 8. Example Pipeline PLONE SYMPOSIUM EAST 2011 [transmogrifier] pipeline = csv_file constructor schemaupdater <html> <body> [csv_file] <h3>Code Sample</h3> blueprint = collective.transmogrifier.sections.csvsource <p>Replace this text!</p> filename = my.migration.import:my_items.csv </body> </html> [constructor] blueprint = collective.transmogrifier.sections.constructor [schemaupdater] blueprint = plone.app.transmogrifier.atschemaupdater
  • 9. my_items.csv PLONE SYMPOSIUM EAST 2011_path , _type , title , description/folder1 , Folder , First Folder , This is folder One/folder2 , Folder , Second Folder , This is folder Two/folder1/foo , Document , One Foo , A document named foo/folder2/foo , Document , Two Foo , Another doc named foo
  • 10. The Result PLONE SYMPOSIUM EAST 2011
  • 11. Items PLONE SYMPOSIUM EAST 2011• Each item is a mapping• Keys are fields• Keys with a leading underscore are controllers
  • 12. Example Item PLONE SYMPOSIUM EAST 2011{_id: a-stronger-connection-to-out-customers, _path: content/stronger-connection-out-customers, _status: 1L, _text_mimetype: text/html, _transitions: publish, _type: Document, allowDiscussion: False, creation_date: 2011/05/14 9:20:50, effectiveDate: 2011/05/14 9:20:50, modification_date: 2011/05/18 9:22:22, subject: betterninterestingnstronger, text: <p>this is some text</p>rn, title: A stronger connection to out customers}
  • 13. Migration Strategy PLONE SYMPOSIUM EAST 2011• Investigate the source• Prepare the destination• Find Transmogrifier blueprints• Write the pipeline
  • 14. Write your own PLONE SYMPOSIUM EAST 2011• Missing blueprint• Write one• Contribute it back
  • 15. GenericSetup PLONE SYMPOSIUM EAST 2011• Make migration part of your release• Ability to package migrations
  • 16. My Migration PLONE SYMPOSIUM EAST 2011• Drupal backed by MySQL• transmogrify.sqlalchemy• Plone 4.0.5• collective.blog.star• plone.app.discussion
  • 17. Package Layout PLONE SYMPOSIUM EAST 2011 my.migration !"" my #   !"" __init__.py #   %"" migration #   !"" __init__.py #   !"" config #   #   !"" articles.cfg #   #   !"" base.cfg #   #   !"" blogs.cfg #   #   !"" comments.cfg #   #   %"" pages.cfg #   !"" configure.zcml #   %"" profiles #      %"" default #      !"" metadata.xml #      %"" transmogrifier.txt !"" setup.cfg %"" setup.py
  • 18. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.base" title="My migration base config" description="Base settings for all transmogrifier imports" configuration="config/base.cfg" />
  • 19. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.pages" title="Drupal pages" description="Import pages from Drupal into Plone" configuration="config/pages.cfg" />
  • 20. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.articles" title="Drupal articles" description="Import articles from Drupal into Plone" configuration="config/articles.cfg" />
  • 21. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.blogs" title="Drupal blog entries" description="Import blog entries from Drupal into Plone" configuration="config/blogs.cfg" />
  • 22. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.comments" title="Drupal comments" description="Import comments from Drupal into Plone" configuration="config/comments.cfg" />
  • 23. transmogrifier.txt PLONE SYMPOSIUM EAST 2011 my.migration.pages my.migration.articles my.migration.blogs my.migration.comments
  • 24. Package Layout PLONE SYMPOSIUM EAST 2011 my.migration !"" my #   !"" __init__.py #   %"" migration #   !"" __init__.py #   !"" config #   #   !"" articles.cfg #   #   !"" base.cfg #   #   !"" blogs.cfg #   #   !"" comments.cfg #   #   %"" pages.cfg #   !"" configure.zcml #   %"" profiles #      %"" default #      !"" metadata.xml #      %"" transmogrifier.txt !"" setup.cfg %"" setup.py
  • 25. base.cfg pipeline PLONE SYMPOSIUM EAST 2011 [transmogrifier] pipeline = drupal portal_type url_normalizer path publication_state text_mimetype mimetype_encapsulator folders constructor commenting comments schema_update workflow reindex_object [settings] # Path to use if there isn’t one given base_path = other-content # Have to escape python string formatting for when # this gets passed into sqlalchemy date_format = %%Y/%%m/%%d %%k:%%i:%%s
  • 26. drupal section PLONE SYMPOSIUM EAST 2011 [drupal] blueprint = transmogrify.sqlalchemy dsn = mysql://user:password@localhost/drupal-transmog
  • 27. articles.cfg PLONE SYMPOSIUM EAST 2011 [transmogrifier] include = my.migration.base [drupal] query = SELECT node.title, node.status AS status, GROUP_CONCAT(tag_data.name SEPARATOR n) AS subject, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.changed, "${settings:date_format}") AS modification_date, body_data.body_value AS text url_alias.alias AS _path FROM node INNER JOIN field_data_field_tags AS tag_field ON tag_field.entity_id = node.nid INNER JOIN taxonomy_term_data AS tag_data ON tag_data.tid = tag_field.field_tags_tid INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias; [portal_type] value = string:Document
  • 28. pages.cfg PLONE SYMPOSIUM EAST 2011 [transmogrifier] include = my.migration.base [drupal] query = my.migration.base SELECT node.title, node.status AS _status, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.changed, "${settings:date_format}") AS modification_date, body_data.body_value AS text, url_alias.alias AS _path FROM node INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) WHERE node.type = "page" GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias; [portal_type] value = string:Document
  • 29. blogs.cfg PLONE SYMPOSIUM EAST 2011 [transmogrifier] include = my.migration.base [drupal] query = SELECT node.title, node.satus AS _status, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.created, "${settings:date_format}") AS modification_date, body_data.body_value AS text, url_alias.alias AS _path FROM node INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) WHERE node.type = "blog" GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias; [portal_type] value = string:BlogEntry [commenting] value = python:True
  • 30. comments.cfg PLONE SYMPOSIUM EAST 2011 [transmogrifier] include = my.migration.base [drupal] query = my.migration.base SELECT comment.subject AS title, FROM_UNIXTIME(comment.created, "${settings:date_format}") AS published, FROM_UNIXTIME(comment.changed, "${settings:date_format}") AS updated, comment.name AS author_name, body_data.comment_body_value AS text, url_alias.alias AS _parent_path FROM comment INNER JOIN field_data_comment_body AS body_data ON body_data.entity_id = comment.cid INNER JOIN node ON node.nid = comment.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) GROUP BY comment.subject, comment.created, comment.changed, comment.name, body_data.comment_body_value, url_alias.alias; [portal_type] # Override the portal type to use the "comment_type" key = string:_comment_type value = string:plone.app.discussion
  • 31. base.cfg overrides PLONE SYMPOSIUM EAST 2011 [path] blueprint = collective.transmogrifier.sections.inserter # only add a path if one does not exist condition = python:_path not in item and not _parent_path in item key = string:_path # Add the value in the extended configuration value = string:${settings:base_path}/${item/_id} [portal_type] blueprint = collective.transmogrifier.sections.inserter key = string:_type # We will add the value in the extended config, but we need a # default set here value = string: [commenting] blueprint = collective.transmogrifier.sections.inserter key = string:allowDiscussion # default to false value = python:False
  • 32. Content Creation PLONE SYMPOSIUM EAST 2011 [comments] blueprint = transmogrify.comments [folders] blueprint = collective.transmogrifier.sections.folders [constructor] blueprint = collective.transmogrifier.sections.constructor [schema_update] blueprint = plone.app.transmogrifier.atschemaupdater [workflow] blueprint = plone.app.transmogrifier.workflowupdater [reindex_object] blueprint = plone.app.transmogrifier.reindexobject
  • 33. Item Modification PLONE SYMPOSIUM EAST 2011 [publication_state] blueprint = collective.transmogrifier.sections.inserter condition = python:_status in item and item[_status] == 1 key = string:_transitions value = string:publish [text_mimetype] # This could probably be taken from the database as well blueprint = collective.transmogrifier.sections.inserter key = string:_text_mimetype value = string:text/html [mimetype_encapsulator] blueprint = plone.app.transmogrifier.mimeencapsulator key = text mimetype = python:item.get(_%s_mimetype, key) field = key condition = mimetype
  • 34. PLONE SYMPOSIUM EAST 2011DEMO
  • 35. Links PLONE SYMPOSIUM EAST 2011• collective.transmogrifierhttp://pypi.python.org/pypi/collective.transmogrifier/• plone.app.transmogrifierhttp://pypi.python.org/pypi/plone.app.transmogrifier/
  • 36. Useful Sources and PLONE SYMPOSIUM EAST 2011Blueprints• plone.app.transmogrifier• transmogrify.filesystem• transmogrify.sqlalchemy• transmogrify.webcrawler• quintagroup.transmogrifier• transmogrify.comments
  • 37. Check out .co m/d emoss ixfeetup