Migrating From Drupal    to Plone with    Transmogrifier   Clayton Parker, Senior Developer       PLONE SYMPOSIUM EAST 2011
Who Am I?                 PLONE SYMPOSIUM EAST 2011• claytron• Python dev since 2003• Plone Core Committer• Foundation Mem...
What will we learn?            PLONE SYMPOSIUM EAST 2011• Transmogrifier basics• Migration Planning Process• Creating a pip...
Migrations                    PLONE SYMPOSIUM EAST 2011• One off scripts• In multiple places• Little to no re-usability
Transmogrifier                  PLONE SYMPOSIUM EAST 2011• A framework for migrations• Re-usable parts
Basics         PLONE SYMPOSIUM EAST 2011• Pipeline• Blueprints• Sources• Section
Sources                         PLONE SYMPOSIUM EAST 2011• A blueprint• First item in your pipeline
Example Pipeline                                PLONE SYMPOSIUM EAST 2011 [transmogrifier] pipeline =     csv_file     con...
my_items.csv                                                 PLONE SYMPOSIUM EAST 2011_path          ,   _type      ,   ti...
The Result   PLONE SYMPOSIUM EAST 2011
Items                                    PLONE SYMPOSIUM EAST 2011• Each item is a mapping• Keys are fields• Keys with a le...
Example Item                                       PLONE SYMPOSIUM EAST 2011{_id: a-stronger-connection-to-out-customers, ...
Migration Strategy                PLONE SYMPOSIUM EAST 2011• Investigate the source• Prepare the destination• Find Transmo...
Write your own         PLONE SYMPOSIUM EAST 2011• Missing blueprint• Write one• Contribute it back
GenericSetup                            PLONE SYMPOSIUM EAST 2011• Make migration part of your release• Ability to package...
My Migration                PLONE SYMPOSIUM EAST 2011• Drupal backed by MySQL• transmogrify.sqlalchemy• Plone 4.0.5• colle...
Package Layout                               PLONE SYMPOSIUM EAST 2011    my.migration    !"" my    #   !"" __init__.py   ...
Registering Configs                               PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig    name="my.migra...
Registering Configs                               PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig    name="my.migra...
Registering Configs                               PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig    name="my.migra...
Registering Configs                               PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig    name="my.migra...
Registering Configs                               PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig    name="my.migra...
transmogrifier.txt           PLONE SYMPOSIUM EAST 2011    my.migration.pages    my.migration.articles    my.migration.blogs...
Package Layout                               PLONE SYMPOSIUM EAST 2011    my.migration    !"" my    #   !"" __init__.py   ...
base.cfg pipeline                                   PLONE SYMPOSIUM EAST 2011    [transmogrifier]    pipeline =        dru...
drupal section                                 PLONE SYMPOSIUM EAST 2011  [drupal]  blueprint = transmogrify.sqlalchemy  d...
articles.cfg                                                                PLONE SYMPOSIUM EAST 2011 [transmogrifier] inc...
pages.cfg                                                                   PLONE SYMPOSIUM EAST 2011  [transmogrifier]  i...
blogs.cfg                                                                   PLONE SYMPOSIUM EAST 2011  [transmogrifier]  i...
comments.cfg                                                               PLONE SYMPOSIUM EAST 2011 [transmogrifier] incl...
base.cfg overrides                                         PLONE SYMPOSIUM EAST 2011 [path] blueprint = collective.transmo...
Content Creation                                          PLONE SYMPOSIUM EAST 2011 [comments] blueprint = transmogrify.co...
Item Modification                                         PLONE SYMPOSIUM EAST 2011 [publication_state] blueprint = collect...
PLONE SYMPOSIUM EAST 2011DEMO
Links                                              PLONE SYMPOSIUM EAST 2011• collective.transmogrifierhttp://pypi.python.o...
Useful Sources and            PLONE SYMPOSIUM EAST 2011Blueprints• plone.app.transmogrifier• transmogrify.filesystem• transm...
Check out          .co m/d emoss ixfeetup
Upcoming SlideShare
Loading in...5
×

Migrating from drupal to plone with transmogrifier

6,229

Published on

Transmogrifier is a migration framework that can help you easily migrate from one platform to another. It has been written in a way that allows re-use of migration code through blueprints. In this talk we will walk through the steps necessary to migrate from Drupal, a popular CMS written in PHP, into Plone. We will see how to use the various blueprints available to build a pipeline that prepares and imports the content into Plone

Published in: Technology
2 Comments
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
6,229
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
31
Comments
2
Likes
3
Embeds 0
No embeds

No notes for slide

Migrating from drupal to plone with transmogrifier

  1. 1. Migrating From Drupal to Plone with Transmogrifier Clayton Parker, Senior Developer PLONE SYMPOSIUM EAST 2011
  2. 2. Who Am I? PLONE SYMPOSIUM EAST 2011• claytron• Python dev since 2003• Plone Core Committer• Foundation Member
  3. 3. What will we learn? PLONE SYMPOSIUM EAST 2011• Transmogrifier basics• Migration Planning Process• Creating a pipeline
  4. 4. Migrations PLONE SYMPOSIUM EAST 2011• One off scripts• In multiple places• Little to no re-usability
  5. 5. Transmogrifier PLONE SYMPOSIUM EAST 2011• A framework for migrations• Re-usable parts
  6. 6. Basics PLONE SYMPOSIUM EAST 2011• Pipeline• Blueprints• Sources• Section
  7. 7. Sources PLONE SYMPOSIUM EAST 2011• A blueprint• First item in your pipeline
  8. 8. Example Pipeline PLONE SYMPOSIUM EAST 2011 [transmogrifier] pipeline = csv_file constructor schemaupdater <html> <body> [csv_file] <h3>Code Sample</h3> blueprint = collective.transmogrifier.sections.csvsource <p>Replace this text!</p> filename = my.migration.import:my_items.csv </body> </html> [constructor] blueprint = collective.transmogrifier.sections.constructor [schemaupdater] blueprint = plone.app.transmogrifier.atschemaupdater
  9. 9. my_items.csv PLONE SYMPOSIUM EAST 2011_path , _type , title , description/folder1 , Folder , First Folder , This is folder One/folder2 , Folder , Second Folder , This is folder Two/folder1/foo , Document , One Foo , A document named foo/folder2/foo , Document , Two Foo , Another doc named foo
  10. 10. The Result PLONE SYMPOSIUM EAST 2011
  11. 11. Items PLONE SYMPOSIUM EAST 2011• Each item is a mapping• Keys are fields• Keys with a leading underscore are controllers
  12. 12. Example Item PLONE SYMPOSIUM EAST 2011{_id: a-stronger-connection-to-out-customers, _path: content/stronger-connection-out-customers, _status: 1L, _text_mimetype: text/html, _transitions: publish, _type: Document, allowDiscussion: False, creation_date: 2011/05/14 9:20:50, effectiveDate: 2011/05/14 9:20:50, modification_date: 2011/05/18 9:22:22, subject: betterninterestingnstronger, text: <p>this is some text</p>rn, title: A stronger connection to out customers}
  13. 13. Migration Strategy PLONE SYMPOSIUM EAST 2011• Investigate the source• Prepare the destination• Find Transmogrifier blueprints• Write the pipeline
  14. 14. Write your own PLONE SYMPOSIUM EAST 2011• Missing blueprint• Write one• Contribute it back
  15. 15. GenericSetup PLONE SYMPOSIUM EAST 2011• Make migration part of your release• Ability to package migrations
  16. 16. My Migration PLONE SYMPOSIUM EAST 2011• Drupal backed by MySQL• transmogrify.sqlalchemy• Plone 4.0.5• collective.blog.star• plone.app.discussion
  17. 17. Package Layout PLONE SYMPOSIUM EAST 2011 my.migration !"" my #   !"" __init__.py #   %"" migration #   !"" __init__.py #   !"" config #   #   !"" articles.cfg #   #   !"" base.cfg #   #   !"" blogs.cfg #   #   !"" comments.cfg #   #   %"" pages.cfg #   !"" configure.zcml #   %"" profiles #      %"" default #      !"" metadata.xml #      %"" transmogrifier.txt !"" setup.cfg %"" setup.py
  18. 18. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.base" title="My migration base config" description="Base settings for all transmogrifier imports" configuration="config/base.cfg" />
  19. 19. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.pages" title="Drupal pages" description="Import pages from Drupal into Plone" configuration="config/pages.cfg" />
  20. 20. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.articles" title="Drupal articles" description="Import articles from Drupal into Plone" configuration="config/articles.cfg" />
  21. 21. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.blogs" title="Drupal blog entries" description="Import blog entries from Drupal into Plone" configuration="config/blogs.cfg" />
  22. 22. Registering Configs PLONE SYMPOSIUM EAST 2011<transmogrifier:registerConfig name="my.migration.comments" title="Drupal comments" description="Import comments from Drupal into Plone" configuration="config/comments.cfg" />
  23. 23. transmogrifier.txt PLONE SYMPOSIUM EAST 2011 my.migration.pages my.migration.articles my.migration.blogs my.migration.comments
  24. 24. Package Layout PLONE SYMPOSIUM EAST 2011 my.migration !"" my #   !"" __init__.py #   %"" migration #   !"" __init__.py #   !"" config #   #   !"" articles.cfg #   #   !"" base.cfg #   #   !"" blogs.cfg #   #   !"" comments.cfg #   #   %"" pages.cfg #   !"" configure.zcml #   %"" profiles #      %"" default #      !"" metadata.xml #      %"" transmogrifier.txt !"" setup.cfg %"" setup.py
  25. 25. base.cfg pipeline PLONE SYMPOSIUM EAST 2011 [transmogrifier] pipeline = drupal portal_type url_normalizer path publication_state text_mimetype mimetype_encapsulator folders constructor commenting comments schema_update workflow reindex_object [settings] # Path to use if there isn’t one given base_path = other-content # Have to escape python string formatting for when # this gets passed into sqlalchemy date_format = %%Y/%%m/%%d %%k:%%i:%%s
  26. 26. drupal section PLONE SYMPOSIUM EAST 2011 [drupal] blueprint = transmogrify.sqlalchemy dsn = mysql://user:password@localhost/drupal-transmog
  27. 27. articles.cfg PLONE SYMPOSIUM EAST 2011 [transmogrifier] include = my.migration.base [drupal] query = SELECT node.title, node.status AS status, GROUP_CONCAT(tag_data.name SEPARATOR n) AS subject, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.changed, "${settings:date_format}") AS modification_date, body_data.body_value AS text url_alias.alias AS _path FROM node INNER JOIN field_data_field_tags AS tag_field ON tag_field.entity_id = node.nid INNER JOIN taxonomy_term_data AS tag_data ON tag_data.tid = tag_field.field_tags_tid INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias; [portal_type] value = string:Document
  28. 28. pages.cfg PLONE SYMPOSIUM EAST 2011 [transmogrifier] include = my.migration.base [drupal] query = my.migration.base SELECT node.title, node.status AS _status, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.changed, "${settings:date_format}") AS modification_date, body_data.body_value AS text, url_alias.alias AS _path FROM node INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) WHERE node.type = "page" GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias; [portal_type] value = string:Document
  29. 29. blogs.cfg PLONE SYMPOSIUM EAST 2011 [transmogrifier] include = my.migration.base [drupal] query = SELECT node.title, node.satus AS _status, FROM_UNIXTIME(node.created, "${settings:date_format}") AS creation_date, FROM_UNIXTIME(node.created, "${settings:date_format}") AS effectiveDate, FROM_UNIXTIME(node.created, "${settings:date_format}") AS modification_date, body_data.body_value AS text, url_alias.alias AS _path FROM node INNER JOIN field_data_body AS body_data ON body_data.entity_id = node.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) WHERE node.type = "blog" GROUP BY node.title, node.status, node.created, node.changed, body_data.body_value, url_alias.alias; [portal_type] value = string:BlogEntry [commenting] value = python:True
  30. 30. comments.cfg PLONE SYMPOSIUM EAST 2011 [transmogrifier] include = my.migration.base [drupal] query = my.migration.base SELECT comment.subject AS title, FROM_UNIXTIME(comment.created, "${settings:date_format}") AS published, FROM_UNIXTIME(comment.changed, "${settings:date_format}") AS updated, comment.name AS author_name, body_data.comment_body_value AS text, url_alias.alias AS _parent_path FROM comment INNER JOIN field_data_comment_body AS body_data ON body_data.entity_id = comment.cid INNER JOIN node ON node.nid = comment.nid INNER JOIN url_alias ON url_alias.source = CONCAT("node/", node.nid) GROUP BY comment.subject, comment.created, comment.changed, comment.name, body_data.comment_body_value, url_alias.alias; [portal_type] # Override the portal type to use the "comment_type" key = string:_comment_type value = string:plone.app.discussion
  31. 31. base.cfg overrides PLONE SYMPOSIUM EAST 2011 [path] blueprint = collective.transmogrifier.sections.inserter # only add a path if one does not exist condition = python:_path not in item and not _parent_path in item key = string:_path # Add the value in the extended configuration value = string:${settings:base_path}/${item/_id} [portal_type] blueprint = collective.transmogrifier.sections.inserter key = string:_type # We will add the value in the extended config, but we need a # default set here value = string: [commenting] blueprint = collective.transmogrifier.sections.inserter key = string:allowDiscussion # default to false value = python:False
  32. 32. Content Creation PLONE SYMPOSIUM EAST 2011 [comments] blueprint = transmogrify.comments [folders] blueprint = collective.transmogrifier.sections.folders [constructor] blueprint = collective.transmogrifier.sections.constructor [schema_update] blueprint = plone.app.transmogrifier.atschemaupdater [workflow] blueprint = plone.app.transmogrifier.workflowupdater [reindex_object] blueprint = plone.app.transmogrifier.reindexobject
  33. 33. Item Modification PLONE SYMPOSIUM EAST 2011 [publication_state] blueprint = collective.transmogrifier.sections.inserter condition = python:_status in item and item[_status] == 1 key = string:_transitions value = string:publish [text_mimetype] # This could probably be taken from the database as well blueprint = collective.transmogrifier.sections.inserter key = string:_text_mimetype value = string:text/html [mimetype_encapsulator] blueprint = plone.app.transmogrifier.mimeencapsulator key = text mimetype = python:item.get(_%s_mimetype, key) field = key condition = mimetype
  34. 34. PLONE SYMPOSIUM EAST 2011DEMO
  35. 35. Links PLONE SYMPOSIUM EAST 2011• collective.transmogrifierhttp://pypi.python.org/pypi/collective.transmogrifier/• plone.app.transmogrifierhttp://pypi.python.org/pypi/plone.app.transmogrifier/
  36. 36. Useful Sources and PLONE SYMPOSIUM EAST 2011Blueprints• plone.app.transmogrifier• transmogrify.filesystem• transmogrify.sqlalchemy• transmogrify.webcrawler• quintagroup.transmogrifier• transmogrify.comments
  37. 37. Check out .co m/d emoss ixfeetup
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×