Upgrades and migrations


Published on

The updated version of this is over here: http://www.slideshare.net/nadavoid/upgrades-and-migrations-bad-camp. I updated it for BAD Camp 2013.

You have an aging Drupal 6 or even a Drupal 5 site. You know it's time to move up to Drupal 7. Now, how? There are two main ways to get there. You can perform a traditional upgrade, or you can migrate the data from the old site to a brand new site. In this session I will show how you can use these methods and discuss their benefits and drawbacks, including a thought process to go through when evaluating these options, drawing from some recent projects.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Upgrades and migrations

  1. 1. Upgrades and Migrations, how? David Lanier @nadavoid ThinkShout Sunday, October 6, 13
  2. 2. Overview • Getting your Drupal 6 site onto Drupal 7 • Use traditional upgrade or migrate data to new site • Look at the benefits and drawbacks of both approaches Sunday, October 6, 13 Your site is unique, so take what you find today and let it help you in your decision making. I won't be saying always do it one way. Both approaches do require work.
  3. 3. An Architecture Metaphor • Changing the foundation (codebase) of your house Sunday, October 6, 13 Do you lift the house and swap out the foundation? Or do you build a new house on a new foundation, then move your stuff in?
  4. 4. Basic pieces • Codebase (core, modules, themes, libraries) • Database • Files (images, pdfs, mp3s, etc.) Sunday, October 6, 13 Files = content. Database & Files together = "data"
  5. 5. Upgrade overview • Swap out the foundation, house in place. • Replace codebase, run upgrade scripts on existing data & files. • Built into core and contrib. • UI based Sunday, October 6, 13 Lift the house, swap in a new foundation, set the house down on the new foundation. The upgrade scripts essentially upgrade the plumbing and wiring to be compatible with the new foundation. UI centered, but drush can be used for most upgrades as well. 'drush updb'
  6. 6. Migrate overview • Build new house on a new foundation. • Build new site. Import data from old site. • Write custom code to define the specifics of the migration. Sunday, October 6, 13 The details of a migration must be handled by writing custom code. There is also helpful support in the UI.
  7. 7. Feeds • Best fit is to regularly import new data. • Not usually a good fit for full migrations. • It is easy to learn, but it usually falls short when getting into the details, such as handling relationships and very large datasets. Sunday, October 6, 13 I really like feeds, and it can be super for importing data into Drupal. But it's just not a module I generally recommend for a full site migration.
  8. 8. Which way do I go? • Why do you want to go? Sunday, October 6, 13 First thing to think about is why specifically do you want to move to Drupal 7? Are there architectural things that need to be changed in your site, in order to take advantage of what you want from D7?
  9. 9. Traditional upgrade if: • Drupal site, only one version back • Simple site • Few contrib modules • End result should mirror current site • Content is infrequently updated • A little downtime is OK Sunday, October 6, 13 Downtime can still be avoided, using good deployment methods. Also, if limited time is available (and the other conditions are met).
  10. 10. Traditional upgrade downsides • Hard to get rid of legacy cruft • Field names • Content type structures • Anything else that seems old or odd Sunday, October 6, 13
  11. 11. Migrate if: • Drupal 5 site, or even a different platform • Complex site • Many contrib modules • New site needs structural changes or enhancements • Content is frequently updated • Minimal downtime is very important Sunday, October 6, 13
  12. 12. Migrate downsides • Very code-centric • Very granular Sunday, October 6, 13 Averse to custom coding? Maybe prefer upgrade. But Migrate isn't so hard, I promise! Granular: you generally have to specify everything (image alt, title, language, etc.). Of course, that could also be a good thing. You're in full control.
  13. 13. Initial preparation • Regular D7 system requirements (drupal.org/requirements) • Local development environment that can support several copies of the site Sunday, October 6, 13 D7: PHP 5.3 recommended, MySQL 5. Local Dev: for testing, experimenting, practicing
  14. 14. Architecture of current site • Inventory of modules, libraries, themes •drush pml | grep Enabled • Content Types, fields • Views, other configuration • Important pages, capabilities, workflows • Reference copy of the site Sunday, October 6, 13 Save lists of directories (list of modules) Save drush output Save certain pages Have a clear idea of exactly what it is you're upgrading from
  15. 15. Determine upgrade path • Module to D7 options • same module (pathauto) • different module • moved into core (CCK to fields) • nothing available, so custom solution Sunday, October 6, 13 This is where most of the work will be. Be sure to check if D7 version of modules have their own dependencies. Views adds a ctools dependency. Others will likely require the libraries module.
  16. 16. Determine upgrade path • Custom modules • Converting 6.x modules to 7.x: http://drupal.org/node/224333 Sunday, October 6, 13 Any custom code will need to be updated to match the Drupal 7 API and coding standards.
  17. 17. Determine upgrade path • Theme • Update or replace base theme • Update or rebuild subtheme • Upgrading 6.x themes to 7.x https://drupal.org/node/254940 Sunday, October 6, 13 A custom theme is custom code, so needs same attention.
  18. 18. Prepare the source • Install a copy of the site to use as the source. • Remove all disabled or unused modules. • Update all remaining modules to their latest D6 versions. Sunday, October 6, 13
  19. 19. How to Upgrade • Practice on a copy of the site. • Pristine copy of site for reference. • Copy of site with everything D6 updated. • Copy that is actually getting upgraded. Sunday, October 6, 13
  20. 20. Today's Example • RidePDW.com Portland Design Works • Portland company that develops bicycle accessories Sunday, October 6, 13 I experimented with an upgrade, but ended up doing a migration instead.
  21. 21. Process • Commit after each step. • Definitely after updating each module. • Include db exports. Sunday, October 6, 13 DB Exports: if not in the repo, in their own folder.
  22. 22. Upgrade Core • Refer back to your list of all enabled modules • Disable all enabled contrib modules • Switch theme to Garland, including the admin theme Sunday, October 6, 13
  23. 23. Upgrade core • Use the copy of the site that you have for actually upgrading • Replace core files manually • Delete core directories & files • Add the new ones • Download Drupal, move the files into place Sunday, October 6, 13 I generally prefer to swap in the new files/folders, rather than building up a new codebase. (List of "incompatible" modules helps me know where I am)
  24. 24. Update the database • Do 'drush updb' • Watch output, observe warnings • Save output to refer back to Sunday, October 6, 13
  25. 25. Sunday, October 6, 13
  26. 26. Sunday, October 6, 13
  27. 27. Sunday, October 6, 13
  28. 28. Sunday, October 6, 13
  29. 29. Core upgrade complete! • Yay! • You're running Drupal 7 now • Commit the current state to a repo • Next step, contrib modules Sunday, October 6, 13
  30. 30. Upgrading contrib modules • Upgrade one at a time • Module's d.o page • Module's README.txt or INSTALL.txt or UPGRADE.txt • Check for special steps, instructions • Check for new dependencies Sunday, October 6, 13 Do a little research on each module before trying. Then go ahead and try. Dependencies: views needs ctools. ubercart needs rules. rules needs entity.
  31. 31. Add the new code • This works: drush dl modulename • Or, delete existing, then add the new Sunday, October 6, 13
  32. 32. Where to start? • Start with the most foundational • ctools • views • Or most standalone • pathauto • devel Sunday, October 6, 13
  33. 33. Views • Replace views code • Update database • Enable the upgraded module Sunday, October 6, 13 This seems to be the usual process. But it's still best to see the module's upgrade documentation first.
  34. 34. Views database updated Sunday, October 6, 13
  35. 35. Enable views Sunday, October 6, 13
  36. 36. Enable views Sunday, October 6, 13
  37. 37. Upgrade CCK • Download D7 version of CCK • Enable the content_migrate module • Use the UI to migrate CCK fields to drupal core fields Sunday, October 6, 13 Kind of a special case, because it's a major architectural change, bringing fields into core. Also common pattern: contrib module providing upgrade path into core.
  38. 38. Available fields Sunday, October 6, 13 Provides suggestions of things to do, and notes about what it will do. Caveat: it lets you migrate fields, even if the relevant content type does not yet exist properly in the upgraded site. Should content types come before fields?
  39. 39. Converted Fields Sunday, October 6, 13
  40. 40. Unavailable Fields Sunday, October 6, 13
  41. 41. Migrate CCK fields • Install required modules • Enable modules that define content types • Sometimes several layers of dependencies emerge Sunday, October 6, 13 nodereference is used to refer to products. Products are defined by ubercart. Ubercart requires Rules. Rules requires entity api.
  42. 42. How to Migrate • Building a whole new house, with its own new foundation. • Then we will move all of our data into it. Sunday, October 6, 13
  43. 43. Plan and get organized • Collect data and files into a safe place, to use as a pristine, read-only source. • Make a spreadsheet or list to plan out your mapping • Build out the architecture of the new site Sunday, October 6, 13 Identify questionable or troublesome pieces, e.g. whether to keep certain fields. Build: at least content types & fields. Anything else that is needed by your migrated data.
  44. 44. Set up environments • Install or get access to a reference site • Install source database • Install a built destination site • An IDE for exploring the migrate API Sunday, October 6, 13 reference site = this is how it currently is. must be inspectable.
  45. 45. migrate_d2d • Not discussing migrate_d2d in depth today. Sunday, October 6, 13 Not migrate_d2d: - complicated things for me as a newcomer to the migrate framework. - Added too many layers and did not streamline things for me. - Had to unmap many things that it automatically mapped for me.
  46. 46. Start a new module • Install the migrate module • Create a new module that extends classes that are in the migrate module Sunday, October 6, 13
  47. 47. Important functions • addFieldMapping() • addSimpleMappings() • defaultValue() • sourceMigration() • addUnmigratedDestinations() • addUnmigratedSources() Sunday, October 6, 13 Just to get a flavor of what you will be using.
  48. 48. Module structure • .info file: file list • .module file: specify migrations via hook_migrate_api() • Migration class files: handle the details and mappings of a migration • name: sitename_migration Sunday, October 6, 13 .info file: Mainly lists files that are used. .module file: register migrations and their classes by using hook_migrate_api(). Migration class files: I generally prefer one class per file, but there's nothing wrong with having multiple classes in one file too.
  49. 49. Directory list Sunday, October 6, 13
  50. 50. .info file Sunday, October 6, 13
  51. 51. .module file Sunday, October 6, 13
  52. 52. class file Sunday, October 6, 13
  53. 53. How did you find out? • Comments in the migrate module • migrate_example module (inside migrate) • articles on btmash.com Sunday, October 6, 13
  54. 54. Additional Docs • Source code • Suggestions in IDE • Migrate handbook at drupal.org/migrate Sunday, October 6, 13
  55. 55. Before enabling custom migrate module Sunday, October 6, 13
  56. 56. Migration class • Set description • Define database source • Create query to get data • Tell migrate that this query is the source of this migration • Do the mappings Sunday, October 6, 13
  57. 57. Set description Sunday, October 6, 13
  58. 58. Set database Sunday, October 6, 13
  59. 59. Define the query Sunday, October 6, 13
  60. 60. Set the source Sunday, October 6, 13
  61. 61. Set the destination Sunday, October 6, 13
  62. 62. Set up table mapping Sunday, October 6, 13 Enables migrate to map source rows to destination rows
  63. 63. Map first field Sunday, October 6, 13
  64. 64. Enable the module Sunday, October 6, 13 Do live demo of migrating this basic amount, using the UI. List of migrations & their status. Status of mappings. Note equivalent tools in drush.
  65. 65. Mapping process • Add fields as needed to the query • Add mappings until the mapping screen clear. Sunday, October 6, 13 Adding fields will usually mean adding joins.
  66. 66. Mapping a subfield Sunday, October 6, 13
  67. 67. Mapping a reference From a ProductHelpMigration Sunday, October 6, 13
  68. 68. Self references • Handles creating references to same import • Uses a createStub function Sunday, October 6, 13
  69. 69. Mapping a file field Sunday, October 6, 13
  70. 70. Term References • Use a dependent migration • Or create as needed Sunday, October 6, 13
  71. 71. Repeats • "Update" completely overwrites • Highwater mark Sunday, October 6, 13 In addition to rollback and reimport. highwater mark, important for bringing in added content. brings in any new items with a greater value of highwater than last recorded. No other comparisons. (no import "changed")
  72. 72. Manipulating source data • prepareRow() function Sunday, October 6, 13
  73. 73. migrate_extras • Provides support for many contrib modules • Addressfield, entity api, more • Excellent examples of custom handlers Sunday, October 6, 13
  74. 74. Migrate • Showed important parts • There's a LOT more there Sunday, October 6, 13
  75. 75. QA. Did it work? • Regardless whether upgrade or migrate • Have someone else check your work • Have the client check your work • Pass any existing acceptance tests Sunday, October 6, 13 Make certain. This will be the content of the next version of this site.
  76. 76. QA Questions • Correct content? • Correct versions of content? • Complete number of fields? • Complete metadata? (urls, relations, dates, authors, tags) • Complete number of nodes? • Images and media? (block live site) Sunday, October 6, 13
  77. 77. Conclusion (?) • traditional upgrade • migrate Sunday, October 6, 13 If you want something that is quick and your site's upgrade path supports it, go with a traditional upgrade. If you want to make a lot of structural changes and need fine control over the upgrade path, go with migrate.
  78. 78. Deployment • Same issues as replacing any site, unless the D6 site is incredibly simple. Sunday, October 6, 13
  79. 79. Future of upgrades • Upgrades are challenging • Migrate is being added to Drupal 8 core, to provide a smooth upgrade path from D6 and D7 core. http://www.drupal4hu.com/node/381 Sunday, October 6, 13
  80. 80. Bookmarks • http://drupalcode.org/project/drupal.git/ blob/refs/heads/7.x:/UPGRADE.txt • https://drupal.org/project/drush_sup • http://drupal.org/project/migrate • http://drupal.org/project/migrate_d2d • http://drupal.org/migrate • http://drupal.org/project/coder (upgrade) • http://drupal.org/project/hacked • http://drupal.org/project/upgrade_status Sunday, October 6, 13