Migration – pain or gain?
Drupal Camp Kyiv
25. May 2019
Plan
2
• VKR:
• About JYSK
• How to sell D8, benefits
• JYSK migration statistics
• Plan migration (code, content, translations), learning before starting
• Migration challenges (general or JYSK specific)
• Migration advices
• Testing (manual, test case): data migration, missing content
• ITA:
• Basic migration tools
• Data structures migration (content types etc) ?
• Configuration migration (variables) ?
• Data migration (content, users, taxonomy), migration groups
• Debug migration
• Custom migrations, plugins (source, process, destination). Gridstack
• Automation script, default content, feeds
• Deploy, release
• Retrospective: code examples module
• Configuration architecture, multi-siting ?
Agenda
1. About us
2. How to sell Drupal 8
3. Migration challenges and
advices
4. Prepare and test migration
5. Do migration
4
About us
Volodymyr
Krasnoshapka
Team Leader,
Drupal Development
JYSK
Igor
Taldenko
Drupal Developer
JYSK
• 10+ years in Drupal
• 6 years in JYSK
• https://www.linkedin.com/in/shlapa
• 7 years in Drupal
• Almost 3 years in JYSK
• Acquia certified for Drupal 7 and 8
• 35 projects
• I don’t know what Javascript is
• First presentation on Drupal Camp
• www.linkedin.com/in/igor-taldenko
VKR
5
About
JYSK
6
1
JYSK Group
JYSK Nordic
JYSK Franchise
DÄNISCHES
BETTENLAGER
JYSK is an international chain of
stores with Scandinavian roots.
JYSK’s annual turnover is 3.58 billion
EUR.
JYSK has more than 2,700 stores and
23,000 employees in 52 countries.
In Germany and Austria the stores
are called DÄNISCHES BETTENLAGER
– in the rest of the world JYSK.
JYSK online platform
8
JYSK online platform
• 8 years starting from 2011 (Drupal 7)
• 20 online stores (jysk.dk, jysk.co.uk, jysk.ua etc)
• 60 million page views per month
• 15 million visitors per month
• ~8% of revenue from online
9
Web technology stack
10
CMS
Drupal
Front-end
ReactJS
Commerce
Java
● products
● text pages
● landing pages
● banners
● blogs
● guides
● faq
● webforms
● text blocks
● SEO
● users
● dynamic
functionality
● products
● orders
● customers
● stock
● stores
● reviews
11
• 10 months
• 10,000 hours of development
• 10 developers
• 7 product owners and testers
• 140 tasks
• 350 test cases
• 760 bugs
• 200,000 nodes
Migration statistics
… and customers noticed nothing :)
How to sell
Drupal 8
12
2
Benefits of Drupal 8 - general
• Customization and flexibility
• Reach site manager’s experience
• Multilingual
• Extensions and integrations
• Open source, free, no vendor lock
• and more...
13
Benefits of Drupal 8 - technical
• Modern paradigms, OOP
• Symfony, Twig, Composer
(fewer #DrupalWTFs)
• Clear data model
• Decoupling and headless
• and more...
14
Benefits of Drupal 8 - enterprise
• Mature
• Long term support
• Security
• Predictable release cycle and
roadmap
• Many vendors
• and more…
15
Enterprise products
16
Specific Why should you migrate
Long life cycle Continuous updates improve
maintainability + LTS
Long term plans No sense to build something
new and big based on old CMS
Many users (customers,
employees, developers)
All will benefit from modern
CMS
Integrations Drupal 8 is better for
integrations and customizations
Selling Drupal 8
• Define requirements
• Consider alternatives
• In-house product benefits
• Do you already use Drupal?
• Developers trained and motivated
• Migration of content and features
• “One-to-one” migration
• Continuous improvements
17
Migration
challenges
and
advices
18
3
19
• New API architecture often
require full module rewrites
• No update.php path
• No migration of
configuration and views
• Missing or different contrib
modules
Migration instead of Update
• Analyze contrib modules
• Estimate more time
• Rewrite instead of porting
Drupal’s next big thing
• New architecture
• Composer
• Multisite installation
• Theme rebuild
• Custom code rewrite
20
• Begin with training
• Theming: redesign OR keep
• Prioritize deadline OR quality
Our challenges
• Complexity and time
• 100% readiness x 20
• No automated tests
• Devices and browsers
• Missing features and content
21
• Estimate more time
• Simplify, consider one-to-one migration
• Automated tests
• Drupal 7 as fallback when missing data
22
Prepare
and
test
migration
4
23
• Analyze content and data
• Createupdate features list
• Estimate contrib modules
• Estimate custom code
• Consider POC
• Project plan
• Training and
documentation
• https://www.drupal.org/docs/8/upgrade
Prepare and plan
Initial setup
External content Internal content
Content migrationContent import
Feature Feature Feature Feature Feature
Test migration
• Test cases
• Test features
• End-to-end test x 20 x 2
• Test critical path
• Pre-production migration
24
• Prepare test cases
• Test functionality
• Test data migration
• Automate tests
ITA
25
Do
migration
26
5
ETL Process
27
Migration sources
• JSON
• CSV
• Drupal to Drupal
• From any database
28
Our case: Drupal 7 to Drupal 8
29
Basic migration tools
• Migrate
• Migrate Drupal
• Migrate Drupal UI
• Migrate Upgrade
• Migrate Tools
• Migrate Plus
• Migrate Manifest
• Migrate Devel
30
Data structure migration and variables
1. Generate migration structure
with migrate upgrade module
2. Get only structure related part
from generated YAML’s
3. Create group
4. Migrate group
5. Prepare variables migration
6. Migrate variables
7. Export all configurations
8. We have migrated all structure
31
Migration Groups
32
Custom plugins. Why do we needed them?
1. Filter source
2. Split source
3. Correct or change data during migrating with help of
process plugins
4. Way of storing data changed for some fields comparing to
Drupal 7
5. Data manipulation for fields...
33
Source plugins. Examples.
1. AliasesSource
2. ImageSources
3. D7PathRedirect
4. D7Webform
5. D7WebformSubmission
6. CustomMenu, CustomMenuLink
7. JyskNode
8. JyskNodeFeeds
9. JyskPanelizedNode
10. TaxonomyTermSource
34
Source plugin. Example. Jysk Node.
Prepare meta tag Prepare Row
35
Process plugins. Examples.
1. BodyTextContentMigration
2. JyskBaseLangcode
3. JyskBasePath
4. JyskGetSvg
5. JyskLinkUri
6. JyskVideo
7. ObjectToArray
36
Debug migration
Migrate devel
Drush mim blog_comments --limit=1 --idlist=24 --migrate-debug
or --migrate-debug-pre almost the same, but before the process runs on the row.
Migration UI and Xdebug
37
38
Data migration manifest
Automation is must
1. Custom install profile
• Set all necessary variables
• Configure blacklist and whitelists for config_split
• Set front page to node
2. Run automation script through Jenkins job
39
Migration steps
1. Collect arguments and check arguments
2. Set environment and general variables
3. Remove public files
4. Re-create database
5. Profile install
6. Enable delete_all module temporary and delete all content, then disable
it
7. Run migration
8. Default Content Deploy Update
9. Import feeds
10. Solr re-index
11. Translation export from Drupal 7 and import into Drupal 8
12. Clean up step, delete all migration tables and uninstall all migration modules
13. Set cron key
14. We are all set
40
Technical challenges
• Panels, custom panel panes
• Default content
• Complex paragraphs migration
• Translation strings
• SEO, meta tags
41
42
Pain
or
gain?
Questions?
• https://www.jysk.com/
• https://www.drupal.org/jysk
• https://jobs.dou.ua/companies/jysk
• https://www.linkedin.com/company/jysk/

MIGRATION - PAIN OR GAIN?

  • 1.
    Migration – painor gain? Drupal Camp Kyiv 25. May 2019
  • 2.
    Plan 2 • VKR: • AboutJYSK • How to sell D8, benefits • JYSK migration statistics • Plan migration (code, content, translations), learning before starting • Migration challenges (general or JYSK specific) • Migration advices • Testing (manual, test case): data migration, missing content • ITA: • Basic migration tools • Data structures migration (content types etc) ? • Configuration migration (variables) ? • Data migration (content, users, taxonomy), migration groups • Debug migration • Custom migrations, plugins (source, process, destination). Gridstack • Automation script, default content, feeds • Deploy, release • Retrospective: code examples module • Configuration architecture, multi-siting ?
  • 3.
    Agenda 1. About us 2.How to sell Drupal 8 3. Migration challenges and advices 4. Prepare and test migration 5. Do migration
  • 4.
    4 About us Volodymyr Krasnoshapka Team Leader, DrupalDevelopment JYSK Igor Taldenko Drupal Developer JYSK • 10+ years in Drupal • 6 years in JYSK • https://www.linkedin.com/in/shlapa • 7 years in Drupal • Almost 3 years in JYSK • Acquia certified for Drupal 7 and 8 • 35 projects • I don’t know what Javascript is • First presentation on Drupal Camp • www.linkedin.com/in/igor-taldenko
  • 5.
  • 6.
  • 7.
    JYSK Group JYSK Nordic JYSKFranchise DÄNISCHES BETTENLAGER JYSK is an international chain of stores with Scandinavian roots. JYSK’s annual turnover is 3.58 billion EUR. JYSK has more than 2,700 stores and 23,000 employees in 52 countries. In Germany and Austria the stores are called DÄNISCHES BETTENLAGER – in the rest of the world JYSK.
  • 8.
  • 9.
    JYSK online platform •8 years starting from 2011 (Drupal 7) • 20 online stores (jysk.dk, jysk.co.uk, jysk.ua etc) • 60 million page views per month • 15 million visitors per month • ~8% of revenue from online 9
  • 10.
    Web technology stack 10 CMS Drupal Front-end ReactJS Commerce Java ●products ● text pages ● landing pages ● banners ● blogs ● guides ● faq ● webforms ● text blocks ● SEO ● users ● dynamic functionality ● products ● orders ● customers ● stock ● stores ● reviews
  • 11.
    11 • 10 months •10,000 hours of development • 10 developers • 7 product owners and testers • 140 tasks • 350 test cases • 760 bugs • 200,000 nodes Migration statistics … and customers noticed nothing :)
  • 12.
  • 13.
    Benefits of Drupal8 - general • Customization and flexibility • Reach site manager’s experience • Multilingual • Extensions and integrations • Open source, free, no vendor lock • and more... 13
  • 14.
    Benefits of Drupal8 - technical • Modern paradigms, OOP • Symfony, Twig, Composer (fewer #DrupalWTFs) • Clear data model • Decoupling and headless • and more... 14
  • 15.
    Benefits of Drupal8 - enterprise • Mature • Long term support • Security • Predictable release cycle and roadmap • Many vendors • and more… 15
  • 16.
    Enterprise products 16 Specific Whyshould you migrate Long life cycle Continuous updates improve maintainability + LTS Long term plans No sense to build something new and big based on old CMS Many users (customers, employees, developers) All will benefit from modern CMS Integrations Drupal 8 is better for integrations and customizations
  • 17.
    Selling Drupal 8 •Define requirements • Consider alternatives • In-house product benefits • Do you already use Drupal? • Developers trained and motivated • Migration of content and features • “One-to-one” migration • Continuous improvements 17
  • 18.
  • 19.
    19 • New APIarchitecture often require full module rewrites • No update.php path • No migration of configuration and views • Missing or different contrib modules Migration instead of Update • Analyze contrib modules • Estimate more time • Rewrite instead of porting
  • 20.
    Drupal’s next bigthing • New architecture • Composer • Multisite installation • Theme rebuild • Custom code rewrite 20 • Begin with training • Theming: redesign OR keep • Prioritize deadline OR quality
  • 21.
    Our challenges • Complexityand time • 100% readiness x 20 • No automated tests • Devices and browsers • Missing features and content 21 • Estimate more time • Simplify, consider one-to-one migration • Automated tests • Drupal 7 as fallback when missing data
  • 22.
  • 23.
    23 • Analyze contentand data • Createupdate features list • Estimate contrib modules • Estimate custom code • Consider POC • Project plan • Training and documentation • https://www.drupal.org/docs/8/upgrade Prepare and plan Initial setup External content Internal content Content migrationContent import Feature Feature Feature Feature Feature
  • 24.
    Test migration • Testcases • Test features • End-to-end test x 20 x 2 • Test critical path • Pre-production migration 24 • Prepare test cases • Test functionality • Test data migration • Automate tests
  • 25.
  • 26.
  • 27.
  • 28.
    Migration sources • JSON •CSV • Drupal to Drupal • From any database 28
  • 29.
    Our case: Drupal7 to Drupal 8 29
  • 30.
    Basic migration tools •Migrate • Migrate Drupal • Migrate Drupal UI • Migrate Upgrade • Migrate Tools • Migrate Plus • Migrate Manifest • Migrate Devel 30
  • 31.
    Data structure migrationand variables 1. Generate migration structure with migrate upgrade module 2. Get only structure related part from generated YAML’s 3. Create group 4. Migrate group 5. Prepare variables migration 6. Migrate variables 7. Export all configurations 8. We have migrated all structure 31
  • 32.
  • 33.
    Custom plugins. Whydo we needed them? 1. Filter source 2. Split source 3. Correct or change data during migrating with help of process plugins 4. Way of storing data changed for some fields comparing to Drupal 7 5. Data manipulation for fields... 33
  • 34.
    Source plugins. Examples. 1.AliasesSource 2. ImageSources 3. D7PathRedirect 4. D7Webform 5. D7WebformSubmission 6. CustomMenu, CustomMenuLink 7. JyskNode 8. JyskNodeFeeds 9. JyskPanelizedNode 10. TaxonomyTermSource 34
  • 35.
    Source plugin. Example.Jysk Node. Prepare meta tag Prepare Row 35
  • 36.
    Process plugins. Examples. 1.BodyTextContentMigration 2. JyskBaseLangcode 3. JyskBasePath 4. JyskGetSvg 5. JyskLinkUri 6. JyskVideo 7. ObjectToArray 36
  • 37.
    Debug migration Migrate devel Drushmim blog_comments --limit=1 --idlist=24 --migrate-debug or --migrate-debug-pre almost the same, but before the process runs on the row. Migration UI and Xdebug 37
  • 38.
  • 39.
    Automation is must 1.Custom install profile • Set all necessary variables • Configure blacklist and whitelists for config_split • Set front page to node 2. Run automation script through Jenkins job 39
  • 40.
    Migration steps 1. Collectarguments and check arguments 2. Set environment and general variables 3. Remove public files 4. Re-create database 5. Profile install 6. Enable delete_all module temporary and delete all content, then disable it 7. Run migration 8. Default Content Deploy Update 9. Import feeds 10. Solr re-index 11. Translation export from Drupal 7 and import into Drupal 8 12. Clean up step, delete all migration tables and uninstall all migration modules 13. Set cron key 14. We are all set 40
  • 41.
    Technical challenges • Panels,custom panel panes • Default content • Complex paragraphs migration • Translation strings • SEO, meta tags 41
  • 42.
  • 43.
    Questions? • https://www.jysk.com/ • https://www.drupal.org/jysk •https://jobs.dou.ua/companies/jysk • https://www.linkedin.com/company/jysk/