Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Managing changes to eZPublish Database

499 views

Published on

Managing changes to eZPublish Databases across the project lifecycle: from manual to fully automated thanks to Kaliop eZMigrationBundle

Published in: Internet
  • Be the first to comment

  • Be the first to like this

Managing changes to eZPublish Database

  1. 1. Visuel à insérer ici EZ CONFERENCE 2016 PARIS Managing Changes to the Database Across the Project Life Cycle
  2. 2. WHO AM I The short version • « the guy who has written a lot of answers on the eZPublish forums » • Working at Kaliop in London since 2014 • Principal consultant at eZ Systems for 7 years • Love coffee and bicycles ? • @gggeek VOTRE SCHEMA
  3. 3. WHAT THIS TALK IS ABOUT The short version, too
  4. 4. Database vs. cms vs. the project lifecycle 01
  5. 5. Project life cycle • Development • Testing • Maintenance
  6. 6. Development phase • Only one source of data: the “dev” db • Size of data set is generally small • Changes to content structure are frequent • Developers need to share the database (mostly the structure)
  7. 7. Testing phase • Two sources of data: the “dev” and “test” db • Size of data set can be small or not • Changes to content structure are less frequent • Developers need to share the dev database • Controlled releases of changes from dev to test • The bulk of test “content” is not overwritten
  8. 8. Maintenance phase • 3 (or more) sources of data: the “dev”, “test” and “prod” db • Size of data set can be large • Changes to content structure are infrequent • Few developers need to share the dev database • Controlled releases of changes • Content might flow the opposite way than changes to structure: “copy prod to test”
  9. 9. What we learned • Different needs during different phases of the project • Changes to Content and to Content-structure have different flows
  10. 10. What the CMS brings in • Content structure can be modified via the GUI!  • Many tables, with many relations • No foreign keys  (import scripts can break referential integrity) • Some data is tied to files stored on disk  (take care when doing backups!) • Impossible to replicate / sync single tables 
  11. 11. Is it all for the better? • Easy to deploy changes to Content Structure after go live Unless the changes takes hours to apply Will never cover 100% of the cases • Hard to automate • Hard to verify
  12. 12. Managing database changes A. I connect to the Admin Interface B. A solved problem 02
  13. 13. Keeping Databases in sync Many possibilities: A. All developers connect to the same db B. Managing changes manually C. The database is committed to git D. Managing changes via scripts E. Live DB replication (really ???) F. More ?
  14. 14. A single, shared database A
  15. 15. Connecting to a shared database • good for teams which are *not* geographically distributed • works when developers do not blow up the database with junk content: needs some developer discipline • works only up to the 1st deployment to TEST env; then 2 dbs have to be managed anyway
  16. 16. Manual change control B
  17. 17. Managing database changes manually • Changes to be applied are documented and applied via GUI • Most Unsafe Option (TM) • Some tools to help you to keep mental sanity: ggsysinfo ezdbintegrity
  18. 18. ggsysinfo https://github.com/gggeek/ggsysinfo • Legacy Extension (no eZPlatform version yet) • Allows to export in a format easy to diff: Content Types definitions Roles and Policies definitions Workflows and Triggers
  19. 19. ggsysinfo https://github.com/gggeek/ggsysinfo
  20. 20. ggsysinfo https://github.com/gggeek/ggsysinfo
  21. 21. ggsysinfo https://github.com/gggeek/ggsysinfo
  22. 22. ezdbintegrity https://github.com/gggeek/ezdbintegrity • Legacy Extension (no eZPlatform version yet) • Allows to verify consistency of data in the database: Foreign keys Content fields Orphaned storage files Custom rules added by the end user • No GUI
  23. 23. ezdbintegrity https://github.com/gggeek/ezdbintegrity
  24. 24. ezdbintegrity https://github.com/gggeek/ezdbintegrity
  25. 25. Database as source code C
  26. 26. Committing the database to git • good for when the development database does not contain a huge number of contents and assets • good for when the development database does not change too frequently • developers might be working on different versions of the db: needs some developer discipline • works only up to the 1st deployment to UAT env; then 2 dbs have to be managed anyway
  27. 27. Committing the database to git TIPS • Database export/import scripts have to developed • Good idea: remove temporary data before export • Bad idea: hardcode database passwords in the scripts • Good idea: read the db passwords from the ezpublish configuration “Kaliop eZPublish 5 installer” • Good idea: have the script manage binary contents & clear caches ex: github.com/kaliop-uk/websummercamp2016/tree/master/site/bin
  28. 28. Automated change control D
  29. 29. Managing database changes via script • Safest option • Good for when the development database does not change too frequently • Perfect after deployment to TEST/PROD • kaliop/ezmigrationbundle
  30. 30. eZ Migration Bundle https://github.com/kalipo-uk/ezmigrationbundle • An eZPublish 5 bundle (no support for pure Legacy mode) • Allows to define “migrations” • Each migration is a set of changes to the DB – content or structure • Migrations are stored as part of the application source code • A console command is used to execute them • A custom table in the db stored information on already executed ones
  31. 31. An example migration https://github.com/kalipo-uk/ezmigrationbundle
  32. 32. What types of migrations are supported https://github.com/kalipo-uk/ezmigrationbundle • creation, update and deletion of Contents • creation, update and deletion of Locations • creation, update and deletion of Users • creation, update and deletion of UserGroups • creation, update and deletion of Roles • creation, update and deletion of ContentTypes • creation and deletion of Languages • creation of Tags (from the Netgen Tags Bundle)
  33. 33. Cool features I https://github.com/kalipo-uk/ezmigrationbundle • Well documented (*) • Stable: functional tests are run on Travis • Future proof: based on the eZPublish Public API * = opinions may vary
  34. 34. Cool features II https://github.com/kalipo-uk/ezmigrationbundle • Supports ids, identifiers and remote-ids to match elements • Supports the concept of ‘references’ to anything which has just been created/updated • Update and delete operations can affect a whole set of elements in a single pass
  35. 35. Cool features III https://github.com/kalipo-uk/ezmigrationbundle For more complex needs, two types of custom migrations: • SQL files • PHP classes with a simple Interface Fully extensible using standard Symfony mechanisms • Event listeners • Tagged services • Service definition takeover
  36. 36. Demo time 03
  37. 37. Going forward 04
  38. 38. Version 3 released today! https://github.com/kalipo-uk/ezmigrationbundle If the demo effect does not strike now…
  39. 39. Is your favourite feature missing? https://github.com/kalipo-uk/ezmigrationbundle The first brick is laid, many scenarios are possible ex: allow a full ETL process / migrate across eZ installations Feature requests and Bugs are managed on Github Pull Requests are welcome (thanks Lolautruche!)
  40. 40. THANK YOU @gggeek https://github.com/kaliop-uk https://github.com/kaliop http://www.kaliop.co.uk/info/ez.html

×