Liquibase få kontroll på dina databasförändringar


Published on

You never develop code without version control, why do you develop your database without it? With Liquibase, database changes are stored in human XML-files and committed to the source control system. Changes are applied to the developers local databases. As changes are committed they are distributed to all other environments including all developers local databases, test databases, staging databases, and even to production databases. This presentation will introduce you to Liquibase and the topic database change management. We will also present some advanced topics based on real life experience and a few tips and tricks as well
Rikard Thulin, Squeed and Roger Nilsson, Altran

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Liquibase få kontroll på dina databasförändringar

  1. 1. Database Change Management Roger Nilsson, Altran Rikard Thulin, Squeed
  2. 2. Rikard Thulin Has been working as a software engineer for 17 years, mostly as an consultant with focus on the Java platform e-mail:, twitter: @rikard_thulin, blog: Roger Nilsson Has been working as a software engineer and consultant for 15 years, with focus on the Java platform e-mail: About the Presenters
  3. 3. Questions? Questions during the presentation is encouraged There will be a Q&A if time permits at the end of the presentation
  4. 4. The Question? You never develop code without version control, why would you develop your database without it?
  5. 5. Liquibase at a glance Liquibase is an Apache 2.0 Licensed tool for tracking, managing and applying database changes Database changes are stored in an XML (or JSON, YAML, plain-ish SQL) file that is version controlled and database independent Liquibase managed and applies changes to the database
  6. 6. Major Concepts - Changeset A changeset is uniquely identified change that should be applied to the database When Liquibase runs, it queries the DATABASECHANGELOG table for the changesets that are marked as executed and then executes all changesets that have not yet been executed
  7. 7. Major Concepts - Changeset
  8. 8. Major Concepts - Changelog file The changelog file contains Change Sets or references to other Changelog files Changelog files can be be arbitrarily nested for better management
  9. 9. Major Concepts - Changelog file <databaseChangeLog> <include file="changelogs/jira-937.xml"/> <include file="changelogs/jira-1023.xml"/> <include file="changelogs/jira-1104.xml"/> <!-- Stored Procedures --> <include file="procedures/uspUpdateSearchIndex.sql"/> </databaseChangeLog>
  10. 10. Major Concepts - Changes Each changeset generally contains a change which describes the change/refactoring to apply to the database Liquibase supports both descriptive changes that generate SQL for supported databases and raw SQL
  11. 11. Major Concepts - Changeset
  12. 12. Major Concepts - Preconditions Preconditions can be applied to either the changelog as a whole or individual change sets columnExists, tableExists, viewExists, foreignKeyConstraintExists, sqlCheck, and more... If a precondition fails, Liquibase will stop execution
  13. 13. Major Concepts - Contexts Contexts can be applied to changesets to control which are ran in different environments For example, some changesets can be tagged as "production" and others as "test" Use Contexts only when there is a good reason
  14. 14. ● Update rollbacks ● Database ”diff“s ● Generating starting change logs from existing databases ● Generating database change documentation ● Code branches and merging Liquibase feature set
  15. 15. Database Change Management Database deltas are part of the code change The correlation between code revisions are ensured Allows the automatisation of database changes Keeps all environments consistent Allows reverting a previous version of a system
  16. 16. Why Liquibase? Source Code
  17. 17. Why Liquibase? 8 developers with 8 local databases/schemas Development server Test / QA server CI Server Staging Server Node 1 Production Node 1 Staging Server Node 2 Production Node 2
  18. 18. Why Liquibase? 8 developers with 8 local databases/schemas Development server Test / QA server CI Server Staging Server Node 1 Production Node 1 Staging Server Node 2 Production Node 2
  19. 19. The four steps 1. Write your database change set 2. Run Liquibase locally to test the SQL 3. Commit your source code and change set 4. Deploy application and database changes Liquibase plays well when merging and working with branches
  20. 20. Structural Refactorings ● Add Column ● Rename Column ● Modify Column ● Drop Column ● Alter Sequence ● Create Table ● Rename Table ● Drop Table ● Create View ● Rename View ● Drop View ● Merge Columns ● Create Stored Procedure Data Quality Refactorings ● Add Lookup Table ● Add Not-Null Constraint ● Remove Not-Null Constraint ● Add Unique Constraint ● Drop Unique Constraint ● Create Sequence ● Drop Sequence ● Add Auto-Increment ● Add Default Value ● Drop Default Value Referential Integrity Refactorings ● Add Foreign Key Constraint ● Drop Foreign Key Constraint ● Drop All Foreign Key Constraints ● Add Primary Key Constraint ● Drop Primary Key Constraint Non-Refactoring Transformations ● Insert Data ● Load Data ● Load Update Data ● Update Data ● Delete Data ● Tag Database ● Stop Architectural Refactorings ● Create Index ● Drop Index Custom Refactorings ● Modifying Generated SQL ● Custom SQL ● Custom SQL File ● Custom Refactoring Class ● Execute Shell Command Supported Refactorings
  21. 21. Running Liquibase ● Manual ○ Command Line ○ Ant ○ Maven ● Automatic ○ Spring ○ Servlet Listener ○ CDI Environment
  22. 22. Tips and Tricks - real life experience You must have a manual way to run Liquibase (command line, ant or maven) during development of changesets Bundle your database changes into your deployment artifact (spring, servlet listener or CDI) It is (mentally) scary to do, but it works!
  23. 23. Demo 1. Update Database with pending changesets 2. Rollback previous changes
  24. 24. Tips and tricks - structure Think about the structure of your changelogs Based on experience, when the changelog xml file is 10.000 lines long it is gets ugly. It is much more difficult when merging and branching. Have one master changelog that includes smaller changelogs in turn, divided by release or issue changelog.xml changelog.xml jira-1.xml changelog-1_0.xml jira-4.xml changelog-1_1.xml
  25. 25. Tips and tricks - structure Liquibase should foremost be used for structural changes but it is also possible to use it for configuration data and for test data Keep in mind that the use case for these are very different! ● Structural changes should be applied together with the corresponding code ● Configuration data has en dependency to the database structure, but is not applied at any time ● Test data has a dependency to the database structure, may have preconditions and is typically applied by a CI server nightly
  26. 26. Tips and tricks - structure ● changelog.xml for structural changes, applied when the system is deployed ● test-data.xml for test data, applied nightly be CI server ● configuration-data.xml for system configuration changes, applied manually when needed
  27. 27. When executing update or rollback there is a handy option which do not apply changes against the database but instead save the resulting SQL This can be usable when* ● the resulting SQL needs to be tweeked ● having the SQL approved by a DBA ● SOX compliance. *) Not a recommended best practices by the authors Advanced Topics SQL Output
  28. 28. Generating RFC to DBA automatically using Jenkins REQUEST FOR CHANGE for sqlserver://sqlprod Planning: 1. Changeset esc-1729.xml::1::rikard::(Checksum: 3:be601b760eaccc5d864da5b3639bb031) Reviewed by: rikard Tested in sqldev, 2013-06-17 23:45:46, OK | Insert Column, New attribute for customer Tested in sqltest, 2013-06-18 12:10:01, OK | Insert Column, New attribute for customer Tested in sqlprod , 2013-09-14 20:57:44, OK | Insert Column, New attribute for customer Implementation: Run script using Jenkins deploy server Implementation & Review: Executed by Jenkins/CI-Server, 2013-09-14 20:57 Scripts successfully applied Advanced Topics SQL Output - real life example
  29. 29. Tips and tricks - wrong is right One of the most difficult challenges is to overcome “wrong is right” principle ● When a changeset has been committed, it can not be modified even if it is “wrong” ● You must write another changeset to correct the previous one ● The “wrong” changeset will hit your production database followed by the “right” changeset
  30. 30. Tips and tricks - stored procedure If you have stored procedures the should obviously be versioned controlled But the CRC does not fit well as that requires the stored procedure to be repeated for every modification The “run always” attribute saves the day. This will make sure that whenever there is a change it is applied -- a better fit for stored procedures
  31. 31. Tips and tricks - stored procedures --liquibase formatted sql --changeset rikard:1 runOnChange:true ALTER procedure javaforum … ● Above we use a plain sql file with two special comments that magically makes it a changeset
  32. 32. Demo Stored procedure 1. Update database with plain SQL file 2. Change procedure and update again..
  33. 33. Tips and Tricks - neutralizing data ● The “run always” attribute can also be used to neutralize data in dev and test environments ● Useful in cases when the database is copied for production to other environments ● The changes that has been applied is stored in the database itself, therefore it is safe to copy clone a database
  34. 34. Tips and Tricks - neutralizing data <changeSet id="1" author="joe" runAlways="true" context="sqld,sqlt,localhost"> <comment>Change users psw on (localhost, dev, test) to 'Secret' and their email to ''</comment> <update tableName="user"> <column name="email" value=""/> <column name="password" value="Secret"/> <where>email != '' OR password != 'Secret'</where> </update> </changeSet>
  35. 35. Real life experience We have used Liquibase in projects for many years, more than 50 man years of development with very little problems Overall score 97% by Roger and Rikard
  36. 36. Alternatives ● Flyway ● c5-db-migration ● dbdeploy ● mybatis ● MIGRATEdb ● migrate4j ● dbmaintain ● AutoPatch
  37. 37. Q & A