Your SlideShare is downloading. ×
0
Database Change Management
Roger Nilsson, Altran
Rikard Thulin, Squeed
Rikard Thulin
Has been working as a software engineer for 17 years,
mostly as an consultant with focus on the Java platfor...
Questions?
Questions during the presentation is
encouraged
There will be a Q&A if time permits at
the end of the presentat...
The Question?
You never develop code without version
control,
why would you develop your
database without it?
Liquibase at a glance
Liquibase is an Apache 2.0 Licensed tool for
tracking, managing and applying database
changes
Databa...
Major Concepts - Changeset
A changeset is uniquely identified change that
should be applied to the database
When Liquibase...
Major Concepts - Changeset
Major Concepts - Changelog file
The changelog file contains Change Sets or
references to other Changelog files
Changelog f...
Major Concepts - Changelog file
<databaseChangeLog>
<include file="changelogs/jira-937.xml"/>
<include file="changelogs/ji...
Major Concepts - Changes
Each changeset generally contains a change
which describes the change/refactoring to apply
to the...
Major Concepts - Changeset
Major Concepts - Preconditions
Preconditions can be applied to either the
changelog as a whole or individual change sets
c...
Major Concepts - Contexts
Contexts can be applied to changesets to
control which are ran in different environments
For exa...
● Update rollbacks
● Database ”diff“s
● Generating starting change logs from
existing databases
● Generating database chan...
Database Change Management
Database deltas are part of the code change
The correlation between code revisions are
ensured
...
Why Liquibase?
Source Code
Why Liquibase?
8 developers with 8 local databases/schemas
Development server
Test / QA server
CI Server
Staging Server No...
Why Liquibase?
8 developers with 8 local databases/schemas
Development server
Test / QA server
CI Server
Staging Server No...
The four steps
1. Write your database change set
2. Run Liquibase locally to test the SQL
3. Commit your source code and c...
Structural Refactorings
● Add Column
● Rename Column
● Modify Column
● Drop Column
● Alter Sequence
● Create Table
● Renam...
Running Liquibase
● Manual
○ Command Line
○ Ant
○ Maven
● Automatic
○ Spring
○ Servlet Listener
○ CDI Environment
Tips and Tricks - real life experience
You must have a manual way to run Liquibase
(command line, ant or maven) during
dev...
Demo
1. Update Database with pending changesets
2. Rollback previous changes
Tips and tricks - structure
Think about the structure of your changelogs
Based on experience, when the changelog xml file ...
Tips and tricks - structure
Liquibase should foremost be used for structural changes
but it is also possible to use it for...
Tips and tricks - structure
● changelog.xml for structural changes, applied when
the system is deployed
● test-data.xml fo...
When executing update or rollback there is a
handy option which do not apply changes
against the database but instead save...
Generating RFC to DBA automatically using Jenkins
REQUEST FOR CHANGE for sqlserver://sqlprod
Planning:
1. Changeset esc-17...
Tips and tricks - wrong is right
One of the most difficult challenges is to
overcome “wrong is right” principle
● When a c...
Tips and tricks - stored procedure
If you have stored procedures the should
obviously be versioned controlled
But the CRC ...
Tips and tricks - stored procedures
--liquibase formatted sql
--changeset rikard:1 runOnChange:true
ALTER procedure javafo...
Demo
Stored procedure
1. Update database with plain SQL file
2. Change procedure and update again..
Tips and Tricks - neutralizing data
● The “run always” attribute can also be used
to neutralize data in dev and test
envir...
Tips and Tricks - neutralizing data
<changeSet id="1" author="joe" runAlways="true"
context="sqld,sqlt,localhost">
<commen...
Real life experience
We have used Liquibase in projects for many years, more
than 50 man years of development with very li...
Alternatives
● Flyway
● c5-db-migration
● dbdeploy
● mybatis
● MIGRATEdb
● migrate4j
● dbmaintain
● AutoPatch
Q & A
Upcoming SlideShare
Loading in...5
×

Liquibase få kontroll på dina databasförändringar

1,606

Published on

You never develop code without version control, why do you develop your database without it? With Liquibase, database changes are stored in human XML-files and committed to the source control system. Changes are applied to the developers local databases. As changes are committed they are distributed to all other environments including all developers local databases, test databases, staging databases, and even to production databases. This presentation will introduce you to Liquibase and the topic database change management. We will also present some advanced topics based on real life experience and a few tips and tricks as well
Rikard Thulin, Squeed and Roger Nilsson, Altran

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,606
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Liquibase få kontroll på dina databasförändringar"

  1. 1. Database Change Management Roger Nilsson, Altran Rikard Thulin, Squeed
  2. 2. Rikard Thulin Has been working as a software engineer for 17 years, mostly as an consultant with focus on the Java platform e-mail: rikard.thulin@squeed.com, twitter: @rikard_thulin, blog: http://squeed.com/blog Roger Nilsson Has been working as a software engineer and consultant for 15 years, with focus on the Java platform e-mail: roger.nilsson@altran.com About the Presenters
  3. 3. Questions? Questions during the presentation is encouraged There will be a Q&A if time permits at the end of the presentation
  4. 4. The Question? You never develop code without version control, why would you develop your database without it?
  5. 5. Liquibase at a glance Liquibase is an Apache 2.0 Licensed tool for tracking, managing and applying database changes Database changes are stored in an XML (or JSON, YAML, plain-ish SQL) file that is version controlled and database independent Liquibase managed and applies changes to the database
  6. 6. Major Concepts - Changeset A changeset is uniquely identified change that should be applied to the database When Liquibase runs, it queries the DATABASECHANGELOG table for the changesets that are marked as executed and then executes all changesets that have not yet been executed
  7. 7. Major Concepts - Changeset
  8. 8. Major Concepts - Changelog file The changelog file contains Change Sets or references to other Changelog files Changelog files can be be arbitrarily nested for better management
  9. 9. Major Concepts - Changelog file <databaseChangeLog> <include file="changelogs/jira-937.xml"/> <include file="changelogs/jira-1023.xml"/> <include file="changelogs/jira-1104.xml"/> <!-- Stored Procedures --> <include file="procedures/uspUpdateSearchIndex.sql"/> </databaseChangeLog>
  10. 10. Major Concepts - Changes Each changeset generally contains a change which describes the change/refactoring to apply to the database Liquibase supports both descriptive changes that generate SQL for supported databases and raw SQL
  11. 11. Major Concepts - Changeset
  12. 12. Major Concepts - Preconditions Preconditions can be applied to either the changelog as a whole or individual change sets columnExists, tableExists, viewExists, foreignKeyConstraintExists, sqlCheck, and more... If a precondition fails, Liquibase will stop execution
  13. 13. Major Concepts - Contexts Contexts can be applied to changesets to control which are ran in different environments For example, some changesets can be tagged as "production" and others as "test" Use Contexts only when there is a good reason
  14. 14. ● Update rollbacks ● Database ”diff“s ● Generating starting change logs from existing databases ● Generating database change documentation ● Code branches and merging Liquibase feature set
  15. 15. Database Change Management Database deltas are part of the code change The correlation between code revisions are ensured Allows the automatisation of database changes Keeps all environments consistent Allows reverting a previous version of a system
  16. 16. Why Liquibase? Source Code
  17. 17. Why Liquibase? 8 developers with 8 local databases/schemas Development server Test / QA server CI Server Staging Server Node 1 Production Node 1 Staging Server Node 2 Production Node 2
  18. 18. Why Liquibase? 8 developers with 8 local databases/schemas Development server Test / QA server CI Server Staging Server Node 1 Production Node 1 Staging Server Node 2 Production Node 2
  19. 19. The four steps 1. Write your database change set 2. Run Liquibase locally to test the SQL 3. Commit your source code and change set 4. Deploy application and database changes Liquibase plays well when merging and working with branches
  20. 20. Structural Refactorings ● Add Column ● Rename Column ● Modify Column ● Drop Column ● Alter Sequence ● Create Table ● Rename Table ● Drop Table ● Create View ● Rename View ● Drop View ● Merge Columns ● Create Stored Procedure Data Quality Refactorings ● Add Lookup Table ● Add Not-Null Constraint ● Remove Not-Null Constraint ● Add Unique Constraint ● Drop Unique Constraint ● Create Sequence ● Drop Sequence ● Add Auto-Increment ● Add Default Value ● Drop Default Value Referential Integrity Refactorings ● Add Foreign Key Constraint ● Drop Foreign Key Constraint ● Drop All Foreign Key Constraints ● Add Primary Key Constraint ● Drop Primary Key Constraint Non-Refactoring Transformations ● Insert Data ● Load Data ● Load Update Data ● Update Data ● Delete Data ● Tag Database ● Stop Architectural Refactorings ● Create Index ● Drop Index Custom Refactorings ● Modifying Generated SQL ● Custom SQL ● Custom SQL File ● Custom Refactoring Class ● Execute Shell Command Supported Refactorings
  21. 21. Running Liquibase ● Manual ○ Command Line ○ Ant ○ Maven ● Automatic ○ Spring ○ Servlet Listener ○ CDI Environment
  22. 22. Tips and Tricks - real life experience You must have a manual way to run Liquibase (command line, ant or maven) during development of changesets Bundle your database changes into your deployment artifact (spring, servlet listener or CDI) It is (mentally) scary to do, but it works!
  23. 23. Demo 1. Update Database with pending changesets 2. Rollback previous changes
  24. 24. Tips and tricks - structure Think about the structure of your changelogs Based on experience, when the changelog xml file is 10.000 lines long it is gets ugly. It is much more difficult when merging and branching. Have one master changelog that includes smaller changelogs in turn, divided by release or issue changelog.xml changelog.xml jira-1.xml changelog-1_0.xml jira-4.xml changelog-1_1.xml
  25. 25. Tips and tricks - structure Liquibase should foremost be used for structural changes but it is also possible to use it for configuration data and for test data Keep in mind that the use case for these are very different! ● Structural changes should be applied together with the corresponding code ● Configuration data has en dependency to the database structure, but is not applied at any time ● Test data has a dependency to the database structure, may have preconditions and is typically applied by a CI server nightly
  26. 26. Tips and tricks - structure ● changelog.xml for structural changes, applied when the system is deployed ● test-data.xml for test data, applied nightly be CI server ● configuration-data.xml for system configuration changes, applied manually when needed
  27. 27. When executing update or rollback there is a handy option which do not apply changes against the database but instead save the resulting SQL This can be usable when* ● the resulting SQL needs to be tweeked ● having the SQL approved by a DBA ● SOX compliance. *) Not a recommended best practices by the authors Advanced Topics SQL Output
  28. 28. Generating RFC to DBA automatically using Jenkins REQUEST FOR CHANGE for sqlserver://sqlprod Planning: 1. Changeset esc-1729.xml::1::rikard::(Checksum: 3:be601b760eaccc5d864da5b3639bb031) Reviewed by: rikard Tested in sqldev, 2013-06-17 23:45:46, OK | Insert Column, New attribute for customer Tested in sqltest, 2013-06-18 12:10:01, OK | Insert Column, New attribute for customer Tested in sqlprod , 2013-09-14 20:57:44, OK | Insert Column, New attribute for customer Implementation: Run script using Jenkins deploy server Implementation & Review: Executed by Jenkins/CI-Server, 2013-09-14 20:57 Scripts successfully applied Advanced Topics SQL Output - real life example
  29. 29. Tips and tricks - wrong is right One of the most difficult challenges is to overcome “wrong is right” principle ● When a changeset has been committed, it can not be modified even if it is “wrong” ● You must write another changeset to correct the previous one ● The “wrong” changeset will hit your production database followed by the “right” changeset
  30. 30. Tips and tricks - stored procedure If you have stored procedures the should obviously be versioned controlled But the CRC does not fit well as that requires the stored procedure to be repeated for every modification The “run always” attribute saves the day. This will make sure that whenever there is a change it is applied -- a better fit for stored procedures
  31. 31. Tips and tricks - stored procedures --liquibase formatted sql --changeset rikard:1 runOnChange:true ALTER procedure javaforum … ● Above we use a plain sql file with two special comments that magically makes it a changeset
  32. 32. Demo Stored procedure 1. Update database with plain SQL file 2. Change procedure and update again..
  33. 33. Tips and Tricks - neutralizing data ● The “run always” attribute can also be used to neutralize data in dev and test environments ● Useful in cases when the database is copied for production to other environments ● The changes that has been applied is stored in the database itself, therefore it is safe to copy clone a database
  34. 34. Tips and Tricks - neutralizing data <changeSet id="1" author="joe" runAlways="true" context="sqld,sqlt,localhost"> <comment>Change users psw on (localhost, dev, test) to 'Secret' and their email to 'test@jforum.se'</comment> <update tableName="user"> <column name="email" value="test@jforum.se"/> <column name="password" value="Secret"/> <where>email != 'test@jforum.se' OR password != 'Secret'</where> </update> </changeSet>
  35. 35. Real life experience We have used Liquibase in projects for many years, more than 50 man years of development with very little problems Overall score 97% by Roger and Rikard
  36. 36. Alternatives ● Flyway ● c5-db-migration ● dbdeploy ● mybatis ● MIGRATEdb ● migrate4j ● dbmaintain ● AutoPatch
  37. 37. Q & A
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×