Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Liquibase for java
developers
By Illia Seleznov
Who am I?
Lead Software Engineer at EPAM Systems
More than 7 years in commercial java development
2 project from scratch t...
Why me?
Working with liquibase since 2014
Have used liquibase on 3 projects
Implemented liquibase on 1 project
Liquibase i...
Agenda
How do we work with DB? What do we really need?
Liquibase artifacts
Liquibase commands
Liquibase with maven
Liquiba...
First DB - first decisions
DB script management
DAO testing
Deployment
Types of DB scripts
Scripts that change existing data or structure in DB
Scripts that change existing logic(procedures, vi...
DB management
Create one file with all sql scripts
Create separate files named 1.sql, 2.sql….
Use spring.jpa.hibernate.ddl...
DAO tests
Run scripts on test DB
Use existing database, populate it with test data before
test running and clean it after ...
Deployment
Simple(stop -> backup -> update -> start)
Simple replication
Replication and zero time deployment
Replication
Database migration tools
Liquibase is open sources
Founded in 2006 year
Author is Nathan Voxland
Github: https://github.com/liquibase
Changesets
Changelog
File structure
/db-migrations
/v-1.0
/2013-03-02--01-initial-schema-import.xml
/2013-03-02--02-core-data.xml
/2013-03-04--...
ChangeLog formats
XML
JSON
YAML
SQL
Liquibase is just a jar file
liquibase [options] [command] [command parameters]
Example:
java -jar liquibase.jar 
--driver...
Liquibase.properties file
1.Create liquibase.properties file near liquibase.jar
2.Add all connection data to this file
dri...
Easy command line
java -jar liquibase.jar --changeLogFile=changelogs/yaml/master.yaml update
java -jar liquibase.jar --cha...
Liquibase system tables
Changests
The changeSet tag is what you use to group database
changes/refactorings together.
Example
<databaseChangeLog>
<changeSet id="1" author="bob">
<comment>A sample change log</comment>
<createTable/>
</change...
Bundled Changes
ADD AUTO INCREMENT
ADD COLUMN
ADD DEFAULT VALUE
ADD FOREIGN KEY CONSTRAINT
ADD LOOKUP TABLE
ADD NOT NULL C...
Load data
<loadData encoding="UTF-8" file="config/liquibase/users.csv"
separator=";" tableName="jhi_user">
<column name="a...
SQL also here
SQL as changelog file
Include sql file
SQL tag in changeset
SQL as changelog file
--liquibase formatted sql
--changeset seleznov:1
create table test1 (
id int primary key,
name varch...
SQL as file
<changeSet id="do_smth" author="IlliaSeleznov">
<sqlFile encoding="utf8"
endDelimiter="/"
path="sql/do_smth.sq...
SQL as part of changeset
<changeSet id="drop_storage_with_index_data" author="Illia_Seleznov" dbms="oracle">
<sql splitSta...
Custom change
<customChange class="com.seleznov.liquibase.example.CustomProcessor">
<param name="relativePath" value="exam...
Changeset attributes
id
author
runAlways
runOnChange
context
failOnError
Contexts
context=”!test”
context=”v1.0 or map”
context=”!qa and !master”
“test, qa” is the same as “test OR qa”
“test, qa ...
Why does not devops sleep?
Sub-tags
comments
preConditions
validCheckSum(not recommended)
rollback
Preconditions
<changeSet id="1" author="bob">
<preConditions onError="MARK_RAN">
<tableExists tableName="angry_devops"/>
<...
AND/OR/NOT Logic
<preConditions>
<or>
<and>
<dbms type="oracle" />
<runningAs username="SYSTEM" />
</and>
<and>
<dbms type...
Available Preconditions
<dbms>
<runningAs>
<changeSetExecuted>
<columnExists>
<tableExists>
<viewExists>
<foreignKeyConstr...
Liquibase commands
Update
Rollback
Diff
Documentation
Maintenance
Update commands
update - updates database to current version
updateCount <value> - applies the next <value> change sets
up...
Diff commands
diff [diff parameters] - writes description of differences to standard out
diffChangeLog [diff parameters] -...
Documentation commands
dbDoc <outputDirectory> - generates Javadoc-like
documentation based on current database and change...
Maintenance commands
tag <tag> - "tags" the current database state for future rollback.
tagExists <tag> - Checks whether t...
Rollback commands
rollback <tag> - rolls back the database to the state it was in when the tag was applied.
rollbackToDate...
RollBack
<changeset id="init-1" author="illia_seleznov">
<insert tablename="Person">
<column name="name" value="John Doe">...
Liquibase maven plugin
<plugin>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-maven-plugin</artifactId>
<version>...
Execute maven liquibase command
mvn liquibase:command
profile with liquibase plugin
Liquibase with Spring-boot
Add liquibase dependency to pom
Add master changelog location to properties
Something to read
https://liquibase.jira.com/wiki/display/CONTRIB/LiquiBa
se+Extensions+Portal
https://habrahabr.ru/post/1...
Contacts
https://github.com/manbe/liquibase-demo
manbe@mail.com
https://www.facebook.com/IlliaSeleznov
Liquibase for java developers
Upcoming SlideShare
Loading in …5
×

Liquibase for java developers

1,462 views

Published on

This presentation is about managing database scripts, why we need to do it from theoretical and practice perspective, how it improves continues integration and delivery process on real projects.

Published in: Engineering
  • Be the first to comment

Liquibase for java developers

  1. 1. Liquibase for java developers By Illia Seleznov
  2. 2. Who am I? Lead Software Engineer at EPAM Systems More than 7 years in commercial java development 2 project from scratch to production Speaker experience at Epam events, Logik Night, UADEVCLUB.
  3. 3. Why me? Working with liquibase since 2014 Have used liquibase on 3 projects Implemented liquibase on 1 project Liquibase is a part of my dream application
  4. 4. Agenda How do we work with DB? What do we really need? Liquibase artifacts Liquibase commands Liquibase with maven Liquibase with Spring Boot
  5. 5. First DB - first decisions DB script management DAO testing Deployment
  6. 6. Types of DB scripts Scripts that change existing data or structure in DB Scripts that change existing logic(procedures, views...)
  7. 7. DB management Create one file with all sql scripts Create separate files named 1.sql, 2.sql…. Use spring.jpa.hibernate.ddl-auto Custom tool DBA will find a solution
  8. 8. DAO tests Run scripts on test DB Use existing database, populate it with test data before test running and clean it after tests running Use embeded db with predefined test data Insert test data Remove test data after test
  9. 9. Deployment Simple(stop -> backup -> update -> start) Simple replication Replication and zero time deployment
  10. 10. Replication
  11. 11. Database migration tools
  12. 12. Liquibase is open sources Founded in 2006 year Author is Nathan Voxland Github: https://github.com/liquibase
  13. 13. Changesets
  14. 14. Changelog
  15. 15. File structure /db-migrations /v-1.0 /2013-03-02--01-initial-schema-import.xml /2013-03-02--02-core-data.xml /2013-03-04--01-notifications.xml /changelog-v.1.0-cumulative.xml /v-2.0 ... /changelog-v.2.0-cumulative.xml /changelog.xml
  16. 16. ChangeLog formats XML JSON YAML SQL
  17. 17. Liquibase is just a jar file liquibase [options] [command] [command parameters] Example: java -jar liquibase.jar --driver=oracle.jdbc.OracleDriver --classpath=pathtoclasses:jdbcdriver.jar --changeLogFile=com/example/db.changelog.xml --url="jdbc:oracle:thin:@localhost:1521:oracle" --username=scott --password=tiger update
  18. 18. Liquibase.properties file 1.Create liquibase.properties file near liquibase.jar 2.Add all connection data to this file driver: org.mariadb.jdbc.Driver url: jdbc:mariadb://localhost:3306/shop_db username: shop password: qwerty classpath: /home/illcko/liquibase/dbdrivers/mariadb-java-client-1.4.6.jar
  19. 19. Easy command line java -jar liquibase.jar --changeLogFile=changelogs/yaml/master.yaml update java -jar liquibase.jar --changeLogFile=changelogs/json/master.json dropAll java -jar liquibase.jar --changeLogFile=changelogs/xml/master.xml dropAll update
  20. 20. Liquibase system tables
  21. 21. Changests The changeSet tag is what you use to group database changes/refactorings together.
  22. 22. Example <databaseChangeLog> <changeSet id="1" author="bob"> <comment>A sample change log</comment> <createTable/> </changeSet> <changeSet id="2" author="bob" runAlways="true"> <alterTable/> <createIndex/> <addPrimaryKey/> </changeSet> </databaseChangeLog>
  23. 23. Bundled Changes ADD AUTO INCREMENT ADD COLUMN ADD DEFAULT VALUE ADD FOREIGN KEY CONSTRAINT ADD LOOKUP TABLE ADD NOT NULL CONSTRAINT ADD PRIMARY KEY ADD UNIQUE CONSTRAINT ALTER SEQUENCE CREATE INDEX CREATE PROCEDURE CREATE SEQUENCE CREATE TABLE CREATE VIEW CUSTOM CHANGE DELETE DROP ALL FOREIGN KEY CONSTRAINTS DROP COLUMN DROP DEFAULT VALUE DROP FOREIGN KEY CONSTRAINT DROP INDEX DROP NOT NULL CONSTRAINT DROP PRIMARY KEY DROP PROCEDURE DROP SEQUENCE DROP TABLE DROP UNIQUE CONSTRAINT DROP VIEW EMPTY EXECUTE COMMAND INSERT LOAD DATA LOAD UPDATE DATA MERGE COLUMNS MODIFY DATA TYPE RENAME COLUMN RENAME TABLE RENAME VIEW SQL SQL FILE STOP TAG DATABASE UPDATE
  24. 24. Load data <loadData encoding="UTF-8" file="config/liquibase/users.csv" separator=";" tableName="jhi_user"> <column name="activated" type="boolean"/> </loadData> id;login;PASSWORD;first_name;last_name;email;activated;lang_key;created_by 1;system;123;System;System;system@localhost;true;en;system 2;123;Anonymous;User;anonymous@localhost;true;en;system
  25. 25. SQL also here SQL as changelog file Include sql file SQL tag in changeset
  26. 26. SQL as changelog file --liquibase formatted sql --changeset seleznov:1 create table test1 ( id int primary key, name varchar(255) ); --rollback drop table test1; --changeset seleznov:2 insert into test1 (id, name) values (1, ‘name 1′); insert into test1 (id, name) values (2, ‘name 2′); --changeset seleznov:3 dbms:oracle create sequence seq_test;
  27. 27. SQL as file <changeSet id="do_smth" author="IlliaSeleznov"> <sqlFile encoding="utf8" endDelimiter="/" path="sql/do_smth.sql" relativeToChangelogFile="true" splitStatements="true" stripComments="true"/> </changeSet>
  28. 28. SQL as part of changeset <changeSet id="drop_storage_with_index_data" author="Illia_Seleznov" dbms="oracle"> <sql splitStatements="false"> <![CDATA[ DECLARE filter_count number; BEGIN select count(*) into filter_count FROM CTXSYS.CTX_PREFERENCES WHERE PRE_NAME = '<MNG_STORAGE>' AND PRE_OWNER in (select user from dual); IF filter_count > 0 THEN ctx_ddl.drop_preference( ''<MNG_STORAGE>'); END IF; END; ]]> </sql> </changeSet>
  29. 29. Custom change <customChange class="com.seleznov.liquibase.example.CustomProcessor"> <param name="relativePath" value="example.json"/> </customChange> interface CustomTaskChange extends CustomChange { String getConfirmationMessage(); void setUp() throws SetupException; void setFileOpener(ResourceAccessor var1); ValidationErrors validate(Database var1); void execute(Database var1) throws CustomChangeException; }
  30. 30. Changeset attributes id author runAlways runOnChange context failOnError
  31. 31. Contexts context=”!test” context=”v1.0 or map” context=”!qa and !master” “test, qa” is the same as “test OR qa” “test, qa and master” is the same as “(test) OR (qa and master)
  32. 32. Why does not devops sleep?
  33. 33. Sub-tags comments preConditions validCheckSum(not recommended) rollback
  34. 34. Preconditions <changeSet id="1" author="bob"> <preConditions onError="MARK_RAN"> <tableExists tableName="angry_devops"/> </preConditions> <comment>Comments should go after preCondition. If they are before then liquibase usually gives error.</comment> <createTable tableName="angry_devops"> <column name="angry" type="int"/> </createTable> </changeSet>
  35. 35. AND/OR/NOT Logic <preConditions> <or> <and> <dbms type="oracle" /> <runningAs username="SYSTEM" /> </and> <and> <dbms type="mssql" /> <runningAs username="sa" /> </and> </or> </preConditions>
  36. 36. Available Preconditions <dbms> <runningAs> <changeSetExecuted> <columnExists> <tableExists> <viewExists> <foreignKeyConstraintExists> <indexExists> <sequenceExists> <primaryKeyExists> <sqlCheck> <changeLogPropertyDefined> <customPrecondition>
  37. 37. Liquibase commands Update Rollback Diff Documentation Maintenance
  38. 38. Update commands update - updates database to current version updateCount <value> - applies the next <value> change sets updateSQL - writes SQL to update database to current version to STDOUT updateCountSQL <value> - writes SQL to apply the next <value> change sets to STDOUT
  39. 39. Diff commands diff [diff parameters] - writes description of differences to standard out diffChangeLog [diff parameters] - writes Change Log XML to update the base database to the target database to standard out Include: ● Version Differences ● Missing/unexpected tables ● Missing/unexpected views ● Missing/unexpected columns ● Missing/unexpected primary keys ● Missing/unexpected unique constraints ● Missing/unexpected foreign Keys ● Missing/unexpected sequences ● Missing/unexpected indexes ● Column definition differences (data type, auto-increment, etc.) ● View definition differences ● Data differences (limited), not checked by default Exclude: ● Non-foreign key constraints (check, etc) ● Stored Procedures ● Data type length
  40. 40. Documentation commands dbDoc <outputDirectory> - generates Javadoc-like documentation based on current database and change log. java -jar liquibase.jar --changeLogFile=changelogs/xml/master.xml dbDoc ../doc
  41. 41. Maintenance commands tag <tag> - "tags" the current database state for future rollback. tagExists <tag> - Checks whether the given tag is already existing. status - Outputs count (list if --verbose) of unrun change sets. validate - checks the changelog for errors. changelogSync - mark all changes as executed in the database. changelogSyncSQL - writes SQL to mark all changes as executed in the database to STDOUT. markNextChangeSetRan - mark the next change set as executed in the database. listLocks - lists who currently has locks on the database changelog. releaseLocks - Releases all locks on the database changelog. dropAll - Drops all database objects owned by the user. Note that functions, procedures and packages are not dropped (limitation in 1.8.1). clearCheckSums - Removes current checksums from database. On next run checksums will be recomputed. generateChangeLog - generateChangeLog of the database to standard out. v1.8 requires the dataDir parameter currently.
  42. 42. Rollback commands rollback <tag> - rolls back the database to the state it was in when the tag was applied. rollbackToDate <date/time> - rolls back the database to the state it was in at the given date/time. rollbackCount <value> - rolls back the last <value> change sets. rollbackSQL <tag> - writes SQL to roll back the database to the state it was in when the tag was applied to STDOUT. rollbackToDateSQL <date/time> - writes SQL to roll back the database to the state it was in at the given date/time version to STDOUT. rollbackCountSQL <value> - writes SQL to roll back the last <value> change sets to STDOUT. futureRollbackSQL - writes SQL to roll back the database to the current state after the changes in the changeslog have been applied. updateTestingRollback - updates the database, then rolls back changes before updating again.
  43. 43. RollBack <changeset id="init-1" author="illia_seleznov"> <insert tablename="Person"> <column name="name" value="John Doe"> </column> </insert> <rollback> DELETE FROM Person WHERE name LIKE 'John Doe'; </rollback> </changeset>
  44. 44. Liquibase maven plugin <plugin> <groupId>org.liquibase</groupId> <artifactId>liquibase-maven-plugin</artifactId> <version>3.0.5</version> <configuration> <propertyFile>src/main/resources/liquibase/liquibase.properties</propertyFile> </configuration> <executions> <execution> <phase>process-resources</phase> <goals> <goal>update</goal> </goals> </execution> </executions> </plugin>
  45. 45. Execute maven liquibase command mvn liquibase:command profile with liquibase plugin
  46. 46. Liquibase with Spring-boot Add liquibase dependency to pom Add master changelog location to properties
  47. 47. Something to read https://liquibase.jira.com/wiki/display/CONTRIB/LiquiBa se+Extensions+Portal https://habrahabr.ru/post/178665/ https://habrahabr.ru/post/179425/ https://habrahabr.ru/post/251617/
  48. 48. Contacts https://github.com/manbe/liquibase-demo manbe@mail.com https://www.facebook.com/IlliaSeleznov

×