SlideShare a Scribd company logo
Database Migrations
What's the problem?
Provide upgrades of applications and databases to our
customers
Some customers require SQL scripts to run via DB admin
tools, e.g. Oracle (sqlplus)
We don't have database schemas that reflect what's been
applied in terms of updates - always the newest
We don't really understand what Hibernate is doing to
create/modify tables based on our persistent objects, we
rely on it for optimization, indexes etc...
Database Migrations
Structure SQL scripts for creation of the
schema, updates of the schema accordingly
Use a tool to run these migrations:
● In a specific order
● Against a specific database (MySQL,
Oracle)
● Recording the actions in a schema version
table
Lots of tools that do this
Flyway
Allows us to do migrations:
● From a shell script
● Programmatically
● When Spring starts
● Using Maven as a build tool
Introducing Flyway into your project
Has been done for MIF - there's a dbtool folder
in the source structure. We can provide a
template in SVN that can be used.
1. Download the dbtool structure
2. Generate an export of the schema that
Hibernate has created for your project
Introducing Flyway into your project
3. Tweak/optimise if necessary
4. Run the following command:
./flyway.sh init
Introducing Flyway into your project
This creates the SCHEMA_VERSION table:
mysql> desc SCHEMA_VERSION;
+-----------------+--------------+------+-----+-------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+--------------+------+-----+-------------------+-------+
| version | varchar(20) | NO | PRI | NULL | |
| description | varchar(100) | YES | | NULL | |
| type | varchar(10) | NO | | NULL | |
| script | varchar(200) | NO | UNI | NULL | |
| checksum | int(11) | YES | | NULL | |
| installed_by | varchar(30) | NO | | NULL | |
| installed_on | timestamp | NO | | CURRENT_TIMESTAMP | |
| execution_time | int(11) | YES | | NULL | |
| state | varchar(15) | NO | | NULL | |
| current_version | tinyint(1) | NO | MUL | NULL | |
+-----------------+--------------+------+-----+-------------------+-------+
Performing a migration
Assume you've added a new script in the
<dbname>/sql folder of dbtool
Run:
./flyway.sh migrate
This will run the script and record the action in
the SCHEMA_VERSION table.
Flyway dbtool structure
├── mysql
│ ├── conf
│ │ └── flyway.properties
│ ├── flyway.cmd
│ ├── flyway.sh
│ ├── jars
│ │ └── mysql-connector-java-5.1.14.jar
│ └── sql
│ ├── V1__MIFBaseSchema.sql
│ └── V2__JOB.DATE_ACKNOWLEDGED.sql
└── oracle
├── conf
│ └── flyway.properties
├── flyway.cmd
├── flyway.sh
├── jars
│ └── ojdbc14-10.2.0.5.jar
└── sql
Process Change
Need to change the way we develop persistent
objects with Hibernate
As well as changing an object we also need to
provide a migration script (and a rollback script)
Put it in the <dbname>/sql folder.
ls -Al mysql/sql
V1__MIFBaseSchema.sql
V2__JOB.DATE_ACKNOWLEDGED.sql
Example
Change a persistent object to add a new field
Write a script with the appropriate ALTER
TABLE ADD COLUMN statement.
Write the rollback script
Test

More Related Content

What's hot

Microservices Design Patterns | Edureka
Microservices Design Patterns | EdurekaMicroservices Design Patterns | Edureka
Microservices Design Patterns | Edureka
Edureka!
 
Java Spring Framework
Java Spring FrameworkJava Spring Framework
Java Spring Framework
Mehul Jariwala
 
Single Page Applications
Single Page ApplicationsSingle Page Applications
Single Page Applications
Massimo Iacolare
 
State management in react applications (Statecharts)
State management in react applications (Statecharts)State management in react applications (Statecharts)
State management in react applications (Statecharts)
Tomáš Drenčák
 
Web Development with Laravel 5
Web Development with Laravel 5Web Development with Laravel 5
Web Development with Laravel 5
Soheil Khodayari
 
Introduction to spring boot
Introduction to spring bootIntroduction to spring boot
Introduction to spring boot
Santosh Kumar Kar
 
Flyway _ A Database Version Management Tool
Flyway _ A Database Version Management ToolFlyway _ A Database Version Management Tool
Flyway _ A Database Version Management Tool
Knoldus Inc.
 
Maven Introduction
Maven IntroductionMaven Introduction
Maven Introduction
Sandeep Chawla
 
The virtual DOM and how react uses it internally
The virtual DOM and how react uses it internallyThe virtual DOM and how react uses it internally
The virtual DOM and how react uses it internally
Clóvis Neto
 
OpenAPI 3.0, And What It Means for the Future of Swagger
OpenAPI 3.0, And What It Means for the Future of SwaggerOpenAPI 3.0, And What It Means for the Future of Swagger
OpenAPI 3.0, And What It Means for the Future of Swagger
SmartBear
 
톰캣 운영 노하우
톰캣 운영 노하우톰캣 운영 노하우
톰캣 운영 노하우
jieunsys
 
Maven Basics - Explained
Maven Basics - ExplainedMaven Basics - Explained
Maven Basics - Explained
Smita Prasad
 
Spring batch overivew
Spring batch overivewSpring batch overivew
Spring batch overivew
Chanyeong Choi
 
Swagger UI
Swagger UISwagger UI
Swagger UI
Walaa Hamdy Assy
 
Vue.js
Vue.jsVue.js
Spring Boot Tutorial
Spring Boot TutorialSpring Boot Tutorial
Spring Boot Tutorial
Naphachara Rattanawilai
 
Angular tutorial
Angular tutorialAngular tutorial
Angular tutorial
Rohit Gupta
 
React JS
React JSReact JS

What's hot (20)

Microservices Design Patterns | Edureka
Microservices Design Patterns | EdurekaMicroservices Design Patterns | Edureka
Microservices Design Patterns | Edureka
 
Maven Overview
Maven OverviewMaven Overview
Maven Overview
 
Java Spring Framework
Java Spring FrameworkJava Spring Framework
Java Spring Framework
 
Single Page Applications
Single Page ApplicationsSingle Page Applications
Single Page Applications
 
State management in react applications (Statecharts)
State management in react applications (Statecharts)State management in react applications (Statecharts)
State management in react applications (Statecharts)
 
Web Development with Laravel 5
Web Development with Laravel 5Web Development with Laravel 5
Web Development with Laravel 5
 
Introduction to spring boot
Introduction to spring bootIntroduction to spring boot
Introduction to spring boot
 
Flyway _ A Database Version Management Tool
Flyway _ A Database Version Management ToolFlyway _ A Database Version Management Tool
Flyway _ A Database Version Management Tool
 
Maven Introduction
Maven IntroductionMaven Introduction
Maven Introduction
 
The virtual DOM and how react uses it internally
The virtual DOM and how react uses it internallyThe virtual DOM and how react uses it internally
The virtual DOM and how react uses it internally
 
OpenAPI 3.0, And What It Means for the Future of Swagger
OpenAPI 3.0, And What It Means for the Future of SwaggerOpenAPI 3.0, And What It Means for the Future of Swagger
OpenAPI 3.0, And What It Means for the Future of Swagger
 
Web workers
Web workersWeb workers
Web workers
 
톰캣 운영 노하우
톰캣 운영 노하우톰캣 운영 노하우
톰캣 운영 노하우
 
Maven Basics - Explained
Maven Basics - ExplainedMaven Basics - Explained
Maven Basics - Explained
 
Spring batch overivew
Spring batch overivewSpring batch overivew
Spring batch overivew
 
Swagger UI
Swagger UISwagger UI
Swagger UI
 
Vue.js
Vue.jsVue.js
Vue.js
 
Spring Boot Tutorial
Spring Boot TutorialSpring Boot Tutorial
Spring Boot Tutorial
 
Angular tutorial
Angular tutorialAngular tutorial
Angular tutorial
 
React JS
React JSReact JS
React JS
 

Viewers also liked

Flyway
FlywayFlyway
Database migrations with Flyway and Liquibase
Database migrations with Flyway and LiquibaseDatabase migrations with Flyway and Liquibase
Database migrations with Flyway and Liquibase
Lars Östling
 
Liquibase & Flyway @ Baltic DevOps
Liquibase & Flyway @ Baltic DevOpsLiquibase & Flyway @ Baltic DevOps
Liquibase & Flyway @ Baltic DevOps
Andrei Solntsev
 
Javaland 2016 - Flyway vs. LiquiBase - Battle der Datenbankmigrationstools
Javaland 2016 - Flyway vs. LiquiBase - Battle der DatenbankmigrationstoolsJavaland 2016 - Flyway vs. LiquiBase - Battle der Datenbankmigrationstools
Javaland 2016 - Flyway vs. LiquiBase - Battle der Datenbankmigrationstools
Stephan Kaps
 
Continuous DB Changes Delivery With Liquibase
Continuous DB Changes Delivery With LiquibaseContinuous DB Changes Delivery With Liquibase
Continuous DB Changes Delivery With Liquibase
Aidas Dragūnas
 
Evoluindo bancos de dados com Flyway
Evoluindo bancos de dados com FlywayEvoluindo bancos de dados com Flyway
Evoluindo bancos de dados com Flyway
Vitor Albuquerque
 
Introdução ao Flyway
Introdução ao FlywayIntrodução ao Flyway
Introdução ao Flyway
Jadson Santos
 
Liquibase
LiquibaseLiquibase
Liquibase
Sergii Fesenko
 
Gitora, Version Control for PL/SQL
Gitora, Version Control for PL/SQLGitora, Version Control for PL/SQL
Gitora, Version Control for PL/SQL
Gerger
 
Gitora, Version Control for PL/SQL
Gitora, Version Control for PL/SQLGitora, Version Control for PL/SQL
Gitora, Version Control for PL/SQL
Gerger
 
Version control for PL/SQL
Version control for PL/SQLVersion control for PL/SQL
Version control for PL/SQL
Gerger
 
Flyway - database migrations made easy
Flyway - database migrations made easyFlyway - database migrations made easy
Flyway - database migrations made easy
jstack
 
Liquibase få kontroll på dina databasförändringar
Liquibase   få kontroll på dina databasförändringarLiquibase   få kontroll på dina databasförändringar
Liquibase få kontroll på dina databasförändringar
Squeed
 
Ansible
AnsibleAnsible
33rd degree talk: open and automatic coding conventions with walkmod
33rd degree talk: open and automatic coding conventions with walkmod33rd degree talk: open and automatic coding conventions with walkmod
33rd degree talk: open and automatic coding conventions with walkmod
walkmod
 
The Challenges & Pitfalls of Database Continuous Delivery
The Challenges & Pitfalls of Database Continuous DeliveryThe Challenges & Pitfalls of Database Continuous Delivery
The Challenges & Pitfalls of Database Continuous Delivery
Perforce
 
Challenges and best practices of database continuous delivery
Challenges and best practices of database continuous deliveryChallenges and best practices of database continuous delivery
Challenges and best practices of database continuous delivery
DBmaestro - Database DevOps
 
walkmod: how it works
walkmod: how it workswalkmod: how it works
walkmod: how it works
walkmod
 
Patterns and Tools for Database Versioning, Migration, Data Loading and Test ...
Patterns and Tools for Database Versioning, Migration, Data Loading and Test ...Patterns and Tools for Database Versioning, Migration, Data Loading and Test ...
Patterns and Tools for Database Versioning, Migration, Data Loading and Test ...
Alan Pinstein
 
Continuous Delivery and Zero Downtime: What your architecture needs to succeed
Continuous Delivery and Zero Downtime: What your architecture needs to succeedContinuous Delivery and Zero Downtime: What your architecture needs to succeed
Continuous Delivery and Zero Downtime: What your architecture needs to succeed
Axel Fontaine
 

Viewers also liked (20)

Flyway
FlywayFlyway
Flyway
 
Database migrations with Flyway and Liquibase
Database migrations with Flyway and LiquibaseDatabase migrations with Flyway and Liquibase
Database migrations with Flyway and Liquibase
 
Liquibase & Flyway @ Baltic DevOps
Liquibase & Flyway @ Baltic DevOpsLiquibase & Flyway @ Baltic DevOps
Liquibase & Flyway @ Baltic DevOps
 
Javaland 2016 - Flyway vs. LiquiBase - Battle der Datenbankmigrationstools
Javaland 2016 - Flyway vs. LiquiBase - Battle der DatenbankmigrationstoolsJavaland 2016 - Flyway vs. LiquiBase - Battle der Datenbankmigrationstools
Javaland 2016 - Flyway vs. LiquiBase - Battle der Datenbankmigrationstools
 
Continuous DB Changes Delivery With Liquibase
Continuous DB Changes Delivery With LiquibaseContinuous DB Changes Delivery With Liquibase
Continuous DB Changes Delivery With Liquibase
 
Evoluindo bancos de dados com Flyway
Evoluindo bancos de dados com FlywayEvoluindo bancos de dados com Flyway
Evoluindo bancos de dados com Flyway
 
Introdução ao Flyway
Introdução ao FlywayIntrodução ao Flyway
Introdução ao Flyway
 
Liquibase
LiquibaseLiquibase
Liquibase
 
Gitora, Version Control for PL/SQL
Gitora, Version Control for PL/SQLGitora, Version Control for PL/SQL
Gitora, Version Control for PL/SQL
 
Gitora, Version Control for PL/SQL
Gitora, Version Control for PL/SQLGitora, Version Control for PL/SQL
Gitora, Version Control for PL/SQL
 
Version control for PL/SQL
Version control for PL/SQLVersion control for PL/SQL
Version control for PL/SQL
 
Flyway - database migrations made easy
Flyway - database migrations made easyFlyway - database migrations made easy
Flyway - database migrations made easy
 
Liquibase få kontroll på dina databasförändringar
Liquibase   få kontroll på dina databasförändringarLiquibase   få kontroll på dina databasförändringar
Liquibase få kontroll på dina databasförändringar
 
Ansible
AnsibleAnsible
Ansible
 
33rd degree talk: open and automatic coding conventions with walkmod
33rd degree talk: open and automatic coding conventions with walkmod33rd degree talk: open and automatic coding conventions with walkmod
33rd degree talk: open and automatic coding conventions with walkmod
 
The Challenges & Pitfalls of Database Continuous Delivery
The Challenges & Pitfalls of Database Continuous DeliveryThe Challenges & Pitfalls of Database Continuous Delivery
The Challenges & Pitfalls of Database Continuous Delivery
 
Challenges and best practices of database continuous delivery
Challenges and best practices of database continuous deliveryChallenges and best practices of database continuous delivery
Challenges and best practices of database continuous delivery
 
walkmod: how it works
walkmod: how it workswalkmod: how it works
walkmod: how it works
 
Patterns and Tools for Database Versioning, Migration, Data Loading and Test ...
Patterns and Tools for Database Versioning, Migration, Data Loading and Test ...Patterns and Tools for Database Versioning, Migration, Data Loading and Test ...
Patterns and Tools for Database Versioning, Migration, Data Loading and Test ...
 
Continuous Delivery and Zero Downtime: What your architecture needs to succeed
Continuous Delivery and Zero Downtime: What your architecture needs to succeedContinuous Delivery and Zero Downtime: What your architecture needs to succeed
Continuous Delivery and Zero Downtime: What your architecture needs to succeed
 

Similar to Database migration with flyway

SQL Server 2008 Integration Services
SQL Server 2008 Integration ServicesSQL Server 2008 Integration Services
SQL Server 2008 Integration Services
Eduardo Castro
 
Flexviews materialized views for my sql
Flexviews materialized views for my sqlFlexviews materialized views for my sql
Flexviews materialized views for my sql
Justin Swanhart
 
WebLogic Scripting Tool made Cool!
WebLogic Scripting Tool made Cool!WebLogic Scripting Tool made Cool!
WebLogic Scripting Tool made Cool!
Maarten Smeets
 
WinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSC
WinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSCWinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSC
WinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSC
WinOps Conf
 
How to instantiate any view controller for free
How to instantiate any view controller for freeHow to instantiate any view controller for free
How to instantiate any view controller for free
BenotCaron
 
Mmik_Powershell_DSC_Azure_DSC
Mmik_Powershell_DSC_Azure_DSCMmik_Powershell_DSC_Azure_DSC
Mmik_Powershell_DSC_Azure_DSC
Mmik Huang
 
Mmik powershell dsc_slideshare_v1
Mmik powershell dsc_slideshare_v1Mmik powershell dsc_slideshare_v1
Mmik powershell dsc_slideshare_v1
Mmik Huang
 
Procedures and triggers in SQL
Procedures and triggers in SQLProcedures and triggers in SQL
Procedures and triggers in SQL
Vikash Sharma
 
VMworld 2016: Getting Started with PowerShell and PowerCLI for Your VMware En...
VMworld 2016: Getting Started with PowerShell and PowerCLI for Your VMware En...VMworld 2016: Getting Started with PowerShell and PowerCLI for Your VMware En...
VMworld 2016: Getting Started with PowerShell and PowerCLI for Your VMware En...
VMworld
 
Mocking with salesforce using Munit
Mocking with salesforce using MunitMocking with salesforce using Munit
Mocking with salesforce using Munit
Son Nguyen
 
Airflow presentation
Airflow presentationAirflow presentation
Airflow presentation
Ilias Okacha
 
WF 4.0 Overview
WF 4.0 OverviewWF 4.0 Overview
WF 4.0 Overview
dannicola
 
VM Job Queues in CloudStack
VM Job Queues in CloudStackVM Job Queues in CloudStack
VM Job Queues in CloudStack
ShapeBlue
 
Apache Spark Streaming: Architecture and Fault Tolerance
Apache Spark Streaming: Architecture and Fault ToleranceApache Spark Streaming: Architecture and Fault Tolerance
Apache Spark Streaming: Architecture and Fault Tolerance
Sachin Aggarwal
 
从 Oracle 合并到 my sql npr 实例分析
从 Oracle 合并到 my sql   npr 实例分析从 Oracle 合并到 my sql   npr 实例分析
从 Oracle 合并到 my sql npr 实例分析
YUCHENG HU
 
Intro to Environment as a Service - Cloudify 5.0.5 Webinar
Intro to Environment as a Service - Cloudify 5.0.5 WebinarIntro to Environment as a Service - Cloudify 5.0.5 Webinar
Intro to Environment as a Service - Cloudify 5.0.5 Webinar
Cloudify Community
 
Chapter09
Chapter09Chapter09
Chapter09
sasa_eldoby
 
AWS re:Invent 2016: Deploying and Managing .NET Pipelines and Microsoft Workl...
AWS re:Invent 2016: Deploying and Managing .NET Pipelines and Microsoft Workl...AWS re:Invent 2016: Deploying and Managing .NET Pipelines and Microsoft Workl...
AWS re:Invent 2016: Deploying and Managing .NET Pipelines and Microsoft Workl...
Amazon Web Services
 

Similar to Database migration with flyway (20)

SQL Server 2008 Integration Services
SQL Server 2008 Integration ServicesSQL Server 2008 Integration Services
SQL Server 2008 Integration Services
 
Simple ETL Solution - Marco Kiesewetter
Simple ETL Solution - Marco KiesewetterSimple ETL Solution - Marco Kiesewetter
Simple ETL Solution - Marco Kiesewetter
 
Flexviews materialized views for my sql
Flexviews materialized views for my sqlFlexviews materialized views for my sql
Flexviews materialized views for my sql
 
WebLogic Scripting Tool made Cool!
WebLogic Scripting Tool made Cool!WebLogic Scripting Tool made Cool!
WebLogic Scripting Tool made Cool!
 
WinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSC
WinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSCWinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSC
WinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSC
 
How to instantiate any view controller for free
How to instantiate any view controller for freeHow to instantiate any view controller for free
How to instantiate any view controller for free
 
Mmik_Powershell_DSC_Azure_DSC
Mmik_Powershell_DSC_Azure_DSCMmik_Powershell_DSC_Azure_DSC
Mmik_Powershell_DSC_Azure_DSC
 
Mmik powershell dsc_slideshare_v1
Mmik powershell dsc_slideshare_v1Mmik powershell dsc_slideshare_v1
Mmik powershell dsc_slideshare_v1
 
Procedures and triggers in SQL
Procedures and triggers in SQLProcedures and triggers in SQL
Procedures and triggers in SQL
 
Mobile
MobileMobile
Mobile
 
VMworld 2016: Getting Started with PowerShell and PowerCLI for Your VMware En...
VMworld 2016: Getting Started with PowerShell and PowerCLI for Your VMware En...VMworld 2016: Getting Started with PowerShell and PowerCLI for Your VMware En...
VMworld 2016: Getting Started with PowerShell and PowerCLI for Your VMware En...
 
Mocking with salesforce using Munit
Mocking with salesforce using MunitMocking with salesforce using Munit
Mocking with salesforce using Munit
 
Airflow presentation
Airflow presentationAirflow presentation
Airflow presentation
 
WF 4.0 Overview
WF 4.0 OverviewWF 4.0 Overview
WF 4.0 Overview
 
VM Job Queues in CloudStack
VM Job Queues in CloudStackVM Job Queues in CloudStack
VM Job Queues in CloudStack
 
Apache Spark Streaming: Architecture and Fault Tolerance
Apache Spark Streaming: Architecture and Fault ToleranceApache Spark Streaming: Architecture and Fault Tolerance
Apache Spark Streaming: Architecture and Fault Tolerance
 
从 Oracle 合并到 my sql npr 实例分析
从 Oracle 合并到 my sql   npr 实例分析从 Oracle 合并到 my sql   npr 实例分析
从 Oracle 合并到 my sql npr 实例分析
 
Intro to Environment as a Service - Cloudify 5.0.5 Webinar
Intro to Environment as a Service - Cloudify 5.0.5 WebinarIntro to Environment as a Service - Cloudify 5.0.5 Webinar
Intro to Environment as a Service - Cloudify 5.0.5 Webinar
 
Chapter09
Chapter09Chapter09
Chapter09
 
AWS re:Invent 2016: Deploying and Managing .NET Pipelines and Microsoft Workl...
AWS re:Invent 2016: Deploying and Managing .NET Pipelines and Microsoft Workl...AWS re:Invent 2016: Deploying and Managing .NET Pipelines and Microsoft Workl...
AWS re:Invent 2016: Deploying and Managing .NET Pipelines and Microsoft Workl...
 

More from Jonathan Holloway

The Role of the Architect
The Role of the ArchitectThe Role of the Architect
The Role of the Architect
Jonathan Holloway
 
Spring boot - an introduction
Spring boot - an introductionSpring boot - an introduction
Spring boot - an introduction
Jonathan Holloway
 
Jenkins CI presentation
Jenkins CI presentationJenkins CI presentation
Jenkins CI presentation
Jonathan Holloway
 
Mockito intro
Mockito introMockito intro
Mockito intro
Jonathan Holloway
 
Debugging
DebuggingDebugging
SOLID principles
SOLID principlesSOLID principles
SOLID principles
Jonathan Holloway
 
Application design for the cloud using AWS
Application design for the cloud using AWSApplication design for the cloud using AWS
Application design for the cloud using AWS
Jonathan Holloway
 
Building data pipelines
Building data pipelinesBuilding data pipelines
Building data pipelines
Jonathan Holloway
 
Introduction to JVM languages and Fantom (very brief)
Introduction to JVM languages and Fantom (very brief)Introduction to JVM languages and Fantom (very brief)
Introduction to JVM languages and Fantom (very brief)
Jonathan Holloway
 
Introduction to using MongoDB with Ruby
Introduction to using MongoDB with RubyIntroduction to using MongoDB with Ruby
Introduction to using MongoDB with Ruby
Jonathan Holloway
 

More from Jonathan Holloway (11)

The Role of the Architect
The Role of the ArchitectThe Role of the Architect
The Role of the Architect
 
Spring boot - an introduction
Spring boot - an introductionSpring boot - an introduction
Spring boot - an introduction
 
Jenkins CI presentation
Jenkins CI presentationJenkins CI presentation
Jenkins CI presentation
 
Mockito intro
Mockito introMockito intro
Mockito intro
 
Debugging
DebuggingDebugging
Debugging
 
SOLID principles
SOLID principlesSOLID principles
SOLID principles
 
Application design for the cloud using AWS
Application design for the cloud using AWSApplication design for the cloud using AWS
Application design for the cloud using AWS
 
Building data pipelines
Building data pipelinesBuilding data pipelines
Building data pipelines
 
Introduction to JVM languages and Fantom (very brief)
Introduction to JVM languages and Fantom (very brief)Introduction to JVM languages and Fantom (very brief)
Introduction to JVM languages and Fantom (very brief)
 
Lightweight web frameworks
Lightweight web frameworksLightweight web frameworks
Lightweight web frameworks
 
Introduction to using MongoDB with Ruby
Introduction to using MongoDB with RubyIntroduction to using MongoDB with Ruby
Introduction to using MongoDB with Ruby
 

Recently uploaded

GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
Elena Simperl
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
Abida Shariff
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
Fwdays
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 

Recently uploaded (20)

GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi"Impact of front-end architecture on development cost", Viktor Turskyi
"Impact of front-end architecture on development cost", Viktor Turskyi
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 

Database migration with flyway

  • 2. What's the problem? Provide upgrades of applications and databases to our customers Some customers require SQL scripts to run via DB admin tools, e.g. Oracle (sqlplus) We don't have database schemas that reflect what's been applied in terms of updates - always the newest We don't really understand what Hibernate is doing to create/modify tables based on our persistent objects, we rely on it for optimization, indexes etc...
  • 3. Database Migrations Structure SQL scripts for creation of the schema, updates of the schema accordingly Use a tool to run these migrations: ● In a specific order ● Against a specific database (MySQL, Oracle) ● Recording the actions in a schema version table Lots of tools that do this
  • 4. Flyway Allows us to do migrations: ● From a shell script ● Programmatically ● When Spring starts ● Using Maven as a build tool
  • 5. Introducing Flyway into your project Has been done for MIF - there's a dbtool folder in the source structure. We can provide a template in SVN that can be used. 1. Download the dbtool structure 2. Generate an export of the schema that Hibernate has created for your project
  • 6. Introducing Flyway into your project 3. Tweak/optimise if necessary 4. Run the following command: ./flyway.sh init
  • 7. Introducing Flyway into your project This creates the SCHEMA_VERSION table: mysql> desc SCHEMA_VERSION; +-----------------+--------------+------+-----+-------------------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------------+--------------+------+-----+-------------------+-------+ | version | varchar(20) | NO | PRI | NULL | | | description | varchar(100) | YES | | NULL | | | type | varchar(10) | NO | | NULL | | | script | varchar(200) | NO | UNI | NULL | | | checksum | int(11) | YES | | NULL | | | installed_by | varchar(30) | NO | | NULL | | | installed_on | timestamp | NO | | CURRENT_TIMESTAMP | | | execution_time | int(11) | YES | | NULL | | | state | varchar(15) | NO | | NULL | | | current_version | tinyint(1) | NO | MUL | NULL | | +-----------------+--------------+------+-----+-------------------+-------+
  • 8. Performing a migration Assume you've added a new script in the <dbname>/sql folder of dbtool Run: ./flyway.sh migrate This will run the script and record the action in the SCHEMA_VERSION table.
  • 9. Flyway dbtool structure ├── mysql │ ├── conf │ │ └── flyway.properties │ ├── flyway.cmd │ ├── flyway.sh │ ├── jars │ │ └── mysql-connector-java-5.1.14.jar │ └── sql │ ├── V1__MIFBaseSchema.sql │ └── V2__JOB.DATE_ACKNOWLEDGED.sql └── oracle ├── conf │ └── flyway.properties ├── flyway.cmd ├── flyway.sh ├── jars │ └── ojdbc14-10.2.0.5.jar └── sql
  • 10. Process Change Need to change the way we develop persistent objects with Hibernate As well as changing an object we also need to provide a migration script (and a rollback script) Put it in the <dbname>/sql folder. ls -Al mysql/sql V1__MIFBaseSchema.sql V2__JOB.DATE_ACKNOWLEDGED.sql
  • 11. Example Change a persistent object to add a new field Write a script with the appropriate ALTER TABLE ADD COLUMN statement. Write the rollback script Test