Mike Willbanks Blog: http://blog.digitalstruct.com Twitter : mwillbanks IRC : lubs Talk: http://joind.in/986 Handling Database Deployment ZendCon Unconference 2009

What We'll Cover (and not cover) The [Somewhat] Prerequisites Why Use Database Change Management Why To Not Use Database Abstraction Layer Migrations Even You Doctrine! DBDeploy and LiquiBase How It Works (under the hood) Basic Syntax Recommended Structure

The [Somewhat] Prerequisites

Why Use Database Change Management

Why To Not Use Database Abstraction Layer Migrations

Even You Doctrine!

DBDeploy and LiquiBase

How It Works (under the hood)

Basic Syntax

Recommended Structure

The [Somewhat] Prerequisites You should be using an issue tracking system. You should be using source code management. You already know how you are deploying your code.

You should be using an issue tracking system.

You should be using source code management.

You already know how you are deploying your code.

Why Database Change Management Do you version your code? Same reasons apply to database deltas (or change sets). Deltas typically are a part of a code change. Ensures the correlation between code revisions. Allows the automation of changes. Ensure you are keeping environments consistent. Allows for reverting back to previous versions (some voodoo magic may be required).

Do you version your code? Same reasons apply to database deltas (or change sets).

Deltas typically are a part of a code change.

Ensures the correlation between code revisions.

Allows the automation of changes.

Ensure you are keeping environments consistent.

Allows for reverting back to previous versions (some voodoo magic may be required).

The Traditional Way - #fail Developer writes code and applies database change locally. Developer forgets to note change on bug tracking ticket or notify anyone of the change or DBA misses the change, etc. Code deployment and database deployment happens. System is broken... #fail Developer attempts to figure out what happened, “oops factor”.

Developer writes code and applies database change locally.

Developer forgets to note change on bug tracking ticket or notify anyone of the change or DBA misses the change, etc.

Code deployment and database deployment happens.

System is broken... #fail

Developer attempts to figure out what happened, “oops factor”.

A Better Way Developer writes code and writes a database delta. Developer tests changes locally. Code and Database Delta is committed against a ticket (or at least together). Pre-Deployment Database Changes are Applied. Code Deployment is pushed out. Post-Deployment Database Changes are Applied. System === happiness.

Developer writes code and writes a database delta.

Developer tests changes locally.

Code and Database Delta is committed against a ticket (or at least together).

Pre-Deployment Database Changes are Applied.

Code Deployment is pushed out.

Post-Deployment Database Changes are Applied.

System === happiness.

Some Process Theory Simplified Database Pre-Deployment Apply database changes compatible with current code, in preparation for new code. Essentially, do not break what is already out there. Code Deployment No more detail needed. Database Post-Deployment Clean up database that was compatible with old code and /or migrate to fulfill new requirements.

Database Pre-Deployment

Apply database changes compatible with current code, in preparation for new code.

Essentially, do not break what is already out there.

Code Deployment

No more detail needed.

Database Post-Deployment

Clean up database that was compatible with old code and /or migrate to fulfill new requirements.

Database Abstraction Migrations Many do not support pre and post migrations. If you change your abstraction layer, you must change your migration logic. Example Case: Code Deployment Database Migrations Run on 1 st Machine What about the n+1 machines?! #fail

Many do not support pre and post migrations.

If you change your abstraction layer, you must change your migration logic.

Example Case:

Code Deployment

Database Migrations Run on 1 st Machine

What about the n+1 machines?! #fail

DBDeploy : dbdeploy.com Java based solution - #port anyone? Utilizes SQL delta files. Handles undo within the delta file. If using Ant – Issue #29 :) Utilizes different table names to handle different deployments. Queries the database for what has not been applied and generates a SQL file to apply towards the database.

Java based solution - #port anyone?

Utilizes SQL delta files.

Handles undo within the delta file.

If using Ant – Issue #29 :)

Utilizes different table names to handle different deployments.

Queries the database for what has not been applied and generates a SQL file to apply towards the database.

DBDeploy : Getting Started Install Java 1.5+ Download DBDeploy (Obvious) Download your Java Connector We will be using the command line version (rather than ant for this talk) It is kind of a blackbox :)

Install Java 1.5+

Download DBDeploy (Obvious)

Download your Java Connector

We will be using the command line version (rather than ant for this talk)

It is kind of a blackbox :)

DBDeploy : Initial Setup Under scripts, install the SQL file for your database Ensure you run this twice with 2 different database names. I suggest creating a pre and post changelog table. Put the Java Connector somewhere that you can reference it.

Under scripts, install the SQL file for your database

Ensure you run this twice with 2 different database names. I suggest creating a pre and post changelog table.

Put the Java Connector somewhere that you can reference it.

DBDeploy : The Command Line java -cp [path/to/driver]:dbdeploy-cli-3.0M2.jar com.dbdeploy.CommandLineTarget Relevant Parameters: -D | --driver Database Driver -d | --dbms Database Type -s | --scriptdirectory -o | --outputfile Output File -u | --url DB URL -U | --userid DB Username -P | --password DB Password -t | --changeLogTableName

java -cp [path/to/driver]:dbdeploy-cli-3.0M2.jar com.dbdeploy.CommandLineTarget

Relevant Parameters:

-D | --driver Database Driver

-d | --dbms Database Type

-s | --scriptdirectory

-o | --outputfile Output File

-u | --url DB URL

-U | --userid DB Username

-P | --password DB Password

-t | --changeLogTableName

DBDeploy : Writing a Script Files run in the format of: 001, 002, 003... with .sql on the end. You can add a comment on the file name: 001_created_test_table.sql Put in your delta, then for handling an undo, put in a comment in the format of: CREATE TABLE `xyz` (id INTEGER); --//@UNDO DROP TABLE `xyz`; Save this file to a directory of the type of script: You did do pre and post changelog right?

Files run in the format of:

001, 002, 003... with .sql on the end.

You can add a comment on the file name:

001_created_test_table.sql

Put in your delta, then for handling an undo, put in a comment in the format of:

CREATE TABLE `xyz` (id INTEGER); --//@UNDO DROP TABLE `xyz`;

Save this file to a directory of the type of script:

You did do pre and post changelog right?

DBDeploy : An Example java -cp mysql-connector-java-5.1.10-bin.jar:dbdeploy-cli-3.0M2.jar com.dbdeploy.CommandLineTarget -Dcom.mysql.jdbc.Driver -dmysql -sexample/ -u"jdbc:mysql://localhost/uncon2009_db" -Uuncon2009 -P***** -tchangelog_pre --outputfile deploy.sql This will create the deploy.sql file to apply to your database. Let's watch this command, understand the output and look at the script.

java -cp mysql-connector-java-5.1.10-bin.jar:dbdeploy-cli-3.0M2.jar com.dbdeploy.CommandLineTarget -Dcom.mysql.jdbc.Driver -dmysql -sexample/ -u"jdbc:mysql://localhost/uncon2009_db" -Uuncon2009 -P***** -tchangelog_pre --outputfile deploy.sql

This will create the deploy.sql file to apply to your database.

Let's watch this command, understand the output and look at the script.

DBDeploy : The Output dbdeploy 3.0M2 Reading change scripts from directory /home/mwillbanks/presentations/zendcon-uncon-2009-db-deployments/bin/dbdeploy/example... Changes currently applied to database: (none) Scripts available: 1, 2 To be applied: 1, 2

DBDeploy : Questions? Any questions on DBDeploy, how it works and anything I may have missed before we go on?

Any questions on DBDeploy, how it works and anything I may have missed before we go on?

LiquiBase : liquibase.org Java based solution - #port anyone? Utilizes XML Files for Changesets Handles undo within the changeset. Utilizes a single table with a primary key on the id, author and filename. Queries the database for what has not been applied and applies the different SQL. Very well documented unlike DBDeploy.

Java based solution - #port anyone?

Utilizes XML Files for Changesets

Handles undo within the changeset.

Utilizes a single table with a primary key on the id, author and filename.

Queries the database for what has not been applied and applies the different SQL.

Very well documented unlike DBDeploy.

LiquiBase : Getting Started Install Java 1.5+ Download LiquiBase (Obvious) Download your Java Connector We will be using the command line version This command line version is full featured.

Install Java 1.5+

Download LiquiBase (Obvious)

Download your Java Connector

We will be using the command line version

This command line version is full featured.

LiquiBase : Initial Setup Put the Java Connector somewhere that you can reference it (usually in the lib folder) That's pretty much all you need initially, besides extracting the tar ball :)

Put the Java Connector somewhere that you can reference it (usually in the lib folder)

That's pretty much all you need initially, besides extracting the tar ball :)

LiquiBase : The Command Line ./liquibase --driver=com.mysql.jdbc.Driver --changeLogFile=../sql/pre-deployment.xml --username=uncon2009 --password=****** --url="jdbc:mysql://localhost/uncon2009_db” Relevant Commands: migrate – applies change sets for migration rollback – to a tag rollbackToDate – rollback to a date generateChangeLog – useful to create the initial version of the change log. changeLogSync – mark all changes as ran. markNextChangeSetRan – mark next change as ran.

./liquibase --driver=com.mysql.jdbc.Driver --changeLogFile=../sql/pre-deployment.xml --username=uncon2009 --password=****** --url="jdbc:mysql://localhost/uncon2009_db”

Relevant Commands:

migrate – applies change sets for migration

rollback – to a tag

rollbackToDate – rollback to a date

generateChangeLog – useful to create the initial version of the change log.

changeLogSync – mark all changes as ran.

markNextChangeSetRan – mark next change as ran.

LiquiBase : Generating a Script Files run in XML files You can include them from a file or keep all changes in a single file. Run the command: ./liquibase --driver=com.mysql.jdbc.Driver --changeLogFile=sql/pre.xml --username=uncon2009 --password=****** --url="jdbc:mysql://localhost/uncon2009_db” generateChangeLog This saves the file to the area specified.

Files run in XML files

You can include them from a file or keep all changes in a single file.

Run the command:

./liquibase --driver=com.mysql.jdbc.Driver --changeLogFile=sql/pre.xml --username=uncon2009 --password=****** --url="jdbc:mysql://localhost/uncon2009_db” generateChangeLog

This saves the file to the area specified.

LiquiBase : Writing a Change Log <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?> <databaseChangeLog xmlns=&quot;http://www.liquibase.org/xml/ns/dbchangelog/1.6&quot; xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot; xsi:schemaLocation=&quot;http://www.liquibase.org/xml/ns/dbchangelog/1.6 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.6.xsd&quot;> </databaseChangeLog> All change logs must define the XML schema: Each change log starts with changeLog: <changeSet id=&quot;1&quot; author=&quot;mike&quot;> <createTable tableName=&quot;uncon2009&quot;> <column name=&quot;id&quot; type=&quot;int&quot;> <constraints primaryKey=&quot;true&quot; nullable=&quot;false&quot;/> </column> </createTable> </changeSet>

All change logs must define the XML schema:

Each change log starts with changeLog:

LiquiBase : The Internals LiquiBase creates tables called: DATABASECHANGELOG – this handles all of the changes, each change has a primary key on id, author and filename. This table also creates an MD5 of each change set, so do not change a changeset unless it has an error and will not run :) DATABASECHANGELOCK – locks to ensure you are not executing the same change sets.

LiquiBase creates tables called:

DATABASECHANGELOG – this handles all of the changes, each change has a primary key on id, author and filename.

This table also creates an MD5 of each change set, so do not change a changeset unless it has an error and will not run :)

DATABASECHANGELOCK – locks to ensure you are not executing the same change sets.

LiquiBase : Examples of Migration Running through the pre deployment file. Running through the post deployment file.

Running through the pre deployment file.

Running through the post deployment file.

LiquiBase : Pre-Deployment File

LiquiBase : Post-Deployment File

Conclusion Use something to manage your database deployments. There are tools, even more than mentioned here. Ensure you can handle pre and post deployments. They are important! Read the manuals of each system There is a bunch of items I didn't cover. Slides will be posted with examples sometime today. Both on my blog and I'll update the joind.in information to reference the presentation.

Use something to manage your database deployments.

There are tools, even more than mentioned here.

Ensure you can handle pre and post deployments.

They are important!

Read the manuals of each system

There is a bunch of items I didn't cover.

Slides will be posted with examples sometime today.

Both on my blog and I'll update the joind.in information to reference the presentation.

Mike Willbanks Blog: http://blog.digitalstruct.com Twitter : mwillbanks IRC : lubs Talk: http://joind.in/986 Questions?