Or: How I Will Replace You With A Very Small Shell Script

Who Daniel Cousineau Senior Software Applications Developer Texas A&M University Division of Student Affairs Department of IT http://www.toosweettobesour.com/ @dcousineau Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Daniel Cousineau

Senior Software Applications Developer

Texas A&M University

Division of Student Affairs

Department of IT

http://www.toosweettobesour.com/

@dcousineau

The Problem Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Human beings make mistakes Automated Deployment With Phing - Daniel Cousineau ZendCon '09

We aren’t as accurate each repetition Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Machines don’t have this problem Automated Deployment With Phing - Daniel Cousineau ZendCon '09

They do the same thing every time (supposedly) Automated Deployment With Phing - Daniel Cousineau ZendCon '09

At Any Given Time New Code New Configuration New Database Schema New Static Files Automated Deployment With Phing - Daniel Cousineau ZendCon '09

New Code

New Configuration

New Database Schema

New Static Files

A Lot To Remember Did you remember to upload ALL new files? Did you remember to update your DB? Did you remember to correct your config? Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Did you remember to upload ALL new files?

Did you remember to update your DB?

Did you remember to correct your config?

Even Worse Did you clear your caches? Did you delete that old file/plugin? In the upload process, was your configuration overwritten? Did you upload ALL the changed files? When did you last upload? Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Did you clear your caches?

Did you delete that old file/plugin?

In the upload process, was your configuration overwritten?

Did you upload ALL the changed files?

When did you last upload?

The Solution Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Automation! Build scripts! We are programmers after all… Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Build scripts!

We are programmers after all…

Don’t Do More Work Than You Have To! Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Work Hard At Being Lazy! Automated Deployment With Phing - Daniel Cousineau ZendCon '09

What is Automation? Automated deployment means a single command Locks your live site Uploads changed files Clears caches and temporary files Updates the database schema Runs other cron tasks Unlocks your live site Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Automated deployment means a single command

Locks your live site

Uploads changed files

Clears caches and temporary files

Updates the database schema

Runs other cron tasks

Unlocks your live site

Why Do We Automate? Deployment is tricky Repetition degrades quality She sells sea shells by the sea shore Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Deployment is tricky

Repetition degrades quality

She sells sea shells by the sea shore

When Is Automation Used? All the time! Staging Live Probably best to use it on your dev box too! Automated Deployment With Phing - Daniel Cousineau ZendCon '09

All the time!

Staging

Live

Probably best to use it on your dev box too!

The Basics Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Tools of the Trade Build System Phing Apache ANT Capastrano Plain PHP or BASH or BAT Files File Transfer Rsync (*NIX Environments) Xcopy (Windows Environments) Configuration Management Database Migration DBDeploy Doctrine Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Build System

Phing

Apache ANT

Capastrano

Plain PHP or BASH or BAT Files

File Transfer

Rsync (*NIX Environments)

Xcopy (Windows Environments)

Configuration Management

Database Migration

DBDeploy

Doctrine

Phing Philosophy Build scripts contains "Targets" Targets should be small and specialized. Example Targets: clean Clear temporary and cached files copy Copy files to their intended destination migrate Upgrade the database schema Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Build scripts contains "Targets"

Targets should be small and specialized.

Example Targets:

clean

Clear temporary and cached files

copy

Copy files to their intended destination

migrate

Upgrade the database schema

Phing Philosophy Targets can have dependencies Target "live" can depend on clean, copy, and migrate Meaning, when we run the "live" target, it first runs clean, copy, then migrate And any tasks they depend on Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Targets can have dependencies

Target "live" can depend on clean, copy, and migrate

Meaning, when we run the "live" target, it first runs clean, copy, then migrate

And any tasks they depend on

Installing Phing pear channel-discover pear.phing.info pear install phing/Phing Want all the little dependencies? pear config-set preferred_state alpha pear install –alldeps phing/Phing pear config-set preferred_state stable Automated Deployment With Phing - Daniel Cousineau ZendCon '09

pear channel-discover pear.phing.info

pear install phing/Phing

Want all the little dependencies?

pear config-set preferred_state alpha

pear install –alldeps phing/Phing

pear config-set preferred_state stable

Running Phing $> phing –v Lists Phing version Phing expects to find a file &quot;build.xml&quot; in the current directory build.xml defines the targets You can use the &quot; -f <filename> &quot; flag to specify an alternate build file like $> phing -f build-live.xml Automated Deployment With Phing - Daniel Cousineau ZendCon '09

$> phing –v

Lists Phing version

Phing expects to find a file &quot;build.xml&quot; in the current directory

build.xml defines the targets

You can use the &quot; -f <filename> &quot; flag to specify an alternate build file like

$> phing -f build-live.xml

Syntax XML If you don’t already know it, you have bigger problems Variables ${variablename} Conventions Psuedo-Namespaces using periods ${namespace.variable.name} Automated Deployment With Phing - Daniel Cousineau ZendCon '09

XML

If you don’t already know it, you have bigger problems

Variables

${variablename}

Conventions

Psuedo-Namespaces using periods

${namespace.variable.name}

Basic Conventions build.xml Central repository of your top-level tasks build-*.xml Can be included into build.xml . Usually for grouping by target (dev, staging, live) or task (migrate, test, etc.) build.properties Technically an INI file that contains variables to be included by build files. Automated Deployment With Phing - Daniel Cousineau ZendCon '09

build.xml

Central repository of your top-level tasks

build-*.xml

Can be included into build.xml .

Usually for grouping by target (dev, staging, live) or task (migrate, test, etc.)

build.properties

Technically an INI file that contains variables to be included by build files.

Basic build.xml <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?> <project name=&quot;ZendCon&quot; default=&quot;default&quot; basedir=&quot;.&quot;> <property file=&quot;build.properties&quot; /> <target name=&quot;default&quot;> <echo message=&quot;Howdy World!&quot; /> </target> </project> Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Basic build.properties source.directory = /src ## Database Configuration db.user = username db.pass = password db.host = localhost db.name = database Automated Deployment With Phing - Daniel Cousineau ZendCon '09

File Transfer Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Techniques Built in Phing Copy Task Cross Platform Slow Not over network Version Control Checkout/Update Syncs deleted files User created files usually ignored Rsync Great for *NIX environments. Almost guaranteed to be installed on Linux, BSD, Mac OSX Can be installed on Windows Xcopy Good for Windows environments Faster than Phing Copy Not over network Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Built in Phing Copy Task

Cross Platform

Slow

Not over network

Version Control Checkout/Update

Syncs deleted files

User created files usually ignored

Rsync

Great for *NIX environments.

Almost guaranteed to be installed on

Linux, BSD, Mac OSX

Can be installed on Windows

Xcopy

Good for Windows environments

Faster than Phing Copy

Not over network

Pitfalls Cleanup Deleted Source Files Usually only a problem when you have * include patterns Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Cleanup Deleted Source Files

Usually only a problem when you have * include patterns

Pitfalls User created files are a HUGE pitfall /htdocs/images/upload Sync programs usually will delete content your deployment machine doesn’t have What if user upload are mixed in with files you want to manage? Solutions Keep user created content completely separated. The further up the file tree and consolidated the better. Separate static file server, Amazon S3, or other style CDNs Automated Deployment With Phing - Daniel Cousineau ZendCon '09

User created files are a HUGE pitfall

/htdocs/images/upload

Sync programs usually will delete content your deployment machine doesn’t have

What if user upload are mixed in with files you want to manage?

Solutions

Keep user created content completely separated. The further up the file tree and consolidated the better.

Separate static file server, Amazon S3, or other style CDNs

Executing External Tools Nearly all file transfer tools will be external commands For this we need the Exec task <exec command=&quot;cp file1 file2&quot; /> Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Nearly all file transfer tools will be external commands

For this we need the Exec task

Rsync So many different options you’re best off finding your own tutorials or reading MAN pages rsync -aCvz -e 'ssh -i /path/to/ssh-key' ${project.basedir}/src user@domain: ${staging.deployment.directory}/ Automated Deployment With Phing - Daniel Cousineau ZendCon '09

So many different options you’re best off finding your own tutorials or reading MAN pages

rsync -aCvz

-e 'ssh -i /path/to/ssh-key'

${project.basedir}/src

user@domain: ${staging.deployment.directory}/

Rsync Options Of Note -a archive-mode Recursive copy, preserve everything, etc. -C cvs-exclude Ignore .cvs, .svn, and other version control artifacts -v verbose Know EXACTLY what’s going on -z compress Compress information transmitted -e alternative remote shell Use a custom SSH command (namely use key-pair) ZendCon '09 Automated Deployment With Phing - Daniel Cousineau

-a archive-mode

Recursive copy, preserve everything, etc.

-C cvs-exclude

Ignore .cvs, .svn, and other version control artifacts

-v verbose

Know EXACTLY what’s going on

-z compress

Compress information transmitted

-e alternative remote shell

Use a custom SSH command (namely use key-pair)

Xcopy Best when your live deployment process involves copying to a network share on Windows machines xcopy ${project.basedir}${source.directory}* ${staging.deployment.directory} /D /I /Y /S /E /Exclude:${project.basedir}staging.deployment.exclude Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Best when your live deployment process involves copying to a network share on Windows machines

xcopy

${project.basedir}${source.directory}*

${staging.deployment.directory}

/D /I /Y /S /E

/Exclude:${project.basedir}staging.deployment.exclude

Xcopy Options Of Note /d Update only those files that are different (timestamps) /i Creates target directories if they don’t exist /y Do not prompt /s Copy all directories and sub-directories… /e … Even if the directories are empty /exclude Exclude files based on glob patterns ZendCon '09 Automated Deployment With Phing - Daniel Cousineau

/d

Update only those files that are different (timestamps)

/i

Creates target directories if they don’t exist

/y

Do not prompt

/s

Copy all directories and sub-directories…

/e

… Even if the directories are empty

/exclude

Exclude files based on glob patterns

Xcopy Exclude File Example ZendCon '09 Automated Deployment With Phing - Daniel Cousineau mp uploads

Configuration Management Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Techniques Check incoming domain www.domain.com means load production configuration localhost/domain.com means load development configuration Check for an environment file Contents of DOCUMENT_ROOT/.env determine configuration Copy over during deployment Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Check incoming domain

www.domain.com means load production configuration

localhost/domain.com means load development configuration

Check for an environment file

Contents of DOCUMENT_ROOT/.env determine configuration

Copy over during deployment

Check Incoming Domain Pros No chance for error Cons Detection is a weak link Especially when development, staging, and live have different web servers (e.g. Apache to IIS) Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Pros

No chance for error

Cons

Detection is a weak link

Especially when development, staging, and live have different web servers (e.g. Apache to IIS)

Check for an Environment File Pros Consistent regardless of Server or OS changes Easy to repair Quick FTP or SSH Easy Testing Just swap file contents Cons Forget to copy changes? Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Pros

Consistent regardless of Server or OS changes

Easy to repair

Quick FTP or SSH

Easy Testing

Just swap file contents

Cons

Forget to copy changes?

Copy Over During Deployment Pros Images and CSS now manageable Reduce redundancy Simple to repurpose regular push command Cons Difficult to manage Forget to copy changes? Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Pros

Images and CSS now manageable

Reduce redundancy

Simple to repurpose regular push command

Cons

Difficult to manage

Forget to copy changes?

Which one to choose? Depends on your application architecture Depends on your environment You’ll know what’s best for your project I use a combination of Environment File and Copy Over During Deployment Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Depends on your application architecture

Depends on your environment

You’ll know what’s best for your project

I use a combination of Environment File and Copy Over During Deployment

Database Migration Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Database Deployment Philosophy “ Versions” often referred to as “ Deltas ” Each Delta has an UP and a DOWN script UP makes the changes DOWN reverts the changes Each change should be a non-destructive schema change Unless of course you need data alteration Automated Deployment With Phing - Daniel Cousineau ZendCon '09

“ Versions” often referred to as “ Deltas ”

Each Delta has an UP and a DOWN script

UP makes the changes

DOWN reverts the changes

Each change should be a non-destructive schema change

Unless of course you need data alteration

DBDeploy Philosophy Each delta is 2 native SQL scripts Separated by --//@undo Database version is stored in a special table Version X was applied at TIMESTAMP… Beware corruption! Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Each delta is 2 native SQL scripts

Separated by --//@undo

Database version is stored in a special table

Version X was applied at TIMESTAMP…

Beware corruption!

DBDeploy Under The Hood Connect to the database, read from changelog table, get the most current revision installed Retrieve all delta files newer that the current revision Revision numbers are based off of alphabetic sorting, name your files accordingly Pull all the “UP” changes and concatenate them to a migration script Up to YOU to run the migration script Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Connect to the database, read from changelog table, get the most current revision installed

Retrieve all delta files newer that the current revision

Revision numbers are based off of alphabetic sorting, name your files accordingly

Pull all the “UP” changes and concatenate them to a migration script

Up to YOU to run the migration script

Installing DBDeploy Is Phing already installed? Great! So is DBDeploy… Create the changelog table Track the current version of your DB CREATE TABLE changelog ( change_number BIGINT NOT NULL, delta_set VARCHAR(10) NOT NULL, start_dt TIMESTAMP NOT NULL, complete_dt TIMESTAMP NULL, applied_by VARCHAR(100) NOT NULL, description VARCHAR(500) NOT NULL ); ALTER TABLE changelog ADD CONSTRAINT Pkchangelog PRIMARY KEY (change_number, delta_set); Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Is Phing already installed? Great! So is DBDeploy…

Create the changelog table

Track the current version of your DB

Basic Delta: 01_init.sql --// CREATE TABLE IF NOT EXISTS `users` ( `id` int(10) unsigned NOT NULL auto_increment, `handle` varchar(25) NOT NULL default '', PRIMARY KEY (`id`), UNIQUE KEY `users_handle_index` (`handle`) ) ENGINE=InnoDB DEFAULT; --//@UNDO DROP TABLE IF EXISTS `users`; --// Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Run Migration First add a task to your Phing build file <target name=&quot;migrate&quot;> </target> Automated Deployment With Phing - Daniel Cousineau ZendCon '09

First add a task to your Phing build file

Run Migration Generate the SQL migration file <target name=&quot;migrate&quot;> <dbdeploy url=&quot;mysql:host=${db.host};dbname=${db.name}&quot; userid=&quot;${db.user}&quot; password=&quot;${db.pass}&quot; dir=&quot;${project.basedir}/db/deltas&quot; outputfile=&quot;${project.basedir}/up.sql&quot; undooutputfile=&quot;${project.basedir}/down.sql&quot; /> </target> Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Generate the SQL migration file

Run Migration Execute the SQL script <target name=&quot;migrate&quot;> <dbdeploy url=&quot;mysql:host=${db.host};dbname=${db.name}&quot; userid=&quot;${db.user}&quot; password=&quot;${db.pass}&quot; dir=&quot;${project.basedir}/db/deltas&quot; outputfile=&quot;${project.basedir}/up.sql&quot; undooutputfile=&quot;${project.basedir}/down.sql&quot; /> <exec command=&quot;/usr/bin/mysql -h${db.local.host} -u${db.local.user} -p ${db.local.pass} ${db.local.name} < ${project.basedir}/up.sql&quot; dir=&quot;${project.basedir}&quot; checkreturn=&quot;true&quot;> </target> Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Execute the SQL script

Pitfalls Make sure you have a good CLI app for loading a SQL file /usr/bin/mysql mysql.exe Build script travelling across operating systems? Write your own Phing task to execute the migration output? Was the changelog table created in the first place? Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Make sure you have a good CLI app for loading a SQL file

/usr/bin/mysql

mysql.exe

Build script travelling across operating systems?

Write your own Phing task to execute the migration output?

Was the changelog table created in the first place?

Doctrine Database Migrations Integrated into the Doctrine CLI tool ./doctrine migrate Same philosophies BUT, deltas are PHP objects that contain an UP and a DOWN method Run using the Phing exec task Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Integrated into the Doctrine CLI tool

./doctrine migrate

Same philosophies

BUT, deltas are PHP objects that contain an UP and a DOWN method

Run using the Phing exec task

Other Database Migration Tools http://www.liquibase.org/ Automated Deployment With Phing - Daniel Cousineau ZendCon '09

http://www.liquibase.org/

Closing Thoughts Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Further Resources The people around you More people than you know automate their deployment There are many, many different techniques My techniques may suck for your particular setup #phpc on freenode Many of the speakers hang out there Many smart people hang out there I HANG OUT THERE! Therefore I am smart Automated Deployment With Phing - Daniel Cousineau ZendCon '09

The people around you

More people than you know automate their deployment

There are many, many different techniques

My techniques may suck for your particular setup

#phpc on freenode

Many of the speakers hang out there

Many smart people hang out there

I HANG OUT THERE!

Therefore I am smart

Further Resources Blogs http://phpdeveloper.org Documentation http://phing.info http://dbdeploy.com Google Experimentation Pushing to a local directory Pushing to a virtual machine Automated Deployment With Phing - Daniel Cousineau ZendCon '09

Blogs

http://phpdeveloper.org

Documentation

http://phing.info

http://dbdeploy.com

Google

Experimentation

Pushing to a local directory

Pushing to a virtual machine

Questions? ZendCon '09 Automated Deployment With Phing - Daniel Cousineau

http://joind.in/937