Automated Deployment With Phing

Automated Deployment With Phing
Or: How I Will Replace You With A Very Small Shell Script
ZendCon '09
Automated Deployment With Phing - Daniel Cousineau
1

Who
Daniel Cousineau
Senior Software Applications Developer
Texas A&M University
Division of Student Affairs
Department of IT
http://www.toosweettobesour.com/
@dcousineau
ZendCon '09
2

The Problem
ZendCon '09
3

Human beings make mistakes
ZendCon '09
4

We aren’t as accurate each repetition
ZendCon '09
5

Machines don’t have this problem
ZendCon '09
6

They do the same thing every time (supposedly)
ZendCon '09
7

At Any Given Time
New Code
New Configuration
New Database Schema
New Static Files
ZendCon '09
8

A Lot To Remember
Did you remember to upload ALL new files?
Did you remember to update your DB?
Did you remember to correct your config?
ZendCon '09
9

Even Worse
Did you clear your caches?
Did you delete that old file/plugin?
In the upload process, was your configuration overwritten?
Did you upload ALL the changed files?
When did you last upload?
ZendCon '09
10

The Solution
ZendCon '09
11

Automation!
Build scripts!
We are programmers after all…
ZendCon '09
12

ZendCon '09
13
Don’t Do More Work Than You Have To!

ZendCon '09
14
Work Hard At Being Lazy!

What is Automation?
Automated deployment means a single command
Locks your live site
Uploads changed files
Clears caches and temporary files
Updates the database schema
Runs other cron tasks
Unlocks your live site
ZendCon '09
15

Why Do We Automate?
Deployment is tricky
Repetition degrades quality
She sells sea shells by the sea shore
ZendCon '09
16

When Is Automation Used?
All the time!
Staging
Live
Probably best to use it on your dev box too!
ZendCon '09
17

The Basics
ZendCon '09
18

Tools of the Trade
Build System
Phing
Apache ANT
Capastrano
Plain PHP or BASH or BAT Files
File Transfer
Rsync (*NIX Environments)
Xcopy (Windows Environments)
Configuration Management
Database Migration
DBDeploy
Doctrine
ZendCon '09
19

Phing Philosophy
Build scripts contains "Targets"
Targets should be small and specialized.
Example Targets:
clean
Clear temporary and cached files
copy
Copy files to their intended destination
migrate
Upgrade the database schema
ZendCon '09
20

Phing Philosophy
Targets can have dependencies
Target "live" can depend on clean, copy, and migrate
Meaning, when we run the "live" target, it first runs clean, copy, then migrate
And any tasks they depend on
ZendCon '09
21

Installing Phing
pear channel-discover pear.phing.info
pear install phing/Phing
Want all the little dependencies?
pear config-set preferred_state alpha
pear install –alldepsphing/Phing
pear config-set preferred_state stable
ZendCon '09
22

Running Phing
$> phing –v
Lists Phing version
Phing expects to find a file "build.xml" in the current directory
build.xml defines the targets
You can use the "-f <filename>" flag to specify an alternate build file like
$> phing -f build-live.xml
ZendCon '09
23

Syntax
XML
If you don’t already know it, you have bigger problems
Variables
${variablename}
Conventions
Psuedo-Namespaces using periods
${namespace.variable.name}
ZendCon '09
24

Basic Conventions
build.xml
Central repository of your top-level tasks
build-*.xml
Can be included into build.xml.
Usually for grouping by target (dev, staging, live) or task (migrate, test, etc.)
build.properties
Technically an INI file that contains variables to be included by build files.
ZendCon '09
25

Basic build.xml
ZendCon '09
26
<?xml version="1.0" encoding="UTF-8"?>
<project name="ZendCon" default="default" basedir=".">
<property file="build.properties" />
<target name="default">
<echo message="Howdy World!" />
</target>
</project>

Basic build.properties
ZendCon '09
27
source.directory = /src
## Database Configuration
db.user = username
db.pass = password
db.host = localhost
db.name = database

File Transfer
ZendCon '09
28

Techniques
Built in Phing Copy Task
Cross Platform
Slow
Not over network
Version Control Checkout/Update
Syncs deleted files
User created files usually ignored
Rsync
Great for *NIX environments.
Almost guaranteed to be installed on
Linux, BSD, Mac OSX
Can be installed on Windows
Xcopy
Good for Windows environments
Faster than Phing Copy
Not over network
ZendCon '09
29

Pitfalls
Cleanup Deleted Source Files
Usually only a problem when you have * include patterns
ZendCon '09
30

Pitfalls
User created files are a HUGE pitfall
/htdocs/images/upload
Sync programs usually will delete content your deployment machine doesn’t have
What if user upload are mixed in with files you want to manage?
Solutions
Keep user created content completely separated. The further up the file tree and consolidated the better.
Separate static file server, Amazon S3, or other style CDNs
ZendCon '09
31

Executing External Tools
Nearly all file transfer tools will be external commands
For this we need the Exec task
ZendCon '09
32
<exec command="cp file1 file2" />

Rsync
So many different options you’re best off finding your own tutorials or reading MAN pages
rsync -aCvz
-e 'ssh -i /path/to/ssh-key'
${project.basedir}/src
user@domain: ${staging.deployment.directory}/
ZendCon '09
33

Rsync Options Of Note
-a archive-mode
Recursive copy, preserve everything, etc.
-C cvs-exclude
Ignore .cvs, .svn, and other version control artifacts
-v verbose
Know EXACTLY what’s going on
-z compress
Compress information transmitted
-e alternative remote shell
Use a custom SSH command (namely use key-pair)
ZendCon '09
34

Xcopy
Best when your live deployment process involves copying to a network share on Windows machines
xcopy
${project.basedir}${source.directory}*
${staging.deployment.directory}
/D /I /Y /S /E
/Exclude:${project.basedir}staging.deployment.exclude
ZendCon '09
35

Xcopy Options Of Note
/d
Update only those files that are different (timestamps)
/i
Creates target directories if they don’t exist
/y
Do not prompt
/s
Copy all directories and sub-directories…
/e
…Even if the directories are empty
/exclude
Exclude files based on glob patterns
ZendCon '09
36

Xcopy Exclude File Example
ZendCon '09
37
mp
uploads

Configuration Management
ZendCon '09
38

Techniques
Check incoming domain
www.domain.com means load production configuration
localhost/domain.com means load development configuration
Check for an environment file
Contents of DOCUMENT_ROOT/.env determine configuration
Copy over during deployment
ZendCon '09
39

Check Incoming Domain
Pros
No chance for error
Cons
Detection is a weak link
Especially when development, staging, and live have different web servers (e.g. Apache to IIS)
ZendCon '09
40

Check for an Environment File
Pros
Consistent regardless of Server or OS changes
Easy to repair
Quick FTP or SSH
Easy Testing
Just swap file contents
Cons
Forget to copy changes?
ZendCon '09
41

Copy Over During Deployment
Pros
Images and CSS now manageable
Reduce redundancy
Simple to repurpose regular push command
Cons
Difficult to manage
Forget to copy changes?
ZendCon '09
42

Which one to choose?
Depends on your application architecture
Depends on your environment
You’ll know what’s best for your project
I use a combination of Environment File and Copy Over During Deployment
ZendCon '09
43

Database Migration
ZendCon '09
44

Database Deployment Philosophy
“Versions” often referred to as “Deltas”
Each Delta has an UP and a DOWN script
UP makes the changes
DOWN reverts the changes
Each change should be a non-destructive schema change
Unless of course you need data alteration
ZendCon '09
45

DBDeploy Philosophy
Each delta is 2 native SQL scripts
Separated by --//@undo
Database version is stored in a special table
Version X was applied at TIMESTAMP…
Beware corruption!
ZendCon '09
46

DBDeploy Under The Hood
Connect to the database, read from changelog table, get the most current revision installed
Retrieve all delta files newer that the current revision
Revision numbers are based off of alphabetic sorting, name your files accordingly
Pull all the “UP” changes and concatenate them to a migration script
Up to YOU to run the migration script
ZendCon '09
47

Installing DBDeploy
Is Phing already installed? Great! So is DBDeploy…
Create the changelog table
Track the current version of your DB
ZendCon '09
48
CREATE TABLE changelog (
change_number BIGINT NOT NULL,
delta_set VARCHAR(10) NOT NULL,
start_dt TIMESTAMP NOT NULL,
complete_dt TIMESTAMP NULL,
applied_by VARCHAR(100) NOT NULL,
description VARCHAR(500) NOT NULL
);
ALTER TABLE changelog
ADD CONSTRAINT Pkchangelog PRIMARY KEY (change_number, delta_set);

Basic Delta: 01_init.sql
ZendCon '09
49
--//
CREATE TABLE IF NOT EXISTS ùsers` (
ìd` int(10) unsigned NOT NULL auto_increment,
`handle` varchar(25) NOT NULL default '',
PRIMARY KEY (ìd`),
UNIQUE KEY ùsers_handle_index` (`handle`)
) ENGINE=InnoDB DEFAULT;
--//@UNDO
DROP TABLE IF EXISTS ùsers`;
--//

Run Migration
First add a task to your Phing build file
ZendCon '09
50
<target name="migrate">
</target>

Run Migration
Generate the SQL migration file
ZendCon '09
51
<dbdeploy
url="mysql:host=${db.host};dbname=${db.name}"
userid="${db.user}"
password="${db.pass}"
dir="${project.basedir}/db/deltas"
outputfile="${project.basedir}/up.sql"
undooutputfile="${project.basedir}/down.sql"
/>
</target>

Run Migration
Execute the SQL script
ZendCon '09
52
<dbdeploy
url="mysql:host=${db.host};dbname=${db.name}"
userid="${db.user}"
password="${db.pass}"
dir="${project.basedir}/db/deltas"
outputfile="${project.basedir}/up.sql"
undooutputfile="${project.basedir}/down.sql"
/>
<exec
command="/usr/bin/mysql -h${db.local.host} -u${db.local.user}
-p ${db.local.pass}
${db.local.name} < ${project.basedir}/up.sql"
dir="${project.basedir}"
checkreturn="true">
</target>

Pitfalls
Make sure you have a good CLI app for loading a SQL file
/usr/bin/mysql
mysql.exe
Build script travelling across operating systems?
Write your own Phing task to execute the migration output?
Was the changelog table created in the first place?
ZendCon '09
53

Doctrine Database Migrations
Integrated into the Doctrine CLI tool
./doctrine migrate
Same philosophies
BUT, deltas are PHP objects that contain an UP and a DOWN method
Run using the Phing exec task
ZendCon '09
54

Other Database Migration Tools
http://www.liquibase.org/
ZendCon '09
55

Closing Thoughts
ZendCon '09
56

Further Resources
The people around you
More people than you know automate their deployment
There are many, many different techniques
My techniques may suck for your particular setup
#phpcon freenode
Many of the speakers hang out there
Many smart people hang out there
I HANG OUT THERE!
Therefore I am smart
ZendCon '09
57

Further Resources
Blogs
http://phpdeveloper.org
Documentation
http://phing.info
http://dbdeploy.com
Google
Experimentation
Pushing to a local directory
Pushing to a virtual machine
ZendCon '09
58

Questions?
ZendCon '09
59

http://joind.in/937

http://www.slideshare.net	110
http://coderwall.com	2
http://static.slidesharecdn.com	1
http://webcache.googleusercontent.com	1
https://twimg0-a.akamaihd.net	1

Automated Deployment With Phing

by Daniel Cousineau on Oct 22, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 115

Statistics

Automated Deployment With Phing — Presentation Transcript