Automated Deployment
Upcoming SlideShare
Loading in...5
×
 

Automated Deployment

on

  • 1,302 views

@phpne (March) talk by @michaelpeacock

@phpne (March) talk by @michaelpeacock

Statistics

Views

Total Views
1,302
Views on SlideShare
1,302
Embed Views
0

Actions

Likes
1
Downloads
11
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Store expected schema, and generate schema array from applied patches.

Automated Deployment Automated Deployment Presentation Transcript

  • Automated Deployment
    Building a simple automated deployment platform with PHP and Linux
    Michael Peacock@michaelpeacockmichaelpeacock.co.uk
  • whois?
    Senior / Lead Web Developer
    Zend Certified Engineer
    Published Author
    PHP 5 Social Networking, PHP 5 E-Commerce development & more
  • Deployment: (an) old style approach
    Take website offline / put into maintenance mode
    Backup everything
    Upload new files - FTP
    Upgrade database
    Put online, and hope for the best
    Do it twice: once for staging and once for deployment
  • http://xkcd.com/303/
  • The problem
    Down time for upgrades
    Manual process
    FTP takes time;
    forgot to CHMOD?
    Clients want to see progress now!
    Bugs and issues can lie dormant for some time
  • What about...
    Many existing solutions are geared towards large projects
    What about...
    the little guy;
    the small agency
    the web app start up on an entry level VPS?
  • What's in store?
    A few simple techniques, scripts and ideas that we currently use to make deployment easy
  • Deployment: the basics
    Get your latest code from version control, and stick it online
    Keep a central record of all the CHMOD / CHOWNing that you need to do
    Swap around your database connection details and other suitable configuration files
  • SVN Export
    Start with a simple svn export
    Store the date/time in a variable
    Create two folders, named with the current date/time. One within the web root, one outside of it
    Two exports: public and private (or one export, and some moving around of folders – up to you!)
  • #!/bin/bash
    DATE=`date +%H-%M-%e-%m-%y`
    mkdir /var/www/staging/$DATE/
    mkdir /var/www/staging-private/$DATE/
    svn export --quiet --username phpne --password PhpN3 httP://localhost/svn/project/trunk /var/www/staging/$DATE/
    svn export --quiet --username phpne --password PhpN3 http://localhost/svn/project/private /var/www/staging-private/$DATE/
  • SVN Export
    Keep your servers svn client happy! It will ask what to do with the svn password, and nobody will listen – so tell it!
    sudonano /var/www/.subversion/servers
    store-plaintext-passwords = no
  • Autonomy
    ln –s /staging /live
  • Autonomy
    When the latest code is checked out, tests have been run, uploads imported, configuration changed and database patched we need to swap this into place instantly
    The answer: symlinks
  • #!/bin/bash
    DATE=`date +%H-%M-%e-%m-%y`
    ...
    rm /home/user/public_html/
    ln –s /var/www/staging/$DATE/ /home/user/public_html/
    Sadly, you can’t edit a symlink, hence rm
  • My user profile pictures aren’t in version control…
  • User contributed files
    Store them elsewhere?
    On a content delivery network?
    On a sub-domain
    Symlink them
    Copy them in post svn export?
    A bit nasty and takes time, and what about new user uploads during the copying process?
  • The database
  • Photo of database table not found, or mysql gone away error message
    http://www.flickr.com/photos/meandmybadself/165846637/
  • Database changes: patches
    For database changes to apply on deploy, you need some deploy aware code in your project.
    Multi-query patch processing
    Schema compare; its easy to forget a database patch!
    Backup database before applying patches
  • public function updateDatabase( $patchID, $some=false )
    {
    // look for the next patch
    if( file_exists( FRAMEWORK_PATH . '../database/patches/' . ++$patchID . '.php' ) )
    {
    $sql = file_get_contents( FRAMEWORK_PATH . '../database/patches/' . $patchID . '.php' );
    // apply the changes from the patch
    mysqli_multi_query( $sql );
    // lather, rinse and repeat
    $this->updateDatabase( $patchID, true );
    }
    else if( $some )
    {
    // All done? Update patch ID in database
    mysqli_query(“UPDATE settings SET `value`=” . $patchID-1 . “ WHERE `key`=‘database-patch-id’ ” );
    exit();
    }
    }
    Apply your database patches
  • $testTables = array();
    mysqli_select_db( $config['patched_db'] );
    $result = mysql_query("SHOW TABLES");
    while( $row = mysql_fetch_row($result) )
    {
    $testTables[ $row[0] ] = array();
    }
    foreach( $testTables as $table => $fields )
    {
    $result = mysql_query("SHOW COLUMNS FROM " . $table );
    while( $row = mysql_fetch_assoc( $result ) )
    {
    $tables[ $table ][ $row['Field'] ] = $row;
    }
    }
    Turn your database schema into an array
  • Compare your patched database to what you expected
    http://joefreeman.co.uk/blog/2009/07/php-script-to-compare-mysql-database-schemas/
  • Databases: Test Database
    If you are applying changes to your database structure, you will need another test database
    Changes are first applied to the test database
    Comparisons run against it
    Unit testing run against code working with that database
    When all is clear, the live database can be patched and upgraded
  • Ask the audience
    Database integration, patching, testing and deployment is probably the weakest link in this deployment chain
  • Unit testing
    While its good practice to only commit code which passes unit tests, sometimes a commit can break existing code if you are a lazy svn updater
    Run the unit tests against sandboxed code before pushing the deployment live
    Did the deployment fail?
  • Unit testing
    Both PHPUnit and PHP SimpleTest have command line interface
    Options:
    Parse the output and look for errors; then continue once its done
    Store a report, and require manual approval before continuing with deployment
    phpunit –testdox-text somefile.txt MyTests
    *this isn’t a stage I’ve actually implemented in our deployment pipeline, just something I’m working on
  • The problem with including Unit Tests
    Running unit tests take time
    We need to log deployment attempts, and try and deploy them once the tests have been run
    We need a central deployment system
  • Photo of USB “kill switch”
    http://www.flickr.com/photos/stevendepolo/3517227492/
  • Triggering deployment: PHP
    echo shell_exec( ‘/var/deploy/deploy.sh ’ . $project . ‘ ‘ . $environment );
    What about root?
    Deployment script requires root access? Update sudoers file
  • PHP Deploy as Root
    Edit the sudoers file
    Sudovisudo
    Create an alias for your deployment scripts
    Cmnd_Alias DPLY = /var/deploy/script1, /var/deploy/script2
    Let the webserver execute as root, without requiring a password
    www-data ALL=(ALL) NOPASSWD: DPLY
  • Automating deployment
    Cron
    Postcommit hooks
    Do this for your bleeding edge staging area; its good to continually test code in its live server environment
    Scheduled deployments
  • Deployment Infrastructure
    Deploying projects across multiple servers?
    Send your commands over SSH to a remote server
    Implement a skeleton deployment system on each server, called from a central deployment area
  • Build a deployment platform
    Projects
    Deployment areas:
    Bleeding
    Staging
    Production
    Configurations, reports and deployment schedules
  • Recap
    Export your repository
    Apply your permission changes
    Swap in/out the appropriate configuration files
    Backup your (test) database
    Patch your database
    Unit test validation
    Swap in/out your configuration files
    Pull in user contributed files
    Backup your environment database
    Patch your live database
    Update your symlinks
  • Rolling back
    Shit! That last deployment didn’t go as planned!
    Symlinks let you keep copies
    Database backup before patches were applied – just incase
    Database patch rollback files – allows you to keep new data but undo structural changes
    Make an undo button in your deployment platform; if you don’t you will need it – if you do, you wont*!
    * OK, I lied, you probably will at some point
  • Caveats
    Queue cheesy stock photo of confused bean figure
  • Caveats
    Some useful pointers when having multiple versions online (bleeding, staging and production)
    Keep robots out (robots.txt meta_robots)
    You don’t want search engines taking your users to the staging environment, nor do you want to be peanalised for duplicate content
    Keep unwanted users out (.htaccess or limited user database)
    Make it clear that the environment is non-production – in case a production user stumbles upon staging!
  • Conclusion
    Deployment needs to take into account a lot of things
    Small and simple home-brew scripts, processes and techniques should help you out
    Look at pulling them together into a simple web-based deployment centre
  • Deploy your projects quickly!
    @michaelpeacock
    mkpeacock@gmail.com
    michaelpeacock.co.uk
    http://slidesha.re/phpdeploy
    http://www.flickr.com/photos/jurvetson/4853963652/sizes/m/in/photostream/