Automated Deployment


Published on

@phpne (March) talk by @michaelpeacock

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Store expected schema, and generate schema array from applied patches.
  • Automated Deployment

    1. 1. Automated Deployment<br />Building a simple automated deployment platform with PHP and Linux<br />Michael<br />
    2. 2. whois?<br />Senior / Lead Web Developer<br />Zend Certified Engineer<br />Published Author<br />PHP 5 Social Networking, PHP 5 E-Commerce development & more<br />
    3. 3. Deployment: (an) old style approach<br />Take website offline / put into maintenance mode<br />Backup everything<br />Upload new files - FTP<br />Upgrade database<br />Put online, and hope for the best<br />Do it twice: once for staging and once for deployment<br />
    4. 4.<br />
    5. 5. The problem<br />Down time for upgrades<br />Manual process<br />FTP takes time; <br />forgot to CHMOD? <br />Clients want to see progress now!<br />Bugs and issues can lie dormant for some time<br />
    6. 6. What about...<br />Many existing solutions are geared towards large projects<br />What about...<br />the little guy;<br />the small agency<br />the web app start up on an entry level VPS?<br />
    7. 7. What's in store?<br />A few simple techniques, scripts and ideas that we currently use to make deployment easy<br />
    8. 8. Deployment: the basics<br />Get your latest code from version control, and stick it online<br />Keep a central record of all the CHMOD / CHOWNing that you need to do<br />Swap around your database connection details and other suitable configuration files<br />
    9. 9. SVN Export<br />Start with a simple svn export<br />Store the date/time in a variable <br />Create two folders, named with the current date/time. One within the web root, one outside of it<br />Two exports: public and private (or one export, and some moving around of folders – up to you!)<br />
    10. 10. #!/bin/bash<br />DATE=`date +%H-%M-%e-%m-%y`<br />mkdir /var/www/staging/$DATE/<br />mkdir /var/www/staging-private/$DATE/<br />svn export --quiet --username phpne --password PhpN3 httP://localhost/svn/project/trunk /var/www/staging/$DATE/<br />svn export --quiet --username phpne --password PhpN3 http://localhost/svn/project/private /var/www/staging-private/$DATE/<br />
    11. 11. SVN Export<br />Keep your servers svn client happy! It will ask what to do with the svn password, and nobody will listen – so tell it!<br />sudonano /var/www/.subversion/servers<br />store-plaintext-passwords = no<br />
    12. 12. Autonomy<br />ln –s /staging /live<br />
    13. 13. Autonomy<br />When the latest code is checked out, tests have been run, uploads imported, configuration changed and database patched we need to swap this into place instantly<br />The answer: symlinks<br />
    14. 14. #!/bin/bash<br />DATE=`date +%H-%M-%e-%m-%y`<br />...<br />rm /home/user/public_html/<br />ln –s /var/www/staging/$DATE/ /home/user/public_html/<br />Sadly, you can’t edit a symlink, hence rm<br />
    15. 15. My user profile pictures aren’t in version control…<br />
    16. 16. User contributed files<br />Store them elsewhere?<br />On a content delivery network?<br />On a sub-domain<br />Symlink them<br />Copy them in post svn export?<br />A bit nasty and takes time, and what about new user uploads during the copying process?<br />
    17. 17. The database<br />
    18. 18. Photo of database table not found, or mysql gone away error message<br /><br />
    19. 19. Database changes: patches<br />For database changes to apply on deploy, you need some deploy aware code in your project. <br />Multi-query patch processing<br />Schema compare; its easy to forget a database patch!<br />Backup database before applying patches<br />
    20. 20. public function updateDatabase( $patchID, $some=false ) <br />{ <br /> // look for the next patch <br /> if( file_exists( FRAMEWORK_PATH . '../database/patches/' . ++$patchID . '.php' ) ) <br /> { <br /> $sql = file_get_contents( FRAMEWORK_PATH . '../database/patches/' . $patchID . '.php' );<br /> // apply the changes from the patch <br />mysqli_multi_query( $sql ); <br /> // lather, rinse and repeat<br /> $this->updateDatabase( $patchID, true ); <br /> } <br /> else if( $some ) <br /> { <br /> // All done? Update patch ID in database<br />mysqli_query(“UPDATE settings SET `value`=” . $patchID-1 . “ WHERE `key`=‘database-patch-id’ ” );<br /> exit(); <br /> } <br />}<br />Apply your database patches<br />
    21. 21. $testTables = array();<br />mysqli_select_db( $config['patched_db'] );<br />$result = mysql_query("SHOW TABLES");<br />while( $row = mysql_fetch_row($result) ) <br />{<br /> $testTables[ $row[0] ] = array();<br />}<br />foreach( $testTables as $table => $fields )<br />{<br /> $result = mysql_query("SHOW COLUMNS FROM " . $table );<br /> while( $row = mysql_fetch_assoc( $result ) ) <br /> {<br /> $tables[ $table ][ $row['Field'] ] = $row;<br /> }<br />}<br />Turn your database schema into an array<br />
    22. 22. Compare your patched database to what you expected<br /><br />
    23. 23. Databases: Test Database<br />If you are applying changes to your database structure, you will need another test database<br />Changes are first applied to the test database<br />Comparisons run against it<br />Unit testing run against code working with that database<br />When all is clear, the live database can be patched and upgraded<br />
    24. 24. Ask the audience<br />Database integration, patching, testing and deployment is probably the weakest link in this deployment chain<br />
    25. 25. Unit testing<br />While its good practice to only commit code which passes unit tests, sometimes a commit can break existing code if you are a lazy svn updater<br />Run the unit tests against sandboxed code before pushing the deployment live<br />Did the deployment fail?<br />
    26. 26. Unit testing<br />Both PHPUnit and PHP SimpleTest have command line interface<br />Options:<br />Parse the output and look for errors; then continue once its done<br />Store a report, and require manual approval before continuing with deployment<br />phpunit –testdox-text somefile.txt MyTests<br />*this isn’t a stage I’ve actually implemented in our deployment pipeline, just something I’m working on<br />
    27. 27. The problem with including Unit Tests<br />Running unit tests take time<br />We need to log deployment attempts, and try and deploy them once the tests have been run<br />We need a central deployment system<br />
    28. 28. Photo of USB “kill switch”<br /><br />
    29. 29. Triggering deployment: PHP<br />echo shell_exec( ‘/var/deploy/ ’ . $project . ‘ ‘ . $environment );<br />What about root?<br />Deployment script requires root access? Update sudoers file<br />
    30. 30. PHP Deploy as Root<br />Edit the sudoers file<br />Sudovisudo<br />Create an alias for your deployment scripts<br />Cmnd_Alias DPLY = /var/deploy/script1, /var/deploy/script2<br />Let the webserver execute as root, without requiring a password<br />www-data ALL=(ALL) NOPASSWD: DPLY<br />
    31. 31. Automating deployment<br />Cron<br />Postcommit hooks<br />Do this for your bleeding edge staging area; its good to continually test code in its live server environment<br />Scheduled deployments<br />
    32. 32. Deployment Infrastructure<br />Deploying projects across multiple servers?<br />Send your commands over SSH to a remote server<br />Implement a skeleton deployment system on each server, called from a central deployment area<br />
    33. 33. Build a deployment platform<br />Projects<br />Deployment areas:<br />Bleeding<br />Staging<br />Production<br />Configurations, reports and deployment schedules<br />
    34. 34. Recap<br />Export your repository<br />Apply your permission changes<br />Swap in/out the appropriate configuration files<br />Backup your (test) database<br />Patch your database<br />Unit test validation<br />Swap in/out your configuration files<br />Pull in user contributed files<br />Backup your environment database<br />Patch your live database<br />Update your symlinks<br />
    35. 35. Rolling back<br />Shit! That last deployment didn’t go as planned!<br />Symlinks let you keep copies<br />Database backup before patches were applied – just incase<br />Database patch rollback files – allows you to keep new data but undo structural changes<br />Make an undo button in your deployment platform; if you don’t you will need it – if you do, you wont*!<br />* OK, I lied, you probably will at some point<br />
    36. 36. Caveats<br />Queue cheesy stock photo of confused bean figure<br />
    37. 37. Caveats<br />Some useful pointers when having multiple versions online (bleeding, staging and production)<br />Keep robots out (robots.txt meta_robots)<br />You don’t want search engines taking your users to the staging environment, nor do you want to be peanalised for duplicate content<br />Keep unwanted users out (.htaccess or limited user database)<br />Make it clear that the environment is non-production – in case a production user stumbles upon staging!<br />
    38. 38. Conclusion<br />Deployment needs to take into account a lot of things<br />Small and simple home-brew scripts, processes and techniques should help you out<br />Look at pulling them together into a simple web-based deployment centre<br />
    39. 39. Deploy your projects quickly!<br />@michaelpeacock<br /><br /><br /> <br /><br />