Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Evolution of deploy.sh

340 views

Published on

Slides from my talk at HackCyprus Summit 2017. It's about how web application deployment evolved, where it is now, and where it is going. Sort of.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Evolution of deploy.sh

  1. 1. Evolution of deploy.sh Leonid Mamchenkov
  2. 2. About Leonid Mamchenkov ● System administrator, web developer ● 20+ years of experience ■ Primetel ■ FxPro ■ Easy Forex ■ Qobo ● Open Source fan and advocate ■ Linux (Fedora, CentOS, Red Hat) ■ PHP (bash, perl) ■ MySQL ■ Nginx ■ Git ■ Vim Qobo Ltd ● Established in February 2013 ● Offices in Nicosia, Limassol, London ● Web design, development, hosting ■ WordPress ■ WooCommerce ■ Amazon AWS ● Software development ■ CakePHP ■ Qobrix Platform ■ CRM Systems ■ Intranets ■ Portals ■ Integrations
  3. 3. What is deployment? Application ● Code (HTML, CSS, JavaScript, PHP) ● Assets (images, fonts) ● Configuration (credentials, defaults, runtime) ● Database (schema, content) Infrastructure ● Hardware ● Operating system ● Filesystem, networking ● Services Deployment: installing, updating or removing the application to or from a particular infrastructure. Software deployment is all of the activities that make a software system available for use. https://en.wikipedia.org/wiki/Software_deployment
  4. 4. Why is deployment automation important? Speed ● Faster deployment ● Frequency ● Feedback loop Release early, release often! https://en.wikipedia.org/wiki/Release_early,_release_often Quality ● Fewer human error ● Consistency ● Predictability Cost ● Machine vs. human ● Reuse ● Troubleshooting
  5. 5. Evolution of the Web (last 20 years) Overall: ● More users ● More data ● More devices ● More features ● More expectations Technologies: Static HTML, Cookies, SSI, CGI, SSL, XML, RPC, API, AJAX, RSS, Web 2.0, Mobile Web, WebSockets, IoT, Big Data, cloud computing, and more ... As the web evolved, so did the deployment of the web applications.
  6. 6. Prehistoric days : before deploy.sh Application ● Very small and basic. ● Often, just a few static HTML and image files. ● Sometimes, an occasional CGI script. Infrastructure ● Shared hosting with limited access. ● Single server. ● No version control. ● No SSH access. Deployment: manual upload via FTP or some sort of web-based control panel. If you are still here, at least look at some FTP deployment / smart upload tools, like SmartFTP.
  7. 7. The birth of deploy.sh #!/bin/bash rsync -re ssh . leonid@example.com:~/public_html/ ssh leonid@example.com << EOF cd ~/public_html/ chown -R leonid.nobody {logs,tmp,uploads} mysql -u leonid example_com < db/mysql.sql EOF echo “All Done” Basic automation ● Simple, step-by-step ● Mostly shell ● No specialized tools ● Basic remote access ● No error handling ● Very specific ● Not portable ● No concurrency control ● No trail ● No changesets ● No documentation ● Often insecure
  8. 8. Version control / The birth of build.sh #!/bin/bash ssh leonid@example.com << EOF cd ~/public_html/ git pull origin master ./bin/build.sh EOF echo “All Done” Improvements ● More reliable file synchronization ● Much faster on larger projects ● Basic permissions management ● Changesets ● Git/Subversion hooks ● Basic concurrency control ● More portability ● Often better security ● Submodules, exports, etc. Separation of concerns between deployment and build processes. Deployment seems simple and easy. Build process - not so much.
  9. 9. The explosion of build tools <?php class DbCreate extends AbstractCommand { public function mysqlDbCreate($host, $port, $name, $user, $pass) { $this->taskMysqlDbCreate() ->host($host)->port($port) ->user($user)->pass($pass) ->db($name)->run(); } } Improvements ● Specialized syntax ● Dependency management ● Improved error handling ● Improved output / verbosity ● Improved portability ● Reusable tasks and modules ● Documentation Tools GNU make, CMake, Apache Ant, Phing, Rake, Phake, Capistrano, Robo, Deployer, many others.
  10. 10. More specialized tools : build parts <?php use SymfonyComponentFinderFinder; $iterator = Finder::create()->files()->name(‘*.php’)->in(‘./src’); $sami = new SamiSami($iterator, [ ‘title’ => ‘My Project’, ‘build_dir’ => ‘build/docs/source’, ‘cache_dir’ => ‘build/docs/source/cache’, ]); return $sami; Build specialization ● Testing frameworks ● Asset management ● Documentation ● Dependency management Tools phpUnit, JUnit, Nightwatch JS, Sass, LESS, minify, phpDocumentor, Doxygen, Sami, composer, npm, bower, gulp, grant, yarn, and many more.
  11. 11. More specialized tools : database management <?php use MigrationsAbstractMigration; class AddEmailFieldToUsers extends AbstractMigration { public function change() { $table = $this->table(‘users’); $table->addColumn(‘email’, ‘string’, [‘limit’ => 255]); $table->update(); } } Database management ● Versioned schema ● SQL abstraction ● Cross-platform / cross-engine ● Testable changes ● Upgrade and rollback ● Initial data import / seeders ● Data manipulation ● Documentation improvements ● Conflict resolution Tools Doctrine, Phinx, DBDeploy, etc.
  12. 12. configure.sh? Infrastructure as Code --- - name: Nginx and PHP setup hosts: web-servers roles: - geerlingguy.nginx - geerlingguy.php - geerlingguy.php-mysql - geerlingguy.composer - geerlingguy.letsencrypt Server provisioning, virtualization, orchestration, configuration management ● Automation ● Documentation ● Testable installations ● Concurrency ● Reusable configurations ● Consistency Tools Ansible, Puppet, Chef, Salt, Docker, Kubernetes, Amazon AWS, Heroku, and many more.
  13. 13. What’s going on? Application Infrastructure Build System Administrator Developer Deploy Configure
  14. 14. DevOps : ChatOps Application Infrastructure Build Developers, SysAdmins, QA, PMs Deploy Configure /deploy install example.com live v2.0
  15. 15. Remote Triggers : Use existing or build your own Chat ● HipChat, Slack, Rocket.Chat, IRC ● Bots, /commands, @mentions Bug Trackers / PM Tools ● Redmine, Request Tracker, JIRA ● Custom ticket types, statuses, workflows Continuous Integration / Delivery ● TravisCI, BitBucket Pipelines, Jenkins ● Workflows, pipelines, stages Source Code Repositories ● GitHub, BitBucket, GitLab ● Hooks, events
  16. 16. Remote Triggers : Webhooks HipChat Redmine Jenkins GitHub JSON Your API /usr/bin/sudo ● Configure ● Deploy ● Build
  17. 17. HipChat Example : Slash Command
  18. 18. HipChat Example : API <?php $payload = $this->app->request->getBody(); $payload = json_decode($payload, true); $user = $payload[‘item’][‘message’][‘from’][‘id’]; $user = hipchatToUnix($user); $command = $payload[‘item’][‘message’][‘message’]; $command = preg_replace(‘#^/deploy #’, ‘’, $command); # $command = “update example.com live v2.0” exec(“/usr/bin/sudo -u $user /usr/local/bin/deploy.sh $command”); Keep in mind ● Security! Security! Security! ● JSON contains context ● Handle failure ● Handle timeouts ● Log excessively! ● Check existing implementations There is more than one way to do it.
  19. 19. Things can get rather complex Deployment Stages ● Install ● Update ● Rollback ● Remove Different Environments ● Development ● Test ● Staging / Pre-production ● Live / Production Layered Infrastructure ● Web servers ● Database servers ● Load balancers ● Caching / proxy servers Atomic Deployments ● Multiple deployed versions ● Common assets, user uploads ● Database migrations ● Caching
  20. 20. KISS : Keep it simple, stupid ● Communicate! Deployment is part of the culture, not just technology. ● Opt for existing solutions and best practices. Do not reinvent the wheel. ● Opt for gradual changes. One step at a time. And baby steps too! ● Smaller, independent units are easier to handle. But don’t overdo it. ● Confidence is the key. ● Keep up! After all, technology constantly evolves.
  21. 21. The End. Thank You
  22. 22. Questions? Leonid Mamchenkov http://mamchenkov.net leonid@mamchenkov.net https://twitter.com/mamchenkov Qobo Ltd https://qobo.biz/ and https://qobrix.com info@qobo.biz and careers@qobo.biz https://github.com/QoboLtd

×