How automated cloud infrastructure setups can help with Continuous Delivery
Upcoming SlideShare
Loading in...5
×
 

How automated cloud infrastructure setups can help with Continuous Delivery

on

  • 431 views

 

Statistics

Views

Total Views
431
Views on SlideShare
395
Embed Views
36

Actions

Likes
0
Downloads
14
Comments
0

4 Embeds 36

http://ehaselwanter.com 17
https://twitter.com 11
http://localhost 7
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    How automated cloud infrastructure setups can help with Continuous Delivery How automated cloud infrastructure setups can help with Continuous Delivery Presentation Transcript

    • Donnerstag, 19. September 13
    • “How automated cloud infrastructure setups can help with Continuous Delivery” Edmund Haselwanter, eh@cloudbau.de, @ehaselwanter Donnerstag, 19. September 13
    • Why Continuous Delivery? For most companies, IT functions as the nervous system and provides an increasing amount of the organizational muscle mass. Most critical business functions are entirely automated within IT, and 95% of all capital projects depend on IT to get done. Today, nearly every business decision will result in at least one IT change. „IT REVOLUTION MANIFESTO“, http://itrevolution.com Theory of Constraints → → E.M. Goldratt, 1984 cloudbau Donnerstag, 19. September 13 Gene Kim, Kevin Behr, George Spafford, 2013
    • Continuous Delivery Wikipedia: http://en.wikipedia.org/wiki/Continuous_delivery ● ● ● ● Version Control Continuous Integration Automated Testing Continuous Deployment cloudbau Donnerstag, 19. September 13
    • We got the software development part right! Request Approve Develop Test WaterScrum Deploy - Fall The core, chronic conflict that every IT leader faces is the need to simultaneously enable faster time to market (i.e., make as many changes as you can), while providing stable, secure and reliable IT services (i.e., make as few changes as you can).  Gene Kim „Lowering risk of change through tools and culture“ John Allspaw, Paul Hammond, 2009 (Flickr) small changes, often reproducible Donnerstag, 19. September 13 culture change Dev+Ops standardized cloudbau expect failure feedback (e.g. tests/metrics/...)
    • The Last Mile: Dev/Test/Prod Parity ● ● ● ● ● ● ● ● Hardware configuration (e.g. amount of RAM, HDD vs. SSD) Software used (e.g. sqlite in dev, mysql in prod) Software/library versions Deployment (e.g. automated for test, manual for prod) System configuration (debug flags, logging) External dependencies (dev: payment mocked out vs. prod: real gateway) Database contents (huge prod DB with millions of entries vs. few entries in dev) Operating system (e.g. dev: Mac OS X, prod: Linux) See also http://12factor.net/ cloudbau Donnerstag, 19. September 13
    • The Last Mile: Dev/Test/Prod Parity ● ● Log level “normal” to chatty on production ● ● Changes in external dependencies (Sun Java to Oracle Java) ● ● ● ● SQL Query no problem in dev environment but too slow for production data ● ● Snowflake back-port: Live Tuning of Kernel/Mysql parameters, got lost after re-install CI build broken due to minor version change in upstream mocking framework (0.4.1 to 0.4.2) Sessions invalid in production due to time difference on multiple servers. No problem in dev environment (just one app and one db server) Different hardware: 32 vs. 64 bit. libraries missing Different hardware: optimized CPU flags on CI system. build not working in prod Staging In-house, Prod AWS Cloud: Latency ok during staging test, problem on production Development on Mac OSX/Windows, deploy on Linux: File not found (case sensitive) See also http://12factor.net/ cloudbau Donnerstag, 19. September 13
    • Getting to the solution... cloudbau Donnerstag, 19. September 13
    • Infrastructure as Code Application Environment Infrastructure cloudbau Donnerstag, 19. September 13
    • Building Blocks: Tools and Culture Config-DB CPUVirt. Binary-Repo StorageVirt. SDN System Description Orchestration Prod Env Programmable Infrastructure (Cloud) Continuos Delivery Anti-Fragile Organizations Cross-Functional Teams Processes and Culture cloudbau Donnerstag, 19. September 13 Dev Env Executeable System Description Continuous Build DevOps Stage Env
    • First... automate and equalize all environments ● ● ● ● ● Chef/Puppet/Cfengine/.... for config Dependency management (for programming languages, packages etc.) Dependency management for config scripts as well (e.g. Berkshelf for chef) Everything in version control Infrastructure can still be different... cloudbau Donnerstag, 19. September 13
    • ... then use a cloud to equalize infrastructure! ● ● ● Encode the infrastructure as well Describe all the pieces of the infrastructure in code Now it’s code: it can be dynamic! cloudbau Donnerstag, 19. September 13
    • But ... There be dragons Infrastructure as Code => We need a process Source Code Test X Compiler Artifact cloudbau Donnerstag, 19. September 13
    • The implementation patterns using OSS ● OpenStack / Eucalyptus / ... ● Chef/Puppet/Cfengine/ ... ● Jenkins (Building and testing the SW but also infrastructures) ● NetflixOSS (Asgard + Aminate) cloudbau Donnerstag, 19. September 13
    • New Question in the Cloud Era: What is my deploy Artifact ● The Software Package itself ● The whole VM ● A prepared container ● ... cloudbau Donnerstag, 19. September 13
    • The Software Package is the Artifact ● The build artifact is a binary. ● Use IaaS API to create the environment ● Deployment is after the server creation ● Deploy with automation or any other tool ● The whole VM ● “from source” is just a special case cloudbau Donnerstag, 19. September 13
    • HP Cloud + Jenkins + Puppet Binary Repo HP Cloud Servers (OpenStack) Jenkins Puppet Modules cloudbau Donnerstag, 19. September 13
    • The VM itself is the Artifact ● Build artifact is a image (optimal for stateless service) ● Only install needed, there is no upgrade/deploy ● IaaS must support Loadbalancer as a service (or something similar) ● Orchestration for deployment steps needed cloudbau Donnerstag, 19. September 13
    • Jenkins + AWS + Asgard + Aminate + Chef Jenkins Aminate AMIs Asgard + Chef AWS EC2 (ASG + ELB) cloudbau Donnerstag, 19. September 13
    • A container is the Artifact ● ● ● ● ● cloudbau Donnerstag, 19. September 13 Optimization of the VM idea Transportable lightweight OS containers (e.g. LXC/docker) Very often back to upgrade/deploy Needs external orchestration as well Very new toolchain emerging (docker/maestro)
    • https://www.docker.io/the_whole_story/ cloudbau Donnerstag, 19. September 13
    • Jenkins + docker + cfengine + Aminate + Chef Jenkins Docker Host VM Maestro + cfengine AWS EC2 (ASG + ELB) cloudbau Donnerstag, 19. September 13
    • Cloud can help with developer environments as well ● ● ● ● cloudbau Donnerstag, 19. September 13 Have your own environment Brings the production setup to the developer One description for local VM and Cloud environment (Dev/Prod) Needs external orchestration as well
    • OpenStack + Vagrant + Chef Chef Repo / Server Binary Repo Jenkins Vagrant cloudbau Donnerstag, 19. September 13 OpenStack Servers
    • One more Thing: Other Areas Cloud can help ● Persistent Data: Copy/Clone Production Data to test migrations ● Debugging: Developer can stand up whole environment (even with data) ● Backup/Restore (snapshot whole images/servers for rollback) cloudbau Donnerstag, 19. September 13
    • What’s missing in the OpenSource Ecosystem? ● Better Orchestration/ Pipeline support in Jenkins (Commercial CI-Servers are advanced here) ● OpenSource Orchestration Engines (like OpenStack Heat) ● NetflixOSS working on OpenStack cloudbau Donnerstag, 19. September 13
    • Don’t forget about Operations ● Cloud/App-Monitoring Solutions (Nagios won’t cut it in dynamic environments) ● Logfiles ● Metric driven autoscaling cloudbau Donnerstag, 19. September 13
    • Thank you! Questions? cloudbau Donnerstag, 19. September 13