- Automated cloud infrastructure setups using tools like Chef, Puppet, and Cfengine can help enable continuous delivery by ensuring development, test, and production environments are standardized and equalized. - Describing infrastructure as code allows environments to be dynamically provisioned in the cloud, eliminating differences between environments. - There are different approaches to the deployment artifact - it can be the software package, a virtual machine image, or a container. The appropriate solution depends on factors like whether the application is stateless. - Open source tools like Jenkins, Docker, OpenStack, and configuration management solutions can be combined to implement continuous delivery pipelines that provision standardized environments for development and production in the cloud.

1. Donnerstag, 19. September 13

2. “How automated cloud infrastructure setups
can help with Continuous Delivery”
Edmund Haselwanter, eh@cloudbau.de, @ehaselwanter
Donnerstag, 19. September 13

3. Why Continuous Delivery?
For most companies, IT functions as the nervous system and provides
an increasing amount of the organizational muscle mass. Most critical
business functions are entirely automated within IT, and 95% of all
capital projects depend on IT to get done. Today, nearly every business
decision will result in at least one IT change.
„IT REVOLUTION MANIFESTO“, http://itrevolution.com
Theory of
Constraints
→
→
E.M. Goldratt, 1984
cloudbau
Donnerstag, 19. September 13
Gene Kim, Kevin Behr,
George Spafford, 2013

4. Continuous Delivery
Wikipedia: http://en.wikipedia.org/wiki/Continuous_delivery
●
●
●
●
Version Control
Continuous Integration
Automated Testing
Continuous Deployment
cloudbau
Donnerstag, 19. September 13

5. We got the software development part right!
Request
Approve
Develop
Test
WaterScrum
Deploy
- Fall
The core, chronic conflict that every IT
leader faces is the need to
simultaneously enable faster time to
market (i.e., make as many changes as
you can), while providing stable, secure
and reliable IT services (i.e., make as
few changes as you can).
Gene Kim
„Lowering risk of change through tools and culture“
John Allspaw, Paul Hammond, 2009 (Flickr)
small changes, often
reproducible
Donnerstag, 19. September 13
culture change Dev+Ops
standardized
cloudbau
expect failure
feedback (e.g. tests/metrics/...)

6. The Last Mile: Dev/Test/Prod Parity
●
●
●
●
●
●
●
●
Hardware configuration (e.g. amount of RAM, HDD vs. SSD)
Software used (e.g. sqlite in dev, mysql in prod)
Software/library versions
Deployment (e.g. automated for test, manual for prod)
System configuration (debug flags, logging)
External dependencies (dev: payment mocked out vs. prod: real gateway)
Database contents (huge prod DB with millions of entries vs. few entries in dev)
Operating system (e.g. dev: Mac OS X, prod: Linux)
See also http://12factor.net/
cloudbau
Donnerstag, 19. September 13

7. The Last Mile: Dev/Test/Prod Parity
●
●
Log level “normal” to chatty on production
●
●
Changes in external dependencies (Sun Java to Oracle Java)
●
●
●
●
SQL Query no problem in dev environment but too slow for production data
●
●
Snowflake back-port: Live Tuning of Kernel/Mysql parameters, got lost after re-install
CI build broken due to minor version change in upstream mocking framework (0.4.1 to
0.4.2)
Sessions invalid in production due to time difference on multiple servers. No problem in
dev environment (just one app and one db server)
Different hardware: 32 vs. 64 bit. libraries missing
Different hardware: optimized CPU flags on CI system. build not working in prod
Staging In-house, Prod AWS Cloud: Latency ok during staging test, problem on
production
Development on Mac OSX/Windows, deploy on Linux: File not found (case sensitive)
See also http://12factor.net/
cloudbau
Donnerstag, 19. September 13

8. Getting to the solution...
cloudbau
Donnerstag, 19. September 13

9. Infrastructure as Code
Application
Environment
Infrastructure
cloudbau
Donnerstag, 19. September 13

10. Building Blocks: Tools and Culture
Config-DB
CPUVirt.
Binary-Repo
StorageVirt.
SDN
System Description
Orchestration
Prod Env
Programmable Infrastructure (Cloud)
Continuos Delivery
Anti-Fragile Organizations
Cross-Functional Teams
Processes and Culture
cloudbau
Donnerstag, 19. September 13
Dev Env
Executeable System Description
Continuous Build
DevOps
Stage Env

11. First... automate and equalize all
environments
●
●
●
●
●
Chef/Puppet/Cfengine/.... for config
Dependency management (for programming languages, packages etc.)
Dependency management for config scripts as well (e.g. Berkshelf for chef)
Everything in version control
Infrastructure can still be different...
cloudbau
Donnerstag, 19. September 13

12. ... then use a cloud to equalize
infrastructure!
●
●
●
Encode the infrastructure as well
Describe all the pieces of the infrastructure in code
Now it’s code: it can be dynamic!
cloudbau
Donnerstag, 19. September 13

13. But ... There be dragons
Infrastructure as Code => We need a process
Source
Code
Test
X
Compiler
Artifact
cloudbau
Donnerstag, 19. September 13

14. The implementation patterns using OSS
● OpenStack / Eucalyptus / ...
● Chef/Puppet/Cfengine/ ...
● Jenkins (Building and testing the SW but also
infrastructures)
● NetflixOSS (Asgard + Aminate)
cloudbau
Donnerstag, 19. September 13

15. New Question in the Cloud Era: What is my
deploy Artifact
● The Software Package itself
● The whole VM
● A prepared container
● ...
cloudbau
Donnerstag, 19. September 13

16. The Software Package is the Artifact
● The build artifact is a binary.
● Use IaaS API to create the environment
● Deployment is after the server creation
● Deploy with automation or any other tool
● The whole VM
● “from source” is just a special case
cloudbau
Donnerstag, 19. September 13

17. HP Cloud + Jenkins + Puppet
Binary Repo
HP Cloud Servers
(OpenStack)
Jenkins
Puppet Modules
cloudbau
Donnerstag, 19. September 13

18. The VM itself is the Artifact
● Build artifact is a image (optimal for stateless service)
● Only install needed, there is no upgrade/deploy
● IaaS must support Loadbalancer as a service (or something similar)
● Orchestration for deployment steps needed
cloudbau
Donnerstag, 19. September 13

19. Jenkins + AWS + Asgard + Aminate + Chef
Jenkins
Aminate
AMIs
Asgard
+ Chef
AWS EC2 (ASG + ELB)
cloudbau
Donnerstag, 19. September 13

20. A container is the Artifact
●
●
●
●
●
cloudbau
Donnerstag, 19. September 13
Optimization of the VM idea
Transportable lightweight OS containers (e.g. LXC/docker)
Very often back to upgrade/deploy
Needs external orchestration as well
Very new toolchain emerging (docker/maestro)

21. https://www.docker.io/the_whole_story/
cloudbau
Donnerstag, 19. September 13

22. Jenkins + docker + cfengine + Aminate +
Chef
Jenkins
Docker
Host VM
Maestro
+ cfengine
AWS EC2 (ASG + ELB)
cloudbau
Donnerstag, 19. September 13

23. Cloud can help with developer environments
as well
●
●
●
●
cloudbau
Donnerstag, 19. September 13
Have your own environment
Brings the production setup to the developer
One description for local VM and Cloud environment (Dev/Prod)
Needs external orchestration as well

24. OpenStack + Vagrant + Chef
Chef Repo / Server
Binary Repo
Jenkins
Vagrant
cloudbau
Donnerstag, 19. September 13
OpenStack Servers

25. One more Thing: Other Areas Cloud can
help
● Persistent Data: Copy/Clone Production Data to test migrations
● Debugging: Developer can stand up whole environment (even with data)
● Backup/Restore (snapshot whole images/servers for rollback)
cloudbau
Donnerstag, 19. September 13

26. What’s missing in the OpenSource
Ecosystem?
● Better Orchestration/ Pipeline support in Jenkins (Commercial CI-Servers
are advanced here)
● OpenSource Orchestration Engines (like OpenStack Heat)
● NetflixOSS working on OpenStack
cloudbau
Donnerstag, 19. September 13

27. Don’t forget about Operations
● Cloud/App-Monitoring Solutions (Nagios won’t cut it in dynamic
environments)
● Logfiles
● Metric driven autoscaling
cloudbau
Donnerstag, 19. September 13

28. Thank you! Questions?
cloudbau
Donnerstag, 19. September 13