Breaking one or two Linux systems has always been pretty easy. But with the advent of configuration management and continuous deployment, we can now break thousands of Linux systems with a single push to Github. Quality testing is required to avoid catastrophe in this new world of automation, but writing good tests and running test infrastructure for config management can be both boring and complex. In this talk, we’ll look at a simple method for implementing automated testing for Puppet using common household items, including Git, Jenkins, Foreman, and shell scripts.

1. Continuous Integration
for Configuration Management
Michael Solberg
Principal Architect, Red Hat

2. 2
AGENDA
Continuous Integration
Configuration Management Testing
Implementing CI for Configuration
Management

3. 3
CONTINUOUS INTEGRATION
Did I just break something?
TEST BEFORE YOU PUSH
Syntax, Unit Tests
TEST AFTER YOU PUSH
Reproducibility, Build Environments, Unit Tests
TEST BEFORE YOU DEPLOY
Integration Tests, Performance Tests, Merge Tests

4. 4
EXAMPLE CI PIPELINE
Used by OpenStack Nova Project
Code moves through a set of manual and automated gates in a feature branch before
being committed to the master branch.

5. 5
HOW DO WE APPLY THIS TO PUPPET?
Configuration Management Testing is a Unique Challenge
● Configuration management systems are distributed.
● Configuration management often relies on external data.
● Configuration management systems rely on lots of 3rd
party
dependencies.
● Configuration management can only be applied the first time once.

6. 6
CI FOR CONFIGURATION MANAGEMENT
Did I just break something?
DOES THE MANIFEST COMPILE?
Syntax, Unit Tests
CAN I APPLY THE MANIFEST TO A SYSTEM?
Reproducibility, Build Environments, Unit Tests
DID IT DO WHAT I EXPECTED IT TO DO?
Integration Tests, Performance Tests, Merge Tests

7. 7
PROCESS
From code commit to tested configuration
● Code is committed to the repository.
● Commit is validated by CI Server
● CI Server updates the CF system with the new commit.
● CI Server provisions a new VM.
● The virtual machine applies the new configuration.
● CI Server checks with config management server that the configuration
was applied without error.
● CI Server runs functional tests against the configured VM.
● CI Server records the results and (optionally) powers off the VM.

8. 8
CONFIGURATION MANAGEMENT CI PIPELINE
For Puppet
Feedback is generated through interactions between the CI Server, CMS, and IaaS.
Configuration Management CI Workflow Virtual Infrastructure
1. commit to scm
3. provisions new system
feedback to devs
CI Server
config management
system
2. updates environment
configured
system
IaaS layer
4. points system
to new config
5. pulls latest config
and sends report
6. runs tests
7. gather feedback and
send to devs

9. 9
WHAT INGREDIENTS DO I NEED?
Essential Functionality
SOURCE CONTROL/CI SERVER
Git, Jenkins
CONFIGURATION MANAGEMENT SERVER
Foreman/Puppet Master
INFRASTRUCTURE AS A SERVICE
OpenStack, oVirt, VMWare

10. DEMO

11. 11
CHALLENGES/SOLUTIONS
Questions we encountered while developing the process.
● How do I know that the Puppet run for a given “build” is the latest
commit?
● Custom Puppet Fact injection by Jenkins
● How do I test upgrades as well as new deployments?
● Apply new changes to pre-provisioned VMs
● How do I manage the Puppet environments?
● Katello/Satellite provides API, r10k
● How do I promote code through different lifecycle environments?
● Katello/Satellite provides lifecycle management.

12. THANK YOU
plus.google.com/+RedHat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHatNewslinkedin.com/company/red-hat