In this talk for CukeUp Jon Topper investigates if we can apply a software testing approach to validate our infrastructure configuration.

1. Test-driven
Infrastructure
Jon Topper

2. What is Infrastructure?
★ Physical Servers
★ Virtual / Cloud Servers
★ Switches
★ Firewalls
★ Routers
★ Load Balancers

3. 3 Year Infrastructure Lifecycle
Change
Build
Risk! 14%
Operate
86%

4. The Rise of DevOps
★ Increased collaboration between developers and
operations staff
★ Improved tooling for automation
★ “Dev” solutions to “Ops” problems

5. Infrastructure as Code

6. Puppet
node 'webserver' {
package { 'httpd':
ensure => latest
}
file { '/etc/httpd/httpd.conf':
require => Package['httpd'],
owner => root,
mode => 644,
content => template('httpd.conf')
}
service { 'httpd':
ensure => running,
enable => true,
require => File['/etc/httpd/httpd.conf']
}
}

7. “Dev” Tooling
★ IDEs, text editors, refactoring tools
★ Version Control Systems
★ Automated documentation generation
★ ... Testing?

8. Automated Infrastructure Testing
★ cucumber-puppet / rspec-puppet
★ cucumber-nagios
★ puppet-lint
★ cucumber-chef
★ vagrant-guard-demo

9. Cucumber Example
Scenario: Basic install of Apache
Given there is a running VM called "server"
When I apply a puppet manifest containing:
"""
include cucumber_defaults
class { sf_apache:
'Port' => '80',
'Children' => '10'
}
"""
Then a second manifest application should do nothing
And there should be 11 processes called “httpd” running
And the Apache module "core_module" should be loaded
And a process called “httpd” should be listening on TCP
port 80
And a GET request to http://localhost/server-status/
should return an http status of 200

10. Given there is a running VM called "server"

11. Vagrant
★ Template (“Box”) based virtual environment
★ Shared filesystem between host and guest
★ Snapshot support via “Sahara” plugin
★ API for scripted interaction
http://vagrantup.com/

12. Given there is a running VM called "server"
Given /^there is a running VM called
"([^"]*)"$/ do |vm_name|
vm_platform.vm( vm_name ).start
vm_platform.vm( vm_name ).snapshot
end

13. attr_reader :last_vm
def initialize
@name_map = {}
end
def vm(name)
if @name_map.has_key?(name)
@last_vm = @name_map[name]
return @name_map[name]
end
vm = create_vm_object_by_name( name )
@name_map[name] = vm
@last_vm = vm
return vm
end
def clean_tainted
@name_map.each { |name,vm|
vm.rollback
@name_map.delete(name)
}
end

14. When I apply a puppet manifest containing:
"""
include cucumber_defaults
class { sf_apache:
'Port' => '80',
'Children' => '10'
}
"""
Then a second manifest application should do nothing
★ Fragment uploaded with SCP
★ Puppet tasks run over Vagrant SSH link
★ Included manifests read from Vagrant shared folder

15. When /^I apply a puppet manifest(#{VMRE}) containing:$/ do |vmre, manifest_content|
vm = identified_vm( vmre )
file = Tempfile.new('cucumber-puppet')
begin
file.write(manifest_content)
file.fsync
vm.upload(file.path,'/tmp/cucumber-puppet.pp')
@puppet_command ="puppet apply --verbose --modulepath=#{$puppet_modulepath} " +
"--manifestdir=#{$puppet_manifestdir} --detailed-exitcodes --color=false " +
"/tmp/cucumber-puppet.pp"
exit_status = vm.sudo( @puppet_command ) do |type,data|
data.chomp!
puts data if data != “”
end
Test::Unit::assert( exit_status == 0 || exit_status == 2,
'Exit code of puppet run not 0 or 2 - errors' )
ensure
file.close
file.unlink
end
end

16. VMRE
VMRE ||= /(?: on the last VM| on the VM(?: called|) "(?:[^"]+)"|)/
def identified_vm( str )
case str
when /^( on the last VM|)$/
return vm_platform.last_vm
when /^ on the VM(?: called|) "([^"]+)"$/
return @vm_platform.vm( $1 )
end
end

17. And a GET request to http://localhost/server-status/
should return an http status of 200
Then /^a GET request to (.+)(#{VMRE}) should return an http status of (d+)$/
do |url,vmre,status|
vm = identified_vm( vmre )
response = vm.freeman.call('http.GET',url)
assert( response['code'].to_i == status.to_i,
"Response code 200 expected from #{url}, " +
"received #{response['code']}" )
end
★ ‘freeman’ is an XML-RPC service
★ First call starts a new XML-RPC server in the guest
★ Code shared over Vagrant’s folder system

18. Full Stack
Host Filesystem
Features
Step Definitions
Cumberbatch Freeman Client
Vagrant Library
Freeman Shared
sshd Server Folder
Vagrant VM

19. Benefits
★ Cleaner interfaces
★ Improved separation of concerns
★ Increased reusability
★ Rapid troubleshooting
★ Empowering for junior engineers

20. Challenges
★ Scenarios slow to run
★ Difficult to debug when snapshot rolled back
★ Multi-VM VirtualBox unstable on OS X
★ Good use of Cucumber not always obvious to the
sysadmin-minded

21. Jon Topper
jon@scalefactory.com
http://www.scalefactory.com/
Twitter: @jtopper