Your SlideShare is downloading. ×
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Testing for Ops: Going Beyond the Manifest - PuppetConf 2013

4,728
views

Published on

"Testing for Ops: Going Beyond the Manifest" by Christopher Webber, Infrastructure Engineer, Demand Media. …

"Testing for Ops: Going Beyond the Manifest" by Christopher Webber, Infrastructure Engineer, Demand Media.

Presentation Overview: This talk aims to show the value of rspec-puppet for those who come from a more Ops-centric background. The focus will be on using tests to go beyond just rewriting manifests in rspec. Instead the focus will be on scenarios like: - Are the baseline security measures in place? - Do the differences between dev and prod get reflected? - Are the config elements that are core to the application present? In addition, tests will help to be a place to help document the oddities of our configurations and ensuring that minor changes don't result in catastrophe.

Speaker Bio: After beginning his career at UC Riverside supporting enterprise operations and bioinformatics research, Chris is now rocking being an infrastructure engineer at Demand Media in Santa Monica. He currently supports large high-traffic sites like eHow.com, LiveSTRONG.com, and Cracked.com. Chris enjoys attending local meetups, writing new Puppet modules, and creating small tools to make his team's lives a little easier. Find him on Twitter as @cwebber.

Published in: Technology

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,728
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
32
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. TESTING FOR OPS: GOING BEYOND THE MANIFEST Christopher Webber (@cwebber) Infrastructure Engineer, Demand Media
  • 2. ABOUT ME • Infrastructure Engineer at Demand Media • Always been in Support or Operations Roles • Been a Puppet user since v 0.24.7 (2008)
  • 3. ROLE MODULES VS LIBRARY MODULES Post By Craig Dunn (http://www.craigdunn.org/2012/05/239/)
  • 4. OPS IMPLICITLY GETS TESTING
  • 5. THEN WE TRY TO DO IT • And it is dumb • Might as well write a script to transform the code from one format to another • Start with unit testing frameworks, looking for integration testing
  • 6. TYPES OF TESTING • Syntax checking (puppet parser validate) • Linting • Unit Tests • Integration Tests
  • 7. SETTING UP OUR ENVIRONMENT • Ruby • Use the version you use in prod • Gems • puppet (use the version you use in prod) • rspec-puppet • puppet-lint • puppetlabs_spec_helper
  • 8. THE TRIFECTA 1 # == Class: ssh 2 # 3 class ssh { 4 5 package { 6 'openssh-server': 7 ensure => installed 8 } 9 10 file { 11 '/etc/ssh/sshd_config': 12 ensure => present, 13 owner => 'root', 14 group => 'root', 15 mode => '0644', 16 content => template('ssh/sshd_config.erb'), 17 require => Package['openssh-server'] 18 } 19 20 service { 21 'sshd': 22 ensure => running, 23 enable => true, 24 hasstatus => true, 25 hasrestart => true, 26 subscribe => File['/etc/ssh/sshd_config'] 27 } 28 29 }
  • 9. THE TESTS 1 require 'spec_helper' 2 3 describe 'ssh' do 4 5 it { should contain_package('openssh-server') } 6 7 it do 8 should contain_file('/etc/ssh/sshd_config') 9 .with_ensure('present') 10 .with_owner('root') 11 .with_group('root') 12 .with_mode('0644') 13 .with_require('Package[openssh-server]') 14 end 15 16 it do 17 should contain_service('sshd') 18 .with_ensure('running') 19 .with_enable(true) 20 .with_hasstatus(true) 21 .with_hasrestart(true) 22 .with_subscribe('File[/etc/ssh/sshd_config]') 23 end 24 end
  • 10. IS THIS USEFUL? At first, I said NO
  • 11. BUT WHY? It really doesn’t capture the things we care about.
  • 12. ENTER INFOSEC • SSH MUST HAVE THE FOLLOWING SETTINGS: • PermitRootLogin no • X11Forwarding no
  • 13. ADD USEFUL TESTS 25 context "InfoSec Requirements" do 26 27 it do 28 should contain_file('/etc/ssh/sshd_config') 29 .with_content(%r{^PermitRootLogin no$}) 30 end 31 32 it do 33 should contain_file('/etc/ssh/sshd_config') 34 .with_content(%r{^X11Forwarding no$}) 35 end 36 37 end
  • 14. AND FAILFailures: 1) ssh InfoSec Requirements Failure/Error: .with_content(%r{^PermitRootLogin no$}) expected that the catalogue would contain File[/etc/ssh/sshd_config] with content matching `/^PermitRootLogin no$/` but its value of `"<file contents>"` does not # ./spec/classes/ssh_spec.rb:29:in `block (3 levels) in <top (required)>' 2) ssh InfoSec Requirements Failure/Error: .with_content(%r{^X11Forwarding no$}) expected that the catalogue would contain File[/etc/ssh/sshd_config] with content matching `/^X11Forwarding no$/` but its value of `"<file contents>"` does not # ./spec/classes/ssh_spec.rb:34:in `block (3 levels) in <top (required)>' Finished in 0.47736 seconds 5 examples, 2 failures Failed examples: rspec ./spec/classes/ssh_spec.rb:27 # ssh InfoSec Requirements rspec ./spec/classes/ssh_spec.rb:32 # ssh InfoSec Requirements
  • 15. WHO CARES? • Can now fix and validate • Becomes one more check and balance • Safety in changes
  • 16. OPERATIONAL EXAMPLES
  • 17. LIBRARY MODULE EXAMPLES https://github.com/cwebberOps/puppetconf-ssh
  • 18. CONTINUING WITH SSH • New Requirements • We have a system that other systems ssh to and drop info • We have determined that we need to increase the MaxStartups to 40
  • 19. NEW TEST CODE 25 it do 26 should contain_file('/etc/ssh/sshd_config') 27 .with_content(%r{^MaxStartups 10$}) 28 end 29 30 context "maxstartups => 40" do 31 32 let (:params) {{ :maxstartups => 40 }} 33 34 it do 35 should contain_file('/etc/ssh/sshd_config') 36 .with_content(%r{^MaxStartups 40$}) 37 end 38 39 end https://github.com/cwebberOps/puppetconf-ssh
  • 20. NEW PUPPET CODE 1 # == Class: ssh 2 # 3 class ssh ( 4 $maxstartups = 10 5 ){ https://github.com/cwebberOps/puppetconf-ssh
  • 21. FEEDBACK LOOP • Did you remember to update the template with your new variable? • Much faster than vagrant destroy && vagrant up or even vagrant provision
  • 22. MOVING ON TO THE ROLE • In the end, it is the module that matters the most • Should test that the config has the right config for the app https://github.com/cwebberOps/puppetconf-infra
  • 23. TEST CODE 1 require 'spec_helper' 2 3 describe 'infra::collector' do 4 5 it do 6 should contain_file('/etc/ssh/sshd_config') 7 .with_content(%r{^MaxStartups 40$}) 8 end 9 10 end https://github.com/cwebberOps/puppetconf-infra
  • 24. PUPPET CODE 1 # Class for setting up the infrastructure collector 2 class infra::collector { 3 4 class { 5 'ssh': 6 maxstartups => 40 7 } 8 9 } https://github.com/cwebberOps/puppetconf-infra
  • 25. BUT THE TESTS... THEY DON’T WORK
  • 26. MORE SETUP • Rakefile • puppet_rspec_helper • .fixtures.yml
  • 27. RAKEFILE & SPEC_HELPER 1 require 'rake' 2 3 require 'rspec/core/rake_task' 4 require 'puppetlabs_spec_helper/rake_tasks' New 1 require 'rake' 2 3 require 'rspec/core/rake_task' 4 5 RSpec::Core::RakeTask.new(:spec) do |t| 6 t.pattern = 'spec/*/*_spec.rb' 7 end Old
  • 28. .FIXTURES.YML 1 fixtures: 2 repositories: 3 ssh: "https://github.com/cwebberOps/puppetconf-ssh.git" 4 symlinks: 5 infra: "#{ source_dir }"
  • 29. FUN EXAMPLES
  • 30. INFRA::JENKINS 1 require 'spec_helper' 2 3 describe 'infra::jenkins' do 4 5 let (:facts) {{ 6 :osfamily => 'RedHat', 7 :operatingsystem => 'CentOS' 8 }} 9 10 it { should include_class('java') } 11 it { should include_class('jenkins') } 12 13 end https://github.com/cwebberOps/puppetconf-infra
  • 31. INFRA::JENKINS 1 # Class for standing up a jenkins box 2 class infra::jenkins { 3 4 class { 5 'java': 6 } -> 7 8 class { 9 'jenkins': 10 } 11 12 } https://github.com/cwebberOps/puppetconf-infra
  • 32. WHAT’S BROKE? http://projects.puppetlabs.com/issues/2053 The Bug The Tweet
  • 33. INFRA::NODEJS & INFRA::NODEJS::DEV1 require 'spec_helper' 2 3 describe 'infra::nodejs' do 4 5 let (:facts) {{ 6 :osfamily => 'RedHat', 7 :operatingsystem => 'CentOS' 8 }} 9 10 it { should include_class('nodejs') } 11 12 it { should_not contain_package('nodejs-dev') } 13 14 it { should contain_file('/app_specific_stuff') } 15 16 end https://github.com/cwebberOps/puppetconf-infra
  • 34. INFRA::NODEJS & INFRA::NODEJS::DEV 1 require 'spec_helper' 2 3 describe 'infra::nodejs::dev' do 4 5 let (:facts) {{ 6 :osfamily => 'RedHat', 7 :operatingsystem => 'CentOS' 8 }} 9 10 it { should include_class('nodejs') } 11 it { should contain_package('nodejs-dev') } 12 13 end https://github.com/cwebberOps/puppetconf-infra
  • 35. INFRA::NODEJS & INFRA::NODEJS::DEV 1 # Demo class that sets up nodejs 2 class infra::nodejs { 3 4 class { 5 '::nodejs': 6 dev_package => false 7 } 8 9 file { 10 '/app_specific_stuff': 11 ensure => directory 12 } 13 14 } https://github.com/cwebberOps/puppetconf-infra
  • 36. INFRA::NODEJS & INFRA::NODEJS::DEV 1 # Override class 2 class infra::nodejs::dev inherits infra::nodejs { 3 4 Class['::nodejs'] { 5 dev_package => true 6 } 7 8 } https://github.com/cwebberOps/puppetconf-infra
  • 37. WHAT’S BROKE? http://projects.puppetlabs.com/issues/5517 The Bug
  • 38. WHY DO THESE EXAMPLES MATTER?
  • 39. WHAT’S NEXT • Integration testing using rspec-system • Continuous Integration
  • 40. PREVIEW
  • 41. RSPEC-SYSTEM1 require 'spec_helper_system' 2 3 describe 'ssh' do 4 5 it 'class should converge' do 6 pp = <<-EOS 7 class { 'ssh': } 8 EOS 9 10 puppet_apply(pp) do |r| 11 r.exit_code.should_not == 1 12 r.refresh 13 r.exit_code.should be_zero 14 end 15 end 16 17 describe service('sshd') do 18 it { should be_enabled } 19 it { should be_running } 20 end 21 end
  • 42. JENKINS
  • 43. QUESTIONS?