Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Test-Driven Puppet Development - PuppetConf 2014

2,451 views

Published on

Test-Driven Puppet Development - Nan Liu, Bodeco

Published in: Technology
  • Be the first to comment

Test-Driven Puppet Development - PuppetConf 2014

  1. 1. Testing Driven Puppet Modules [Nan Liu @sesshin] Copyright 2014 1 / 45
  2. 2. Overview 2 / 45
  3. 3. Overview Who This talk is not for people who: Write perfect code (manifests) Never upgrade Puppet Don't have SLA Used Puppet since 0.2x 3 / 45
  4. 4. Overview Who Why Testing Puppet on Production is __ 4 / 45
  5. 5. Overview Who Why Positive feedback loop: 5 / 45
  6. 6. Overview Who Why Creating a virtuous cycle: 6 / 45
  7. 7. Overview Who Why Creating a virtuous cycle: 7 / 45
  8. 8. Overview Who Why What Developing Puppet Modules Development feedback puppet-lint puppet-syntax rspec-puppet Testing Packer Vagrant beaker 8 / 45
  9. 9. Development Feedback 9 / 45
  10. 10. Development puppet-lint Manifest lint check: $ puppet lint manifests/**/* WARNING: top-scope variable being used without an explicit namespace on line 4 WARNING: class inheriting from params class on line 26 WARNING: class not documented on line 1 WARNING: line has more than 80 characters on line 52 10 / 45
  11. 11. Development puppet-lint Manifest lint cleanup: $ puppet-lint -f demo.pp FIXED: string containing only a variable on line 1 FIXED: variable not enclosed in {} on line 5 FIXED: indentation of => is not properly aligned on line 2 FIXED: indentation of => is not properly aligned on line 3 FIXED: indentation of => is not properly aligned on line 5 WARNING: ensure found on line but it's not the first attribute on line 4 11 / 45
  12. 12. Development puppet-lint Configure puppet-lint behavior PuppetLint.configuration.disable_80chars PuppetLint.configuration.disable_arrow_alignment PuppetLint.configuration.disable_class_inherits_from_params_class PuppetLint.configuration.disable_class_parameter_defaults PuppetLint.configuration.fail_on_warnings = true PuppetLint.configuration.ignore_paths = ['spec/**/*.pp', 'pkg/**/*.pp'] 12 / 45
  13. 13. Development puppet-lint puppet-syntax Manifest validation: $ puppet parser validate manifests/bad.pp Error: Could not parse for environment production: Syntax error at 'demo'; expected '}' at /Users/nan/src/puppet-demo/manifests/bad.pp:4 13 / 45
  14. 14. Development puppet-lint puppet-syntax Rakefile require 'puppet-syntax/tasks/puppet-syntax' PuppetSyntax.exclude_paths = ["spec/fixtures/**/*"] PuppetSyntax.future_parser = true 14 / 45
  15. 15. Development puppet-lint puppet-syntax Puppet module folder: manifests/ tests/ Rakefile require 'puppetlabs_spec_helper/rake_tasks' PuppetLint.configuration.disable_80chars PuppetLint.configuration.disable_arrow_alignment PuppetLint.configuration.disable_class_inherits_from_params_class PuppetLint.configuration.disable_class_parameter_defaults PuppetLint.configuration.fail_on_warnings = true PuppetLint.configuration.ignore_paths = ['spec/**/*.pp', 'pkg/**/*.pp'] require 'puppet-syntax/tasks/puppet-syntax' PuppetSyntax.exclude_paths = ["spec/fixtures/**/*"] PuppetSyntax.future_parser = true 15 / 45
  16. 16. Development puppet-lint puppet-syntax rspec-puppet RSpec-Puppet Compile catalog module dependency system fact class parameters Verifies catalog Class/Resource specification Relationships Expectation 16 / 45
  17. 17. Development puppet-lint puppet-syntax rspec-puppet .fixtures.yml fixtures: repositories: apt: "https://github.com/puppetlabs/puppetlabs-apt.git" stdlib: "https://github.com/puppetlabs/puppetlabs-stdlib.git" firewall: "https://github.com/puppetlabs/puppetlabs-firewall.git" concat: "https://github.com/puppetlabs/puppetlabs-concat.git" symlinks: postgresql: "#{source_dir}" 17 / 45
  18. 18. Development puppet-lint puppet-syntax rspec-puppet Supply system facts and class parameters: let(:facts) {{ :osfamily => 'redhat' }} let(:params) {{ :keys_enable => true, :keys_file => '/etc/ntp/ntp.keys', :keys_trusted => ['1', '2', '3'], :keys_controlkey => '2', :keys_requestkey => '3', }} 18 / 45
  19. 19. Development puppet-lint puppet-syntax rspec-puppet require 'spec_helper' describe 'ntp' do Dir.glob('tests/*.pp').each do |file| let(:facts) {{ :osfamily => 'redhat' }} context file do let(:pre_condition) { File.read(file) } it{ should compile } end end end 19 / 45
  20. 20. Development puppet-lint puppet-syntax rspec-puppet Puppet Class: describe 'ntp' do let(:facts) {{ :osfamily => 'RedHat' }} let(:params) {{ :iburst_enable => true, }} it do should contain_file('/etc/ntp.conf').with({ 'content' => /iburstn/, }) end end 20 / 45
  21. 21. Development puppet-lint puppet-syntax rspec-puppet Define Type describe 'mysql::db', :type => :define do let(:facts) {{ :osfamily => 'RedHat' }} let(:title) { 'test_db' } let(:params) { { 'user' => 'testuser', 'password' => 'testpass', } } ... end 21 / 45
  22. 22. Development puppet-lint puppet-syntax rspec-puppet Puppet module folder: .fixtures.yml Rakefile manifests/ tests/ spec/classes/* spec/defines/* 22 / 45
  23. 23. Development puppet-lint puppet-syntax rspec-puppet Gemfile source "https://rubygems.org" group :test do gem "rake" gem "puppet", ENV['GEM_PUPPET_VERSION'] gem "puppet-lint" gem "rspec-puppet", :git => 'https://github.com/rodjek/rspec-puppet.git' gem "puppet-syntax" gem "puppetlabs_spec_helper" end 23 / 45
  24. 24. Development puppet-lint puppet-syntax rspec-puppet Customize How many modules do you have? modulesync 24 / 45
  25. 25. Development puppet-lint puppet-syntax rspec-puppet Customize Gem::Specification.new do |s| ... if facter_version = ENV['GEM_FACTER_VERSION'] s.add_runtime_dependency 'facter', facter_version else s.add_runtime_dependency 'facter' end if puppet_version = ENV['GEM_PUPPET_VERSION'] || ENV['PUPPET_GEM_VERSION'] s.add_runtime_dependency 'puppet', puppet_version else s.add_runtime_dependency 'puppet' end s.add_runtime_dependency 'rake' s.add_runtime_dependency 'rspec', '~> 2.11.0' s.add_runtime_dependency 'mocha', '~> 0.10.5' s.add_runtime_dependency 'puppetlabs_spec_helper', '0.7' s.add_runtime_dependency 'rspec-puppet' s.add_runtime_dependency 'puppet-lint', '~> 1.0' s.add_runtime_dependency 'puppet-syntax' s.files = Dir.glob('lib/**/*') + %w(LICENSE) s.require_path = 'lib' end 25 / 45
  26. 26. Development puppet-lint puppet-syntax rspec-puppet Customize Gemfile group :development, :test do gem 'bodeco_module_helper', :git => 'https://github.com/bodeco/bodeco_module_helper.git' end Rakefile require 'bodeco_module_helper/rake_tasks' 26 / 45
  27. 27. Development puppet-lint puppet-syntax rspec-puppet Customize $ rake -T rake beaker # Run beaker acceptance tests rake beaker_nodes # List available beaker nodesets rake build # Build puppet module package rake clean # Clean a built module package rake coverage # Generate code coverage inform... rake help # Display the list of available... rake lint # Check puppet manifests with p... rake spec # Run spec tests in a clean fix... rake spec_clean # Clean up the fixtures directory rake spec_prep # Create the fixtures directory rake spec_standalone # Run spec tests on an existing... rake syntax # Syntax check Puppet manifests... rake syntax:hiera # Syntax check Hiera config files rake syntax:manifests # Syntax check Puppet manifests rake syntax:templates # Syntax check Puppet templates rake validate # Check syntax of Ruby files an... 27 / 45
  28. 28. Development puppet-lint puppet-syntax rspec-puppet Summary puppet-lint: style enforcer puppet-syntax: code parse rspec-puppet: catalog verify pin your versions 28 / 45
  29. 29. Testing 29 / 45
  30. 30. Testing Packer Say no to mystery boxes Virtualbox or VMware fusion/workstation Amazon, Digital Ocean, GCE, ... 30 / 45
  31. 31. Testing Packer "builders": [{ "vm_name": "centos70", "type": "vmware-iso", "guest_os_type": "centos-64", "http_directory": "http", "iso_url": "{{ user `iso_url` }}", "iso_checksum": "{{ user `iso_checksum` }}", "tools_upload_flavor": "linux", "boot_command": [ "<tab> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort}}/ks7.cfg<enter>" ], "disk_size": 10140, "vmx_data": { "memsize": "512", "numvcpus": "1", "cpuid.coresPerSocket": "1" } }] 31 / 45
  32. 32. Testing Packer "provisioners": [{ "type": "shell", "environment_vars": [ "CM={{user `cm`}}", "CM_VERSION={{user `cm_version`}}", "CM_SET_PATH={{user `cm_set_path`}}", "CLEANUP_PAUSE={{user `cleanup_pause`}}" ], "execute_command": "echo 'vagrant' | {{.Vars}} sudo -E -S bash '{{.Path}}'", "scripts": [ "script/fix-slow-dns.sh", "script/sshd.sh", "script/vagrant.sh", "script/vmtool.sh", "script/cmtool.sh", "script/cleanup.sh" ] }], 32 / 45
  33. 33. Testing Packer https://github.com/puppetlabs/puppet-vagrant-boxes https://github.com/mitchellh/veewee-to-packer https://github.com/box-cutter https://github.com/hashicorp/puppet-bootstrap 33 / 45
  34. 34. Testing Packer Vagrant Vagrant.configure('2') do |conf| conf.vm.define 'demo' do |mod| mod.vm.box = 'centos64.box' mod.vm.synced_folder './modules', "/opt/puppet/share/puppet/modules" mod.vm.synced_folder './manifests', "/etc/puppetlabs/puppet/manifests" mod.vm.synced_folder './data', "/etc/puppetlabs/puppet/data" mod.vm.provision :puppet do |p| p.module_path = 'spec/fixtures/modules' p.manifests_path = 'manifests' p.manifest_file = ENV['VAGRANT_MANIFEST'] || 'init.pp' p.options = '--verbose' end end end 34 / 45
  35. 35. Testing Packer Vagrant https://github.com/adrienthebo/vagrant-config_builder $ vagrant plugin install vagrant-config_builder ├── config │ ├── roles.yaml │ └── vms.yaml └── Vagrantfile --- vms: - name: db private_networks: [ {ip: '10.20.1.2'} ] box: centos-5-i386 hostname: db.puppetlabs.vm synced_folders: - host_path: '.' guest_path: '/vagrant' disabled: true provisioners: - type: puppet manifests_path: 'tests' module_path: 'spec/fixtures/modules' manifest_file: <%= ENV['VAGRANT_MANIFEST'] || 'init.pp' %> 35 / 45
  36. 36. Testing Packer Vagrant def vm(opt) module_name = opt.fetch(:module).to_s || raise(ArgumentError, 'Must provide puppet module hostname = opt.fetch(:hostname, module_name).to_s memory = opt.fetch(:memory, 512) cpu = opt.fetch(:cpu, 1) os_type = opt.fetch(:type, :linux) Vagrant.configure('2') do |conf| conf.vm.network(:forwarded_port, guest: port, host: port, auto_correct: true) if port if os_type == :windows conf.ssh.username = 'vagrant' conf.winrm.username = 'vagrant' conf.winrm.password = 'vagrant' end ... 36 / 45
  37. 37. Testing Packer Vagrant vm( :hostname => 'oel', :module => 'application', :memory => 8096, :box => 'oracle65-pe3.2.3', :port => 8080 ) 37 / 45
  38. 38. Testing Packer Vagrant $ vagrant up $ vagrant provision $ vagrant ssh $ vagrant destroy 38 / 45
  39. 39. Testing Packer Vagrant Beaker Puppet Labs testing framework spec/acceptance ├── class_spec.rb ├── disable_monitoring_spec.rb ├── nodesets │ ├── centos-59-x64.yml │ ├── centos-64-x64-pe.yml │ ├── centos-64-x64.yml │ ├── centos-65-x64.yml │ ├── default.yml │ ├── fedora-18-x64.yml │ ├── sles-11-x64.yml │ ├── ubuntu-server-10044-x64.yml │ ├── ubuntu-server-12042-x64.yml │ └── ubuntu-server-1404-x64.yml ├── ntp_config_spec.rb ├── ntp_install_spec.rb ├── ntp_parameters_spec.rb ├── ntp_service_spec.rb ├── preferred_servers_spec.rb ├── restrict_spec.rb └── unsupported_spec.rb 39 / 45
  40. 40. Testing Packer Vagrant Beaker HOSTS: centos-64-x64: roles: - master platform: el-6-x86_64 box : centos-64-x64-vbox4210-nocm box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210-nocm.box hypervisor : vagrant CONFIG: type: foss 40 / 45
  41. 41. Testing Packer Vagrant Beaker it 'should run successfully' do pp = "class { 'ntp': }" # Apply twice to ensure no errors the second time. apply_manifest(pp, :catch_failures => true) do |r| expect(r.stderr).not_to match(/error/i) end apply_manifest(pp, :catch_failures => true) do |r| expect(r.stderr).not_to eq(/error/i) expect(r.exit_code).to be_zero end end 41 / 45
  42. 42. Testing Packer Vagrant Beaker it 'starts the service' do pp = <<-EOS class { 'ntp': service_enable => true, service_ensure => running, service_manage => true, service_name => '#{servicename}' } EOS apply_manifest(pp, :catch_failures => true) end describe service(servicename) do it { should be_running } it { should be_enabled } end 42 / 45
  43. 43. Testing Packer Vagrant Beaker Summary Packer: VM build Vagrant: VM clone and testing Beaker: Automated testing 43 / 45
  44. 44. Questions 44 / 45
  45. 45. Thank You! [nan@bodeco.io] 45 / 45

×