Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

modern module development - Ken Barber 2012 Edinburgh Puppet Camp


Published on

Published in: Technology, Spiritual
  • Be the first to comment

modern module development - Ken Barber 2012 Edinburgh Puppet Camp

  1. 1. Modern Module Development 2012-03-23 Puppet Camp Edinburgh Ken Barber Professional Services Engineer IRC & Twitter: @ken_barber
  2. 2. What is a module?• A re-usable way of sharing common components: • Classes • Defined Resources • Ruby Resources • Files • Templates • Functions • Facts
  3. 3. We need more modules!
  4. 4. and they need tobe good quality ...
  5. 5. Whats new(ish) inModule Development? • Puppet Module Face • rspec-puppet • puppet-lint • Kwalify
  6. 6. But lets talkpatterns first ...
  7. 7. Why are patterns important?
  8. 8. Class Patterns• Most installation and management problems consist of: • Installing software using files or packages • Modifying configuration files • Starting services
  9. 9. We commonly referto this pattern as: Package, Config, Service
  10. 10. Package Classclass bind::package { package { $bind::packages: ensure => installed, }}
  11. 11. Config Classclass bind::config { file { $bind::config_file: content => template(“${module_name}/named.conf”), owner => $bind::user, group => $bind::group, mode => ‘0644’, }}
  12. 12. Service Classclass bind::service { service { $bind::services: ensure => running, enable => true, hasstatus => true, hasrestart => true, }}
  13. 13. Main Classclass bind { anchor { ‘bind::start’: }-> class { ‘bind::package’: }~> class { ‘bind::config’: }~> class { ‘bind::service’: }~> anchor { ‘bind::end’: }}
  14. 14. What are anchors?• Provided in stdlib• Anchors are resource that ‘do nothing’• They ensure the edge of the classes are demarcated properly so the graph executes in the expected order
  15. 15. But what aboutdefined resources?
  16. 16. Defined Resourcesdefine bind::zone ( $value = ‘nil’ ){ file { “/etc/bind/config.d/${name}.conf”: content => “${name} = ${value}n”, require => Class[‘bind::package’], notify => Class[‘bind::service’], }}
  17. 17. Package, Config, Service Pattern• This pattern is good for ~80% of situations.• Deals with ordering in a non-complex way, working with defined resources as well.
  18. 18. Params Pattern
  19. 19. Default Parameters• Most parameters have OS dependant defaults.• In the past this was done with case statements in the main modules.
  20. 20. Class Parameters• I generally use two types of class parameters: • User-tuneable parameters • Advanced parameters for overriding items such as package name, service name configuration file paths
  21. 21. Params PatternThe trick is to move your OS based param lookup intomanifests/params.pp: class bind::params { case $::operatingsystem { ubuntu, debian: { $package = bind9 $service = bind9 $config_dir = /etc/bind } ‘centos’, ‘redhat’: { $package = ‘bind’ $service = ‘bind’ $config_dir = ‘/etc/named’ } default: { fail(“Unknown OS: $::operatingsystem”) } } }
  22. 22. Params PatternThen inherit this in your base class like so: class bind ( $package = $bind::params::package, $service = $bind::params::service, $config_dir = $bind::params::config_dir ) inherits bind::params { ... }
  23. 23. Params PatternThen elsewhere in your code, reference the fullyqualified parameter from your bind class: class bind::package { package { $bind::packages: ensure => installed, } }
  24. 24. Puppet ModuleTool will now be a Puppet Face!
  25. 25. Puppet Module Tool# puppet help moduleUSAGE: puppet module <action>This subcommand can find, install, and manage modules from the Puppet Forge,a repository of user-contributed Puppet code. It can also generate emptymodules, and prepare locally developed modules for release on the Forge.OPTIONS: --mode MODE - The run mode to use (user, agent, or master). --render-as FORMAT - The rendering format to use. --verbose - Whether to log verbosely. --debug - Whether to log debug information.ACTIONS: build Build a module release package. changes Show modified files of an installed module. clean Clean the module download cache. generate Generate boilerplate for a new module. install Install a module from a repository or release archive. list List installed modules search Search a repository for a module. uninstall Uninstall a puppet module. upgrade Upgrade a puppet module.See puppet man module or man puppet-module for full help.
  26. 26. Puppet Module Face• Obviously still subject to design changes before release ....
  27. 27. ValidatingParameters
  28. 28. Validating Parameters• Your allowed set of class parameters are like an Interface Contract to your users• A lot of bugs in code are to do with bad input• We have three mechanisms for validating input: • parameter definitions in the class • stdlib validate_* functions • kwalify
  29. 29. Kwalify Validation• Uses the kwalify library - which is a language independent way of validating JSON/YAML style data structures.• Kwalify has binding in many languages - so its a nice universal choice for structured data validation.
  30. 30. Kwalify• Provides a single declarative way of defining what parameters are valid.• When validation fails, it shows all cases of failure.• Doesn’t handle logical validation across components - you’ll have to do this yourself.
  31. 31. A Kwalify Schema in Puppet DSL $schema = { type => map, mapping => { name => { type => str, pattern => /^w+s*w+$/, }, age => { type => str, pattern => /^d+$/, }, }}
  32. 32. KwalifyValidation in DSLclass bind ( ... ){ $args = get_scope_args() $schema = { ... } kwalify($schema, $args)}
  33. 33. Module Testing
  34. 34. rspec-puppet• Tim Sharpe (@rodjek) from Github created it• Rspec is often used by Puppet for testing, so it made sense for us to use rspec-puppet• The same facility can be used for testing Puppet DSL and Ruby code as well: providers, types, facts etc.
  35. 35. rspec-puppet • Is able to test: • Classes • Defined Resources • Nodes • Functions
  36. 36. rspec-puppet• Paths for tests: • Classes: • spec/classes/<name_of_class>_spec.rb • Defined Resources • spec/defines/<name_of_define>_spec.rb
  37. 37. rspec-puppetFor class testing, the idea is to test your compiledcatalogue. The following just tests if your classcompiles with no parameters: # spec/classes/bind_spec.rb require spec_helper describe bind, :type => :class do let(:facts) do { :operatingsystem => "CentOS" } end describe when only mandatory parameters are provided do let(:params) do {} end it class should get included do subject.should contain_class(bind) end end end
  38. 38. rspec-puppetYou also want to make sure your expectations of howthe catalogue should look after compilation are met: it ‘bind package should be defined’ do subject.should contain_package(‘bind’) end it ‘bind service should be defined’ do subject.should contain_service(‘bind’) end
  39. 39. rspec-puppet For testing templates, we can do something a little different:it changing authnxdomain should modify template do params = { :config_options = "/tmp/named.conf.options" :options = { "authnxdomain" => "yes", } } content = param_value("file", "/tmp/named.conf.options", "content") content.should =~ /authnxdomain yes/end
  40. 40. rspec-puppet Requires a spec/spec_helper.rb file with appropriate settings:require rspecrequire rspec-puppetrequire puppetrequire mochafixture_path = File.expand_path(File.join(File.dirname(__FILE__), fixtures))RSpec.configure do |c| c.module_path = File.join(fixture_path, modules) c.manifest_dir = File.join(fixture_path, manifests)end
  41. 41. rspec-puppetAlso the following should be added to your Rakefileto get ‘rake spec’ to work: require rubygems require rspec/core/rake_task do |t| t.pattern = spec/*/*_spec.rb end
  42. 42. Importance ofGood Coding Style
  43. 43. Importance ofGood Coding Style• Code is read much more often then it is written (did Guido van Rossum say this?)• Other people being able to read your code is important to attract contributors• Contributors make your code better and ultimately more successful
  44. 44. Style Guide• Available here: •
  45. 45. puppet-lint• Also made by Tim Sharpe from Github• Uses the Puppet Labs style guide• Saves time, as it tells you what is wrong with your content before a code review
  46. 46. Using it in your RakefileDownload the gem: gem install puppet-lintThen just add the following to your modules Rakefile: require puppet-lint/tasks/puppet-lint
  47. 47. Running it:# rake lintEvaluating manifests/init.ppEvaluating manifests/config.ppEvaluating manifests/package.ppEvaluating manifests/service.pp
  48. 48. Packaging for the Forge
  49. 49. The Modulefile• Provides meta data for your module• Includes dependencies • In the future the module tool can use these dependencies to download extra modules as needed• Is used to generate metadata.json, which is used by the forge
  50. 50. The Modulefilename puppetlabs-bindversion 0.0.1source Puppet Labs Inc.license ASL 2.0summary ISC Bind Moduledescription Installs and configures the ISC BIND DNS serverproject_page puppetlabs/stdlib, >= 2.2.1dependency puppetlabs/kwalify, >= 0.0.1dependency ripienaar/concat, >= 20100507
  51. 51. Packaging using the module tool# puppet-module build===================================================Building /Users/ken/Development/puppetlabs-bind for release-----------------------------------------------------------Done. Built: pkg/puppetlabs-bind-0.0.1.tar.gz
  52. 52. Uploading To the Forge• Create an account - this username will be publicly visible, so if you are a company create a shared one• Create a project - this is the short name of the module• Create a revision - this is the revision you wish to upload. Use semantic versioning if you can ie. 1.2.0, 3.1.2 etc.• Upload your revision, this is the tar.gz file created in pkg/ with puppet-module build
  53. 53. Using a forge moduleUsing the new module face you can install the forgemodule: # puppet module search bind # puppet module install puppetlabs-bind
  54. 54. Future WorkNeeded for Modules• Acceptance Testing • ‘Real’ build testing, perhaps using monitoring frameworks for acceptance?• Hiera and data lookup methodology • Using Hiera to emulate the params pattern
  55. 55. AnyQuestions?