modern module development - Ken Barber 2012 Edinburgh Puppet Camp


Published on

Published in: Technology, Spiritual
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • Which is what this talk will hopefully help you all achieve.\n
  • New(ish) because most of these ideas have been kicking around for a while, but only used in anger recently. Kwalify is new, only published to the forge on Wednesday.\n
  • I’ll show you some patterns, and jump back and forth between code and this slidepack. In my examples, I’ll but using my in-development ‘bind’ module as the basis of some of my examples where applicable.\n
  • Providing a consistent mechanism for writing a module means we get the boring stuff out of the way earlier. Less fapping about with style & layout of code. It also means we can learn from others mistakes - and wins.\n
  • If you’ve ever been to any of out training courses - we talk about this quite a bit.\n
  • Or Package, File, Service - but file is a bit ambigous - since a package is a file right?\nThis pattern breaks these three major items into consecutive chunks, in Puppet terms - sub classes.\n
  • Contains all ‘package’ related data.\n
  • Contains all ‘configuration’ related resources.\n
  • Contains all ‘service’ related resources. Possibly even Cron or Scheduled jobs ...\nNext slide: main class\n
  • This class ‘brings it all together’ using anchors. This provides course grained relationships between the Package, Config & Service classes. You can see our usage of chained resources, this is mainly so it looks sexy.\n
  • Anchors may look strange, but you need them to surround classes to ensure ordering is done correctly. This is due to ambiguous boundaries for classes in the graph. This is more or less a work-around for a ‘bug’ or design flaw, and is more obvious when you have multiple layers of classes.\n
  • Defined resources fit well with this pattern because you can now just build up relationships with the classes. ie. you require the package class, and notify the service class.\n
  • This is a contrived example, and not really how you would manage a zone - but I just want to show how a resource inside a defined resource can establish relationships with the course grained package,config,service classes. Also - the beauty of this - is that you can notify and require these classes _outside_ of the module.\n
  • I generally drop this pattern in as a template for customers so they can just copy it and modify to taste.\nNext section: Params Pattern\n
  • \n
  • \n
  • \n
  • The default will stop the code from working on non-supported operating systems. This may not always be desirable, but sometime its hard to find reasonable defaults in this case.\n
  • Yes - it looks wordy. Now you can use these variables directly in your class - $package, $service, $config_dir.\n
  • By doing it this way, you allow users who define class parameters to override the params defaults.\nAlso - check out RI Pienaars patterns around using Hiera and his take on the params pattern using the Hiera Puppet backend.\nNext Topic: Puppet Module as a Face?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • At this point I’ll just drop to the console and show it in action.\n
  • \n
  • \n
  • \n
  • \n
  • These are conventions for layouts, but its recommended sticking to these conventions for the examples to work.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • modern module development - Ken Barber 2012 Edinburgh Puppet Camp

    1. 1. Modern Module Development 2012-03-23 Puppet Camp Edinburgh Ken Barber Professional Services Engineer IRC & Twitter: @ken_barber
    2. 2. What is a module?• A re-usable way of sharing common components: • Classes • Defined Resources • Ruby Resources • Files • Templates • Functions • Facts
    3. 3. We need more modules!
    4. 4. and they need tobe good quality ...
    5. 5. Whats new(ish) inModule Development? • Puppet Module Face • rspec-puppet • puppet-lint • Kwalify
    6. 6. But lets talkpatterns first ...
    7. 7. Why are patterns important?
    8. 8. Class Patterns• Most installation and management problems consist of: • Installing software using files or packages • Modifying configuration files • Starting services
    9. 9. We commonly referto this pattern as: Package, Config, Service
    10. 10. Package Classclass bind::package { package { $bind::packages: ensure => installed, }}
    11. 11. Config Classclass bind::config { file { $bind::config_file: content => template(“${module_name}/named.conf”), owner => $bind::user, group => $bind::group, mode => ‘0644’, }}
    12. 12. Service Classclass bind::service { service { $bind::services: ensure => running, enable => true, hasstatus => true, hasrestart => true, }}
    13. 13. Main Classclass bind { anchor { ‘bind::start’: }-> class { ‘bind::package’: }~> class { ‘bind::config’: }~> class { ‘bind::service’: }~> anchor { ‘bind::end’: }}
    14. 14. What are anchors?• Provided in stdlib• Anchors are resource that ‘do nothing’• They ensure the edge of the classes are demarcated properly so the graph executes in the expected order
    15. 15. But what aboutdefined resources?
    16. 16. Defined Resourcesdefine bind::zone ( $value = ‘nil’ ){ file { “/etc/bind/config.d/${name}.conf”: content => “${name} = ${value}n”, require => Class[‘bind::package’], notify => Class[‘bind::service’], }}
    17. 17. Package, Config, Service Pattern• This pattern is good for ~80% of situations.• Deals with ordering in a non-complex way, working with defined resources as well.
    18. 18. Params Pattern
    19. 19. Default Parameters• Most parameters have OS dependant defaults.• In the past this was done with case statements in the main modules.
    20. 20. Class Parameters• I generally use two types of class parameters: • User-tuneable parameters • Advanced parameters for overriding items such as package name, service name configuration file paths
    21. 21. Params PatternThe trick is to move your OS based param lookup intomanifests/params.pp: class bind::params { case $::operatingsystem { ubuntu, debian: { $package = bind9 $service = bind9 $config_dir = /etc/bind } ‘centos’, ‘redhat’: { $package = ‘bind’ $service = ‘bind’ $config_dir = ‘/etc/named’ } default: { fail(“Unknown OS: $::operatingsystem”) } } }
    22. 22. Params PatternThen inherit this in your base class like so: class bind ( $package = $bind::params::package, $service = $bind::params::service, $config_dir = $bind::params::config_dir ) inherits bind::params { ... }
    23. 23. Params PatternThen elsewhere in your code, reference the fullyqualified parameter from your bind class: class bind::package { package { $bind::packages: ensure => installed, } }
    24. 24. Puppet ModuleTool will now be a Puppet Face!
    25. 25. Puppet Module Tool# puppet help moduleUSAGE: puppet module <action>This subcommand can find, install, and manage modules from the Puppet Forge,a repository of user-contributed Puppet code. It can also generate emptymodules, and prepare locally developed modules for release on the Forge.OPTIONS: --mode MODE - The run mode to use (user, agent, or master). --render-as FORMAT - The rendering format to use. --verbose - Whether to log verbosely. --debug - Whether to log debug information.ACTIONS: build Build a module release package. changes Show modified files of an installed module. clean Clean the module download cache. generate Generate boilerplate for a new module. install Install a module from a repository or release archive. list List installed modules search Search a repository for a module. uninstall Uninstall a puppet module. upgrade Upgrade a puppet module.See puppet man module or man puppet-module for full help.
    26. 26. Puppet Module Face• Obviously still subject to design changes before release ....
    27. 27. ValidatingParameters
    28. 28. Validating Parameters• Your allowed set of class parameters are like an Interface Contract to your users• A lot of bugs in code are to do with bad input• We have three mechanisms for validating input: • parameter definitions in the class • stdlib validate_* functions • kwalify
    29. 29. Kwalify Validation• Uses the kwalify library - which is a language independent way of validating JSON/YAML style data structures.• Kwalify has binding in many languages - so its a nice universal choice for structured data validation.
    30. 30. Kwalify• Provides a single declarative way of defining what parameters are valid.• When validation fails, it shows all cases of failure.• Doesn’t handle logical validation across components - you’ll have to do this yourself.
    31. 31. A Kwalify Schema in Puppet DSL $schema = { type => map, mapping => { name => { type => str, pattern => /^w+s*w+$/, }, age => { type => str, pattern => /^d+$/, }, }}
    32. 32. KwalifyValidation in DSLclass bind ( ... ){ $args = get_scope_args() $schema = { ... } kwalify($schema, $args)}
    33. 33. Module Testing
    34. 34. rspec-puppet• Tim Sharpe (@rodjek) from Github created it• Rspec is often used by Puppet for testing, so it made sense for us to use rspec-puppet• The same facility can be used for testing Puppet DSL and Ruby code as well: providers, types, facts etc.
    35. 35. rspec-puppet • Is able to test: • Classes • Defined Resources • Nodes • Functions
    36. 36. rspec-puppet• Paths for tests: • Classes: • spec/classes/<name_of_class>_spec.rb • Defined Resources • spec/defines/<name_of_define>_spec.rb
    37. 37. rspec-puppetFor class testing, the idea is to test your compiledcatalogue. The following just tests if your classcompiles with no parameters: # spec/classes/bind_spec.rb require spec_helper describe bind, :type => :class do let(:facts) do { :operatingsystem => "CentOS" } end describe when only mandatory parameters are provided do let(:params) do {} end it class should get included do subject.should contain_class(bind) end end end
    38. 38. rspec-puppetYou also want to make sure your expectations of howthe catalogue should look after compilation are met: it ‘bind package should be defined’ do subject.should contain_package(‘bind’) end it ‘bind service should be defined’ do subject.should contain_service(‘bind’) end
    39. 39. rspec-puppet For testing templates, we can do something a little different:it changing authnxdomain should modify template do params = { :config_options = "/tmp/named.conf.options" :options = { "authnxdomain" => "yes", } } content = param_value("file", "/tmp/named.conf.options", "content") content.should =~ /authnxdomain yes/end
    40. 40. rspec-puppet Requires a spec/spec_helper.rb file with appropriate settings:require rspecrequire rspec-puppetrequire puppetrequire mochafixture_path = File.expand_path(File.join(File.dirname(__FILE__), fixtures))RSpec.configure do |c| c.module_path = File.join(fixture_path, modules) c.manifest_dir = File.join(fixture_path, manifests)end
    41. 41. rspec-puppetAlso the following should be added to your Rakefileto get ‘rake spec’ to work: require rubygems require rspec/core/rake_task do |t| t.pattern = spec/*/*_spec.rb end
    42. 42. Importance ofGood Coding Style
    43. 43. Importance ofGood Coding Style• Code is read much more often then it is written (did Guido van Rossum say this?)• Other people being able to read your code is important to attract contributors• Contributors make your code better and ultimately more successful
    44. 44. Style Guide• Available here: •
    45. 45. puppet-lint• Also made by Tim Sharpe from Github• Uses the Puppet Labs style guide• Saves time, as it tells you what is wrong with your content before a code review
    46. 46. Using it in your RakefileDownload the gem: gem install puppet-lintThen just add the following to your modules Rakefile: require puppet-lint/tasks/puppet-lint
    47. 47. Running it:# rake lintEvaluating manifests/init.ppEvaluating manifests/config.ppEvaluating manifests/package.ppEvaluating manifests/service.pp
    48. 48. Packaging for the Forge
    49. 49. The Modulefile• Provides meta data for your module• Includes dependencies • In the future the module tool can use these dependencies to download extra modules as needed• Is used to generate metadata.json, which is used by the forge
    50. 50. The Modulefilename puppetlabs-bindversion 0.0.1source Puppet Labs Inc.license ASL 2.0summary ISC Bind Moduledescription Installs and configures the ISC BIND DNS serverproject_page puppetlabs/stdlib, >= 2.2.1dependency puppetlabs/kwalify, >= 0.0.1dependency ripienaar/concat, >= 20100507
    51. 51. Packaging using the module tool# puppet-module build===================================================Building /Users/ken/Development/puppetlabs-bind for release-----------------------------------------------------------Done. Built: pkg/puppetlabs-bind-0.0.1.tar.gz
    52. 52. Uploading To the Forge• Create an account - this username will be publicly visible, so if you are a company create a shared one• Create a project - this is the short name of the module• Create a revision - this is the revision you wish to upload. Use semantic versioning if you can ie. 1.2.0, 3.1.2 etc.• Upload your revision, this is the tar.gz file created in pkg/ with puppet-module build
    53. 53. Using a forge moduleUsing the new module face you can install the forgemodule: # puppet module search bind # puppet module install puppetlabs-bind
    54. 54. Future WorkNeeded for Modules• Acceptance Testing • ‘Real’ build testing, perhaps using monitoring frameworks for acceptance?• Hiera and data lookup methodology • Using Hiera to emulate the params pattern
    55. 55. AnyQuestions?