Puppet Camp Sydney
The (im)perfect Puppet module
2AGENDA
Who we are
The common pattern
Pitfalls
Module standards
Module walk-through‘
WHO IS ODECEE?
Innovation for the Enterprise
A digital world where technology is simple
and business gets what it wants
170+ staff
Banking/financial
services
Communications Logistics Insurance Government
Offices in
Melbourne &
Sydney
Digital
DevOps
Sprout is designed to collaborate on new ideas and develop
innovative ‘real world’ solutions.
4WHO WE ARE
PHILIP JAYPETER HALL
•  Ten years industry experience
•  Odecee Lead Engineer
•  Swinburne University Centre for
Astrophysics & Supercomputing
•  realestate.com.au
•  catchoftheday.com.au
@peterkh
•  Eight years industry experience
•  Odecee Senior Engineer
•  Ericsson Mobile Packet Core
•  Softel Systems
•  Puppet certified professional #454
@ps_jay
5
Data
SCOPE
Business logic
goes here
(not sharable)
Node
Role
Profile
No business logic
Sharable on the
Puppet Forge
Class
Module
Hiera
Explicit
Hiera lookups
Automatic
lookups only
Scope of our talk
6
Style guide:
•  v1.1.2: Existed 2011 – 2015
•  v2.0.1: Current 2015 onwards
Guiding principles:
•  Readability matters
•  Scoping and simplicity are key
•  Your module is a piece of
software
Covers writing code but not how to
structure modules
CODE STANDARDS
docs.puppetlabs.com/guides/style_guide.html
7MODULE STANDARDS
Two documents
•  Module fundamentals: covers file-system layout
docs.puppetlabs.com/puppet/latest/reference/modules_fundamentals.html
•  Beginners guide to modules: step-by-step guide
docs.puppetlabs.com/guides/module_guides/bgtm.html
8
>	
  puppet	
  module	
  generate	
  pcsyd15-­‐foo	
  
	
  
pcsyd15-­‐foo/Gemfile	
  
pcsyd15-­‐foo/metadata.json	
  
pcsyd15-­‐foo/Rakefile	
  
pcsyd15-­‐foo/README.md	
  
pcsyd15-­‐foo/manifests/init.pp	
  
pcsyd15-­‐foo/spec/spec_helper.rb	
  
pcsyd15-­‐foo/spec/classes/init_spec.rb	
  
pcsyd15-­‐foo/tests/init.pp	
  
MODULE SKELETON
9AUTOMATIC PARAMETER LOOKUP
Puppet 3 – automatic lookup
class	
  foo	
  (	
  
	
  	
  param_1,	
  	
  #	
  no	
  default	
  
	
  	
  param_2	
  =	
  “a	
  default”,	
  
)	
  
Since puppet 3, parameters get their value automatically
Explicitly set in a resource-like declaration
Hiera lookup (automatic)
Default provided by class
If no default, catalog compile fails
Equivalent Puppet 2 code
class	
  foo	
  (	
  
	
  	
  param_1	
  =	
  hiera(“foo::param_1”),	
  
	
  	
  param_2	
  =	
  hiera(“foo::param_2”,	
  “a	
  default”)	
  
)	
  
10THE COMMON MODULE PATTERN
params.pp
•  Defaults
•  Handle operating system
differences
init.pp
•  Inherit from params.pp
•  Contain (or anchor) other
subclasses (install,
config, …)
Hiera
config.pp, install.pp, etc.
•  Inherit from init.pp
•  Contained by init.pp
init.pp variables
referenced by all other
classes/defines
puppet-stdlib used for
param validation
Well known Puppet Forge
contributors following this
pattern:
•  puppetlabs
•  example42
•  garethr
11PITFALL #1: Looking up Hiera directly
•  Module less sharable: not all installations use Hiera
•  Harder to debug: more areas of code to identify Hiera
lookups
•  Non-obvious: Module user needs to know all “magic”
Hiera values to set for desired outcome
•  Complex unit testing: Need to feed Hiera values as well
as parameters
12PITFALL #1: Hunting for Hiera lookups
class	
  nginx::params	
  {	
  
	
  	
  $worker_processes	
  =	
  $::processorcount	
  
	
  	
  $worker_connections	
  =	
  ‘1024’	
  
	
  	
  $conf_template	
  =	
  ‘nginx/nginx.conf.erb’	
  
	
  	
  $error_pages_default	
  =	
  false	
  
	
  	
  $set_real_ip	
  =	
  hiera(“nginx::real_ip_range”)	
  
	
  	
  $real_ip_header	
  =	
  ‘X-­‐Forwarded-­‐For’	
  
	
  	
  $blocked_ips	
  =	
  []	
  
	
  	
  $certs	
  =	
  {}	
  
	
  	
  $server_template	
  =	
  'nginx/server-­‐template.conf.erb’	
  
...	
  
13PITFALL #2: Module Scope
•  A module should only have one area of responsibility
•  That module should be the only module with that area of
responsibility
•  Anti-pattern:
•  Apache module for installing httpd
•  MyPHPApp module installs a vhost config in:
/etc/httpd/conf.d/
14PITFALL #3: Dependencies
•  Don’t have “too many” dependencies
•  puppet-stdlib is okay & expected
•  Aim for zero dependencies
•  Testing is more complex: need to include the dependency
•  Module less sharable: now dependent on additional classes
that others may not have
•  From Module Fundamentals:
“Be wary of having classes declare classes from other modules”
15
Example of a class including a class:
#	
  Class	
  for	
  creating	
  the	
  PuppetDB	
  postgresql	
  database	
  
class	
  puppetdb::database::postgresql(	
  
	
  	
  …	
  
)	
  inherits	
  puppetdb::params	
  {	
  
	
  	
  …	
  
	
  	
  #	
  create	
  the	
  puppetdb	
  database	
  
	
  	
  postgresql::server::db	
  {	
  $database_name:	
  
	
  	
  	
  	
  …	
  
	
  	
  }	
  
}	
  
Must also have
puppetlabs-postgresql
module
Alternatively:
•  Document in the README
and then link in a
“profile” class
•  Create module parameter
for inclusion of the
external class
PITFALL #3: puppetlabs-puppetdb
Module Walk-through
puppetlabs-ntp
github.com/puppetlabs/puppetlabs-ntp
CONSIDERATIONS
Anchor pattern or
contain function
needed for classes
inside classes
Hiera needed to
keep profiles and
roles sensible
Puppet version
> 2.6.2 needed for
param inherits
18
Separation between
modules keeps
testing simple
Common patterns
reduces staff
ramp up time
Mix puppet forge
and local modules
with ease
IN SUMMARY
Questions?
@odecee www.facebook.com/odeceesocialwww.odecee.com.au www.linkedin.com/company/odecee
Philip JayPeter Hall
Lead DevOps engineer
@peterkh @ps_jay
Senior DevOps Engineer

Puppet Camp Sydney 2015: The (Im)perfect Puppet Module

  • 1.
    Puppet Camp Sydney The(im)perfect Puppet module
  • 2.
    2AGENDA Who we are Thecommon pattern Pitfalls Module standards Module walk-through‘
  • 3.
    WHO IS ODECEE? Innovationfor the Enterprise A digital world where technology is simple and business gets what it wants 170+ staff Banking/financial services Communications Logistics Insurance Government Offices in Melbourne & Sydney Digital DevOps Sprout is designed to collaborate on new ideas and develop innovative ‘real world’ solutions.
  • 4.
    4WHO WE ARE PHILIPJAYPETER HALL •  Ten years industry experience •  Odecee Lead Engineer •  Swinburne University Centre for Astrophysics & Supercomputing •  realestate.com.au •  catchoftheday.com.au @peterkh •  Eight years industry experience •  Odecee Senior Engineer •  Ericsson Mobile Packet Core •  Softel Systems •  Puppet certified professional #454 @ps_jay
  • 5.
    5 Data SCOPE Business logic goes here (notsharable) Node Role Profile No business logic Sharable on the Puppet Forge Class Module Hiera Explicit Hiera lookups Automatic lookups only Scope of our talk
  • 6.
    6 Style guide: •  v1.1.2:Existed 2011 – 2015 •  v2.0.1: Current 2015 onwards Guiding principles: •  Readability matters •  Scoping and simplicity are key •  Your module is a piece of software Covers writing code but not how to structure modules CODE STANDARDS docs.puppetlabs.com/guides/style_guide.html
  • 7.
    7MODULE STANDARDS Two documents • Module fundamentals: covers file-system layout docs.puppetlabs.com/puppet/latest/reference/modules_fundamentals.html •  Beginners guide to modules: step-by-step guide docs.puppetlabs.com/guides/module_guides/bgtm.html
  • 8.
    8 >  puppet  module  generate  pcsyd15-­‐foo     pcsyd15-­‐foo/Gemfile   pcsyd15-­‐foo/metadata.json   pcsyd15-­‐foo/Rakefile   pcsyd15-­‐foo/README.md   pcsyd15-­‐foo/manifests/init.pp   pcsyd15-­‐foo/spec/spec_helper.rb   pcsyd15-­‐foo/spec/classes/init_spec.rb   pcsyd15-­‐foo/tests/init.pp   MODULE SKELETON
  • 9.
    9AUTOMATIC PARAMETER LOOKUP Puppet3 – automatic lookup class  foo  (      param_1,    #  no  default      param_2  =  “a  default”,   )   Since puppet 3, parameters get their value automatically Explicitly set in a resource-like declaration Hiera lookup (automatic) Default provided by class If no default, catalog compile fails Equivalent Puppet 2 code class  foo  (      param_1  =  hiera(“foo::param_1”),      param_2  =  hiera(“foo::param_2”,  “a  default”)   )  
  • 10.
    10THE COMMON MODULEPATTERN params.pp •  Defaults •  Handle operating system differences init.pp •  Inherit from params.pp •  Contain (or anchor) other subclasses (install, config, …) Hiera config.pp, install.pp, etc. •  Inherit from init.pp •  Contained by init.pp init.pp variables referenced by all other classes/defines puppet-stdlib used for param validation Well known Puppet Forge contributors following this pattern: •  puppetlabs •  example42 •  garethr
  • 11.
    11PITFALL #1: Lookingup Hiera directly •  Module less sharable: not all installations use Hiera •  Harder to debug: more areas of code to identify Hiera lookups •  Non-obvious: Module user needs to know all “magic” Hiera values to set for desired outcome •  Complex unit testing: Need to feed Hiera values as well as parameters
  • 12.
    12PITFALL #1: Huntingfor Hiera lookups class  nginx::params  {      $worker_processes  =  $::processorcount      $worker_connections  =  ‘1024’      $conf_template  =  ‘nginx/nginx.conf.erb’      $error_pages_default  =  false      $set_real_ip  =  hiera(“nginx::real_ip_range”)      $real_ip_header  =  ‘X-­‐Forwarded-­‐For’      $blocked_ips  =  []      $certs  =  {}      $server_template  =  'nginx/server-­‐template.conf.erb’   ...  
  • 13.
    13PITFALL #2: ModuleScope •  A module should only have one area of responsibility •  That module should be the only module with that area of responsibility •  Anti-pattern: •  Apache module for installing httpd •  MyPHPApp module installs a vhost config in: /etc/httpd/conf.d/
  • 14.
    14PITFALL #3: Dependencies • Don’t have “too many” dependencies •  puppet-stdlib is okay & expected •  Aim for zero dependencies •  Testing is more complex: need to include the dependency •  Module less sharable: now dependent on additional classes that others may not have •  From Module Fundamentals: “Be wary of having classes declare classes from other modules”
  • 15.
    15 Example of aclass including a class: #  Class  for  creating  the  PuppetDB  postgresql  database   class  puppetdb::database::postgresql(      …   )  inherits  puppetdb::params  {      …      #  create  the  puppetdb  database      postgresql::server::db  {  $database_name:          …      }   }   Must also have puppetlabs-postgresql module Alternatively: •  Document in the README and then link in a “profile” class •  Create module parameter for inclusion of the external class PITFALL #3: puppetlabs-puppetdb
  • 16.
  • 17.
    CONSIDERATIONS Anchor pattern or containfunction needed for classes inside classes Hiera needed to keep profiles and roles sensible Puppet version > 2.6.2 needed for param inherits
  • 18.
    18 Separation between modules keeps testingsimple Common patterns reduces staff ramp up time Mix puppet forge and local modules with ease IN SUMMARY
  • 19.
    Questions? @odecee www.facebook.com/odeceesocialwww.odecee.com.au www.linkedin.com/company/odecee PhilipJayPeter Hall Lead DevOps engineer @peterkh @ps_jay Senior DevOps Engineer