Puppet | Custom Modules & Using the Forge

PUPPET

Custom Modules & Using the Forge

INTRODUCTION
Aaron Bernstein – Aspiring Engineer

Scott Eichner – Aspiring Engineer

▪ One of the lead developers in the Workspace
Team at GoDaddy

▪ One of the lead developers in the Workspace
Team at GoDaddy

▪

Been working with GoDaddy for about a year and
a half.

▪

Specifically tasked on our FaxThruEmail product.

▪ Experience includes:
▪

Fifteen years of programming in a multitude of
languages…

▪

Responsible for system administration over the last
decade.

▪

Currently working with operations/engineering on:
▪
▪
▪

Automating our configuration and package
management (RPM/YUM, Puppet);
Improving our build (deploy) process (Jenkins,
Spacewalk, Koji/Sake);
Implementing continuous integration efforts for
our product (Github, Unit Testing).

▪ Been working with GoDaddy for about 9
years.
▪ Specifically tasked on our Webmail
product for the past 6 years.
▪ Experience includes:
▪ Nine years of programming in a
multitude of languages…
▪ Recently tasked with DevOps role for
workspace platform team.
▪ Beginning to work on the Cloud
Platform team.

PRESENTATION STYLES
& DISCLAIMERS
•
•
•
•
•

Bold = Keywords
Italicized = Applicable Terms
Courier = Code Samples
Green = GoDaddy
Underlined = Links

!! All contents within this presentation have come from open sources, all credit goes to those who created it. I have merely become a messenger of the knowledge. It’s open source, pass it around !!
The MIT License (MIT)
Copyright (c) 2013 Aaron Bernstein & Scott Eichner
Permission is hereby granted, free of charge, to any person obtaining a copy of this presentation and associated documentation files (the “Presentation"), to deal in the Presentation without restriction,
including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Presentation, and to permit persons to whom the Presentation is furnished to
do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Presentation.
THE PRESENTATION IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT
OR OTHERWISE, ARISING FROM, OUT OF OR INCONNECTION WITH THE PRESENTATION OR THE USE OR OTHER DEALINGS IN THE PRESENTATION.

WHAT ARE MODULES?
▪ The smallest unit of measure that Puppet builds from.
▪ Generally thought of as a configuration containing each of our three core
requirements, installation, configuration and monitoring.
▪ Typically breaks down to classes,
▪ Classes are singleton collections of resources.

▪ Directories with files, arranged in a specific, predictable structure.
▪ Manifest files within a module have to obey certain naming restrictions.
▪ This set of directories is known as the modulepath, which is a configurable setting.

▪ If a class is defined in a module, you can declare that class by name
in any manifest. Puppet will automatically find and load the manifest that
contains the class definition.

GIVE YOUR MODULES PURPOSE
▪ Classes are Puppet’s way of separating
out chunks of code, and modules are
Puppet’s way of organizing classes so that
you can refer to them by name.
▪ What task do you need your module to
accomplish?
▪ What work is your module addressing?
▪ What higher function should your module
have within your Puppet environment?
▪ Puppet Labs best practice strongly
recommends creating multiple modules
when and where applicable. The practice
of having many small, focused modules is
encouraged, as it promotes code reuse
and turns modules into building blocks
rather than full solutions.

Base

Web

Application

Fax

User Profiles

Webserver

Tomcat

Fax Software

Basic System
Packages

API Client

Caching
Systems

Modems

Security
Patches

Caching
Systems

DSN Broker

DSN Broker

Configurations

Configurations

Configurations

DESIGN PATTERN:
NODE/ROLE/PROFILES/MODULES/RESOURCES

▪ A node includes one role, and one only;
▪ A role includes one or more profiles to define the type of server;

▪ A profile includes and manages modules to define a logical technical stack;
▪ Modules manage resources and should only be responsible for managing
aspects of the component they are written for.

MODULE DIRECTORY STRUCTURE
▪ my_module — This outermost directory’s name matches the name of the module.
▪

manifests/ — Contains all of the manifests in the module.
▪
▪
▪
▪

init.pp — Contains a class definition. This class’s name must match the module’s name.
other_class.pp — Contains a class named my_module::other_class.
my_defined_type.pp — Contains a defined type named my_module::my_defined_type.
implementation/ — This directory’s name affects the class names beneath it.
▪ foo.pp — Contains a class named my_module::implementation::foo.
▪ bar.pp — Contains a class named my_module::implementation::bar.

▪

files/ — Contains static files, which managed nodes can download.
▪

service.conf — This file’s URL would be puppet:///modules/my_module/service.conf.

▪

lib/ — Contains plugins, like custom facts and custom resource types.

▪

templates/ — Contains templates, which the module’s manifests can use.
▪

▪

tests/ — Contains examples showing how to declare the module’s classes and defined types.
▪
▪

▪

component.erb — A manifest can render this template with template('my_module/component.erb').
init.pp
other_class.pp — Each class or type should have an example in the tests directory.

spec/ — Contains spec tests for any plugins in the lib directory.

DESIGN PATTERN:
PACKAGE/FILE/SERVICE
▪ Also referred to as “THE TRIFECTA”

# Package/File/Service SSH Example

▪ Learn it, live it, love it.

package { 'openssh-server':
ensure => installed,
}

▪ If you can only do this, you can still do a lot.

file { '/etc/ssh/sshd_config':
source => 'puppet:///modules/sshd/sshd_config',
owner => 'root',
group => 'root',
mode => '640',
# will restart whenever modified.
notify => Service['sshd'],
require => Package['openssh-server'],
}
service { 'sshd':
ensure => running,
enable => true,
hasstatus => true,
hasrestart => true,
}

MODULE CLASSES
▪

Naming Convention
▪

▪

The file must be named the same as the class or definition that is
contained within (with the sole exception of the main class), and
classes must be named after their function.

Modules
▪

The main class of a module is its interface point, and ought to be
the only parameterized class if possible. Limiting the
parameterized classes to just the main class allows you to
control usage of the entire module with the inclusion of a single
class.

▪

The main class of any module must share the name of the
module and be located in the ìnit.pp` file.

▪

The install class must be located in the ìnstall.pp` file, and
should contain all of the resources related to getting the
software the module manages onto the node.

▪

The resources related to configuring the installed software
should be placed in a config class and must be located in the
`config.pp` file.

▪

The remaining service resources, and anything else related to the
running state of the software, should be contained in the service
class. The service class must located in the `service.pp` file.

▪

Once a class is stored in a module, there are actually several
ways to declare or assign it
▪

You can declare classes by putting ìnclude ntp` in your main
manifest. The include function declares a class, if it hasn’t
already been declared somewhere else. If a class HAS already
been declared, include will notice that and do nothing.

MODULE PARAMETERS
▪

Parameters form the public API of your
module. They are the most important interface you
expose, and care should be taken to give a good
balance to the number and variety of parameters so
users can customize their interactions with the
module.

class ntp::params {
$config_template =
$package_ensure =
$service_enable =
$service_ensure =
$service_manage =

▪

Naming consistency is imperative for community
comprehension and assists in troubleshooting and
collaborating on module development.

▪

Best practices recommend the pattern of
`thing_property` for naming parameters.

case $::osfamily {
'RedHat': {
$config
= '/etc/ntp.conf'
$driftfile
= '/var/lib/ntp/drift'
$keys_file
= '/etc/ntp/keys'
$package_name = [ 'ntp' ]
$service_name = 'ntpd'
$servers
= [
'0.centos.pool.ntp.org',
]
}
default: {
fail("The ${module_name} is not supported on an
${::osfamily} based system.")
}
}
}

▪

Puppet Standard Module Initiative
▪

▪

A community driven effort to standardize on parameters.

If you want to maximize the usability of your module
without requiring users to modify it, you have to
make it more flexible through the addition of
parameters. Adding parameters enables more
customized use of your module.

'ntp/ntp.conf.erb'
'present'
true
'running'
true

PUPPET DYNAMIC DATA
Params.pp
class ntp::params {
$config_template
$package_ensure
$service_enable
$service_ensure
$service_manage

= 'ntp/ntp.conf.erb'
= 'present'
= true
= 'running'
= true

case $::osfamily {
'RedHat': {
$config
= '/etc/ntp.conf'
$driftfile
= '/var/lib/ntp/drift'
$keys_file
= '/etc/ntp/keys'
$package_name = [ 'ntp' ]
$service_name = 'ntpd'
$servers
=[
]
}
default: {
fail("The ${module_name} module is not supported on an
${::osfamily} based system.")
}
}
}

Hiera (./data/operatingsystem/RedHat.yaml)
ntp::config_template: 'ntp/ntp.conf.erb'
ntp::package_ensure: 'present'
ntp::service_enable: = true
ntp::service_ensure: = 'running'
ntp::service_manage: = true
ntp::config: '/etc/ntp.conf'
ntp::driftfile: '/var/lib/ntp/drift'
ntp::keys_file: '/etc/ntp/keys'
ntp::package_name: [ 'ntp' ]
ntp::service_name: 'ntpd'
ntp::servers: [
]

MODULE INHERITANCE
▪ Modules can inherit from other
modules using the `inherits` in
the class construct.

class ntp::params { … }
class ntp (
$config_template = $ntp::params::config_template,
$package_ensure = $ntp::params::package_ensure,
$service_enable = $ntp::params::service_enable,
$service_ensure = $ntp::params::service_ensure,
$service_manage = $ntp::params::service_manage
) inherits ntp::params { … }
class ntp::verne inherits ntp { … }

MODULE DEPENDENCIES
▪ If your module's functionality
depends on another module, then
you must list these dependencies
and include them directly.
▪ This means you must `include x` in
your `init.pp` to ensure the
dependency is included in the
catalog.
▪ You must also add the dependency
to the Modulefile and
`.fixtures.yml`.

class openssh::install {
…
package { 'openssh-server':
ensure => installed,
}
…
}
class openssh::config {
…
file { '/etc/ssh/sshd_config':
source => 'puppet:///modules/sshd/sshd_config',
owner => 'root',
group => 'root',
mode => '640',
# will restart whenever modified.
notify => Service['sshd'],
require => Package['openssh-server'],
}
…
}
class openssh::service {
…
service { 'sshd':
ensure => running,
enable => true,
hasstatus => true,
hasrestart => true,
}
…
}

MODULE ORDERING
▪ Class-based ordering allows you
to shield the implementation
details of each class from the
other classes.

# modules/module/manifests/init.pp
Anchor['begin'] ->
Class['module::install'] ->
Class['module::config'] ~>
Class['module::service']->
Anchor['end']

▪ Due to a long-standing Puppet
bug (#8040), there is a need to
anchor classes within a module
to stop them floating around in
the ordering graph and
potentially doing things in the
wrong order.
▪ To this end we recommend you
anchor classes in the`init.pp` file for
your module.

(Note: anchoring requires puppetlabs-stdlib.)

TESTING PUPPET MODULES
▪ Rspec-Puppet
▪ A unit-testing framework for Puppet.
It extends RSpec to allow the testing
framework to understand Puppet
catalogs, the artifact it specializes in
testing.
▪ RSpec lets you provide facts, like
`osfamily`, in order to test the
module in various scenarios.

▪ Rspec-system
▪ An acceptance/integration testing
framework that provisions,
configures, and uses various Vagrant
virtual machines to apply your
puppet module to a real machine
and then test things from the
command line within the VM.

▪ serverspec
▪ Serverspec provides additional
testing constructs (such as
`be_running` and `be_installed`) for
rspec-system, and allows you to
abstract away details of the
underlying distribution when testing.

USING THE MODULEFILE
▪

Modulefiles support the following pieces of metadata:
▪

name — REQUIRED. The full name of the module, including the
username

▪

version — REQUIRED. The current version of the module. This
should be a semantic version.

▪

summary — REQUIRED. A one-line description of the module.

▪

description — REQUIRED. A more complete description of the
module.

▪

dependency — A module that this module depends on. Unlike
the other fields, the dependency method accepts up to three
comma-separated arguments:
▪

a module name (with a slash between the user and name, not a
hyphen), a version requirement, and a repository.

▪

project_page — The module’s website.

▪

license — The license under which the module is made available.

▪

author — The module’s author. If not provided, this field will
default to the username portion of the module’s name field.

▪

source — The module’s source. This field’s purpose is not
specified.

In your module’s main directory, create a text
file named Modulefile.
The Modulefile resembles a configuration or
data file, but is actually a simple Ruby domainspecific language (DSL), which is executed when
you build a tarball of the module.
# Modulefile
name ‘examplecorp-mymodule’
version ‘0.0.1’
summary ‘My module is an example of a
summary’
description “This is a full description of
the module, and is being written as a
multi-line string.“
dependency ‘puppetlabs/mysql’, ‘1.2.3’

DOCUMENTING PUPPET MODULES
▪ You are encouraged to document any
module you write, whether you intend the
module solely for internal use or for
publication on the Puppet Forge.
▪ Documenting your module will help futureyou remember what your module was built
to do and can help explain why you chose
to do things one way versus another. And
anyone else using your module will
definitely appreciate it.
▪ If you use the `puppet module generate`
command it will create a default
`README.md` template.
▪ Refer to the Puppet module style guide for
more on documentation.

#modulename
####Table of Contents
1. [Overview](#overview)
2. [Module Description - What the module does and why it is
useful](#module-description)
3. [Setup - The basics of getting started with
[Modulename]](#setup)
* [What [Modulename] affects](#what-[modulename]-affects)
* [Setup requirements](#setup-requirements)
* [Beginning with [Modulename]](#beginning-with[Modulename])
4. [Usage - Configuration options and additional
functionality](#usage)
5. [Reference - An under-the-hood peek at what the module is
doing and how](#reference)
5. [Limitations - OS compatibility, etc.](#limitations)
6. [Development - Guide for contributing to the
module](#development)
7. [Release Notes/Contributors/Etc] - **Optional**
##Overview
##Module Description
##Setup
###What [Modulename] affects

VERSIONING PUPPET MODULES
▪ Once you've decided on the new version
number, you must increase the version
number metadata in the Modulefile.
▪ Versioning within the Modulefile allows you
to build Puppet environments by picking and
choosing specific versions of a module or
set of modules.

▪ It also allows you to create a list of
dependencies in the Modulefile of your
modules with specific versions of
dependent modules, which ensures your
module isn't used with an ancient
dependency that won't work.

▪ Versioning also enables workflow
management by allowing you to easily
use different versions of modules in
different environments.

A Note on Semantic Versioning
▪ When writing your Modulefile, you’re setting a
version for your own module and optionally
expressing dependencies on others’ module
versions.
▪ We strongly recommend following the Semantic
Versioning specification. Doing so allows others to
rely on your modules without unexpected change.
▪ Many other users already use semantic versioning,
and you can take advantage of this in your
modules’ dependencies.
▪ For example, if you depend on
puppetlabs/stdlib and want to allow updates
while avoiding breaking changes, you could
write the following line in your Modulefile
(assuming a current stdlib version of 2.2.1):
▪ dependency 'puppetlabs/stdlib', '2.x'

THE FORGE
▪ REUSE
▪ Puppet Forge, a repository of modules
written by the community for Puppet
Open Source and Puppet Enterprise IT
automation software.
▪ These modules solve a wide variety of
problems so using them can save you
time and effort.
▪ SHARE
▪ Register an account, create a module,
upload a release of it and your
automation code is now shared with
the Puppet community.
▪

Learn how to create and share
modules using the puppet module tool.

MANAGING PUPPET MODULES
▪ The `puppet module` subcommand,
which ships with Puppet, is a tool
for finding and managing new
modules from the Forge. Its
interface is similar to several
common package managers, and
makes it easy to search for and
install new modules from the
command line.
▪ If you have used a command line
package manager tool (like gem,
apt-get, or yum) before, these
actions will generally do what you
expect.

/*
Use the --tree option with module list
to view the modules arranged by dependency
instead of by location on disk
*/
# puppet module list
# puppet module search apache
# puppet module install puppetlabs-apache --version
0.0.2
# puppet module upgrade puppetlabs-apache --version
0.0.3
# puppet module uninstall puppetlabs-apache

GENERATE & BUILD PUPPET MODULES
▪ Use the `puppet module
generate` action to generate a
template layout.
▪ An example Modulefile and
README, and will also include a
copy of the spec_helper tool for
writing rspec-puppet tests.

▪ Once your content and
Modulefile are ready, you can
build a package of your module.
▪ This will generate a .tar.gz package,
which will be saved in the module’s
pkg/ subdirectory.

# puppet module generate <USERNAME>-<MODULE NAME>
# puppet module build <MODULE DIRECTORY>

PUBLISHING PUPPET MODULES
▪ Puppet Forge requires module names to have
a username prefix.
▪
▪

Because many users have published their own
versions of modules with common names
(“mysql,” “bacula,” etc.)
Be sure to use this long name in your
module’s Modulefile.

▪ Now that you have a compiled tar.gz package,
you can upload it to the Puppet Forge. There is
currently no command line tool for publishing;
you must use the Puppet Forge’s web
interface.
▪ Your module has now been published to the
Puppet Forge. The Forge will pull your
README, Changelog, and License files from
your tarball to display on your module’s page.
To confirm that it was published correctly, you
can install it on a new system using the puppet
module install action.

To publish your module, you will need to:
1.

Create a Puppet Forge account (if you don’t already have one)

2.

Prepare your module

3.

Write a Modulefile with the required metadata

4.

Build an uploadable tarball of your module

5.

Upload your module using the Puppet Forge’s web
interface.
▪

If you have never published this module before, you
must create a new page for it.

SHOWCASE
▪ Roles/Profiles Repository.
▪ Present shared repository for
reusable roles/profiles.

▪ User Profile Module.
▪ Present a way to manage
configurations on many different
systems using Puppet.

▪ Using Vagrant and Puppet
▪ Demonstration of a complete build
out of a VM

▪ Current efforts in configuring and
managing our systems using
Puppet.

Thank you! Questions?
Need additional guidance or reference material?

Download or print a copy of our Core Types Cheat
Sheet and Modules Cheat Sheet for fast reference
when writing your first manifests.
Experienced Puppet users spend most of their time
in the type reference and the language guide.
Get acquainted with them early!

CONTACT
INFORMATION
Provide feel free to contact me at the following
addresses.
m: abernstein@godaddy.com
gd_github: abernstein
twitter: @bernstein_aaron
m: seichner@godaddy.com
gd_github: seichner

RESOURCES (DOCUMENTATION)
▪ Official Site

▪ Puppet Patterns

▪ Download Learning Puppet VM

▪ Modules Documentation

▪ Learning Guide

▪ Beginners Guide to Modules

▪ Style Guide

▪ Puppet 2.7.x Reference Manual

▪ Glossary

▪ Puppet Standard Module
Initiative

RESOURCES (DOCUMENTATION)
▪ Why Puppet Should Manage
Your Infrastructure
▪ Puppet Forge

▪ PuppetLabs Github
▪ Writing Custom Types
▪ Git Workflow and Puppet
Environments

▪ Rspec-Puppet
▪ Rspec-system
▪ serverspec
▪ Vagrant
▪ Mcollective
▪ Scaling Recommendations

RESOURCES (PRESENTATIONS)
▪ Slideshare - Puppetlabs
▪ Slideshare - PuppetConf 2013
▪ PuppetLabs - Roles/Profiles
▪ Puppet and Windows
▪ Getting Started with the
Learning Puppet VM

▪ What’s New and Awesome in
Puppet Enterprise 3
▪ Introduction to Mcollective

▪ Troubleshooting the Puppet
Enterprise Stack

Puppet | Custom Modules & Using the Forge

More Related Content

Similar to Puppet | Custom Modules & Using the Forge

Recently uploaded

Puppet | Custom Modules & Using the Forge