Automated System Configuration with Puppet

Puppet
Automated System Conﬁguration Management

Martin Alfke <martin.alfke@buero20.org>

1
Wednesday, December 8, 2010

Agenda
• Part I - Puppet Basics
• General + Communication
• Manifests, Modules, Templates + Functions

• Part II - Puppet Workshop
• Part III - Working with Puppet
• GIT/SVN for Puppet
• Production / Test / Development
• Monitoring

2

General
• “Put simply, Puppet is a system for automating
system administration tasks”

• Puppet...

• is a declarative language for expressing system
conﬁguration

• is a client-server distribution

• Requirements:

• Ruby > 1.8.1 < 1.9

• Facter

3

Communication
• Security

• SSL certificate based authentication

• manual signing of certificate requests

• Layers:

• Configuration Language

• Transaction layer

• Resource Abstraction Layer

4

Supported Platforms
• Linux

• Debian / Ubuntu / Fedora / CentOS / RHEL /
OEL / Mandriva / SuSE / Gentoo

• BSD

• FreeBSD / OpenBSD

• Other Unix

• OS X / Solaris / HP-UX

• Windows - coming in 2010

5

Functional Overview
• Clients connect to
Puppet Master
• Puppet Master send
clients description of
tasks
• Puppet Master stores
Clients reports
• Reports can be imported
into dashboard database
• Dashboard web interface
to reports

6

Facter
/usr/bin/facter
architecture => amd64
domain => buero20.local
facterversion => 1.5.7
fqdn => puppet.buero20.local
...
interfaces => eth0,eth1
ipaddress => 10.0.2.15
...
operatingsystem => Debian
processorcount => 1

7

Puppet Configuration Language - 1-6

• manifests/site.pp

• Global file with node definitions

• modules/<name>/manifests/init.pp

• Module initialization

• Use lower case for names (modules, templates, functions,
defines, exec, resources,...)

8


• Resources

• user - create or remove users

• group - create or remove groups

• package install or remove distribution packages

• ﬁle - create directories, symlinks, copy ﬁles

• cron - add cron jobs

• service - run or stop services like daemons

9


• Classes

• aggregate resources for easier use

• subclasses (=nested classes) for modularity

• parameterised classes for more ﬂexible handling

• classes support inheritance

10


• Deﬁnitions

• reusable objects

• Modules

• combine collections of resources, classes and
deﬁnitions

11


• Chaining resources

• make sure that a service is restarted after
filechange

• make sure that config file is copied prior
starting a service

• make sure that a package is installed prior
starting the service

12


• Nodes

• connect modules and clases to systems

• nodenames are short hostname, fqdn or
“default”

13

Manifests

• Deﬁne static resources
ﬁle { “/etc/passwd”: • Static resources have
owner => root, full path and name.
group => root,
mode => 644,
}

14

Manifests with facter Variables
• Using facter
variables inside
file { “sshconfig”:
a definition
name => $operatingsystem ? {
solaris => “/usr/local/etc/ssh/sshd_config”,
default => “/etc/ssh/sshd_config”,
},
owner => root,
group => root,
mode => 644,
}

15

Manifest with Sub-Classes
class mysql {
class client {
class packages {
package { "mysql-client": ensure => installed }
}
}
class server {
class packages {
package { "mysql-server": ensure => installed }
package { "mysql-common": ensure => installed }

16

Manifests with Exec

ﬁle {"/etc/apt/keys/pgp_key.asc":
owner => root, group => root, mode => 640,
source => "puppet://$server/ﬁles/etc/apt/keys/pgp_key.asc"
}
exec { "/usr/bin/apt-key add /etc/apt/keys/pgp_key.asc":
unless => "/bin/sh -c '[ `/usr/bin/apt-key list | grep buildd |
wc -l` -eq 1 ]'"
}

17

Manifests with Subscription
file {"/etc/apt/keys/puppet.key":
owner => root, group => root, mode => 640,
source => "puppet:///files/etc/apt/keys/puppet.key"
}
exec { subscribe-base-config-puppet-key:
command => "/usr/bin/apt-key add /etc/apt/keys/puppet.key;
/usr/bin/apt-get update",
logoutput => false,
refreshonly => true,
subscribe => File["/etc/apt/keys/puppet.key"]
}

18

Modules - Directory structure

• Directory structure - e.g. /etc/ssh/sshd_config
module/sshd/
manifests/
init.pp
files/
etc/ • Modules require strict
directories naming.
ssh/
sshd_config

19

Modules - Initialization Manifest

• init.pp manifest will be integrated
automatically when class name is
equal to module name
• modules/manifests/sshd/init.pp
class sshd {
file { “/etc/ssh/sshd_config”:
mode => 644,
source => “puppet:///modules/sshd/etc/ssh/sshd_config”,
}
}

20

Templates - Directory Structure

• Templates require strict
directory naming (like modules)

• Directory structure + content - e.g. Network settings
network/
manifests/
init.pp
templates/
network.erb

21

Templates - Initialization Manifest
• Templates may use facter variables

• Manifests - init.pp
ﬁle { “/etc/sysconﬁg/network”:
content => template(“templates/network.erb”),
}
• Templatess - network.erb
NETWORKING=yes
HOSTNAME=<%= hostname %>
NOZEROCONF=yes

22

Functions
• Directory structure e.g. read parameter
from conﬁguration ﬁle using facter:
lib/
facter/
function.rb
• Content of library functions function.rb:
require ‘facter’
Facter.add(“PUPPET_FUNCTION”) do
%x{/bin/grep -E “^PUPPET_FUNCTION=” /etc/
puppet_function | sed -e ‘s/*.=//’ } .chomp
end
end

23

Agenda

• Monitoring

24

Puppet Workshop
• Installation - Puppet master and client on puppet master only

• Initialization

• Installation - Puppet client on puppet client only

• Modules

• User Management

• Apache sites conﬁguration

• Templating for /etc/hosts

• Setup Reporting and Dashboard

25

Puppet Workshop - Installation - 1-5

•check requirements:
• ruby --version
• ruby -rshadow -e’print “OKn”’

26

•from source
• fetch and extract source
• wget http://puppetlabs.com/downloads/facter/
facter-1.5.8.tar.gz

• wget http://puppetlabs.com/downloads/puppet/
puppet-2.6.2.tar.gz

27


• install

• ruby install.rb

• mkdir /etc/puppet

28

•conﬁguration
• puppet --mkuser
• puppet --genconﬁg > /etc/puppet/puppet.conf
• vi /etc/hosts - add entry for nodename puppet if
not existing

29

•manifests/site.pp
• add empty section for default node
node default {
notice(“default node”)
}

30

Puppet Workshop - Initialization
•ﬁrst start of puppet:
• puppetd --test
•puppet CA
• check client certiﬁcate
• puppetca --list
• puppetca --list --all

31

Puppet Workshop - Modules - 1-2
•File Structure
• mkdir -p modules/<name>/{manifests,ﬁles}

•modules/<name>/manifests/init.pp
class <name> {
notice(“module <name>”)
}

32

Puppet Workshop - Modules - 2-2

•including modules in manifests/site.pp
node default {
include <name>
}

33

Puppet Workshop - Account Module - 1-6
• User Management

• create your personal login

• create home directory

1. Module directories
mkdir -p modules/users/{manifests,ﬁles}

34

2. Module init.pp

vi modules/users/manifests/init.pp

class users {
user{ "martin":
! home!! ! ! ! ! => "/home/martin",
! managehome! => true,
! shell! ! ! ! ! ! => "/bin/bash",
! comment!! ! ! => "Martin Alfke",
! ensure!! ! ! ! => present,

35


#!uid! ! ! ! ! ! => 0,
#!gid !! ! ! ! ! => 0,
# password ! ! => '0OfNn.f5krlF2',
#!allowdupe !! => true,
}
}

36


3. modify site.pp

vi manifests/site.pp

node default {
! include users
}

37

1. create new file
mkdir -p modules/users/files/home/martin/www/

cat > modules/users/files/home/martin/www/index.html
<< EOF
<html>
<head><title>My testsite</title></head>
<body>
foo
</body>
</html>
EOF

38

2. Module init.pp
add to modules/users/manifests/init.pp
class users {
......
ﬁle {“/home/martin/www”:
! ensure => directory,
}
ﬁle{“home/martin/www/index.html”:
! source => “puppet:///modules/users/home/martin/
www/index.html”,
}
}

39

Puppet Workshop - Apache Module - 1-6
• Apache sites Management

• packages

• your own vhost config

mkdir -p modules/apache/{manifests,files}
mkdir -p modules/apache/files/etc/apache2/sites-available/

40

2. your vhost deﬁnition

cat > modules/apache/ﬁles/etc/apache2/sites-available/blit-
test << EOF
Listen 88
NameVirtualHost *:88
<VirtualHost *:88>
! DocumentRoot /home/martin/www
</VirtualHost>
EOF

41

2. Module init.pp
vi modules/apache/manifests/init.pp

class apache {
! package{“apache2”: ensure! ! => present }
!
! package{“php5-mysql”: ensure! => present }
! ﬁle{“/etc/apache2/sites-available/blit-test”:
! ! source => “puppet:///modules/apache/etc/apache2/sites-
available/blit-test”,
! }
}

42


3. Add to node default manifest site.pp

include apache

43

• Apache sites Management

1. Add to apache init.pp • enabling sites with function
class apache {
...
deﬁne vhost ($ensure = ʻpresentʼ) {
! case $ensure {
! ! ʻpresentʼ: {
! ! exec { “/usr/sbin/a2ensite $name”:
! ! ! unless => “/bin/readlink -e /etc/apache2/sites-enabled/$name”
! ! }
! ! }

44


! ! ʻabsentʼ: {
! ! ! exec { “/usr/sbin/a2dissite $name”:
! ! ! ! onlyif => “/bin/readlink -e /etc/apache2/sites-enabled/$name”
! ! ! }
! ! }
! ! default: { err (“Unknown ensure value: $ensure) }
! }

45


! vhost {“blit-test”:
! ! ensure => “present”,
! }
! vhost{“000-default”:
! ! ensure => absent,
! }
}

46

Puppet Workshop - Templates - 1-2

• File Structure

• mkdir -p modules/<name>/{manifests,templates}

• modules/<name>/manifests/init.pp
class <name> {
notice(“module <name>”)
}

47

Puppet Workshop - Templates - 2-2

• including modules in manifests/site.pp
node default {
include <name>
}

48

Puppet Workshop - Hosts Template - 1-3
• Hosts Template

• conﬁgure entries in /etc/host

mkdir -p modules/hosts/{manifests,templates}

49

2. Module init.pp

vi modules/hosts/manifests/init.pp

class hosts {
! ﬁle{“/etc/hosts”:
! ! owner! => root,
! ! group! => root,
!
! ! content!=> template(hosts.erb),
! }
}

50


3. template hosts.erb

vi templates/hosts.erb

127.0.0.1!localhost
<%= ipaddress %>!<%= fqdn %> <%= hostname %>
192.168.0.2! puppet
192.168.0.4! mysql! mysqlmaster

51

Puppet Workshop - Functions - 1-2

• File Structure

• mkdir -p modules/<name>/lib/

52

Puppet Workshop - Functions - 2-2

• including modules in manifests/site.pp
node default {
include <name>
}

53

Puppet Workshop - Facter Function - 1-4
• Facter Function

• provide additional fact

mkdir -p modules/facter/lib/facter

54

2. function.rb

vi modules/facter/lib/facter/function.rb

require ʻfacterʼ
Facter.add(“PUPPET_FUNCTION”) do
! setcode do
! ! %x{/bin/grep -E “^PUPPET_FUNCTION=” /etc/puppet_function |
sed -e ʻs/.*=//ʼ}.chomp
! end
end

55


3. puppet.conf
section [main]
pluginsync = true

4. puppet run
puppetd --test

5. call facter puppet function
facter --puppet | grep puppet_function

56


3. use custom facts in manifests
case $puppet_function {
! “MYSQL”:! ! { include mysql }
! “APACHE”:!! { include apache }
! “PROXY”:! ! { include proxy }
...
}

57

Puppet Workshop - Dashboard - 1-5

• Installation

• fetch and extract sourc

• wget http://puppetlabs.com/downloads/
dashboard/puppet-dashboard-1.0.4.tgz

• install mysql-server

58


• Conﬁguration

• edit /usr/share/puppet-dashboard/conﬁg/database.yaml

• Create Database

• cd /usr/share/puppet-dashboard; rake RAILS_ENV
db:create or

• mysql -Ne ‘create database dashboard;’

59


• Initialize Database

• cd /usr/share/puppet-dashboard; rake
RAILS_ENV db:migrate

• Import Reports

RAILS_ENV=production reports:import

60


• Start service

• cd /usr/share/puppet-dashboard; ./bin/server
-e production -d

• Review your Dashboard in browser

• http://<your puppetmaster ip>:3000/

61

• add error to manifest (e.g. point source to a
non existing ﬁle)

• run puppetd

• puppetd --test

• import data

RAILS_ENV=production reports:import

• review dashboard

62

Agenda

• Monitoring

63

Puppet into GIT/SVN

• Why revision control system?

• Co-working

• Branches

• Which RCS System?

• Which ever you prefer

64

Puppet Staging
• Production, Test and Development

• /etc/puppet/puppet.conf

• [main] - environment = ...

• [development] - modulepath=/etc/puppet/
development/modules

• [testing] - modulepath=/etc/puppet/testing/
modules

• [production] - modulepath=/etc/puppet/
production/modules

65

Puppet Monitoring

• Puppet Dashboard

• Conﬁgure puppet to store results

• [master] section: reports=http, store

• [agent] (v2.6) or [puppetd] section: report=true

• Conﬁgure Database (e.g. MySQL)

66

Puppet Dashboard

67

Puppet
Automated System Conﬁguration Management

Thank you !
Questions ?

Martin Alfke <martin.alfke@buero20.org>

68

Automated System Configuration with Puppet

Recommended

Recommended

More Related Content

Similar to Automated System Configuration with Puppet

Similar to Automated System Configuration with Puppet (20)

More from Martin Alfke

More from Martin Alfke (18)

Recently uploaded

Recently uploaded (20)

Automated System Configuration with Puppet