A Presentation about Puppet that I've made at the OSSPAC conference

Infrastructure Automation Ohad Levy – Infineon Technologies [email_address]

Automated
”… a process which may once have been performed manually but has been altered in some way which allows a machine or computer to either wholly or partially manipulate the process to save time”

Infrastructure
“ Infrastructure is generally a set of interconnected structural elements that provide the framework supporting an entire structure”

Automated Infrastructure
“ Having the basic services necessary for your infrastructure to operate largely without the aid of a keeper.”

Grow – from This: http://i68.photobucket.com/albums/i2/RealConnections/googlecomputers.jpg

Large Scale Environment

Where does it fit in?

Typical System Lifecycle Pre/ Installation DNS/DHCP … Initial Configuration Kickstart Jumpstart Custom Scripts Fixes Updates Audits

System configuration possibilities
Manual System Configuration
„ log in and „just do it!““
+ OK for small number of servers
+ Quick and easy
- Very difficult to ensure similar configurations
- No auditing capability
- No change history
- No release documentation
- No way to deploy multiple servers quickly.
- No way to rebuild critical servers quickly.

System configuration possibilities
Install Time Auto-Configuration
„ Deploy using Kickstart, Jumpstart etc“
+ Ensures that new systems are brought into proper state as part of installation process
+ Possible to deploy multiple servers quickly
- No Validation that settings remain correct
- No easy way to deploy changes
- Auto-Configuration process is different between the sites  leads to different configurations
- No Auditing
- No change history
- No release documentation

Typical System Lifecycle Pre/ Installation DNS/DHCP … Initial Configuration Kickstart Jumpstart Custom Scripts Fixes Updates Audits Puppet

Puppet
What is Puppet ?
A product designed to deploy system configurations. It is –
Open source based on Ruby
Policy based
Runs every 30 minutes
An abstraction layer between the system administrator and the system
Capable to run on any UNIX operating system

Our Challenges
Keep our systems "harmonized“
Audit / Know what's going on each system
Replace a server if it dies or to be able to add another server that is exactly like it
Similar Applications, different OS's
Push out changes to all the servers that need a particular change
Stop duplicating effort
Go home early

How does Puppet works?

Puppet Types
A Type is a particular element that Puppet knows how to configure
Files (content, permissions, ownership)
Packages (installations, updates..)
Services (enabled/disabled, running/stopped)
Exec (run commands)
Full List: cron, exec, file, filebucket, group, host, interface, k5login, mailalias, maillist, mount, nagios*, package, service, sshkey, tidy, user, yumrepo, zone

Example: Managing sudoers file
file { /etc/sudoers:
owner => root,
group => root,
mode => 600,
source => puppet://server/files/sudoer
}

Dependencies
" require " and " before " / " after " settings ensures that types are
applied in the correct order
file { "/etc/sudoers":
...
require => Package[sudo]
}
package { "sudo":
ensure => present,
before => File["/etc/sudoers"]
}

Dependencies - continued
" notify " and " subscribe " settings can trigger cascaded updates
Particularly useful in services, exec
file { "/etc/ssh/sshd_conf":
...
notify => Service["sshd"]
}
service { "sshd":
subscribe => File["/etc/ssh/sshd_conf“]
}

What is Facter?
Facter gathers information about the client, which can be used as variables within puppet.
You can add custom facts as needed.
package {"sshd":
ensure => installed,
name => $operatingsystem ? {
solaris => "IFKLssh",
default => “openssh-server”
}
}

Example Facts
$ sudo facter
architecture => amd64
domain => sin.infineon.com
facterversion => 1.3.8
fqdn => sinn1636.sin.infineon.com
hardwaremodel => x86_64
hostname => sinn1636
ipaddress => 172.20.89.113
kernel => Linux
kernelrelease => 2.6.24-21-generic
lsbdistcodename => hardy
lsbdistdescription => Ubuntu 8.04.1
lsbdistid => Ubuntu
lsbdistrelease => 8.04
macaddress => 00:1c:25:14:26:ab
manufacturer => LENOVO
max speed => 2000 MHz
maximum memory module size => 4096 MB
maximum total memory size => 8192 MB memoryfree => 988.30 MB memorysize => 1.94 GB operatingsystem => Debian operatingsystemrelease => 2.6.24-21-generic processor0 => Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz processor1 => Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz processorcount => 2 productname => 6458B43 ps => ps -ef puppetversion => 0.24.5 rubysitedir => /usr/local/lib/site_ruby/1.8 rubyversion => 1.8.6 serialnumber => L3A8632 swapfree => 912.23 MB swapsize => 956.99 MB type => Notebook uniqueid => 007f0101 vendor => LENOVO

What is a Class?
A named collection of type objects
Can include or inherit from other classes
class sudo_class {
include foo_class
file { "/etc/sudoers":
...
}
package { "sudo":
...
}
}

Class inheritance
class afile {
file { "/tmp/foo":
ensure => file
source => "/src/versionA"
}
}
class another_file inherits afile {
File ["/tmp/foo"] {
source => "/src/versionB"
}
}

What is a Node ?
A configuration block matching a client
Can contain types, classes
"default" node matches any client without a node block
node "ohad.myself" {
include sudo_class
include other_class
}

External Nodes
Node definitions can be defined outside of puppet – e.g. LDAP, external script…
Ideal when you have too many nodes..

Classes and definitions
Classes are groups of resources.
Definitions are similar to classes, but they can be instantiated multiple times with different arguments on the same node.
class apache2 {
define simple-vhost ( $admin = "webmaster", $aliases, $docroot) {
file { "/etc/apache2/sites-available/$name":
mode => "644",
require => [ Package["apache2"], Service["apache2"] ],
content => template ("apache/vhost.conf"),
}
}
}

Classes and definitions - Continued
node mywebserver {
include apache2
apache2::simple-vhost { "test.example.com":
docroot => "/var/www/test“
}
apache2::simple-vhost { "debian.example.com":
docroot =>"/var/www/debian“,
alias => [“debiantest.example.com”,”debian”],
admin => “system@example.com”
}
}

vhost.conf template
Puppet uses Ruby's ERB template system:
<VirtualHost *>
ServerAdmin <%= admin %>
DocumentRoot <%= docroot %>
ServerName <%= name %>
<% aliases.each do |al| -%>
ServerAlias <%= al %>
<% end -%>
ErrorLog "|/usr/bin/cronolog /var/log/apache/<%=
name %>/%Y-%m/error-%d"
CustomLog "|/usr/bin/cronolog /var/log/apache/<%=
name %>/%Y-%m/access %d" sane
</VirtualHost>

Template output
more /etc/apache2/sites-available/debian.example.com
<VirtualHost *>
ServerAdmin [email_address]
DocumentRoot /var/www/debian
ServerName debian.example.com
ServerAlias debiantest.example.com
ServerAlias debian
ErrorLog |/usr/bin/cronolog
/var/log/apache/debian.example.com/%Y-%m/error-%d“
CustomLog "|/usr/bin/cronolog
/var/log/apache/debian.example.com/%Y-%m/access-%d“ sane
</VirtualHost>

OS API - It also works the other way around:
$ ralsh user levyo
user { 'levyo':
groups => ['tidc'],
ensure => 'present',
password => 'x',
shell => '/bin/bash',
uid => '49960',
comment => 'Ohad Levy, +65 xxxx xxxx',
gid => '11100',
home => '/home/levyo'
}

Puppet Modules
Modules allow you to group both the logic and the files for an application together.
Puppet automatically searches its module path to find modules.
Modules can contain four types of files, each of which must be stored in a separate subdirectory:
Manifests – must be stored in manifests/, and if you create manifests/init.pp then that file will be loaded if you import that moudle name directly (e.g. import “mymodule”)
Templates – must be stored under templates/, put in your manifest some where template(“mymodule/mytemplate”)
Files – stored in files/, can be accessed as: source => puppet://”mymodule”/myfile
Plugins – Additional types, provides or facts

File server and File Bucket
Puppet also includes a file server which you can use for transferring files from the server to the client.
If you configure it, puppet can also save a backup of each file that is changed on the client to the server. The backups go in a filebucket and can be retrieved later.

Getting Started
Install puppet (yum,pkg-get,…) or manually install ruby, facter and puppet
Setup a puppet server (puppetmaster / puppet-server).
Write a manifest for your node (classes etc)
Start puppet master on the server
Start puppetd on the client

Some more info
Puppet uses SSL for all communications, therefor it includes a CA, you can automatically sign CSR or use puppetca tool to mange them.
Storeconfig, option to save machine states(facts, configuration runs) and special facts (e.g. SSH keys) in a database.
Reporting, puppet has a few reporting options, most common are emails with changes, RRD files, yaml files and puppetshow web interface.
Puppet HA, load balancing etc

What are the benefits ?
Costs
Based on open source tools – No license cost
Reduced Labor  Reduced Cost
Productivity
Reproducibility, Accuracy and homogenous environment
(Reduce needed knowledge to operate our environment)
Allow easy ramp up of new sites and services
Change Management
Testing Processes, Auditing
Global control over servers
Harmonization  Consolidation
…

Conclusions
We're all stuck on the hamster wheel
Makes easy stuff easier, hard stuff possible
Harmonize! (not just today, also in the future)
Enable the business for consolidation
Similar projects
cfengine
bcfg2
Additional Resources
http://reductivelabs.com/trac/puppet
http://reductivelabs.com/trac/puppet/wiki/LanguageTutorial
http://reductivelabs.com/trac/puppet/wiki/CompleteConfiguration
#puppet on irc.freenode.org

Backup Slides Additional information

Configuration DB - Terminology
Module – A bundle of settings related to a single application
At Infineon we use 3 types of modules:
Host Type - e.g. Login Servers, Compute nodes…
Service - module which is needed by other modules, e.g. ssh, x11 etc.
Sites Modules - sites customizations
Versioning – The ability to refer (tag) to a certain state/PiT
Host Type and Service modules are associated with a version, e.g. Login Server v1.0
Site Modules are not tagged in order to reduce complexity
* Every change in a module, requires a check in action, which is identified by a user, and a comment (change log)
* A user access restriction should be applied to each module and to the tagging mechanism

Environments – The ability to select a group of modules in a certain version
Each site is associated with at least two environments – Production and Testing:
Testing – an environment which refer to versioned modules under a test tag.
Production – A selection of modules in a certain version that has already been approved for Production usage.
Development environment – The state where the modules are in the latest check-in version, but are not tagged (e.g. not a testing candidate yet).
Development  Testing  Production (Per Host Type)

Global modules / owners
Each owner is responsible for his module (or modules)
A special attentions is required for Service module as many other modules might depend on them
Site modules
As Site modules are not tagged (versioned) any change in the site specific settings, will be rolled out immediately.

Risks & Challenges
If the right processes are not in place, it would be possible to implement the wrong changes globally
Learning Curve
How our platform works
How to use version control
How to work in the new processes/workflows
Fear (Am I obsolete?, will I continue to learn?)

Why Did we choose Puppet
Open source
More flexible than other open source alternatives
Really works (Declarative language, Abstracts configuration as resources, Allows relationships, Transactional, Modular)
Provides an API to the OS, making it very simple to support other OS’s in the future
Puppet is now being adopted in many projects
For more info see http://reductivelabs.com/trac/puppet/wiki/CfengineVsPuppet
http://reductivelabs.com/trac/puppet/wiki/WhosUsingPuppet

Datacenters Challenges
Harmonization
Simplify Servers management
Implement and maintain Global standards
Accurate Configuration Database
Rapid recovery and deployment of services globally
Eliminates duplicate / unnecessary effort
Same look and feel everywhere (enables consolidation / global operation )
Audits / Compliance

Infineon Puppet Infrastructure
Puppeteer (Puppet Master of site PM’s)
Puppet Master per site (or based on size/load)
SQL Database for Web portal, Inventory
Configuration database (Version control & Ticketing system)

Gini’s Features
AD/LDAP Authentication
DHCP/DNS Connectors (API, + MS(yacks) DHCP, MS(yacks) DNS and infoblox)
Puppet Host type management
Disk Layout Management
Host Hardware management (needed for Solaris DHCP jumpstart, and general host info)
Multi OS management (e.g. RH, CentOs, Solaris.... in their various versions)
User role management (who is allowed to install, in which site, which host type etc)
Subnet and IP Management (auto assignment of IP address, DHCP reservation DNS records etc)
Multi Domain / site management
Hands-free auto OS installation (currently Kickstart and Jumpstart (sparc) using DHCP)
Management of Puppet Environments, choose an env per host (e.g. we have different production states for different sites, or for different host types).
Management of puppet certificates (also support chained certificates).
Import of Machine Facts (we cant use store config due to the large setup of our env)
Nice Graphs (with Gruff)
Puppet modules selection per host(for those who needs more than the default host type)
SP (Service Processor, iLOM etc) management
TFTP management
Status indications for machine state (pre installation, during puppet first run, etc)
... and probably a lot of other small stuff that I cant think of now

Our Solution Web Portal Puppet Configuration DB Inventory DHCP/DNS .. Harmonized

Where does GINI live? GINI DB DNS TFTP CERTIFICATES CMDB DHCP LOGS LSF Future work

http://www.slideshare.net	15
http://basavrajb.blogspot.com	2
http://www.techgig.com	2

A Presentation about Puppet that I've made at the OSSPAC conference

by ohadlevy

Accessibility

Upload Details

Usage Rights

3 Embeds 19

Statistics