Infrastructure as Code
with Chef / Puppet
Edmund Haselwanter (e.haselwanter@cloudbau.de)

Samstag, 16. November 13
Introduction

Edmund Haselwanter

●Co-Founder and CSO cloudbau GmbH
●Dell Professional Services (Openstack/Crowbar/
Automation)

●Software Development Background
●10 Years Infrastructure Automation
●@ehaselwanter on twitter/github/irc/...
cloudbau
Samstag, 16. November 13
Business-Agility needs IT-Agility
For most companies, IT functions as the nervous system and provides
an increasing amount of the organizational muscle mass. Most critical
business functions are entirely automated within IT, and 95% of all
capital projects depend on IT to get done. Today, nearly every business
decision will result in at least one IT change.
„IT REVOLUTION MANIFESTO“, http://itrevolution.com

Theory of
Constraints

→

→
E.M. Goldratt, 1984

cloudbau
Samstag, 16. November 13

Gene Kim, Kevin Behr,
George Spafford, 2013
IT-Agility and the „Risk of Change“
Request

Approve

Develop

Test

WaterScrum

Deploy

- Fall

The core, chronic conflict that every IT
leader faces is the need to
simultaneously enable faster time to
market (i.e., make as many changes as
you can), while providing stable, secure
and reliable IT services (i.e., make as
few changes as you can). 
Gene Kim

„Lowering risk of change through tools and culture“
John Allspaw, Paul Hammond, 2009

small changes, often
reproducible

Samstag, 16. November 13

cooperative culture

process standards

cloudbau

expect failure
metrics
Tools and Culture
Config-DB

CPUVirt.
Binary-Repo

StorageVirt.
SDN

Orchestration

Prod Env

Programmable Infrastructure

Description in Code

Automatic Build
DevOps

Cross-Functional Teams

Process and Culture

cloudbau
Samstag, 16. November 13

Stage Env

Anti-Fragile Organizations

Continuos Delivery

Description

Dev Env
Configuration Management

cloudbau
Samstag, 16. November 13
Evolving towards Configuration
Management

●Just build it
●Keep notes in server.txt
●Move notes to the wiki
●Custom scripts (in scm?!)
●Snapshot & Clone
cloudbau
Samstag, 16. November 13
Applications

http://www.flickr.com/photos/steffenz/
337700069/
http://www.flickr.com/photos/kky/
704056791/

cloudbau
Samstag, 16. November 13
Infrastructure

cloudbau
http://www.flickr.com/photos/sbh/462754460/
Samstag, 16. November 13
Collection of Resources

•
•
•
•
•

Networking
Files
Directories
Symlinks
Mounts

•
•
•
•
•
•
•
•
•

cloudbau
http://www.flickr.com/photos/philliecasablanca/3354734116/
Samstag, 16. November 13

Routes
Users
Groups
Tasks
Packages
Software
Services
Configuration
Other Stuff
Acting in Concert

cloudbau
Samstag, 16. November 13

http://www.flickr.com/photos/glowjangles/4081048126/
To Provide a Service

cloudbau
Samstag, 16. November 13

http://www.flickr.com/photos/28309157@N08/3743455858/
And it Evolves

cloudbau
Samstag, 16. November 13

http://www.flickr.com/photos/16339684@N00/2681435235/
See Node

Application Server

cloudbau
Samstag, 16. November 13
See Nodes

Application Server

Application Database

cloudbau
Samstag, 16. November 13
See Nodes Grow

Application Server

Application Databases

cloudbau
Samstag, 16. November 13
See Nodes Grow

Application Servers

Application Databases

cloudbau
Samstag, 16. November 13
See Nodes Grow

Load Balancer

Application Servers

Application Databases

cloudbau
Samstag, 16. November 13
See Nodes Grow

Load Balancers

Application Servers

Application Databases

cloudbau
Samstag, 16. November 13
See Nodes Grow

Load Balancers

Application Servers

Application Database Cache

Application Databases

cloudbau
Samstag, 16. November 13
Tied Together with Configuration

Load Balancers

Application Servers

Application Database Cache

Application Databases

cloudbau
Samstag, 16. November 13
So when this

Graphite

Nagios

Jboss App

Memcache

Postgres Slaves

cloudbau
Samstag, 16. November 13

Postgres Master
Becomes This

Graphite

Nagios

Jboss App

Memcache

Postgres Slaves

cloudbau
Samstag, 16. November 13

Postgres Master
This can happen automagically

Graphite

Nagios

Jboss App

Memcache

Postgres Slaves

cloudbau
Samstag, 16. November 13

Postgres Master
Count the Resources

Graphite

Nagios

Jboss App

Memcache

Postgres Slaves

•

•
•
•
•
•
•
•
•
•
•
•
•

Load balancer config
Nagios host ping
Nagios host ssh
Nagios host HTTP
Nagios host app health
Graphite CPU
Graphite Memory
Graphite Disk
Graphite SNMP
Memcache firewall
Postgres firewall
Postgres authZ config

12+ resource changes for 1 node addition
cloudbau

Samstag, 16. November 13
Focus Today: Chef and Puppet

Series A: 2,5 M$ (2009)
Series B: 11 M$ (2010)
Series C: 19,5 M$ (2012)

Technologie

cloudbau
Samstag, 16. November 13

Series A: 2 M$ (2009)
Series B: 5 M$ (2010)
Series C: 8,5 M$ (2011)
Series D: 30 M$ (2013)

Technologie

*alle Angaben von www.crunchbase.com
Chef & Puppet

since 2005

since 2009

Declarative DSL

Ruby DSL

Customers:

Customers:

• CERN

•

Facebook

• Zynga

•

SAP

•VMware

•

IBM (Smartcloud)

•

Samstag, 16. November 13

AWS (OpsWorks)

• Twitter

cloudbau

•

Microsoft
What is Chef?
Chef is an IT automation platform for developers & systems engineers to continuously
define, build, and manage infrastructure.
CHEF USES:

Recipes and
Cookbooks
that describe and deliver code.
Chef enables people to easily build &
manage complex & dynamic applications
at massive scale.

•  New model for describing infrastructure that
promotes reuse

•  Programmatically provision and configure
•  Reconstruct business from code repository,
data backup, and bare metal resources

cloudbau
Samstag, 16. November 13

“!

”!
cloudbau
Samstag, 16. November 13
cloudbau
Samstag, 16. November 13
What is Puppet?
Puppet is IT automation software that helps system administrators manage infrastructure
throughout its lifecycle, from provisioning and configuration to orchestration and reporting.

How Puppet Works
Puppet uses a declarative, model-based
approach to IT automation.
1. Define the desired state of the
infrastructure's configuration using
Puppet's declarative configuration
language.
2. Simulate configuration changes before
enforcing them.
3. Enforce the deployed desired state
automatically, correcting any configuration
drift.
4. Report on the differences between actual
and desired states and any changes made
enforcing the desired state.

cloudbau

Samstag, 16. November 13
cloudbau
Samstag, 16. November 13
cloudbau
Samstag, 16. November 13
Concepts

cloudbau
Samstag, 16. November 13
Declarative Language

Package A
Template 1

File 1

Package B

cloudbau
Samstag, 16. November 13

Service I
Convergence to desired State

State X

State Y
“run”

Package A

Package A

Template 1

File 1

File 1
Package B

cloudbau
Samstag, 16. November 13

Service I
Idempotence

State Y

State Y
“run”

Package A

Template 1

File 1

Package B

cloudbau
Samstag, 16. November 13

Service I

Package A

Template 1

File 1

Package B

Service I
The Implementations
Puppet and Chef

cloudbau
Samstag, 16. November 13
Terminology

Java

Puppet

Chef

*
.java

manifests *
.pp

recipes *
.rb

package

module

cookbook

cloudbau
Samstag, 16. November 13
Terminology

Java

Puppet

Chef

singleton

parametrized
classes

recipes with
attributes

defines, classes

definitions,
libraries

class
cloudbau
Samstag, 16. November 13
Syntax

Puppet

Chef

Limited DSL (by design)

DSL + full ruby

Extended by Definitions

Custom Resource providers

ERB & Custom Libraries

cloudbau
Samstag, 16. November 13

Definitions, Resource/Providers, Light
Weight Resource
Definitions, Resource/Providers, Light
Weight Resource
Providers, Libraries
Syntax Examples

Puppet

class myapache {
package “apache2”
service “apache2”:
ensure => “running”,
require => Package[“apache2”]
}

cloudbau
Samstag, 16. November 13

Chef

package “apache2”
service “apache2 do
action [:enable , :start]
end
Variables

Puppet

Chef

#special syntax

#native ruby

$bla = “aa”

bla = “aa”

#string interpolation

#string interpolation

$bla_string = “${bla}”

bla_string = #{bla}

cloudbau
Samstag, 16. November 13
Loops

Puppet

Chef

pass array of elements
erb template for loops
create_resource (:type, hash)

cloudbau
Samstag, 16. November 13

do while , repeat etc. available
through native ruby
Conditionals

Puppet (since 2.7)

Chef

if
case (execute block)
selectors (get value)

cloudbau
Samstag, 16. November 13

if, unless, case etc. available
through native ruby
Project Structure

Puppet Modules

Chef
attributes/
definitions/

files/
lib/

files/
libraries/

README.rdoc

metadata.rb

manifests/

providers/

templates/

README.rdoc
recipes/
resources/

cloudbau
Samstag, 16. November 13

templates/
Execution Model

Puppet

Chef

Compile catalog (directed graph model)
Then Execute

Top Down execution

only code for state transition is visited

every line of code is visited all the time

cloudbau
Samstag, 16. November 13
Execution Model

Puppet

Chef

standalone

standalone

client - server

cloudbau
Samstag, 16. November 13

client - server
hosted
System Information

Puppet/Factor => Flat Key/Value

Chef/Ohai => Nested Hash
"hostname": "server-1",
"fqdn": "server-1.example.com",
"domain": "example.com",
"network": {
"interfaces": {
"eth0": {
"type": "eth",
"number": "0",
"encapsulation": "Ethernet",
"addresses": {
"00:0C:29:43:26:C5": {
"family": "lladdr"
},
"192.168.177.138": {
"family": "inet",
"broadcast": "192.168.177.255",
"netmask": "255.255.255.0"
},

cloudbau
Samstag, 16. November 13
System Information

●Selects Package Provider
●Selects Service Provider
●....

cloudbau
Samstag, 16. November 13
System Information (Chef Example)

execute "load sysctl" do
command "/sbin/sysctl -p"
action :nothing
end
bytes = node['memory']['total'].split("kB")[0].to_i * 1024 / 3,
pages = node['memory']['total'].split("kB")[0].to_i * 1024 / 3 / 2048
# adjust shared memory and semaphores
template "/etc/sysctl.conf" do
source "sysctl.conf.erb"
variables(
:shmmax_in_bytes => bytes,
:shmall_in_pages => pages
)
notifies :run, "execute[load sysctl]", :immediately
end

cloudbau
Samstag, 16. November 13
Templates

# Set up application listeners here.

listen application 0.0.0.0:80
balance roundrobin
<% @pool_members.each do |member| -%>
server <%= member[:hostname] %> <%= member[:ipaddress]
weight 1 maxconn 1 check
<% end -%>
<% if node["haproxy"]["enable_admin"] -%>
listen admin 0.0.0.0:22002
mode http
stats uri /
<% end -%>
cloudbau
Samstag, 16. November 13
Lint Tools

http://acrmp.github.io/foodcritic/

http://puppet-lint.com/

cloudbau
Samstag, 16. November 13
Dependency Management
https://github.com/applicationsonline/librarian-chef
http://berkshelf.com

http://librarian-puppet.com

cloudbau
Samstag, 16. November 13
Reports

cloudbau
Samstag, 16. November 13
Web Interface

cloudbau
Samstag, 16. November 13
Tools

●Razor for bare metal provisioning
●mcollectiv for remote execution/
orchestration

●test-kitchen for infrastructure testing
●vagrant for rapid development
● ....

cloudbau
Samstag, 16. November 13
Networking

●Switch Config
●Driven by SDN Adoption
●Lots of Vendors (Arista ..)

cloudbau
Samstag, 16. November 13
Community

cloudbau
Samstag, 16. November 13
Chef Community
Community Overview!
25,000+ Community
Members!
!
1,000+ Community "
Cookbooks!
!
250,000+ Cookbook
Downloads!
!
400+ Public Training
Attendees in the last year!
!
30+ Meetup Groups!
!

cloudbau
Samstag, 16. November 13

Over 200 Corporate Contributors
http://community.opscode.com/cookbooks
approx 1200 co0kbooks
134 maintained by Opscode

cloudbau
Samstag, 16. November 13
http://forge.puppetlabs.com
approx 1700 modules
72 maintained by Puppetlabs

cloudbau
Samstag, 16. November 13
The Price Tag

cloudbau
Samstag, 16. November 13
Puppet: How to Buy
Cumulative # of
Per Node
Per Node Premium
Nodes
Standard Support
Support*
1-10

Download FREE

NA

11-99

$ 99

Contact Sales

100-249

$ 93

Contact Sales

250-499

$ 88

$ 152

500-999

$ 83

$ 119

1000-2499

$ 79

$ 99

2500+

Contact Sales

Contact Sales

cloudbau
Samstag, 16. November 13

https://puppetlabs.com/puppet/how-to-buy
Puppet Enterprise vs. OpenSource

cloudbau
Samstag, 16. November 13
Chef: How to Buy
Free

Launch

Free

$ 120

$ 300

$ 600

Nodes

5

20

50

100

Standard
Support

—

✔

✔

✔

Price per
Month

Standard Premium

http://www.opscode.com/enterprise-chef/#plans-pricing

cloudbau
Samstag, 16. November 13
Enterprise Chef vs. OpenSource Chef

cloudbau
Samstag, 16. November 13
Danke!
cloudbau
Edmund Haselwanter
@ehaselwanter
e.haselwanter@cloudbau.de
+49 30 57701800
www.cloudbau.de

Samstag, 16. November 13

cloudbau GmbH
Körnerstr. 7-10
10785 Berlin
Credits:
• Patrick Debois (http://www.jedi.be/blog/) for some of his Puppet/Chef Slides
• Opscode (http://www.opscode.com) for product info and some slides
• Puppetlabs (http://puppetlabs.com) for product info

Samstag, 16. November 13

Infrastructure as Code with Chef / Puppet