Puppet at GitHub

and other adventures in ChatOps
Puppet at GitHub

@wfarr
GitHub Operations
Known Aliases: King of Kebabs
The Computer Guy
Mr. RealTalk
Probably a delinquent

The State of Puppet at GitHub


 The State of Puppet at GitHub
boxen/puppet-*

~1 YEAR OLD

OPEN SOURCED 3 MONTHS AGO

~180 PUBLIC MODULES

2 EXTERNAL MAINTAINERS
@fromonesrc @JHaals

THE STACK


PUPPET 3.2


MASTER-LESS


RUBY 1.8.7


RUN MANUALLY


github/puppet

~5 YEARS OLD

CODE COMMITS

CODE ADDITIONS

CODE DELETIONS

CONTRIBUTIONS (PAST YEAR)
~2k commits
~150 commits

CONTRIBUTIONS (PAST YEAR)
~23% of all commits

AN AVERAGE WEEK
About 50 Pull Requests and 25 Issues comprising
300 commits across 24 authors


PUPPET 2.7.GITHUB


SINGLE PUPPETMASTER


RUN VIA CRON JOB


PUPPETDB


CUSTOM NODE DEFINITIONS


 How GitHub Writes Puppet
PUPPETLABS/STDLIB

PARAMETER VALIDATION

class redis::server(
$data_dir = '/var/lib/redis',
$manage_service = false,
$package = 'redis-server'
) {
validate_bool($manage_service)
validate_absolute_path($data_dir)
validate_re($package, '^redis2?-server$',
"Redis::Server[${name}]: package must be either redis-server or redis2-server: $
{package}")
}

DATA MUNGING

define ruby::version(
$env = {}
) {
$default_environment = {
'CC' => 'clang',
'CFLAGS' => '-O2'
}
$ruby_build_environment = merge($default_environment, $env)
exec { "install ruby version ${name}":
environment => join_keys_to_values($ruby_build_environment, '=')
}
}

RESOURCE HANDLING

class redis::server(
$data_dir = '/var/lib/redis'
) {
if ! defined_with_params(User[redis], { ensure => 'present' }) {
user { 'redis': ensure => 'present', group => 'redis' }
}
}

$latest_tcs_version = "${::ruby::root}/versions/1.9.3-p231-tcs-github-1.0.30"
$tcs_alias = "${::ruby::root}/versions/1.9.3-p231-tcs-github"
$desired_params = {
'ensure' => 'link',
'target' => $latest_tcs_version,
'force' => true
}
File <| title == $tcs_version |> {
ensure => link,
target => $latest_tcs_version,
force => true
}
ensure_resource('file', $tcs_alias, $desired_params)

GITHUB::ROLE::*

NODE CONFIGURATION

node /^github-redisd+[a-z]?-rs1-prd.iad.github.net$/ {
class { 'github::role::redis':
env => 'production',
private_ipv4 => $::ipaddress,
}
}

ROLE CONFIGURATION

class github::role::redis($env, $private_ipv4) {
validate_re($env, '^(vagrant|staging|production)$')
validate_re($private_ipv4, '^d+.d+.d+.d+$')
$monitor = $env ? { 'production' => true, default => false }
class {
'github::core':
monitor => $monitor,
private_address => $private_ipv4 ;
'redis::server':
bind_address => $private_ipv4,
monitor => $monitor ;
}
}


ABSTRACTION
How GitHub Writes Puppet

class github::core($monitor) {
include github::common_packages
include github::staff
class {
'github::ssh':
monitor => $monitor ;
'github::ipv6':
ensure => absent ;
}
}

class github::ipv6($ensure = present) {
if $::lsbdistcodename != 'squeeze' {
file { '/etc/modprobe.d/ipv6':
ensure => $ensure,
mode => '0444',
source => 'puppet:///modules/github//etc/modprobe.d/ipv6',
}
} else {
$value = $ensure ? { present => 0, default => 1 }
sysctl { 'net.ipv6.conf.all.disable_ipv6':
value => $value,
}
}
}


AUGEAS

class redis::server($bind_address, $data_dir, $monitor, $port) {
redis::config {
'dir':
value => $data_dir,
require => File[$data_dir];
'bind':
value => $bind_address;
'port':
value => $port;
'daemonize':
value => 'yes';
}
}

define redis::config($value, $ensure = present) {
validate_re($ensure, '^(present|absent)$')
$changes = $ensure ? { present => "set ${name} ${value}", default => "rm ${name}" }
augeas { "Set Redis config '${name}' to '${value}'":
changes => $changes,
context => '/files/etc/redis/redis.conf',
lens => 'Redis.lns',
incl => '/etc/redis/redis.conf',
require => File['/etc/redis/redis.conf']
}
}

CODE SHARE


LIBRARIAN-PUPPET


HENSON

KEEP IT CLEAN
 How GitHub Deploys Puppet

rodjek/puppet-lint

KEEP IT GREEN

rodjek/rspec-puppet

tmm1/test-queue

KEEP IT LEAN

$ git commit -am "can't lint this"
modules/github/manifests/role/redis.pp: syntax ok
modules/github/manifests/role/redis.pp - WARNING: => is not properly aligned on line 118
1 errors found, aborting commit.

CHATOPS

/puppet env worker
#=>
worker1.rs.github.com: production

/puppet run worker2
#=> Running puppet on worker2.rs.github.com/production

/puppet noop feature_branch worker2
#=> Running puppet on worker2.rs.github.com/feature_branch --noop

/puppet force feature_branch worker2
#=> Running puppet on worker2.rs.github.com/feature_branch

/puppet disable worker2
#=> Disabling puppet on worker2.rs.github.com

/puppet enable worker2
#=> Disabling puppet on worker2.rs.github.com

/puppet last_run worker2

/puppet certs
#=>
"wills-macbook-pro.local" (A4:5B:AC:B9:E1:85:8B:2B:0E:8B:62:F9:03:32:C9:03)

The Future of Puppet at GitHub


boxen/puppet-*
The Future of Puppet at GitHub

SHELL "REMEMBER TO RUN" SUPPORT

You haven't run Boxen in over a week. =(
We really recommend running Boxen regularly. It's way better that way!
Do you want to run boxen now? (y/N) y
# boxen
Updating Boxen.
...

OPT-OUT

touch $HOME/.boxen-never-prompt-for-updates

PUPPETMASTER SUPPORT

OPT-IN ONLY

HIERA SUPPORT

"uncomfortunities"

UBUNTU SUPPORT

UBUNTU PRECISE

github/puppet

RUBY 1.9.3


PUPPET 3.X


GPANEL: ENC


lol censored

MCOLLECTIVE


REPLACING PARTS OF GITHUB/SHELL


STRUCTURED DATA > SED/AWK


HIERA


THANKS
speakerdeck.com/wfarr/
puppet-at-github-
puppetcamp-raleigh-2013

Puppet at GitHub

More Related Content

What's hot

Viewers also liked

Similar to Puppet at GitHub

More from Puppet

Recently uploaded

Puppet at GitHub