Infrastructure as Software - PuppetConf 2014

Infrastructure as Software http://people.v.igoro.us/~dustin/ias-slides/index.h...
INFRASTRUCTURE AASS SSOOFFTTWWAARREE
Dustin J. Mitchell
dustin@mozilla.com
Sept 24, 2014
1 of 37 09/24/2014 03:32 PM

Infrastructure as Software CCOODDEE http://people.v.igoro.us/~dustin/ias-slides/index.h...
2 of 37 09/24/2014 03:32 PM

CCOODDEE == SSNNIIPPPPEETT
var width=150; // width of the eyes in pixels
var colour="#06f"; // colour of the eye - bluey green in this case
var iris="#000"; // colour of the iris (normally black);
/***************************
* Moving Eyeballs Effect *
*(c)2012-3 mf2fm web-design *
* http://www.mf2fm.com/rv *
* DON'T EDIT BELOW THIS BOX *
***************************/
var swide=800;
function addLoadEvent(funky) {
var oldonload=window.onload;
if (typeof(oldonload)!='function') window.onload=funky;
else window.onload=function() {
...
3 of 37 09/24/2014 03:32 PM

CCOODDEE == SSNNIIPPPPEETT
# == Class: baseconfig
##
Performs initial configuration tasks for all Vagrant boxes.
# class baseconfig {
exec { 'apt-get update':
command => '/usr/bin/apt-get update';
}
host { 'hostmachine':
ip => '192.168.0.1';
}
file {
'/home/vagrant/.bashrc':
owner => 'vagrant',
group => 'vagrant',
mode => '0644',
source => 'puppet:///modules/baseconfig/bashrc';
}
}
4 of 37 09/24/2014 03:32 PM

CCOODDEE == SSCCRRIIPPTT
#! /usr/bin/env python
class RequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_POST(self):
content_length = int(self.headers['content-length'])
data = self.rfile.read(content_length)
self.send_response(200)
self.end_headers()
self.wfile.write("accepted.n")
self.wfile.close()
now = int(time.time())
filename = os.path.join(REPORT_DIR, "report-%s.yaml" % (now,))
fd = os.open(filename, os.O_EXCL|os.O_CREAT|os.O_WRONLY)
os.fdopen(fd, "w").write(data)
def main():
logging.basicConfig(format="%(asctime)s %(message)s", level=logging.DEBUG)
SocketServer.TCPServer.allow_reuse_address = True
httpd = SocketServer.TCPServer(("", PORT), RequestHandler)
httpd.serve_forever()
main()
5 of 37 09/24/2014 03:32 PM

“My friend Clift Norris has identified a fundamental constant
that I call Norris’ number, the average amount of code an
untrained programmer can write before he or she hits a wall. Clift
estimates this as 1,500 lines. Beyond that the code becomes so
tangled that the author cannot debug or modify it without
herculean effort.”
John D. Cook
--
6 of 37 09/24/2014 03:32 PM

SSOOFFTTWWAARREE IINN PPUUPPPPEETT??
Let me tell you a story..
7 of 37 09/24/2014 03:32 PM

tinyurl.MMEEEETT PPUUPPPPEETTAAGGAAIINN com/puppetagain
Open Source
Written in Puppet
Cross-Platform
Highly Available
Secure
Designed to Manage Job-Runners
8 of 37 09/24/2014 03:32 PM

SSOOFFTTWWAARREE
Architectural models
9 of 37 09/24/2014 03:32 PM

OPENSTACK Infrastructure as Software AARRCCHHhttIIpTT://pEEeCoCplTTe.vUU.igRRoroEE.us/~dustin/ias-slides/index.h...
10 of 37 09/24/2014 03:32 PM

AAPPPPRROOAACCHHAABBIILLIITTYY
If I change this, what will happen?
Where should I start reading code?
11 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: TTOOPPLLEEVVEELL
class toplevel::base {
include users::root
} class toplevel::server inherits toplevel::base {
include puppet::periodic
include cron
} class toplevel::server::mozpool inherits toplevel::server {
include bmm
include mozpool
}
12 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: TTOOPPLLEEVVEELL
node "mobile-imaging1.p1.releng.scl3.mozilla.com" {
include toplevel::server::mozpool
}
13 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: CCOONNFFIIGGUURRAATTIIOONN
# modules/config/manifests/base.pp
class config::base {
$ntp_server = ''
}#
manifests/moco-config.pp
class config inherits config::base {
$ntp_server = 'time.mozilla.org'
}
14 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: CCOONNFFIIGGUURRAATTIIOONN
class ntp::config {
include ::config
if ($::config::ntp_server) {
..
}
}
15 of 37 09/24/2014 03:32 PM

SSOOFFTTWWAARREE
Controlled Interdependencies
16 of 37 09/24/2014 03:32 PM

Infrastructure as Software AAPPAACCHHEE:: MMOOhDDttUpU://LLpeEEopSSle.v.igoro.us/~dustin/ias-slides/index.h...
17 of 37 09/24/2014 03:32 PM

PPUUPPPPEETTAAGGAAIINN:: DDIIRRSS
class dirs::builds {
file { "/builds": ensure => directory; }
} class dirs::builds::slave {
include dirs::builds
file { "/builds/slave": ensure => directory; }
}
18 of 37 09/24/2014 03:32 PM

PPUUPPPPEETTAAGGAAIINN:: DDIIRRSS
class talos {
include dirs::builds::slave
file {
"/builds/slave/talos-slave":
ensure => directory;
}
}
19 of 37 09/24/2014 03:32 PM

SSOOFFTTWWAARREE
Organizing Principles
20 of 37 09/24/2014 03:32 PM

PA: PRINCIPLE OOFF LLEEAASSTT SSUURRPPRRIISSEE
.. installs mig-agent .. or dies trying
include mig_agent::install
21 of 37 09/24/2014 03:32 PM

PA: PRINCIPLE OOFF LLEEAASSTT SSUURRPPRRIISSEE
class mig_agent::install {
case $operatingsystem {
CentOS: { .. }
default: {
fail("Cannot install on $operatingsystem")
}
}
}
22 of 37 09/24/2014 03:32 PM

SSOOFFTTWWAARREE
Organizing Principles
Abstractions
23 of 37 09/24/2014 03:32 PM

PPUUPPPPEETT:: PPAACCKKAAGGEESS
Write once, run everywhere, right?
package {
'httpd':
ensure => '2.2.15';
}
24 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: PPAACCKKAAGGEESS
include packages::httpd
25 of 37 09/24/2014 03:32 PM

PUPPETAGAIN: PPAACCKKAAGGEESS
class packages::httpd {
case $::operatingsystem {
CentOS: {
package { "httpd": ensure => latest; }
}
Ubuntu: {
package { "apache2": ensure => latest; }
}
Darwin: {
# installed by default
}
default: {
fail("cannot install on $::operatingsystem")
}
}
}
26 of 37 09/24/2014 03:32 PM

WWRRIITTIINNGG SSOOFFTTWWAARREE
IISS HHAARRDD
27 of 37 09/24/2014 03:32 PM

CCHHAALLLLEENNGGEESS
Package repositories are part of the code
28 of 37 09/24/2014 03:32 PM

RREEPPOOSSIITTOORRIIEESS::
PPIINNNNIINNGG IISS HHAARRDD
Un-specified prerequisite packages aren't pinned
Can confuse package managers
Modifying the repo makes production changes
29 of 37 09/24/2014 03:32 PM

MMIIRRRROORRIINNGG IISS HHAARRDD
“Can we update mirrors now?”
“No.”
30 of 37 09/24/2014 03:32 PM

SSHHAARRIINNGG IISS HHAARRDD
31 of 37 09/24/2014 03:32 PM

TTHHEEYY''RREE HHUUGGEE
[root@releng-puppet2.srv.releng.scl3.mozilla.com dmitchell]# df -h /data
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_relengpuppet2-lv_data
414G 315G 78G 81% /data
32 of 37 09/24/2014 03:32 PM

Interactions are hard to model
33 of 37 09/24/2014 03:32 PM

MMOODDUULLEE IINNTTEERRAACCTTIIOONNSS
'httpd' module installs Apache
'rsyslog' module installs rsyslog
When both are installed, we want to send access logs to rsyslog
34 of 37 09/24/2014 03:32 PM

MMOODDUULLEE IINNTTEERRAACCTTIIOONNSS
Httpd has to know about rsyslog?
class httpd::logging {
include rsyslog::config_dir
file {
"${rsyslog::config_dir::dir}/httpd.conf":
content => template("${module_name}/rsyslogd_httpd.conf.erb");
}
}
35 of 37 09/24/2014 03:32 PM

Interactions are hard to model
Acceptance-level testing is hard
36 of 37 09/24/2014 03:32 PM

GGOO FFOORRTTHH AANNDD
WWRRIITTEE SSOOFFTTWWAARREE
37 of 37 09/24/2014 03:32 PM

Infrastructure as Software - PuppetConf 2014

More Related Content

Similar to Infrastructure as Software - PuppetConf 2014

More from Puppet

Recently uploaded

Infrastructure as Software - PuppetConf 2014