From recipes to
    running systems
managing production servers as you write code
Goals
Centralise configuration changes
Formalise configuration
Minimise efforts of maintaining systems
Minimise risks and fear of changes
Test infrastructure changes
Principles
Version control recipes
Puppet, Chef to code standard recipes
Re-use recipes across environments
Deploy often and in smaller chunks
Test changes on testing environment
Tools
• Git, Github
• Puppet, Chef
• Cucumber-nagios, Cucumber-puppet,
  Rspec-puppet
• Jenkins,Vagrant, virtualisation, linux
  containers
• Puppet dashboard, Foreman
Puppet

• Configuration management utility
• Written in Ruby (^_^)
• Model driven
• Abstract OS layer
• Focus on high level details and relationships
General overview
   Modules               Puppet Master        Node configuration




               General infrastructure
 Monitoring
                        LDAP / Kerberos           SSH gateway
Nagios, Munin



          Applications infrastructure
Development                  Staging               Production
                                              app servers, web servers,
 httpd, mysqld, ...      VMs, httpd, mysqld
                                                    db servers, ...




                      Puppet clients
Anatomy of a
          puppet run
1. Requests a signed certificate (first run only)
2. Collects facts (hardware, OS, hostname, ...)
3. Obtains latest catalog from master
4. Apply changes
5. Sends report of changes and failure to master
Puppet organisation
  External node
                                        Nodes
    classifier


              Top level classes
              app_server, web_server,
                db_server, db_slave,
                  sphinx_server...



Modules      Apache              Ruby           Mysql
             Nagios               Git           Sphinx
             Munin          Memcached             ....
Structure of a module
                             define how the module
manifests/                           works
                                 static content
files/                          notonthehighstreet.cert
                                dynamic templates
templates/                         httpd.conf.erb
                               ruby based libraries
lib/                            package providers, ...

spec/, tests/, features/... tests for classes, definitions
Manifests
a collection of resources & relationships
Dynamic templates
Pre-defined resources
File        Host
Package     Mount
Service     Exec (commands)
User        SSH key
Group       SSH authorised key
Cron(job)
Resource collections
Classes, parameterised classes, defined resource types



  Classes:
• single inheritance
• singleton
• are typically used to group all the resources
  of a particular package, service, ...
Example of a class
Resource collections
Classes, parameterised classes, defined resource types




  Parameterised classes:
• similar to classes
• accept parameters to customise their
  behavior
Example of a parameterised class
Resource collections
Classes, parameterised classes, defined resource types



  Defined Resource Types:
• no inheritance
• reusable on same system
• cannot define same resources more than
  once
Example of a defined resource type
The Foreman

• Provides web interface to Puppet
• Client inventory
• Collects reports and failures
• External node classifier
• Provision new nodes
Web interface of Puppet system
Server and high-level configuration
            inventory
Collect reports from Puppet master
Log and report errors for immediate
              analysis
Provide nodes configuration
     to Puppet master
Provisioning new node from web
Present

• Most of our services managed by Puppet
• Most of our environments managed by
  Puppet
• Basic testing process in place on Jenkins
Future
•   Centralise host provisioning with Foreman
•   Complete testing cycle with production-like
    testing servers
•   Deploy fully working application servers from
    scratch with Puppet
•   Puppet 3.0 / Ruby 1.9
•   Provide development instance configurable
    with Puppet

Puppet

  • 1.
    From recipes to running systems managing production servers as you write code
  • 2.
    Goals Centralise configuration changes Formaliseconfiguration Minimise efforts of maintaining systems Minimise risks and fear of changes Test infrastructure changes
  • 3.
    Principles Version control recipes Puppet,Chef to code standard recipes Re-use recipes across environments Deploy often and in smaller chunks Test changes on testing environment
  • 4.
    Tools • Git, Github •Puppet, Chef • Cucumber-nagios, Cucumber-puppet, Rspec-puppet • Jenkins,Vagrant, virtualisation, linux containers • Puppet dashboard, Foreman
  • 5.
    Puppet • Configuration managementutility • Written in Ruby (^_^) • Model driven • Abstract OS layer • Focus on high level details and relationships
  • 6.
    General overview Modules Puppet Master Node configuration General infrastructure Monitoring LDAP / Kerberos SSH gateway Nagios, Munin Applications infrastructure Development Staging Production app servers, web servers, httpd, mysqld, ... VMs, httpd, mysqld db servers, ... Puppet clients
  • 7.
    Anatomy of a puppet run 1. Requests a signed certificate (first run only) 2. Collects facts (hardware, OS, hostname, ...) 3. Obtains latest catalog from master 4. Apply changes 5. Sends report of changes and failure to master
  • 8.
    Puppet organisation External node Nodes classifier Top level classes app_server, web_server, db_server, db_slave, sphinx_server... Modules Apache Ruby Mysql Nagios Git Sphinx Munin Memcached ....
  • 9.
    Structure of amodule define how the module manifests/ works static content files/ notonthehighstreet.cert dynamic templates templates/ httpd.conf.erb ruby based libraries lib/ package providers, ... spec/, tests/, features/... tests for classes, definitions
  • 10.
    Manifests a collection ofresources & relationships
  • 11.
  • 12.
    Pre-defined resources File Host Package Mount Service Exec (commands) User SSH key Group SSH authorised key Cron(job)
  • 13.
    Resource collections Classes, parameterisedclasses, defined resource types Classes: • single inheritance • singleton • are typically used to group all the resources of a particular package, service, ...
  • 14.
  • 15.
    Resource collections Classes, parameterisedclasses, defined resource types Parameterised classes: • similar to classes • accept parameters to customise their behavior
  • 16.
    Example of aparameterised class
  • 17.
    Resource collections Classes, parameterisedclasses, defined resource types Defined Resource Types: • no inheritance • reusable on same system • cannot define same resources more than once
  • 18.
    Example of adefined resource type
  • 19.
    The Foreman • Providesweb interface to Puppet • Client inventory • Collects reports and failures • External node classifier • Provision new nodes
  • 20.
    Web interface ofPuppet system
  • 21.
    Server and high-levelconfiguration inventory
  • 22.
    Collect reports fromPuppet master
  • 23.
    Log and reporterrors for immediate analysis
  • 24.
  • 25.
  • 26.
    Present • Most ofour services managed by Puppet • Most of our environments managed by Puppet • Basic testing process in place on Jenkins
  • 27.
    Future • Centralise host provisioning with Foreman • Complete testing cycle with production-like testing servers • Deploy fully working application servers from scratch with Puppet • Puppet 3.0 / Ruby 1.9 • Provide development instance configurable with Puppet