SlideShare a Scribd company logo
1 of 29
Download to read offline
Systems Automation


                           With Puppet




Ohad Levy                                Tel Aviv Linux Club
Senior Staff Engineer at                 07/09/2008
Infineon Technologies

                                  
Warning




Many of the slides are copied! :)




                
Typical System Life cycle




 Installation




                     
Typical System Life cycle




                       Initial
 Installation       Configuration




                              
Typical System Life cycle




                                      Fixes
                       Initial      Updates
 Installation       Configuration    Audits




                              
The Challenges
    Keep our systems quot;harmonizedquot;
●



    Know whats going on on each system
●



    Replace a server if it dies or to be able to add another server
●

    that is exactly like it
    Similar Applications, different OS's
●



    Push out changes to all the servers that need a particular
●

    change
    Stop duplicating effort
●



    Go home early
●




                                  
How to solve the problem?

    The Manual way
●


         Log in and do it
     ●


         Thats OK only for a limited set of machines...
     ●



    Install time auto configure 
●


         Kickstart, jumpstart etc, with a post installation scripts
     ●



    But then what?
●


         How to push a change?
     ●



         No History of changes, audit etc...
     ●



    Or....
●

                                     
Puppet Life cycle




                                    Puppet

                                               Fixes
                       Initial               Updates
 Installation       Configuration             Audits




                              
What is Puppet?

    A GPL Open Source Project written in Ruby
●



    A declarative language for expressing system       
●


    configuration
    A Client and server
●



    A library to realize the configuration
●



    Puppet is the abstraction layer between the system 
●


    administrator and the system
    Puppet requires only Ruby and Facter
●



    Client runs every 30 minutes by default
●
                             
Puppet components




             
Puppet Types

A Type is a particular element that Puppet knows how 
 to configure
    Files (content, permissions, ownership)
●



    Packages (ensure installed or absent)
●



    Services (enabled/disabled, running/stopped)
●



    Exec (run commands)
●



    Full List: cron, exec, file, filebucket, group, host, 
●


    interface, k5login, mailalias, maillist, mount, 
    nagios*, package, service, sshkey, tidy, user, 
    yumrepo, zone                
Example: Managing sudoers file

file { “/etc/sudoers”:
    ensure => file,
    owner  => root,
    group  => root,
    mode   => 600,
    source => “puppet://server/files/sudoer”
}



                                
Dependencies
“require” and “before” / “after” settings ensures that types are 
  applied in the correct order


file { “/etc/sudoers”:
    ...
    require => Package[sudo]
}
package { “sudo”:
    ensure => present,
    before => File[“/etc/sudoers”]
}
                                 
Dependencies - continued
    “notify” and “subscribe” settings can trigger cascaded updates
●



    Particularly useful in services, exec
●




file { “/etc/ssh/sshd_conf”:
     ...
     notify => Service[“sshd”]
}


service { “sshd”:
     subscribe => File[“/etc/ssh/sshd_conf”
}
                                     
What is Facter?
     Facter gathers information about the client, which can be 
●


     used as variables within puppet.
      You can add custom facts as needed.
●



  package {quot;sshdquot;:

    ensure   => installed,
    name     => $operatingsystem ? {
                  solaris => quot;IFKLsshquot;,
                  default => quot;openssh­serverquot;
       }
 }                                         
Example Facts
$ sudo facter
                                     lsbdistid => Ubuntu
architecture => amd64
                                     lsbdistrelease => 8.04
domain => sin.infineon.com
                                     macaddress => 00:1c:25:14:26:ab
facterversion => 1.3.8
                                     manufacturer => LENOVO
fqdn => sinn1636.sin.infineon.com
hardwaremodel => x86_64              memorysize => 1.94 GB
hostname => sinn1636                 processorcount => 2
ipaddress => 172.20.88.132
                                     puppetversion => 0.24.4
kernel => Linux
                                     rubysitedir =>
kernelrelease => 2.6.24­16­generic      /usr/local/lib/site_ruby/1.8
lsbdistcodename => hardy             rubyversion => 1.8.6
lsbdistdescription => Ubuntu 8.04
                                      
What is a Class?
    A named collection of type objects
●



    Can include or inherit from other classes
●


class sudo_class {
     include foo_class
     file { “/etc/sudoers”:
         ...
     }
     package{ “sudo”:
         ...
     }
}
                                  
Class inheritance
class afile {
    file { “/tmp/foo”:
        ensure => file
        source => “/src/versionA”
    }
}
class another_file inherits afile {
    File[“/tmp/foo”] {
        source => “/src/versionB”
    }
}                            
What is a Node ?

    A configuration block matching a client
●



    Can contain types, classes
●



    “default” node matches any client without a node 
●


    block


node “ohad.myself” {
    include sudo_class
    include other_class
}

                              
External Node

    Node definitions can be defined outside of puppet ­ 
●


    LDAP, external script
    Ideal for sites with too many nodes to bother pre­
●


    creating




                              
Classes and definitions
    Classes are groups of resources.
●




    Definitions are similar to classes, but they can be instantiated multiple times with different arguments on 
●


    the same node.

class apache2 {

     define simple-vhost ( $admin = quot;webmasterquot;, $aliases, $docroot) {

       file { quot;/etc/apache2/sites-available/$namequot;:

          mode         => quot;644quot;,

          require => [ Package[quot;apache2quot;], Service[quot;apache2quot;] ],

          content => template(quot;apache/vhost.confquot;),

    }}}

node debiantest {

       include apache2
       apache2::simple-vhost { quot;debian.example.comquot;:                             docroot =>
       quot;/var/www/debianquot;}
       apache2::simple-vhost { quot;test.example.comquot;: docroot =>
                                       
       quot;/var/www/testquot;}
vhost.conf template


    Puppet uses Ruby's ERB template system:
●



<VirtualHost *>
           ServerAdmin           <%= admin %>
           DocumentRoot          <%= docroot %>
           ServerName            <%= name %>
<% aliases.each do |al| -%>
           ServerAlias           <%= al %>
<% end -%>
          ErrorLog quot;|/usr/bin/cronolog /var/log/apache/<%=
    name %>/%Y-%m/error-%dquot;
          CustomLog quot;|/usr/bin/cronolog /var/log/apache/<%=
    name %>/%Y-%m/access %dquot; sane
</VirtualHost>                            
Templates output
# more /etc/apache2/sites-available/debian.example.com
<VirtualHost *>
       ServerAdmin     system@example.com
       DocumentRoot    /var/www/debian
       ServerName      debian.example.com
       ServerAlias     debiantest.example.com
       ServerAlias     debian


        ErrorLog quot;|/usr/bin/cronolog
  /var/log/apache/debian.example.com/%Y-%m/error-%dquot;
        CustomLog quot;|/usr/bin/cronolog
  /var/log/apache/debian.example.com/%Y-%m/access-%dquot; sane
</VirtualHost>
                              
OS API - It also works the
other way around:
$ ralsh user levyo
user { 'levyo':
    password => 'absent',
    shell => '/bin/bash',
    ensure => 'present',
    uid => '49960',
    gid => '49960',
    home => '/home/levyo',
    comment => 'Ohad Levy',
    groups => 
    ['adm','dialout','fax','cdrom','floppy','tape','audio','dip','plugdev','scanner','fuse','lp
    admin','admin']
                                                
Getting Started

    Install puppet (yum/apt­get install puppet) or install 
●


    ruby, gem install facter/puppet.
    Setup the puppet server (puppetmaster) – use 
●


    version control!
    Write a manifest for your node.
●



    Start puppet master on the server
●



    Run puppetd on the client
●




                               
Next steps - modules
    Modules allow you to group both the logic and the files for an application together. 
●



    Puppet automatically searches its module path to find modules.
●



    Modules can contain four types of files, each of which must be stored in a separate 
●


    subdirectory:
         Manifests ­ must be stored in manifests/, and if you create manifests/init.pp then 
    ●


        that file will be loaded if you import the module name directly, e.g. import 
        quot;mymodulequot;.
        Templates ­ must be stored in templates/, and the module name must be added to 
    ●


        the template name: template(quot;mymodule/mytemplate.erbquot;)
        Files ­ stored in files/, these are available from the file server under 
    ●


        modules/<module name>/files/<file name>.
        Plugins – additional types, providers or facts.
    ●




                                                 
File server and File Bucket
    Puppet also includes a file server which you can use for transferring files from the 
●


    server to the client.
    If you configure it, puppet can also save a backup of each file that is changed on the 
●


    client to the server.  The backups go in a filebucket and can be retrieved later.




                                              
Some more info

    Puppet uses SSL for all communications, therefor it 
●


    includes a CA, you can automatically sign CSR or 
    use puppetca tool to mange them.
    Storeconfig, option to save machine states(facts, 
●


    configuration runs) and special facts (e.g. SSH keys) 
    in a database.
    Reporting, puppet has a few reporting options, most 
●


    common are emails with changes, RRD files, yaml 
    files and puppetshow web interface.
    Puppet HA, load balancing etc.
●
                              
Conclusions
    We're all stuck on the hamster wheel
●



    Makes easy stuff easy, hard stuff possible
●



    Similar projects
●



         cfengine
     ●


         bcfg2
     ●


    Additional Resources
●


         http://reductivelabs.com/trac/puppet
     ●



         http://reductivelabs.com/trac/puppet/wiki/LanguageTutorial
     ●



         http://reductivelabs.com/trac/puppet/wiki/CompleteConfiguration
     ●



         #puppet on irc.freenode.org
     ●

                                          

More Related Content

What's hot

Automating the Network
Automating the NetworkAutomating the Network
Automating the NetworkPuppet
 
SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...
SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...
SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...SaltStack
 
SaltConf 2014: Safety with powertools
SaltConf 2014: Safety with powertoolsSaltConf 2014: Safety with powertools
SaltConf 2014: Safety with powertoolsThomas Jackson
 
“warpdrive”, making Python web application deployment magically easy.
“warpdrive”, making Python web application deployment magically easy.“warpdrive”, making Python web application deployment magically easy.
“warpdrive”, making Python web application deployment magically easy.Graham Dumpleton
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdminsPuppet
 
Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014Puppet
 
Deploying on the cutting edge
Deploying on the cutting edgeDeploying on the cutting edge
Deploying on the cutting edgeericholscher
 
Building a Cauldron for Chef to Cook In
Building a Cauldron for Chef to Cook InBuilding a Cauldron for Chef to Cook In
Building a Cauldron for Chef to Cook Inasync_io
 
CentOS 6 to CentOS 7 Upgrade Procedure
CentOS 6 to CentOS 7 Upgrade ProcedureCentOS 6 to CentOS 7 Upgrade Procedure
CentOS 6 to CentOS 7 Upgrade ProcedureVEXXHOST Private Cloud
 
Managing Puppet using MCollective
Managing Puppet using MCollectiveManaging Puppet using MCollective
Managing Puppet using MCollectivePuppet
 
Making the most of your Test Suite
Making the most of your Test SuiteMaking the most of your Test Suite
Making the most of your Test Suiteericholscher
 
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014Puppet
 
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014Puppet
 
SaltConf14 - Ben Cane - Using SaltStack in High Availability Environments
SaltConf14 - Ben Cane - Using SaltStack in High Availability EnvironmentsSaltConf14 - Ben Cane - Using SaltStack in High Availability Environments
SaltConf14 - Ben Cane - Using SaltStack in High Availability EnvironmentsSaltStack
 
The Puppet Master on the JVM - PuppetConf 2014
The Puppet Master on the JVM - PuppetConf 2014The Puppet Master on the JVM - PuppetConf 2014
The Puppet Master on the JVM - PuppetConf 2014Puppet
 
Introduction to Ansible (Pycon7 2016)
Introduction to Ansible (Pycon7 2016)Introduction to Ansible (Pycon7 2016)
Introduction to Ansible (Pycon7 2016)Ivan Rossi
 
Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)Soshi Nemoto
 
FullStack London - Cloud Native Node.js
FullStack London - Cloud Native Node.jsFullStack London - Cloud Native Node.js
FullStack London - Cloud Native Node.jsBethany Nicolle Griggs
 

What's hot (20)

Automating the Network
Automating the NetworkAutomating the Network
Automating the Network
 
SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...
SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...
SaltConf14 - Eric johnson, Google - Orchestrating Google Compute Engine with ...
 
vBACD - Introduction to Opscode Chef - 2/29
vBACD - Introduction to Opscode Chef - 2/29vBACD - Introduction to Opscode Chef - 2/29
vBACD - Introduction to Opscode Chef - 2/29
 
SaltConf 2014: Safety with powertools
SaltConf 2014: Safety with powertoolsSaltConf 2014: Safety with powertools
SaltConf 2014: Safety with powertools
 
“warpdrive”, making Python web application deployment magically easy.
“warpdrive”, making Python web application deployment magically easy.“warpdrive”, making Python web application deployment magically easy.
“warpdrive”, making Python web application deployment magically easy.
 
Puppet for SysAdmins
Puppet for SysAdminsPuppet for SysAdmins
Puppet for SysAdmins
 
Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014
 
Deploying on the cutting edge
Deploying on the cutting edgeDeploying on the cutting edge
Deploying on the cutting edge
 
Building a Cauldron for Chef to Cook In
Building a Cauldron for Chef to Cook InBuilding a Cauldron for Chef to Cook In
Building a Cauldron for Chef to Cook In
 
CentOS 6 to CentOS 7 Upgrade Procedure
CentOS 6 to CentOS 7 Upgrade ProcedureCentOS 6 to CentOS 7 Upgrade Procedure
CentOS 6 to CentOS 7 Upgrade Procedure
 
Tp install anything
Tp install anythingTp install anything
Tp install anything
 
Managing Puppet using MCollective
Managing Puppet using MCollectiveManaging Puppet using MCollective
Managing Puppet using MCollective
 
Making the most of your Test Suite
Making the most of your Test SuiteMaking the most of your Test Suite
Making the most of your Test Suite
 
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
Beaker: Automated, Cloud-Based Acceptance Testing - PuppetConf 2014
 
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
Performance Tuning Your Puppet Infrastructure - PuppetConf 2014
 
SaltConf14 - Ben Cane - Using SaltStack in High Availability Environments
SaltConf14 - Ben Cane - Using SaltStack in High Availability EnvironmentsSaltConf14 - Ben Cane - Using SaltStack in High Availability Environments
SaltConf14 - Ben Cane - Using SaltStack in High Availability Environments
 
The Puppet Master on the JVM - PuppetConf 2014
The Puppet Master on the JVM - PuppetConf 2014The Puppet Master on the JVM - PuppetConf 2014
The Puppet Master on the JVM - PuppetConf 2014
 
Introduction to Ansible (Pycon7 2016)
Introduction to Ansible (Pycon7 2016)Introduction to Ansible (Pycon7 2016)
Introduction to Ansible (Pycon7 2016)
 
Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)Fabric workshop(1) - (MOSG)
Fabric workshop(1) - (MOSG)
 
FullStack London - Cloud Native Node.js
FullStack London - Cloud Native Node.jsFullStack London - Cloud Native Node.js
FullStack London - Cloud Native Node.js
 

Similar to Systems Automation with Puppet

A Presentation about Puppet that I've made at the OSSPAC conference
A Presentation about Puppet that I've made at the OSSPAC conferenceA Presentation about Puppet that I've made at the OSSPAC conference
A Presentation about Puppet that I've made at the OSSPAC conferenceohadlevy
 
Deploy Rails Application by Capistrano
Deploy Rails Application by CapistranoDeploy Rails Application by Capistrano
Deploy Rails Application by CapistranoTasawr Interactive
 
Adventures in infrastructure as code
Adventures in infrastructure as codeAdventures in infrastructure as code
Adventures in infrastructure as codeJulian Simpson
 
Apache and PHP: Why httpd.conf is your new BFF!
Apache and PHP: Why httpd.conf is your new BFF!Apache and PHP: Why httpd.conf is your new BFF!
Apache and PHP: Why httpd.conf is your new BFF!Jeff Jones
 
Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013grim_radical
 
Turbogears Presentation
Turbogears PresentationTurbogears Presentation
Turbogears Presentationdidip
 
L.A.M.P Installation Note --- CentOS 6.5
L.A.M.P Installation Note --- CentOS 6.5L.A.M.P Installation Note --- CentOS 6.5
L.A.M.P Installation Note --- CentOS 6.5William Lee
 
JUDCon London 2011 - Bin packing with drools planner by example
JUDCon London 2011 - Bin packing with drools planner by exampleJUDCon London 2011 - Bin packing with drools planner by example
JUDCon London 2011 - Bin packing with drools planner by exampleGeoffrey De Smet
 
yusukebe in Yokohama.pm 090909
yusukebe in Yokohama.pm 090909yusukebe in Yokohama.pm 090909
yusukebe in Yokohama.pm 090909Yusuke Wada
 
Harmonious Development: Via Vagrant and Puppet
Harmonious Development: Via Vagrant and PuppetHarmonious Development: Via Vagrant and Puppet
Harmonious Development: Via Vagrant and PuppetAchieve Internet
 
Making Spinnaker Go @ Stitch Fix
Making Spinnaker Go @ Stitch FixMaking Spinnaker Go @ Stitch Fix
Making Spinnaker Go @ Stitch FixDiana Tkachenko
 
Python Deployment with Fabric
Python Deployment with FabricPython Deployment with Fabric
Python Deployment with Fabricandymccurdy
 

Similar to Systems Automation with Puppet (20)

A Presentation about Puppet that I've made at the OSSPAC conference
A Presentation about Puppet that I've made at the OSSPAC conferenceA Presentation about Puppet that I've made at the OSSPAC conference
A Presentation about Puppet that I've made at the OSSPAC conference
 
Deploy Rails Application by Capistrano
Deploy Rails Application by CapistranoDeploy Rails Application by Capistrano
Deploy Rails Application by Capistrano
 
Sinatra
SinatraSinatra
Sinatra
 
Adventures in infrastructure as code
Adventures in infrastructure as codeAdventures in infrastructure as code
Adventures in infrastructure as code
 
Apache and PHP: Why httpd.conf is your new BFF!
Apache and PHP: Why httpd.conf is your new BFF!Apache and PHP: Why httpd.conf is your new BFF!
Apache and PHP: Why httpd.conf is your new BFF!
 
Os Wilhelm
Os WilhelmOs Wilhelm
Os Wilhelm
 
Cooking with Chef
Cooking with ChefCooking with Chef
Cooking with Chef
 
Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013Puppet: Eclipsecon ALM 2013
Puppet: Eclipsecon ALM 2013
 
Capistrano
CapistranoCapistrano
Capistrano
 
Turbogears Presentation
Turbogears PresentationTurbogears Presentation
Turbogears Presentation
 
L.A.M.P Installation Note --- CentOS 6.5
L.A.M.P Installation Note --- CentOS 6.5L.A.M.P Installation Note --- CentOS 6.5
L.A.M.P Installation Note --- CentOS 6.5
 
JUDCon London 2011 - Bin packing with drools planner by example
JUDCon London 2011 - Bin packing with drools planner by exampleJUDCon London 2011 - Bin packing with drools planner by example
JUDCon London 2011 - Bin packing with drools planner by example
 
Logstash
LogstashLogstash
Logstash
 
yusukebe in Yokohama.pm 090909
yusukebe in Yokohama.pm 090909yusukebe in Yokohama.pm 090909
yusukebe in Yokohama.pm 090909
 
Cooking with Chef
Cooking with ChefCooking with Chef
Cooking with Chef
 
T5 Oli Aro
T5 Oli AroT5 Oli Aro
T5 Oli Aro
 
Harmonious Development: Via Vagrant and Puppet
Harmonious Development: Via Vagrant and PuppetHarmonious Development: Via Vagrant and Puppet
Harmonious Development: Via Vagrant and Puppet
 
Making Spinnaker Go @ Stitch Fix
Making Spinnaker Go @ Stitch FixMaking Spinnaker Go @ Stitch Fix
Making Spinnaker Go @ Stitch Fix
 
Python Deployment with Fabric
Python Deployment with FabricPython Deployment with Fabric
Python Deployment with Fabric
 
Capistrano
CapistranoCapistrano
Capistrano
 

More from elliando dias

Clojurescript slides
Clojurescript slidesClojurescript slides
Clojurescript slideselliando dias
 
Why you should be excited about ClojureScript
Why you should be excited about ClojureScriptWhy you should be excited about ClojureScript
Why you should be excited about ClojureScriptelliando dias
 
Functional Programming with Immutable Data Structures
Functional Programming with Immutable Data StructuresFunctional Programming with Immutable Data Structures
Functional Programming with Immutable Data Structureselliando dias
 
Nomenclatura e peças de container
Nomenclatura  e peças de containerNomenclatura  e peças de container
Nomenclatura e peças de containerelliando dias
 
Polyglot and Poly-paradigm Programming for Better Agility
Polyglot and Poly-paradigm Programming for Better AgilityPolyglot and Poly-paradigm Programming for Better Agility
Polyglot and Poly-paradigm Programming for Better Agilityelliando dias
 
Javascript Libraries
Javascript LibrariesJavascript Libraries
Javascript Librarieselliando dias
 
How to Make an Eight Bit Computer and Save the World!
How to Make an Eight Bit Computer and Save the World!How to Make an Eight Bit Computer and Save the World!
How to Make an Eight Bit Computer and Save the World!elliando dias
 
A Practical Guide to Connecting Hardware to the Web
A Practical Guide to Connecting Hardware to the WebA Practical Guide to Connecting Hardware to the Web
A Practical Guide to Connecting Hardware to the Webelliando dias
 
Introdução ao Arduino
Introdução ao ArduinoIntrodução ao Arduino
Introdução ao Arduinoelliando dias
 
Incanter Data Sorcery
Incanter Data SorceryIncanter Data Sorcery
Incanter Data Sorceryelliando dias
 
Fab.in.a.box - Fab Academy: Machine Design
Fab.in.a.box - Fab Academy: Machine DesignFab.in.a.box - Fab Academy: Machine Design
Fab.in.a.box - Fab Academy: Machine Designelliando dias
 
The Digital Revolution: Machines that makes
The Digital Revolution: Machines that makesThe Digital Revolution: Machines that makes
The Digital Revolution: Machines that makeselliando dias
 
Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.elliando dias
 
Hadoop and Hive Development at Facebook
Hadoop and Hive Development at FacebookHadoop and Hive Development at Facebook
Hadoop and Hive Development at Facebookelliando dias
 
Multi-core Parallelization in Clojure - a Case Study
Multi-core Parallelization in Clojure - a Case StudyMulti-core Parallelization in Clojure - a Case Study
Multi-core Parallelization in Clojure - a Case Studyelliando dias
 

More from elliando dias (20)

Clojurescript slides
Clojurescript slidesClojurescript slides
Clojurescript slides
 
Why you should be excited about ClojureScript
Why you should be excited about ClojureScriptWhy you should be excited about ClojureScript
Why you should be excited about ClojureScript
 
Functional Programming with Immutable Data Structures
Functional Programming with Immutable Data StructuresFunctional Programming with Immutable Data Structures
Functional Programming with Immutable Data Structures
 
Nomenclatura e peças de container
Nomenclatura  e peças de containerNomenclatura  e peças de container
Nomenclatura e peças de container
 
Geometria Projetiva
Geometria ProjetivaGeometria Projetiva
Geometria Projetiva
 
Polyglot and Poly-paradigm Programming for Better Agility
Polyglot and Poly-paradigm Programming for Better AgilityPolyglot and Poly-paradigm Programming for Better Agility
Polyglot and Poly-paradigm Programming for Better Agility
 
Javascript Libraries
Javascript LibrariesJavascript Libraries
Javascript Libraries
 
How to Make an Eight Bit Computer and Save the World!
How to Make an Eight Bit Computer and Save the World!How to Make an Eight Bit Computer and Save the World!
How to Make an Eight Bit Computer and Save the World!
 
Ragel talk
Ragel talkRagel talk
Ragel talk
 
A Practical Guide to Connecting Hardware to the Web
A Practical Guide to Connecting Hardware to the WebA Practical Guide to Connecting Hardware to the Web
A Practical Guide to Connecting Hardware to the Web
 
Introdução ao Arduino
Introdução ao ArduinoIntrodução ao Arduino
Introdução ao Arduino
 
Minicurso arduino
Minicurso arduinoMinicurso arduino
Minicurso arduino
 
Incanter Data Sorcery
Incanter Data SorceryIncanter Data Sorcery
Incanter Data Sorcery
 
Rango
RangoRango
Rango
 
Fab.in.a.box - Fab Academy: Machine Design
Fab.in.a.box - Fab Academy: Machine DesignFab.in.a.box - Fab Academy: Machine Design
Fab.in.a.box - Fab Academy: Machine Design
 
The Digital Revolution: Machines that makes
The Digital Revolution: Machines that makesThe Digital Revolution: Machines that makes
The Digital Revolution: Machines that makes
 
Hadoop + Clojure
Hadoop + ClojureHadoop + Clojure
Hadoop + Clojure
 
Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.Hadoop - Simple. Scalable.
Hadoop - Simple. Scalable.
 
Hadoop and Hive Development at Facebook
Hadoop and Hive Development at FacebookHadoop and Hive Development at Facebook
Hadoop and Hive Development at Facebook
 
Multi-core Parallelization in Clojure - a Case Study
Multi-core Parallelization in Clojure - a Case StudyMulti-core Parallelization in Clojure - a Case Study
Multi-core Parallelization in Clojure - a Case Study
 

Recently uploaded

SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentationphoebematthew05
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 

Recently uploaded (20)

SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
costume and set research powerpoint presentation
costume and set research powerpoint presentationcostume and set research powerpoint presentation
costume and set research powerpoint presentation
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 

Systems Automation with Puppet

  • 1. Systems Automation With Puppet Ohad Levy Tel Aviv Linux Club Senior Staff Engineer at 07/09/2008 Infineon Technologies    
  • 2. Warning Many of the slides are copied! :)    
  • 3. Typical System Life cycle Installation    
  • 4. Typical System Life cycle Initial Installation Configuration    
  • 5. Typical System Life cycle Fixes Initial Updates Installation Configuration Audits    
  • 6. The Challenges Keep our systems quot;harmonizedquot; ● Know whats going on on each system ● Replace a server if it dies or to be able to add another server ● that is exactly like it Similar Applications, different OS's ● Push out changes to all the servers that need a particular ● change Stop duplicating effort ● Go home early ●    
  • 7. How to solve the problem? The Manual way ● Log in and do it ● Thats OK only for a limited set of machines... ● Install time auto configure  ● Kickstart, jumpstart etc, with a post installation scripts ● But then what? ● How to push a change? ● No History of changes, audit etc... ● Or.... ●    
  • 8. Puppet Life cycle Puppet Fixes Initial Updates Installation Configuration Audits    
  • 9. What is Puppet? A GPL Open Source Project written in Ruby ● A declarative language for expressing system        ● configuration A Client and server ● A library to realize the configuration ● Puppet is the abstraction layer between the system  ● administrator and the system Puppet requires only Ruby and Facter ● Client runs every 30 minutes by default ●    
  • 11. Puppet Types A Type is a particular element that Puppet knows how  to configure Files (content, permissions, ownership) ● Packages (ensure installed or absent) ● Services (enabled/disabled, running/stopped) ● Exec (run commands) ● Full List: cron, exec, file, filebucket, group, host,  ● interface, k5login, mailalias, maillist, mount,  nagios*, package, service, sshkey, tidy, user,  yumrepo, zone   
  • 12. Example: Managing sudoers file file { “/etc/sudoers”: ensure => file, owner  => root, group  => root, mode   => 600, source => “puppet://server/files/sudoer” }    
  • 13. Dependencies “require” and “before” / “after” settings ensures that types are  applied in the correct order file { “/etc/sudoers”: ... require => Package[sudo] } package { “sudo”: ensure => present, before => File[“/etc/sudoers”] }    
  • 14. Dependencies - continued “notify” and “subscribe” settings can trigger cascaded updates ● Particularly useful in services, exec ● file { “/etc/ssh/sshd_conf”: ... notify => Service[“sshd”] } service { “sshd”: subscribe => File[“/etc/ssh/sshd_conf” }    
  • 15. What is Facter? Facter gathers information about the client, which can be  ● used as variables within puppet.  You can add custom facts as needed. ●   package {quot;sshdquot;:     ensure   => installed,     name     => $operatingsystem ? {                   solaris => quot;IFKLsshquot;,                   default => quot;openssh­serverquot;        }  }    
  • 16. Example Facts $ sudo facter lsbdistid => Ubuntu architecture => amd64 lsbdistrelease => 8.04 domain => sin.infineon.com macaddress => 00:1c:25:14:26:ab facterversion => 1.3.8 manufacturer => LENOVO fqdn => sinn1636.sin.infineon.com hardwaremodel => x86_64 memorysize => 1.94 GB hostname => sinn1636 processorcount => 2 ipaddress => 172.20.88.132 puppetversion => 0.24.4 kernel => Linux rubysitedir => kernelrelease => 2.6.24­16­generic /usr/local/lib/site_ruby/1.8 lsbdistcodename => hardy rubyversion => 1.8.6 lsbdistdescription => Ubuntu 8.04    
  • 17. What is a Class? A named collection of type objects ● Can include or inherit from other classes ● class sudo_class { include foo_class file { “/etc/sudoers”: ... } package{ “sudo”: ... } }    
  • 18. Class inheritance class afile { file { “/tmp/foo”: ensure => file source => “/src/versionA” } } class another_file inherits afile { File[“/tmp/foo”] { source => “/src/versionB” } }    
  • 19. What is a Node ? A configuration block matching a client ● Can contain types, classes ● “default” node matches any client without a node  ● block node “ohad.myself” { include sudo_class include other_class }    
  • 20. External Node Node definitions can be defined outside of puppet ­  ● LDAP, external script Ideal for sites with too many nodes to bother pre­ ● creating    
  • 21. Classes and definitions Classes are groups of resources. ● Definitions are similar to classes, but they can be instantiated multiple times with different arguments on  ● the same node. class apache2 { define simple-vhost ( $admin = quot;webmasterquot;, $aliases, $docroot) { file { quot;/etc/apache2/sites-available/$namequot;: mode => quot;644quot;, require => [ Package[quot;apache2quot;], Service[quot;apache2quot;] ], content => template(quot;apache/vhost.confquot;), }}} node debiantest { include apache2 apache2::simple-vhost { quot;debian.example.comquot;: docroot => quot;/var/www/debianquot;} apache2::simple-vhost { quot;test.example.comquot;: docroot =>     quot;/var/www/testquot;}
  • 22. vhost.conf template Puppet uses Ruby's ERB template system: ● <VirtualHost *> ServerAdmin <%= admin %> DocumentRoot <%= docroot %> ServerName <%= name %> <% aliases.each do |al| -%> ServerAlias <%= al %> <% end -%> ErrorLog quot;|/usr/bin/cronolog /var/log/apache/<%= name %>/%Y-%m/error-%dquot; CustomLog quot;|/usr/bin/cronolog /var/log/apache/<%= name %>/%Y-%m/access %dquot; sane </VirtualHost>    
  • 23. Templates output # more /etc/apache2/sites-available/debian.example.com <VirtualHost *> ServerAdmin system@example.com DocumentRoot /var/www/debian ServerName debian.example.com ServerAlias debiantest.example.com ServerAlias debian ErrorLog quot;|/usr/bin/cronolog /var/log/apache/debian.example.com/%Y-%m/error-%dquot; CustomLog quot;|/usr/bin/cronolog /var/log/apache/debian.example.com/%Y-%m/access-%dquot; sane </VirtualHost>    
  • 24. OS API - It also works the other way around: $ ralsh user levyo user { 'levyo':     password => 'absent',     shell => '/bin/bash',     ensure => 'present',     uid => '49960',     gid => '49960',     home => '/home/levyo',     comment => 'Ohad Levy',     groups =>  ['adm','dialout','fax','cdrom','floppy','tape','audio','dip','plugdev','scanner','fuse','lp admin','admin']    
  • 25. Getting Started Install puppet (yum/apt­get install puppet) or install  ● ruby, gem install facter/puppet. Setup the puppet server (puppetmaster) – use  ● version control! Write a manifest for your node. ● Start puppet master on the server ● Run puppetd on the client ●    
  • 26. Next steps - modules Modules allow you to group both the logic and the files for an application together.  ● Puppet automatically searches its module path to find modules. ● Modules can contain four types of files, each of which must be stored in a separate  ● subdirectory:  Manifests ­ must be stored in manifests/, and if you create manifests/init.pp then  ● that file will be loaded if you import the module name directly, e.g. import  quot;mymodulequot;. Templates ­ must be stored in templates/, and the module name must be added to  ● the template name: template(quot;mymodule/mytemplate.erbquot;) Files ­ stored in files/, these are available from the file server under  ● modules/<module name>/files/<file name>. Plugins – additional types, providers or facts. ●    
  • 27. File server and File Bucket Puppet also includes a file server which you can use for transferring files from the  ● server to the client. If you configure it, puppet can also save a backup of each file that is changed on the  ● client to the server.  The backups go in a filebucket and can be retrieved later.    
  • 28. Some more info Puppet uses SSL for all communications, therefor it  ● includes a CA, you can automatically sign CSR or  use puppetca tool to mange them. Storeconfig, option to save machine states(facts,  ● configuration runs) and special facts (e.g. SSH keys)  in a database. Reporting, puppet has a few reporting options, most  ● common are emails with changes, RRD files, yaml  files and puppetshow web interface. Puppet HA, load balancing etc. ●    
  • 29. Conclusions We're all stuck on the hamster wheel ● Makes easy stuff easy, hard stuff possible ● Similar projects ● cfengine ● bcfg2 ● Additional Resources ● http://reductivelabs.com/trac/puppet ● http://reductivelabs.com/trac/puppet/wiki/LanguageTutorial ● http://reductivelabs.com/trac/puppet/wiki/CompleteConfiguration ● #puppet on irc.freenode.org ●