Being a Puppet Master: Automating Amazon EC2 with Puppet & Friends
Upcoming SlideShare
Loading in...5
×
 

Being a Puppet Master: Automating Amazon EC2 with Puppet & Friends

on

  • 20,462 views

 

Statistics

Views

Total Views
20,462
Views on SlideShare
20,245
Embed Views
217

Actions

Likes
59
Downloads
705
Comments
0

14 Embeds 217

http://kurapov.name 133
http://localhost 19
https://twitter.com 14
http://tweetedtimes.com 12
http://twitter.com 10
http://confluence.matomyrnd.com 8
http://www.techgig.com 5
http://us-w1.rockmelt.com 4
http://a0.twimg.com 3
http://trunk.ly 3
http://col.xtend.int 2
https://si0.twimg.com 2
http://mstanislav.posterous.com 1
https://twimg0-a.akamaihd.net 1
More...

Accessibility

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Being a Puppet Master: Automating Amazon EC2 with Puppet & Friends Being a Puppet Master: Automating Amazon EC2 with Puppet & Friends Presentation Transcript

  • Being a Puppet MasterAutomating Amazon EC2 with Puppet & Friends Mark Stanislav <mark.stanislav@gmail.com>
  • Puppet: A Quick Overview
  • Puppet: A Quick OverviewStop administrating your environment and start developing it...
  • Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurations
  • Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.
  • Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.Supports Linux, Solaris, BSD, OS X; Windows in process!
  • Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.Supports Linux, Solaris, BSD, OS X; Windows in process!Project ran by Luke Kanies; Founder/CEO of Puppet Labs $5M Series B in July 2010; ~$7M total funding
  • Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.Supports Linux, Solaris, BSD, OS X; Windows in process!Project ran by Luke Kanies; Founder/CEO of Puppet Labs $5M Series B in July 2010; ~$7M total fundingCFEngine & Chef are similar projects; both are quality, too.
  • Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.Supports Linux, Solaris, BSD, OS X; Windows in process!Project ran by Luke Kanies; Founder/CEO of Puppet Labs $5M Series B in July 2010; ~$7M total fundingCFEngine & Chef are similar projects; both are quality, too.Sun, Stanford, Match.com, Media Temple, & Digg all use it!
  • High-Level Puppet Overview
  • High-Level Puppet Overview Modules Puppet Master Configuration
  • High-Level Puppet Overview Modules Puppet Master Configuration Puppet Clients
  • High-Level Puppet Overview Modules Puppet Master Configuration General Cloud InfrastructureMonitoring DNS Syslog LDAP Nagios/Munin BIND Nameserver rsyslog Server OpenLDAP Server Puppet Clients
  • High-Level Puppet Overview Modules Puppet Master Configuration General Cloud Infrastructure Monitoring DNS Syslog LDAP Nagios/Munin BIND Nameserver rsyslog Server OpenLDAP Server Software Development EnvironmentsDevelopment Testing Review ProductionApache, Tomcat, Passenger Apache, Tomcat, Passenger Apache, Tomcat, Passenger Apache, Tomcat, Passenger Puppet Clients
  • Puppet Network Overview
  • Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet Clients
  • Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet ClientsConfiguration allows for manual synchronizations or a set increment
  • Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet ClientsConfiguration allows for manual synchronizations or a set incrementClient or server initiated synchronizations
  • Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet ClientsConfiguration allows for manual synchronizations or a set incrementClient or server initiated synchronizationsClient/Server configuration leverages a Certificate Authority (CA) on thePuppet Master to sign client certificates to verify authenticity
  • Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet ClientsConfiguration allows for manual synchronizations or a set incrementClient or server initiated synchronizationsClient/Server configuration leverages a Certificate Authority (CA) on thePuppet Master to sign client certificates to verify authenticityTransmissions of all data between a master & client are encrypted
  • Why EC2 IaaS is Tiring... =
  • Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexible =
  • Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexibleBuilding and deploying anew AMI is time consuming =
  • Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexibleBuilding and deploying anew AMI is time consuming“What do you mean you =want to update a file? Wecan’t just do that...”
  • Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexibleBuilding and deploying anew AMI is time consuming“What do you mean you =want to update a file? Wecan’t just do that...”Auto-scaling is fantastic butmanaging the scaling hostsis not
  • Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexibleBuilding and deploying anew AMI is time consuming“What do you mean you =want to update a file? Wecan’t just do that...”Auto-scaling is fantastic butmanaging the scaling hostsis notTime to deploy & configureoffsets benefits of IaaS
  • Puppet is an EC2 Superhero
  • Puppet is an EC2 SuperheroDeployment of a “base” EC2AMI - just what you alwaysneed on any standard image
  • Puppet is an EC2 SuperheroDeployment of a “base” EC2AMI - just what you alwaysneed on any standard imageLeverage EC2 securitygroups to give context to anew instance
  • Puppet is an EC2 SuperheroDeployment of a “base” EC2AMI - just what you alwaysneed on any standard imageLeverage EC2 securitygroups to give context to anew instancePuppet knows what youwant out of the box;configure a new instancewithout interaction
  • Puppet is an EC2 SuperheroDeployment of a “base” EC2AMI - just what you alwaysneed on any standard imageLeverage EC2 securitygroups to give context to anew instancePuppet knows what youwant out of the box;configure a new instancewithout interactionUpdate a package orconfiguration file at anytime
  • New EC2 Puppet Client Flow
  • New EC2 Puppet Client FlowEC2 Instance WithPuppet Spawned
  • New EC2 Puppet Client FlowEC2 Instance With Puppet ServicePuppet Spawned Starts For First Time
  • New EC2 Puppet Client FlowEC2 Instance With Puppet Service Client GeneratesPuppet Spawned Starts For First Time SSL Certificate
  • New EC2 Puppet Client FlowEC2 Instance With Puppet Service Client GeneratesPuppet Spawned Starts For First Time SSL Certificate Client Sends SSL Certificate to Master
  • New EC2 Puppet Client FlowEC2 Instance With Puppet Service Client GeneratesPuppet Spawned Starts For First Time SSL Certificate Master Signs SSL Client Sends SSL Certificate Certificate to Master
  • New EC2 Puppet Client FlowEC2 Instance With Puppet Service Client GeneratesPuppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to Master
  • New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates:
  • New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates: Puppet Master can allow certain domain scopes (*.example.com) to be auto-signed when asked by a valid hostname
  • New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates: Puppet Master can allow certain domain scopes (*.example.com) to be auto-signed when asked by a valid hostname Create a crontab script that executes every minute looking for new SSL certificates in a certain directory and signs them
  • New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates: Puppet Master can allow certain domain scopes (*.example.com) to be auto-signed when asked by a valid hostname Create a crontab script that executes every minute looking for new SSL certificates in a certain directory and signs them Auto-sign everything you are asked to sign without question
  • New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates: Puppet Master can allow certain domain scopes (*.example.com) to be auto-signed when asked by a valid hostname Create a crontab script that executes every minute looking for new SSL certificates in a certain directory and signs them Auto-sign everything you are asked to sign without question Manually sign each certificate when you add a new Puppet Client
  • Puppet Module Structure
  • Puppet Module Structure Module Folder
  • Puppet Module Structure Module Foldermanifests/ Tell the module how to work
  • Puppet Module Structure Module Foldermanifests/ Tell the module how to work files/ Static files needed for deployment
  • Puppet Module Structure Module Foldermanifests/ Tell the module how to work files/ Static files needed for deploymenttemplates/ Dynamic Ruby-based templates
  • Puppet Module Structure Module Foldermanifests/ Tell the module how to work files/ Static files needed for deploymenttemplates/ Dynamic Ruby-based templates lib/ Relevant Ruby-based libraries
  • A Partial List of Puppet ‘types’
  • A Partial List of Puppet ‘types’Files & Directories
  • A Partial List of Puppet ‘types’Files & DirectoriesUsers & Groups
  • A Partial List of Puppet ‘types’Files & DirectoriesUsers & GroupsServices
  • A Partial List of Puppet ‘types’Files & DirectoriesUsers & GroupsServicesPackages
  • A Partial List of Puppet ‘types’Files & Directories CrontabsUsers & GroupsServicesPackages
  • A Partial List of Puppet ‘types’Files & Directories CrontabsUsers & Groups /etc/hostsServicesPackages
  • A Partial List of Puppet ‘types’Files & Directories CrontabsUsers & Groups /etc/hostsServices Mail AliasesPackages
  • A Partial List of Puppet ‘types’Files & Directories CrontabsUsers & Groups /etc/hostsServices Mail AliasesPackages Mount Points
  • A Partial List of Puppet ‘types’Files & Directories Crontabs NagiosUsers & Groups /etc/hostsServices Mail AliasesPackages Mount Points
  • A Partial List of Puppet ‘types’Files & Directories Crontabs NagiosUsers & Groups /etc/hosts SELinuxServices Mail AliasesPackages Mount Points
  • A Partial List of Puppet ‘types’Files & Directories Crontabs NagiosUsers & Groups /etc/hosts SELinuxServices Mail Aliases SSH KeysPackages Mount Points
  • A Partial List of Puppet ‘types’Files & Directories Crontabs NagiosUsers & Groups /etc/hosts SELinuxServices Mail Aliases SSH KeysPackages Mount Points Yum Repos
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Supports 23 different package providers
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Supports 23 different package providers Abstracted for your OS automatically
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Supports 23 different package providers Abstracted for your OS automatically Specify ‘installed’, ‘absent’, or ‘latest’ for desired state
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Supports 23 different package providers Abstracted for your OS automatically Specify ‘installed’, ‘absent’, or ‘latest’ for desired state Change from ‘installed’ to ‘latest’ and deploy for quick
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Supports 23 different Supports 10 different package providers ‘init’ frameworks Abstracted for your OS automatically Specify ‘installed’, ‘absent’, or ‘latest’ for desired state Change from ‘installed’ to ‘latest’ and deploy for quick
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Supports 23 different Supports 10 different package providers ‘init’ frameworks Abstracted for your Control whether a OS automatically service starts on boot or is required to Specify ‘installed’, be running always ‘absent’, or ‘latest’ for desired state Change from ‘installed’ to ‘latest’ and deploy for quick
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Supports 23 different Supports 10 different package providers ‘init’ frameworks Abstracted for your Control whether a OS automatically service starts on boot or is required to Specify ‘installed’, be running always ‘absent’, or ‘latest’ for desired state A service can be notified to restart if a Change from configuration file has ‘installed’ to ‘latest’ and deploy for quick
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a OS automatically service starts on boot or is required to Specify ‘installed’, be running always ‘absent’, or ‘latest’ for desired state A service can be notified to restart if a Change from configuration file has ‘installed’ to ‘latest’ and deploy for quick
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a Load content from OS automatically service starts on ‘files/’, ‘templates/’ boot or is required to or custom strings Specify ‘installed’, be running always ‘absent’, or ‘latest’ for desired state A service can be notified to restart if a Change from configuration file has ‘installed’ to ‘latest’ and deploy for quick
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a Load content from OS automatically service starts on ‘files/’, ‘templates/’ boot or is required to or custom strings Specify ‘installed’, be running always ‘absent’, or ‘latest’ Create symlinks for desired state A service can be notified to restart if a Change from configuration file has ‘installed’ to ‘latest’ and deploy for quick
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a Load content from OS automatically service starts on ‘files/’, ‘templates/’ boot or is required to or custom strings Specify ‘installed’, be running always ‘absent’, or ‘latest’ Create symlinks for desired state A service can be notified to restart if a Supports 5 types to Change from configuration file has verify a file checksum ‘installed’ to ‘latest’ and deploy for quick
  • A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a Load content from OS automatically service starts on ‘files/’, ‘templates/’ boot or is required to or custom strings Specify ‘installed’, be running always ‘absent’, or ‘latest’ Create symlinks for desired state A service can be notified to restart if a Supports 5 types to Change from configuration file has verify a file checksum ‘installed’ to ‘latest’ and deploy for quick Purge a directory of
  • General Puppet Syntax
  • General Puppet SyntaxClass Configuration: Single Class: class ntp { ... } Inherited Class: class sftp inherits ssh { ... } Nested Class: class foo { class bar { ... } } Scoped Class: class ntp::base { ... }
  • General Puppet SyntaxClass Configuration: Single Class: class ntp { ... } Inherited Class: class sftp inherits ssh { ... } Nested Class: class foo { class bar { ... } } Scoped Class: class ntp::base { ... }Selectors: $admin = $user_id ? { ‘0’ => ‘root’, }
  • General Puppet SyntaxClass Configuration: If-Else Conditionals: Single Class: if ($ec2_security_groups == ‘DNS’) { class ntp { ... } include bind::server Inherited Class: } else { include bind::client class sftp inherits ssh { ... } } Nested Class: class foo { class bar { ... } } Scoped Class: class ntp::base { ... }Selectors: $admin = $user_id ? { ‘0’ => ‘root’, }
  • General Puppet SyntaxClass Configuration: If-Else Conditionals: Single Class: if ($ec2_security_groups == ‘DNS’) { class ntp { ... } include bind::server Inherited Class: } else { include bind::client class sftp inherits ssh { ... } } Nested Class: Case Statements: class foo { case $ec2_security_groups { class bar { ... } Monitoring: { include nagios } Developer: { include mercurial } } } Scoped Class: class ntp::base { ... }Selectors: $admin = $user_id ? { ‘0’ => ‘root’, }
  • General Puppet SyntaxClass Configuration: If-Else Conditionals: Single Class: if ($ec2_security_groups == ‘DNS’) { class ntp { ... } include bind::server Inherited Class: } else { include bind::client class sftp inherits ssh { ... } } Nested Class: Case Statements: class foo { case $ec2_security_groups { class bar { ... } Monitoring: { include nagios } Developer: { include mercurial } } } Scoped Class: Set a Variable: class ntp::base { ... } $lib_path = “/usr/local/lib64/”Selectors: $admin = $user_id ? { ‘0’ => ‘root’, }
  • General Puppet SyntaxClass Configuration: If-Else Conditionals: Single Class: if ($ec2_security_groups == ‘DNS’) { class ntp { ... } include bind::server Inherited Class: } else { include bind::client class sftp inherits ssh { ... } } Nested Class: Case Statements: class foo { case $ec2_security_groups { class bar { ... } Monitoring: { include nagios } Developer: { include mercurial } } } Scoped Class: Set a Variable: class ntp::base { ... } $lib_path = “/usr/local/lib64/”Selectors: $admin = $user_id ? { Basic Math: ‘0’ => ‘root’, $file_size = $bytes * 1024 }
  • A Simple NTP Puppet Module
  • A Simple NTP Puppet Modulentpd/manifests/init.pp: class ntp { package { "ntp": ensure => latest } service { "ntpd": ensure => running, enable => true, hasrestart => true, hasstatus => true, require => Package["ntp"], } file { "/etc/ntp.conf": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ ntp.conf", notify => Service["ntpd"]; "/etc/sysconfig/ntpd": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ntpd", notify => Service["ntpd"]; } }
  • A Simple NTP Puppet Modulentpd/manifests/init.pp: class ntp { package { "ntp": ensure => latest } service { "ntpd": ensure => running, enable => true, hasrestart => true, hasstatus => true, require => Package["ntp"], } file { "/etc/ntp.conf": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ ntp.conf", notify => Service["ntpd"]; "/etc/sysconfig/ntpd": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ntpd", notify => Service["ntpd"]; } }
  • A Simple NTP Puppet Modulentpd/manifests/init.pp: ntpd/files/ntp.conf: class ntp { restrict default kod nomodify notrap nopeer noquery package { "ntp": ensure => latest } restrict 127.0.0.1 service { "ntpd": ensure => running, server nist.netservicesgroup.com enable => true, server time.nist.gov hasrestart => true, server time-a.nist.gov hasstatus => true, server time-b.nist.gov require => Package["ntp"], } server 127.127.1.0 fudge 127.127.1.0 stratum 10 file { "/etc/ntp.conf": driftfile /var/lib/ntp/drift ensure => present, owner => root, keys /etc/ntp/keys group => root, mode => 0644, source => "puppet:///modules/ntp/ ntp.conf", notify => Service["ntpd"]; "/etc/sysconfig/ntpd": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ntpd", notify => Service["ntpd"]; } }
  • A Simple NTP Puppet Modulentpd/manifests/init.pp: ntpd/files/ntp.conf: class ntp { restrict default kod nomodify notrap nopeer noquery package { "ntp": ensure => latest } restrict 127.0.0.1 service { "ntpd": ensure => running, server nist.netservicesgroup.com enable => true, server time.nist.gov hasrestart => true, server time-a.nist.gov hasstatus => true, server time-b.nist.gov require => Package["ntp"], } server 127.127.1.0 fudge 127.127.1.0 stratum 10 file { "/etc/ntp.conf": driftfile /var/lib/ntp/drift ensure => present, owner => root, keys /etc/ntp/keys group => root, mode => 0644, source => "puppet:///modules/ntp/ ntp.conf", ntpd/files/ntpd: notify => Service["ntpd"]; OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid" "/etc/sysconfig/ntpd": SYNC_HWCLOCK=yes ensure => present, NTPDATE_OPTIONS="-g -x" owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ntpd", notify => Service["ntpd"]; } }
  • EC2 Security Group Magic
  • EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstance
  • EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstancePuppet can learn about EC2meta-data very easily
  • EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstancePuppet can learn about EC2meta-data very easilyTell Puppet to configureinstances based on theirsecurity group
  • EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstancePuppet can learn about EC2meta-data very easilyTell Puppet to configureinstances based on theirsecurity groupScales for 1 instance or 100
  • EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstancePuppet can learn about EC2meta-data very easilyTell Puppet to configureinstances based on theirsecurity groupScales for 1 instance or 100Rinse and repeat for eachservice group you have
  • EC2 Security Groups + Puppet
  • EC2 Security Groups + Puppet‘DNS’ EC2 Security Group:
  • EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: 22/TCP for SSH for remote access 53/{TCP,UDP} for DNS nameserver
  • EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver
  • EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver
  • EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver The Puppet type ‘file’ allows for variable-replacement in filenames and use-on-first-match
  • EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver The Puppet type ‘file’ allows for variable-replacement in filenames and use-on-first-match file { "/etc/ssh/sshd_config": source => [ “puppet:///modules/ssh/{$ec2_security_groups}-sshd_config”, "puppet:///modules/ssh/sshd_config" ]; }
  • EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver The Puppet type ‘file’ allows for variable-replacement in filenames and use-on-first-match file { "/etc/ssh/sshd_config": source => [ “puppet:///modules/ssh/{$ec2_security_groups}-sshd_config”, "puppet:///modules/ssh/sshd_config" ]; } Puppet will use ‘DNS-sshd_config’ if it exists. If the file does not exist, it will use ‘sshd_config’
  • Client Meta-Data with Facter
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples:
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime $selinux - Configure packages based on SELinux contexts
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime $selinux - Configure packages based on SELinux contexts $operatingsystemrelease - Run OS version specific tasks
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime $selinux - Configure packages based on SELinux contexts $operatingsystemrelease - Run OS version specific tasks $is_virtual - Configure hosts based on VM vs. Physical
  • Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime $selinux - Configure packages based on SELinux contexts $operatingsystemrelease - Run OS version specific tasks $is_virtual - Configure hosts based on VM vs. Physical $ec2_ami_id - Update configuration for the EC2 AMI used
  • Nagios ‘Type’
  • Nagios ‘Type’Puppet natively supports creating Nagios configuration
  • Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automatically
  • Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupings
  • Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupings
  • Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupingsNagios Service:@@nagios_service { "load_check_${hostname}": service_description => "Load Averages", check_command => "load_check!3!5", host_name => "$fqdn", use => "generic-service";}
  • Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupingsNagios Service:@@nagios_service { "load_check_${hostname}": service_description => "Load Averages", check_command => "load_check!3!5", host_name => "$fqdn", use => "generic-service";}Nagios Service Group:@@nagios_servicegroup { "apache_servers": alias => "Apache Servers";}
  • Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupingsNagios Service: Nagios Host:@@nagios_service { "load_check_${hostname}": @@nagios_host { $fqdn: service_description => "Load ensure => present, Averages", hostgroups => "ldap", check_command => "load_check!3!5", use => "generic-host"; host_name => "$fqdn", } use => "generic-service";}Nagios Service Group:@@nagios_servicegroup { "apache_servers": alias => "Apache Servers";}
  • Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupingsNagios Service: Nagios Host:@@nagios_service { "load_check_${hostname}": @@nagios_host { $fqdn: service_description => "Load ensure => present, Averages", hostgroups => "ldap", check_command => "load_check!3!5", use => "generic-host"; host_name => "$fqdn", } use => "generic-service";}Nagios Service Group: Nagios Host Group:@@nagios_servicegroup { @@nagios_hostgroup { "apache_servers": "load_balancers": alias => "Apache Servers"; alias => "Load Balancers";} }
  • Puppet Generated Host/Service Checks
  • Puppet Generated Munin Metrics/Groupings
  • The Foreman: A Heavy Lifter
  • The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasks
  • The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easily
  • The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groups
  • The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groupsLDAP authentication
  • The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groupsLDAP authenticationStatistical graphs for metrics
  • The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groupsLDAP authenticationStatistical graphs for metricsExecute puppetrun on hosts
  • The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groupsLDAP authenticationStatistical graphs for metricsExecute puppetrun on hostsProvision hosts from the web
  • The Foreman ‘Overview’ Page
  • Interact with ‘Facter Facts’
  • Evaluate Puppet Efficiency with Reports
  • General Statistics for Puppet Clients
  • Marionette Collective
  • Marionette CollectiveManage/Control/Execute: Services Packages Process Information Facter Facts Pings
  • Marionette CollectiveManage/Control/Execute: Services Packages Process Information Facter Facts PingsDecide which hosts you actupon by any Facter Fact
  • Marionette CollectiveManage/Control/Execute: Services Packages Process Information Facter Facts PingsDecide which hosts you actupon by any Facter FactEasily manage a largeamount of diverse hosts
  • View Any Service’s Status Across Hosts
  • Check Versions That Are Installed
  • View Processes On Hosts Matching a ‘Fact’
  • Quickly Retrieve a List of MCollective Hosts
  • Consider This Scenario
  • Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts
  • Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP
  • Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)
  • Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)4. Hosts that become ‘WWW’ servers automatically are added to the Elastic Load Balancer (ELB) instance
  • Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)4. Hosts that become ‘WWW’ servers automatically are added to the Elastic Load Balancer (ELB) instance5. Nagios & Munin configuration is done automatically
  • Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)4. Hosts that become ‘WWW’ servers automatically are added to the Elastic Load Balancer (ELB) instance5. Nagios & Munin configuration is done automatically6. If an instance dies, the next time a new instance starts it is given the old host’s IP and that service is fulfilled again
  • Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)4. Hosts that become ‘WWW’ servers automatically are added to the Elastic Load Balancer (ELB) instance5. Nagios & Munin configuration is done automatically6. If an instance dies, the next time a new instance starts it is given the old host’s IP and that service is fulfilled again ...most importantly, you’ve done nothing :)
  • Take Your Environment
  • Take Your EnvironmentPuppet: Provides you with the means to handle ad-hocEC2 instance scaling with granular updates/configurationchanges based on any ‘Fact’ you can supply.
  • Take Your EnvironmentPuppet: Provides you with the means to handle ad-hocEC2 instance scaling with granular updates/configurationchanges based on any ‘Fact’ you can supply.The Foreman: Manage your hosts from a well designedfront-end. View reports, check for deployment efficiency,get the ‘big picture’ on your infrastructure; even deployhosts from scratch!
  • Take Your EnvironmentPuppet: Provides you with the means to handle ad-hocEC2 instance scaling with granular updates/configurationchanges based on any ‘Fact’ you can supply.The Foreman: Manage your hosts from a well designedfront-end. View reports, check for deployment efficiency,get the ‘big picture’ on your infrastructure; even deployhosts from scratch!MCollective: Handle your mass administrative tasks withconsistency and structure. Utilize ‘Facter’ to intelligentlyexecute tasks only against certain sub-sets of hosts.
  • Take Your EnvironmentPuppet: Provides you with the means to handle ad-hocEC2 instance scaling with granular updates/configurationchanges based on any ‘Fact’ you can supply.The Foreman: Manage your hosts from a well designedfront-end. View reports, check for deployment efficiency,get the ‘big picture’ on your infrastructure; even deployhosts from scratch!MCollective: Handle your mass administrative tasks withconsistency and structure. Utilize ‘Facter’ to intelligentlyexecute tasks only against certain sub-sets of hosts.Nagios/Munin: Automatically deploy full monitoring &metrics for hosts without ever hand configuring a file.
  • Thanks! Questions?mark.stanislav@gmail.comuncompiled.com@markstanislav