Being a Puppet Master: Automating Amazon EC2 with Puppet & Friends

24,304 views
24,086 views

Published on

0 Comments
73 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
24,304
On SlideShare
0
From Embeds
0
Number of Embeds
326
Actions
Shares
0
Downloads
848
Comments
0
Likes
73
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Being a Puppet Master: Automating Amazon EC2 with Puppet & Friends

    1. 1. Being a Puppet MasterAutomating Amazon EC2 with Puppet & Friends Mark Stanislav <mark.stanislav@gmail.com>
    2. 2. Puppet: A Quick Overview
    3. 3. Puppet: A Quick OverviewStop administrating your environment and start developing it...
    4. 4. Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurations
    5. 5. Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.
    6. 6. Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.Supports Linux, Solaris, BSD, OS X; Windows in process!
    7. 7. Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.Supports Linux, Solaris, BSD, OS X; Windows in process!Project ran by Luke Kanies; Founder/CEO of Puppet Labs $5M Series B in July 2010; ~$7M total funding
    8. 8. Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.Supports Linux, Solaris, BSD, OS X; Windows in process!Project ran by Luke Kanies; Founder/CEO of Puppet Labs $5M Series B in July 2010; ~$7M total fundingCFEngine & Chef are similar projects; both are quality, too.
    9. 9. Puppet: A Quick OverviewStop administrating your environment and start developing it...Re-usable code for managing your software & configurationsProvides a Domain Specific Language (DSL) to script with Classes, conditionals, selectors, variables, basic math, etc.Supports Linux, Solaris, BSD, OS X; Windows in process!Project ran by Luke Kanies; Founder/CEO of Puppet Labs $5M Series B in July 2010; ~$7M total fundingCFEngine & Chef are similar projects; both are quality, too.Sun, Stanford, Match.com, Media Temple, & Digg all use it!
    10. 10. High-Level Puppet Overview
    11. 11. High-Level Puppet Overview Modules Puppet Master Configuration
    12. 12. High-Level Puppet Overview Modules Puppet Master Configuration Puppet Clients
    13. 13. High-Level Puppet Overview Modules Puppet Master Configuration General Cloud InfrastructureMonitoring DNS Syslog LDAP Nagios/Munin BIND Nameserver rsyslog Server OpenLDAP Server Puppet Clients
    14. 14. High-Level Puppet Overview Modules Puppet Master Configuration General Cloud Infrastructure Monitoring DNS Syslog LDAP Nagios/Munin BIND Nameserver rsyslog Server OpenLDAP Server Software Development EnvironmentsDevelopment Testing Review ProductionApache, Tomcat, Passenger Apache, Tomcat, Passenger Apache, Tomcat, Passenger Apache, Tomcat, Passenger Puppet Clients
    15. 15. Puppet Network Overview
    16. 16. Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet Clients
    17. 17. Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet ClientsConfiguration allows for manual synchronizations or a set increment
    18. 18. Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet ClientsConfiguration allows for manual synchronizations or a set incrementClient or server initiated synchronizations
    19. 19. Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet ClientsConfiguration allows for manual synchronizations or a set incrementClient or server initiated synchronizationsClient/Server configuration leverages a Certificate Authority (CA) on thePuppet Master to sign client certificates to verify authenticity
    20. 20. Puppet Network Overview Puppet Master 8140/TCP 8139/TCP Client Initiated SSL Server Initiated puppetd -t puppetrun Puppet ClientsConfiguration allows for manual synchronizations or a set incrementClient or server initiated synchronizationsClient/Server configuration leverages a Certificate Authority (CA) on thePuppet Master to sign client certificates to verify authenticityTransmissions of all data between a master & client are encrypted
    21. 21. Why EC2 IaaS is Tiring... =
    22. 22. Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexible =
    23. 23. Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexibleBuilding and deploying anew AMI is time consuming =
    24. 24. Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexibleBuilding and deploying anew AMI is time consuming“What do you mean you =want to update a file? Wecan’t just do that...”
    25. 25. Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexibleBuilding and deploying anew AMI is time consuming“What do you mean you =want to update a file? Wecan’t just do that...”Auto-scaling is fantastic butmanaging the scaling hostsis not
    26. 26. Why EC2 IaaS is Tiring...An Amazon Machine Image(AMI) is very inflexibleBuilding and deploying anew AMI is time consuming“What do you mean you =want to update a file? Wecan’t just do that...”Auto-scaling is fantastic butmanaging the scaling hostsis notTime to deploy & configureoffsets benefits of IaaS
    27. 27. Puppet is an EC2 Superhero
    28. 28. Puppet is an EC2 SuperheroDeployment of a “base” EC2AMI - just what you alwaysneed on any standard image
    29. 29. Puppet is an EC2 SuperheroDeployment of a “base” EC2AMI - just what you alwaysneed on any standard imageLeverage EC2 securitygroups to give context to anew instance
    30. 30. Puppet is an EC2 SuperheroDeployment of a “base” EC2AMI - just what you alwaysneed on any standard imageLeverage EC2 securitygroups to give context to anew instancePuppet knows what youwant out of the box;configure a new instancewithout interaction
    31. 31. Puppet is an EC2 SuperheroDeployment of a “base” EC2AMI - just what you alwaysneed on any standard imageLeverage EC2 securitygroups to give context to anew instancePuppet knows what youwant out of the box;configure a new instancewithout interactionUpdate a package orconfiguration file at anytime
    32. 32. New EC2 Puppet Client Flow
    33. 33. New EC2 Puppet Client FlowEC2 Instance WithPuppet Spawned
    34. 34. New EC2 Puppet Client FlowEC2 Instance With Puppet ServicePuppet Spawned Starts For First Time
    35. 35. New EC2 Puppet Client FlowEC2 Instance With Puppet Service Client GeneratesPuppet Spawned Starts For First Time SSL Certificate
    36. 36. New EC2 Puppet Client FlowEC2 Instance With Puppet Service Client GeneratesPuppet Spawned Starts For First Time SSL Certificate Client Sends SSL Certificate to Master
    37. 37. New EC2 Puppet Client FlowEC2 Instance With Puppet Service Client GeneratesPuppet Spawned Starts For First Time SSL Certificate Master Signs SSL Client Sends SSL Certificate Certificate to Master
    38. 38. New EC2 Puppet Client FlowEC2 Instance With Puppet Service Client GeneratesPuppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to Master
    39. 39. New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates:
    40. 40. New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates: Puppet Master can allow certain domain scopes (*.example.com) to be auto-signed when asked by a valid hostname
    41. 41. New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates: Puppet Master can allow certain domain scopes (*.example.com) to be auto-signed when asked by a valid hostname Create a crontab script that executes every minute looking for new SSL certificates in a certain directory and signs them
    42. 42. New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates: Puppet Master can allow certain domain scopes (*.example.com) to be auto-signed when asked by a valid hostname Create a crontab script that executes every minute looking for new SSL certificates in a certain directory and signs them Auto-sign everything you are asked to sign without question
    43. 43. New EC2 Puppet Client Flow EC2 Instance With Puppet Service Client Generates Puppet Spawned Starts For First Time SSL Certificate Puppet Client Master Signs SSL Client Sends SSL Synchronizes Certificate Certificate to MasterMethods to Sign Client SSL Certificates: Puppet Master can allow certain domain scopes (*.example.com) to be auto-signed when asked by a valid hostname Create a crontab script that executes every minute looking for new SSL certificates in a certain directory and signs them Auto-sign everything you are asked to sign without question Manually sign each certificate when you add a new Puppet Client
    44. 44. Puppet Module Structure
    45. 45. Puppet Module Structure Module Folder
    46. 46. Puppet Module Structure Module Foldermanifests/ Tell the module how to work
    47. 47. Puppet Module Structure Module Foldermanifests/ Tell the module how to work files/ Static files needed for deployment
    48. 48. Puppet Module Structure Module Foldermanifests/ Tell the module how to work files/ Static files needed for deploymenttemplates/ Dynamic Ruby-based templates
    49. 49. Puppet Module Structure Module Foldermanifests/ Tell the module how to work files/ Static files needed for deploymenttemplates/ Dynamic Ruby-based templates lib/ Relevant Ruby-based libraries
    50. 50. A Partial List of Puppet ‘types’
    51. 51. A Partial List of Puppet ‘types’Files & Directories
    52. 52. A Partial List of Puppet ‘types’Files & DirectoriesUsers & Groups
    53. 53. A Partial List of Puppet ‘types’Files & DirectoriesUsers & GroupsServices
    54. 54. A Partial List of Puppet ‘types’Files & DirectoriesUsers & GroupsServicesPackages
    55. 55. A Partial List of Puppet ‘types’Files & Directories CrontabsUsers & GroupsServicesPackages
    56. 56. A Partial List of Puppet ‘types’Files & Directories CrontabsUsers & Groups /etc/hostsServicesPackages
    57. 57. A Partial List of Puppet ‘types’Files & Directories CrontabsUsers & Groups /etc/hostsServices Mail AliasesPackages
    58. 58. A Partial List of Puppet ‘types’Files & Directories CrontabsUsers & Groups /etc/hostsServices Mail AliasesPackages Mount Points
    59. 59. A Partial List of Puppet ‘types’Files & Directories Crontabs NagiosUsers & Groups /etc/hostsServices Mail AliasesPackages Mount Points
    60. 60. A Partial List of Puppet ‘types’Files & Directories Crontabs NagiosUsers & Groups /etc/hosts SELinuxServices Mail AliasesPackages Mount Points
    61. 61. A Partial List of Puppet ‘types’Files & Directories Crontabs NagiosUsers & Groups /etc/hosts SELinuxServices Mail Aliases SSH KeysPackages Mount Points
    62. 62. A Partial List of Puppet ‘types’Files & Directories Crontabs NagiosUsers & Groups /etc/hosts SELinuxServices Mail Aliases SSH KeysPackages Mount Points Yum Repos
    63. 63. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Supports 23 different package providers
    64. 64. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Supports 23 different package providers Abstracted for your OS automatically
    65. 65. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Supports 23 different package providers Abstracted for your OS automatically Specify ‘installed’, ‘absent’, or ‘latest’ for desired state
    66. 66. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Supports 23 different package providers Abstracted for your OS automatically Specify ‘installed’, ‘absent’, or ‘latest’ for desired state Change from ‘installed’ to ‘latest’ and deploy for quick
    67. 67. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Supports 23 different Supports 10 different package providers ‘init’ frameworks Abstracted for your OS automatically Specify ‘installed’, ‘absent’, or ‘latest’ for desired state Change from ‘installed’ to ‘latest’ and deploy for quick
    68. 68. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Supports 23 different Supports 10 different package providers ‘init’ frameworks Abstracted for your Control whether a OS automatically service starts on boot or is required to Specify ‘installed’, be running always ‘absent’, or ‘latest’ for desired state Change from ‘installed’ to ‘latest’ and deploy for quick
    69. 69. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Supports 23 different Supports 10 different package providers ‘init’ frameworks Abstracted for your Control whether a OS automatically service starts on boot or is required to Specify ‘installed’, be running always ‘absent’, or ‘latest’ for desired state A service can be notified to restart if a Change from configuration file has ‘installed’ to ‘latest’ and deploy for quick
    70. 70. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a OS automatically service starts on boot or is required to Specify ‘installed’, be running always ‘absent’, or ‘latest’ for desired state A service can be notified to restart if a Change from configuration file has ‘installed’ to ‘latest’ and deploy for quick
    71. 71. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a Load content from OS automatically service starts on ‘files/’, ‘templates/’ boot or is required to or custom strings Specify ‘installed’, be running always ‘absent’, or ‘latest’ for desired state A service can be notified to restart if a Change from configuration file has ‘installed’ to ‘latest’ and deploy for quick
    72. 72. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a Load content from OS automatically service starts on ‘files/’, ‘templates/’ boot or is required to or custom strings Specify ‘installed’, be running always ‘absent’, or ‘latest’ Create symlinks for desired state A service can be notified to restart if a Change from configuration file has ‘installed’ to ‘latest’ and deploy for quick
    73. 73. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a Load content from OS automatically service starts on ‘files/’, ‘templates/’ boot or is required to or custom strings Specify ‘installed’, be running always ‘absent’, or ‘latest’ Create symlinks for desired state A service can be notified to restart if a Supports 5 types to Change from configuration file has verify a file checksum ‘installed’ to ‘latest’ and deploy for quick
    74. 74. A Partial List of Puppet ‘types’ Files & Directories Crontabs Nagios Users & Groups /etc/hosts SELinux Services Mail Aliases SSH Keys Packages Mount Points Yum ReposPackages: Services: Files/Directories: Supports 23 different Supports 10 different Specify ownership & package providers ‘init’ frameworks permissions Abstracted for your Control whether a Load content from OS automatically service starts on ‘files/’, ‘templates/’ boot or is required to or custom strings Specify ‘installed’, be running always ‘absent’, or ‘latest’ Create symlinks for desired state A service can be notified to restart if a Supports 5 types to Change from configuration file has verify a file checksum ‘installed’ to ‘latest’ and deploy for quick Purge a directory of
    75. 75. General Puppet Syntax
    76. 76. General Puppet SyntaxClass Configuration: Single Class: class ntp { ... } Inherited Class: class sftp inherits ssh { ... } Nested Class: class foo { class bar { ... } } Scoped Class: class ntp::base { ... }
    77. 77. General Puppet SyntaxClass Configuration: Single Class: class ntp { ... } Inherited Class: class sftp inherits ssh { ... } Nested Class: class foo { class bar { ... } } Scoped Class: class ntp::base { ... }Selectors: $admin = $user_id ? { ‘0’ => ‘root’, }
    78. 78. General Puppet SyntaxClass Configuration: If-Else Conditionals: Single Class: if ($ec2_security_groups == ‘DNS’) { class ntp { ... } include bind::server Inherited Class: } else { include bind::client class sftp inherits ssh { ... } } Nested Class: class foo { class bar { ... } } Scoped Class: class ntp::base { ... }Selectors: $admin = $user_id ? { ‘0’ => ‘root’, }
    79. 79. General Puppet SyntaxClass Configuration: If-Else Conditionals: Single Class: if ($ec2_security_groups == ‘DNS’) { class ntp { ... } include bind::server Inherited Class: } else { include bind::client class sftp inherits ssh { ... } } Nested Class: Case Statements: class foo { case $ec2_security_groups { class bar { ... } Monitoring: { include nagios } Developer: { include mercurial } } } Scoped Class: class ntp::base { ... }Selectors: $admin = $user_id ? { ‘0’ => ‘root’, }
    80. 80. General Puppet SyntaxClass Configuration: If-Else Conditionals: Single Class: if ($ec2_security_groups == ‘DNS’) { class ntp { ... } include bind::server Inherited Class: } else { include bind::client class sftp inherits ssh { ... } } Nested Class: Case Statements: class foo { case $ec2_security_groups { class bar { ... } Monitoring: { include nagios } Developer: { include mercurial } } } Scoped Class: Set a Variable: class ntp::base { ... } $lib_path = “/usr/local/lib64/”Selectors: $admin = $user_id ? { ‘0’ => ‘root’, }
    81. 81. General Puppet SyntaxClass Configuration: If-Else Conditionals: Single Class: if ($ec2_security_groups == ‘DNS’) { class ntp { ... } include bind::server Inherited Class: } else { include bind::client class sftp inherits ssh { ... } } Nested Class: Case Statements: class foo { case $ec2_security_groups { class bar { ... } Monitoring: { include nagios } Developer: { include mercurial } } } Scoped Class: Set a Variable: class ntp::base { ... } $lib_path = “/usr/local/lib64/”Selectors: $admin = $user_id ? { Basic Math: ‘0’ => ‘root’, $file_size = $bytes * 1024 }
    82. 82. A Simple NTP Puppet Module
    83. 83. A Simple NTP Puppet Modulentpd/manifests/init.pp: class ntp { package { "ntp": ensure => latest } service { "ntpd": ensure => running, enable => true, hasrestart => true, hasstatus => true, require => Package["ntp"], } file { "/etc/ntp.conf": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ ntp.conf", notify => Service["ntpd"]; "/etc/sysconfig/ntpd": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ntpd", notify => Service["ntpd"]; } }
    84. 84. A Simple NTP Puppet Modulentpd/manifests/init.pp: class ntp { package { "ntp": ensure => latest } service { "ntpd": ensure => running, enable => true, hasrestart => true, hasstatus => true, require => Package["ntp"], } file { "/etc/ntp.conf": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ ntp.conf", notify => Service["ntpd"]; "/etc/sysconfig/ntpd": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ntpd", notify => Service["ntpd"]; } }
    85. 85. A Simple NTP Puppet Modulentpd/manifests/init.pp: ntpd/files/ntp.conf: class ntp { restrict default kod nomodify notrap nopeer noquery package { "ntp": ensure => latest } restrict 127.0.0.1 service { "ntpd": ensure => running, server nist.netservicesgroup.com enable => true, server time.nist.gov hasrestart => true, server time-a.nist.gov hasstatus => true, server time-b.nist.gov require => Package["ntp"], } server 127.127.1.0 fudge 127.127.1.0 stratum 10 file { "/etc/ntp.conf": driftfile /var/lib/ntp/drift ensure => present, owner => root, keys /etc/ntp/keys group => root, mode => 0644, source => "puppet:///modules/ntp/ ntp.conf", notify => Service["ntpd"]; "/etc/sysconfig/ntpd": ensure => present, owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ntpd", notify => Service["ntpd"]; } }
    86. 86. A Simple NTP Puppet Modulentpd/manifests/init.pp: ntpd/files/ntp.conf: class ntp { restrict default kod nomodify notrap nopeer noquery package { "ntp": ensure => latest } restrict 127.0.0.1 service { "ntpd": ensure => running, server nist.netservicesgroup.com enable => true, server time.nist.gov hasrestart => true, server time-a.nist.gov hasstatus => true, server time-b.nist.gov require => Package["ntp"], } server 127.127.1.0 fudge 127.127.1.0 stratum 10 file { "/etc/ntp.conf": driftfile /var/lib/ntp/drift ensure => present, owner => root, keys /etc/ntp/keys group => root, mode => 0644, source => "puppet:///modules/ntp/ ntp.conf", ntpd/files/ntpd: notify => Service["ntpd"]; OPTIONS="-u ntp:ntp -p /var/run/ntpd.pid" "/etc/sysconfig/ntpd": SYNC_HWCLOCK=yes ensure => present, NTPDATE_OPTIONS="-g -x" owner => root, group => root, mode => 0644, source => "puppet:///modules/ntp/ntpd", notify => Service["ntpd"]; } }
    87. 87. EC2 Security Group Magic
    88. 88. EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstance
    89. 89. EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstancePuppet can learn about EC2meta-data very easily
    90. 90. EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstancePuppet can learn about EC2meta-data very easilyTell Puppet to configureinstances based on theirsecurity group
    91. 91. EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstancePuppet can learn about EC2meta-data very easilyTell Puppet to configureinstances based on theirsecurity groupScales for 1 instance or 100
    92. 92. EC2 Security Group MagicEC2 security groups are anamed set of inboundfirewall rules for a giveninstancePuppet can learn about EC2meta-data very easilyTell Puppet to configureinstances based on theirsecurity groupScales for 1 instance or 100Rinse and repeat for eachservice group you have
    93. 93. EC2 Security Groups + Puppet
    94. 94. EC2 Security Groups + Puppet‘DNS’ EC2 Security Group:
    95. 95. EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: 22/TCP for SSH for remote access 53/{TCP,UDP} for DNS nameserver
    96. 96. EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver
    97. 97. EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver
    98. 98. EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver The Puppet type ‘file’ allows for variable-replacement in filenames and use-on-first-match
    99. 99. EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver The Puppet type ‘file’ allows for variable-replacement in filenames and use-on-first-match file { "/etc/ssh/sshd_config": source => [ “puppet:///modules/ssh/{$ec2_security_groups}-sshd_config”, "puppet:///modules/ssh/sshd_config" ]; }
    100. 100. EC2 Security Groups + Puppet‘DNS’ EC2 Security Group: Inbound Firewall Rules: Puppet Modules Enabled: 22/TCP for SSH for remote access ssh - SSH server configuration 53/{TCP,UDP} for DNS nameserver bind - BIND nameserver The Puppet type ‘file’ allows for variable-replacement in filenames and use-on-first-match file { "/etc/ssh/sshd_config": source => [ “puppet:///modules/ssh/{$ec2_security_groups}-sshd_config”, "puppet:///modules/ssh/sshd_config" ]; } Puppet will use ‘DNS-sshd_config’ if it exists. If the file does not exist, it will use ‘sshd_config’
    101. 101. Client Meta-Data with Facter
    102. 102. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!
    103. 103. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples:
    104. 104. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture
    105. 105. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry
    106. 106. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime
    107. 107. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime $selinux - Configure packages based on SELinux contexts
    108. 108. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime $selinux - Configure packages based on SELinux contexts $operatingsystemrelease - Run OS version specific tasks
    109. 109. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime $selinux - Configure packages based on SELinux contexts $operatingsystemrelease - Run OS version specific tasks $is_virtual - Configure hosts based on VM vs. Physical
    110. 110. Client Meta-Data with FacterRetrieve useful ‘facts’ about a client host to determine how tointeract with it. ‘facter’ quickly inventories all system metrics!Examples: $architecture - Create files that are based on architecture $hostname/$ip_address_eth0 - Create an /etc/hosts entry $uptime_days - Update all packages after 30 days uptime $selinux - Configure packages based on SELinux contexts $operatingsystemrelease - Run OS version specific tasks $is_virtual - Configure hosts based on VM vs. Physical $ec2_ami_id - Update configuration for the EC2 AMI used
    111. 111. Nagios ‘Type’
    112. 112. Nagios ‘Type’Puppet natively supports creating Nagios configuration
    113. 113. Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automatically
    114. 114. Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupings
    115. 115. Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupings
    116. 116. Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupingsNagios Service:@@nagios_service { "load_check_${hostname}": service_description => "Load Averages", check_command => "load_check!3!5", host_name => "$fqdn", use => "generic-service";}
    117. 117. Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupingsNagios Service:@@nagios_service { "load_check_${hostname}": service_description => "Load Averages", check_command => "load_check!3!5", host_name => "$fqdn", use => "generic-service";}Nagios Service Group:@@nagios_servicegroup { "apache_servers": alias => "Apache Servers";}
    118. 118. Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupingsNagios Service: Nagios Host:@@nagios_service { "load_check_${hostname}": @@nagios_host { $fqdn: service_description => "Load ensure => present, Averages", hostgroups => "ldap", check_command => "load_check!3!5", use => "generic-host"; host_name => "$fqdn", } use => "generic-service";}Nagios Service Group:@@nagios_servicegroup { "apache_servers": alias => "Apache Servers";}
    119. 119. Nagios ‘Type’Puppet natively supports creating Nagios configurationEasily generate specific configuration for n hosts automaticallyNever again manually include hosts/services in groupingsNagios Service: Nagios Host:@@nagios_service { "load_check_${hostname}": @@nagios_host { $fqdn: service_description => "Load ensure => present, Averages", hostgroups => "ldap", check_command => "load_check!3!5", use => "generic-host"; host_name => "$fqdn", } use => "generic-service";}Nagios Service Group: Nagios Host Group:@@nagios_servicegroup { @@nagios_hostgroup { "apache_servers": "load_balancers": alias => "Apache Servers"; alias => "Load Balancers";} }
    120. 120. Puppet Generated Host/Service Checks
    121. 121. Puppet Generated Munin Metrics/Groupings
    122. 122. The Foreman: A Heavy Lifter
    123. 123. The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasks
    124. 124. The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easily
    125. 125. The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groups
    126. 126. The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groupsLDAP authentication
    127. 127. The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groupsLDAP authenticationStatistical graphs for metrics
    128. 128. The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groupsLDAP authenticationStatistical graphs for metricsExecute puppetrun on hosts
    129. 129. The Foreman: A Heavy LifterEasy-to-use Puppet webinterface for many tasksReview Puppet reportsregarding your hosts easilyEdit host facts and groupsLDAP authenticationStatistical graphs for metricsExecute puppetrun on hostsProvision hosts from the web
    130. 130. The Foreman ‘Overview’ Page
    131. 131. Interact with ‘Facter Facts’
    132. 132. Evaluate Puppet Efficiency with Reports
    133. 133. General Statistics for Puppet Clients
    134. 134. Marionette Collective
    135. 135. Marionette CollectiveManage/Control/Execute: Services Packages Process Information Facter Facts Pings
    136. 136. Marionette CollectiveManage/Control/Execute: Services Packages Process Information Facter Facts PingsDecide which hosts you actupon by any Facter Fact
    137. 137. Marionette CollectiveManage/Control/Execute: Services Packages Process Information Facter Facts PingsDecide which hosts you actupon by any Facter FactEasily manage a largeamount of diverse hosts
    138. 138. View Any Service’s Status Across Hosts
    139. 139. Check Versions That Are Installed
    140. 140. View Processes On Hosts Matching a ‘Fact’
    141. 141. Quickly Retrieve a List of MCollective Hosts
    142. 142. Consider This Scenario
    143. 143. Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts
    144. 144. Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP
    145. 145. Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)
    146. 146. Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)4. Hosts that become ‘WWW’ servers automatically are added to the Elastic Load Balancer (ELB) instance
    147. 147. Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)4. Hosts that become ‘WWW’ servers automatically are added to the Elastic Load Balancer (ELB) instance5. Nagios & Munin configuration is done automatically
    148. 148. Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)4. Hosts that become ‘WWW’ servers automatically are added to the Elastic Load Balancer (ELB) instance5. Nagios & Munin configuration is done automatically6. If an instance dies, the next time a new instance starts it is given the old host’s IP and that service is fulfilled again
    149. 149. Consider This Scenario1. You reserve 10 Elastic IPs for a network of hosts2. Each instance starts and Puppet gives it an elastic IP3. Based on an ‘IP -> NEED’ map, each new instance is created for a specific need (DNS, WWW, IMAP, etc.)4. Hosts that become ‘WWW’ servers automatically are added to the Elastic Load Balancer (ELB) instance5. Nagios & Munin configuration is done automatically6. If an instance dies, the next time a new instance starts it is given the old host’s IP and that service is fulfilled again ...most importantly, you’ve done nothing :)
    150. 150. Take Your Environment
    151. 151. Take Your EnvironmentPuppet: Provides you with the means to handle ad-hocEC2 instance scaling with granular updates/configurationchanges based on any ‘Fact’ you can supply.
    152. 152. Take Your EnvironmentPuppet: Provides you with the means to handle ad-hocEC2 instance scaling with granular updates/configurationchanges based on any ‘Fact’ you can supply.The Foreman: Manage your hosts from a well designedfront-end. View reports, check for deployment efficiency,get the ‘big picture’ on your infrastructure; even deployhosts from scratch!
    153. 153. Take Your EnvironmentPuppet: Provides you with the means to handle ad-hocEC2 instance scaling with granular updates/configurationchanges based on any ‘Fact’ you can supply.The Foreman: Manage your hosts from a well designedfront-end. View reports, check for deployment efficiency,get the ‘big picture’ on your infrastructure; even deployhosts from scratch!MCollective: Handle your mass administrative tasks withconsistency and structure. Utilize ‘Facter’ to intelligentlyexecute tasks only against certain sub-sets of hosts.
    154. 154. Take Your EnvironmentPuppet: Provides you with the means to handle ad-hocEC2 instance scaling with granular updates/configurationchanges based on any ‘Fact’ you can supply.The Foreman: Manage your hosts from a well designedfront-end. View reports, check for deployment efficiency,get the ‘big picture’ on your infrastructure; even deployhosts from scratch!MCollective: Handle your mass administrative tasks withconsistency and structure. Utilize ‘Facter’ to intelligentlyexecute tasks only against certain sub-sets of hosts.Nagios/Munin: Automatically deploy full monitoring &metrics for hosts without ever hand configuring a file.
    155. 155. Thanks! Questions?mark.stanislav@gmail.comuncompiled.com@markstanislav

    ×