Puppet for dummies - PHPBenelux UG edition

5,062 views
4,906 views

Published on

0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,062
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
140
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Puppet for dummies - PHPBenelux UG edition

    1. 1. Puppet for Dummies ZendCon - October 2011 Santa Clara - United Stateshttp://joind.in/3781
    2. 2. Who am I? Joshua Thijssen Senior Software Engineer @ Enrise (Netherlands) Development in PHP, Python, Perl, C, Java, and system & DB admin. Blog: http://www.adayinthelifeof.nl Email: joshua@enrise.com Twitter: @jaytaphhttp://www.flickr.com/photos/akrabat/5422369749/in/photostream/
    3. 3. The question of the day
    4. 4. The question of the day What is puppet and why should I care?
    5. 5. Why use puppet? “People are finally figuring out puppet and how it gets you to the pub by 4pm. Note that I’ve been at this pub since 2pm.” - Jorge Castro
    6. 6. Why use puppet?
    7. 7. What is puppet? Puppet is a (not necessarily the) solution for the following problem: How do we setup, manage, synchronize, and upgrade our internal and external infrastructure?
    8. 8. What is puppet? But isn’t that a sysadmin problem??
    9. 9. What is puppet? Short answer:
    10. 10. What is puppet? Short answer: NO
    11. 11. How do we manage our infrastructure?
    12. 12. How do we manage our infrastructure? ‣ Solution 1: We don’t,
    13. 13. How do we manage our infrastructure? ‣ Solution 1: We don’t, ‣ Solution 2: We outsource,
    14. 14. How do we manage our infrastructure? ‣ Solution 1: We don’t, ‣ Solution 2: We outsource, ‣ Solution 3: We automate the process.
    15. 15. How do we manage our infrastructure? (1)
    16. 16. How do we manage our infrastructure? (1) ‣ It’s not funny: you find it more often than not. Especially inside small development companies.
    17. 17. How do we manage our infrastructure? (1) ‣ It’s not funny: you find it more often than not. Especially inside small development companies. ‣ Internal sysadmin, but he’s too busy with development to do sysadmin.
    18. 18. How do we manage our infrastructure? (1) ‣ It’s not funny: you find it more often than not. Especially inside small development companies. ‣ Internal sysadmin, but he’s too busy with development to do sysadmin. ‣ We only act on escalation
    19. 19. How do we manage our infrastructure? (1) ‣ It’s not funny: you find it more often than not. Especially inside small development companies. ‣ Internal sysadmin, but he’s too busy with development to do sysadmin. ‣ We only act on escalation ‣ reactive, not proactive
    20. 20. How do we manage our infrastructure? (2)
    21. 21. How do we manage our infrastructure? (2) ‣ Expensive $LA’s.
    22. 22. How do we manage our infrastructure? (2) ‣ Expensive $LA’s. ‣ What about INTERNAL servers like your development systems and infrastructure?
    23. 23. How do we manage our infrastructure? (2) ‣ Expensive $LA’s. ‣ What about INTERNAL servers like your development systems and infrastructure? ‣ Fight between stability and agility.
    24. 24. How do we manage our infrastructure? (2) ‣ Expensive $LA’s. ‣ What about INTERNAL servers like your development systems and infrastructure? ‣ Fight between stability and agility. ‣ Does your hosting company decide on whether you can use PHP5.3???
    25. 25. How do we manage our infrastructure? (3)
    26. 26. How do we manage our infrastructure? (3) ‣ We are in charge.
    27. 27. How do we manage our infrastructure? (3) ‣ We are in charge. ‣ Dedicated package repositories, tools, etc,..
    28. 28. How do we manage our infrastructure? (3) ‣ We are in charge. ‣ Dedicated package repositories, tools, etc,.. ‣ Use: cfEngine, chef, puppet.
    29. 29. How do we manage our infrastructure? (3) ‣ We are in charge. ‣ Dedicated package repositories, tools, etc,.. ‣ Use: cfEngine, chef, puppet. ‣ It’s actually not that hard.
    30. 30. What is puppet? ‣ Open source configuration management tool. ‣ Written in Ruby ‣ Open source https://github.com/puppetlabs ‣ Commercial version available (puppet enterprise)
    31. 31. What is puppet? ¹ ‣ Don’t tell HOW to do stuff. ‣ Tell WHAT to do.¹ It’s not actually true, but good enough for now...
    32. 32. Architectural overview
    33. 33. Architectural overview Puppet
    34. 34. Architectural overview Puppet Puppet CA Master https Puppet Agent
    35. 35. Architectural overview Puppet Puppet CA Master https Puppet Puppet Puppet Agent Agent Agent
    36. 36. Puppet structure ‣ Puppet master (puppetmasterd) ‣ Puppet cert (puppetca) ‣ Puppet agent (puppetd) ‣ Facter
    37. 37. Puppet master (puppetmasterd) ‣ Central server ‣ File & configuration server ‣ REST over HTTPS interface
    38. 38. Puppet cert (puppet CA) ‣ Certificate signing server ‣ Creates, signs, checks x509 certificates ‣ So you don’t have to worry about it
    39. 39. Puppet cert (puppet CA)Check all systems that have connected to our CA serverroot@puppetmaster:~# puppet cert --list --all+ puppetmaster.noxlogic.local(74:A7:C8:27:72:0D:C1:DD:B8:71:0D:4F:37:69:3D:0C)puppetnode1.noxlogic.local(09:9D:1E:01:D0:A7:BA:FB:8C:F4:2D:96:78:34:54:44)
    40. 40. Puppet cert (puppet CA) Let’s sign our first noderoot@puppetmaster:~# puppet cert --sign puppetnode1.noxlogic.local....root@puppetmaster:~# puppet cert --list --all+ puppetmaster.noxlogic.local(74:A7:C8:27:72:0D:C1:DD:B8:71:0D:4F:37:69:3D:0C)+ puppetnode1.noxlogic.local(CC:50:49:98:1D:F9:06:36:0E:6E:31:F5:27:D8:50:D8)
    41. 41. Puppet agent (puppetd) ‣ Runs on every node that will be managed by puppet. ‣ Calls the puppet master every 30 minutes with system information. ‣ Receives and executes a catalog.
    42. 42. Facter ‣ Runs on nodes to gather system information. ‣ Returns $variables to be used in configuration.
    43. 43. Facter (1)[root@puppetnode1 ~]# facter --puppetarchitecture => x86_64fqdn => puppetnode1.noxlogic.localinterfaces => eth1,eth2,loipaddress_eth1 => 192.168.1.114ipaddress_eth2 => 192.168.56.200kernel => Linuxkernelmajversion => 2.6operatingsystem => CentOSoperatingsystemrelease => 6.0processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHzpuppetversion => 2.6.9‣ A simple list with info (also useable in your own tools)
    44. 44. Facter (2) ‣ You can add your own facts: ‣ project name ‣ master / slave database server ‣ zend server ‣ directadmin / plesk‣ Very simple to add new facts (in ruby, that is)
    45. 45. Facter (3) zendstudio.rb: Facter.add(“Zendserver”) do confine :kernel => :linux setcode do if FileTest.exists?(“/usr/local/zend/bin”) “true” else “false” end end end‣ Crude, but effective enough for us
    46. 46. How does it work Check cert Master Return facts Client Returns catalog
    47. 47. Puppet manifests ‣ Manifests are puppet definitions ‣ <filename>.pp ‣ Puppet DSL ‣ De-cla-ra-tive language ‣ Version your manifests! (git/svn)
    48. 48. Puppet manifests package { “mc” : ensure => present, } file { “/home/jaytaph/secret-ingredient.txt” : ensure => present, mode => 0600, user => ‘jaytaph’, group => ‘noxlogic’, source => “puppet:///secret.txt”, }
    49. 49. Puppet manifests package { “httpd” : ensure => present, } service { “httpd”: running => true, enable => true, require => Package[“httpd”], }‣ Spot the problem....
    50. 50. Puppet manifests Centos / Redhat service: httpd package: httpd config: /etc/httpd/conf/httpd.conf vhosts: /etc/httpd/conf.d/*.conf Debian / Ubuntu service: apache2 package: apache2 config: /etc/apache2/httpd.conf vhosts: /etc/apache2/sites-available‣ Different distributions, different names
    51. 51. Puppet manifests service { “apache”: case $operatingsystem { centos, redhat { $apache = “httpd” } debian, ubuntu { $apache = “apache2” } default : { fail(‘I don’t know this OS/distro’) } } name => $apache, running => true, }‣ $operatingsystem is a FACT
    52. 52. Puppet manifests /etc/puppet/manifests/site.pp: node default { $def_packages = [ “mc”, “strace”, “sysstat” ] package { $def_packages : ensure => latest, } }‣ “Main” manifest
    53. 53. Puppet manifests node ‘web.noxlogic.local’ { package { “httpd” : ensure => latest, } } node ‘db.noxlogic.local’ { package { “mysql-server” : ensure => installed, } }‣ Defining nodes
    54. 54. Puppet manifests node basenode { user { “jaytaph” : ensure => present, gid => 1000, uid => 1000, home => “/home/jaytaph”, shell => “/bin/sh”, password => “supersecrethashedpassword”, } } node *.noxlogic.local inherits basenode { ... }
    55. 55. Puppet manifests
    56. 56. Puppet manifests class webserver { service { “apache”: ensure => running, require => Package[“httpd”], } package { “apache” : ensure => installed, } }
    57. 57. Puppet manifests class webserver { service { “apache”: ensure => running, require => Package[“httpd”], } package { “apache” : ensure => installed, } file { “vhost_$hostname” : path => “/etc/httpd/conf/10-vhost.conf”, content => template(“vhost.template.erb”), notify => Service[“httpd”], } }
    58. 58. Puppet manifests vhost.template.erb <virtualHost <%= ipaddress %>:80> ServerName <%= webserver_name %> ServerAlias <%= webserver_alias %> DocumentRoot <%= webserver_docroot %> </virtualHost>‣ ERB Templates can use custom variables and facts
    59. 59. Puppet manifests node “web01.noxlogic.local” inherits base { $webserver_name = “web01.noxlogic.local” $webserver_alias = “www.noxlogic.local” $webserver_docroot = “/var/www/web01” import webserver } node “web02.noxlogic.local” inherits base { $webserver_name = “web02.noxlogic.local” $webserver_alias = “crm.noxlogic.local” $webserver_docroot = “/var/www/web02” import webserver }
    60. 60. What can puppet manage ‣ Almost everything. ‣ standard 48 different resource types ‣ Ranging from “file” to “cron” to “ssh_key” to “user” to “selinux”. ‣ Can control your Cisco routers and windows machines too (sortakinda)‣ http://docs.puppetlabs.com/references/stable/type.html
    61. 61. Puppet modules ‣ A puppet module is a collection of resources, classes, templates. ‣ Used for easy distribution and code-reuse. ‣ Self-contained, run out-of-the-box
    62. 62. Puppet modules ‣ puppetforge / github ‣ Create your own (and share!). ‣ Use the ones from puppet enterprise edition. ‣ Use the standard layout / best practices
    63. 63. Puppet modules MODULE_PATH/ └──downcased_module_name/ ├──files/ ├──manifests/ │ ├──init.pp │ └──foo.pp ├──lib/ │ ├──puppet/ │ │ ├──parser/ │ │ │ └──functions/ │ │ ├──provider/ │ │ └──type/ │ └──facter/ ├──templates/ ├──tests │ ├──init.pp │ └──foo.pp └──README
    64. 64. Puppet modules class ntp::install { package{"ntpd": ensure => latest } } class ntp::config { File{ require => Class["ntp::install"], notify => Class["ntp::service"], owner => "root", group => "root", mode => 644 } file{"/etc/ntp.conf": source => "puppet:///ntp/ntp.conf"; "/etc/ntp/step-tickers": source => "puppet:///ntp/step-tickers"; } } class ntp::service { service{"ntp": ensure => running, enable => true, require => Class["ntp::config"], } } class ntp { include ntp::install, ntp::config, ntp::service }
    65. 65. Test your modules ‣ (Unit)test your modules ‣ Test them with: puppet apply --noop ‣ More advanced testing: cucumber / cucumber-puppet (BDD)
    66. 66. External Node Configuration (1) ‣ Split modules and nodes ‣ Nodes should be classes - params only (best case scenario?) ‣ Nodes can be configured through YAML
    67. 67. External Node Configuration (2)node1.enrise.local.yaml---classes: - baseparameters: puppetserver: puppet.enrise.local
    68. 68. External Node Configuration (2)node1.enrise.local.yaml---classes: - baseparameters: puppetserver: puppet.enrise.local node node1.enrise.local { $puppetserver = ‘puppet.enrise.local’ include base }
    69. 69. External Node Configuration (3)Puppet doesn’t care how you create YAML files. ‣ Ruby, PHP, Python, Perl, Pony, shellscript. ‣ REST, SOAP, XMLRPC. ‣ Use a database backend. ‣ Or use LDAP instead of YAML.
    70. 70. Confusing puppet things
    71. 71. Confusing puppet things ‣ Puppet went from v0.25 to v2.6. ‣ REST interface since 2.6. XMLRPC before that. ‣ One binary to rule them all (puppet). ‣ Puppet v2.7 switched from GPLv2 to apache2.0 license.
    72. 72. Confusing puppet things ‣ --test does not mean dry-run! (--noop does). ‣ It’s not object oriented. (puppet class != php class) ‣ It’s a declarative language.
    73. 73. Puppet dashboardhttp://media.techtarget.com/digitalguide/images/Misc/puppetDashboard.gif
    74. 74. Live demo | MCollective?
    75. 75. MCollective‣ Puppet agent “calls” the master every 30 minutes.‣ But what about realtime command & control?‣ “Puppet kick”... (meh)‣ MCollective (Marionette Collective)
    76. 76. MCollective ‣ Which systems running a database and have 16GB or less? ‣ Which systems are using <50% of available memory? ‣ Restart all apache services in timezone GMT+5.‣ How do we handle large number of nodes?
    77. 77. MCollective Client Middleware Node MCollective Server MCollective Client ACTIVEMQ Server MCollective Server Collective‣ Middleware takes care of distribution,‣ queued, broadcast etc..
    78. 78. MCollective ‣ The collectivehttp://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html
    79. 79. MCollective$ mc-facts operatingsystemReport for fact: operatingsystem CentOS found 3 times Debian found 14 times Solaris found 4 times$ mc-facts -W operatingsystem=Centos operatingsystemreleaseReport for fact: operatingsystemrelease 6.0 found 1 times 5.6 found 2 times‣ Filter out nodes based on facts
    80. 80. MCollective - cool stuff ‣ Display all running processes ‣ Run or deploy software ‣ Restart services ‣ Start puppet agent ‣ Upgrade your systems
    81. 81. Recap -ETOOMUCHINFO Let’s recap
    82. 82. Recap (1) ‣ Configuration management tool. ‣ Focusses on “what” instead of “how”. ‣ Scales from 1 to 100K+ systems. ‣ Uses descriptive manifests. ‣ Can use external node configurations.
    83. 83. Recap (2) ‣ Useful for sysadmins and developers. ‣ Keeps your infrastructure in sync. ‣ Keeps your infrastructure versioned. ‣ MCollective controls your hosts based on facts, not names.
    84. 84. Any questions?http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg
    85. 85. ‣ THANK YOU FOR YOUR ATTENTION

    ×