Puppet for dummies - ZendCon 2011 Edition

14,063 views

Published on

Puppet is a configuration management tool which allows easy deployment and configuration ranging from 1 to 1 thousand servers (and even more). Even though its common knowledge for devops, puppet is still a strange piece of software for developers. How does it work and what can it do for you as a developer?

0 Comments
20 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
14,063
On SlideShare
0
From Embeds
0
Number of Embeds
197
Actions
Shares
0
Downloads
668
Comments
0
Likes
20
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • anagram\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Puppet for dummies - ZendCon 2011 Edition

    1. 1. Puppet for Dummies ZendCon - October 2011 Santa Clara - United Stateshttp://joind.in/3781
    2. 2. Who am I? Joshua Thijssen Senior Software Engineer @ Enrise (Netherlands) Development in PHP, Python, Perl, C, Java, and System & DB admin. Blog: http://adayinthelifeof.nl Email: joshua@enrise.com Twitter: @jaytaphhttp://www.flickr.com/photos/akrabat/5422369749/in/photostream/
    3. 3. Joind.in‣ http://joind.in/3781
    4. 4. The question of the day
    5. 5. The question of the day What is puppet and why should I care?
    6. 6. Why should I care? “People are finally figuring out puppet and how it gets you to the pub by 4pm. Note that I’ve been at this pub since 2pm.” - Jorge Castro
    7. 7. Why should I care (really)?
    8. 8. What is puppet? Puppet is a (not necessarily the) solution for the following problem: How do we setup, manage, synchronize, and upgrade our internal and external infrastructure?
    9. 9. But isn’t that a sysadmin problem? Sysadmin! Y U no fix problem!
    10. 10. But isn’t that a sysadmin problem? Sysadmin! Y U no fix problem! NO
    11. 11. What is puppet? LAMP-stack
    12. 12. What is puppet? Linux PHP LAMP-stack Apache MySQL
    13. 13. What is puppet? LAMPGMVNMCSTRAH-stack
    14. 14. What is puppet?Hadoop MongoDB ActiveMQ Gearman Linux VarnishTika PHP LAMPGMVNMCSTRAH-stack Solr Apache Ngnix CouchDBRedis Memcache MySQL
    15. 15. How do we manage our infrastructure?
    16. 16. How do we manage our infrastructure? ‣ Solution 1: We don’t,
    17. 17. How do we manage our infrastructure? ‣ Solution 1: We don’t, ‣ Solution 2: We outsource,
    18. 18. How do we manage our infrastructure? ‣ Solution 1: We don’t, ‣ Solution 2: We outsource, ‣ Solution 3: We automate the process.
    19. 19. How do we manage our infrastructure? (1)‣ Solution 1: we don’t
    20. 20. How do we manage our infrastructure? (1) ‣ It’s not funny: you find it more often than not. Especially inside small development companies.‣ Solution 1: we don’t
    21. 21. How do we manage our infrastructure? (1) ‣ It’s not funny: you find it more often than not. Especially inside small development companies. ‣ Internal sysadmin, but he’s too busy with development to do sysadmin.‣ Solution 1: we don’t
    22. 22. How do we manage our infrastructure? (1) ‣ It’s not funny: you find it more often than not. Especially inside small development companies. ‣ Internal sysadmin, but he’s too busy with development to do sysadmin. ‣ We only act on escalation‣ Solution 1: we don’t
    23. 23. How do we manage our infrastructure? (1) ‣ It’s not funny: you find it more often than not. Especially inside small development companies. ‣ Internal sysadmin, but he’s too busy with development to do sysadmin. ‣ We only act on escalation ‣ reactive, not proactive‣ Solution 1: we don’t
    24. 24. How do we manage our infrastructure? (2)‣ Solution 2: we outsource
    25. 25. How do we manage our infrastructure? (2) ‣ Expensive $LA’s.‣ Solution 2: we outsource
    26. 26. How do we manage our infrastructure? (2) ‣ Expensive $LA’s. ‣ What about INTERNAL servers like your development systems and infrastructure?‣ Solution 2: we outsource
    27. 27. How do we manage our infrastructure? (2) ‣ Expensive $LA’s. ‣ What about INTERNAL servers like your development systems and infrastructure? ‣ Fight between stability and agility.‣ Solution 2: we outsource
    28. 28. How do we manage our infrastructure? (2) ‣ Expensive $LA’s. ‣ What about INTERNAL servers like your development systems and infrastructure? ‣ Fight between stability and agility. ‣ Does your hosting company decide on whether you can use PHP5.3???‣ Solution 2: we outsource
    29. 29. How do we manage our infrastructure? (3)‣ Solution 3: we do it ourselves and automate
    30. 30. How do we manage our infrastructure? (3) ‣ We are in charge.‣ Solution 3: we do it ourselves and automate
    31. 31. How do we manage our infrastructure? (3) ‣ We are in charge. ‣ You can do what you like‣ Solution 3: we do it ourselves and automate
    32. 32. How do we manage our infrastructure? (3) ‣ We are in charge. ‣ You can do what you like ‣ Use: cfEngine, chef, puppet.‣ Solution 3: we do it ourselves and automate
    33. 33. How do we manage our infrastructure? (3) ‣ We are in charge. ‣ You can do what you like ‣ Use: cfEngine, chef, puppet. ‣ When done right, maintenance should not be difficult.‣ Solution 3: we do it ourselves and automate
    34. 34. What is puppet?‣ PUPPET TO THE RESCUE
    35. 35. What is puppet? ‣ Open source configuration management tool. ‣ Written in Ruby ‣ Open source: https://github.com/puppetlabs ‣ Commercial version available (puppet enterprise)
    36. 36. What is puppet? ¹ ‣ Don’t tell HOW to do stuff. ‣ Tell WHAT to do.¹ It’s not actually true, but good enough for now...
    37. 37. What is puppet? “yum install httpd” “apt-get install apache2” ¹ ‣ Don’t tell HOW to do stuff. ‣ Tell WHAT to do. “install and run the apache webserver”¹ It’s not actually true, but good enough for now...
    38. 38. Architectural overview
    39. 39. Architectural overview Puppet
    40. 40. Architectural overview Puppet Puppet CA Master https Puppet Agent
    41. 41. Architectural overview Puppet Puppet CA Master https Puppet Puppet Puppet Agent Agent Agent
    42. 42. How does it work Puppet Puppet master client
    43. 43. How does it work Check credentials Puppet Puppet master client
    44. 44. How does it work Check credentials Puppet Send facts Puppet master client
    45. 45. How does it work Check credentials Puppet Send facts Puppet master Returns “catalog” client
    46. 46. How does it work Check credentials Puppet Send facts Puppet master Returns “catalog” client Report results
    47. 47. Puppet manifests ‣ Manifests are puppet definitions ‣ <filename>.pp ‣ Puppet DSL ‣ De-cla-ra-tive language ‣ Version your manifests! (git/svn)
    48. 48. Puppet manifests package { “strace” : ensure => present, } file { “/home/jaytaph/secret-ingredient.txt” : ensure => present, mode => 0600, user => ‘jaytaph’, group => ‘noxlogic’, source => “puppet:///secret.txt”, }
    49. 49. Puppet manifests package { “httpd” : ensure => present, } service { “httpd”: running => true, enable => true, require => Package[“httpd”], }‣ Spot the problem....
    50. 50. Puppet manifests Centos / Redhat service: httpd package: httpd config: /etc/httpd/conf/httpd.conf vhosts: /etc/httpd/conf.d/*.conf Debian / Ubuntu service: apache2 package: apache2 config: /etc/apache2/httpd.conf vhosts: /etc/apache2/sites-available‣ Different distributions, different names
    51. 51. Puppet manifests package { “webserver”: case $operatingsystem { centos, redhat { $apache = “httpd” } debian, ubuntu { $apache = “apache2” } default : { fail(‘I don’t know this OS/distro’) } } name => $apache, ensure => installed, }‣ $operatingsystem is a FACT
    52. 52. Facter[root@puppetnode1 ~]# facter --puppetarchitecture => x86_64fqdn => puppetnode1.noxlogic.localinterfaces => eth1,eth2,loipaddress_eth1 => 192.168.1.114ipaddress_eth2 => 192.168.56.200kernel => Linuxkernelmajversion => 2.6operatingsystem => CentOSoperatingsystemrelease => 6.0processor0 => Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHzpuppetversion => 2.6.9‣ A simple list with info (also useable in your own tools)
    53. 53. Puppet manifests /etc/puppet/manifests/site.pp: node default { $def_packages = [ “mc”, “strace”, “sysstat” ] package { $def_packages : ensure => latest, } }‣ “Main” manifest
    54. 54. Puppet manifests node /^webd+.example.local$/ { package { “httpd” : ensure => latest, } } node /^dbd+.example.local$/ { package { “mysql-server” : ensure => installed, } }‣ Defining nodes - regular expressions
    55. 55. Puppet manifests node basenode { user { “jaytaph” : ensure => present, gid => 1000, uid => 1000, home => “/home/jaytaph”, shell => “/bin/sh”, password => “supersecrethashedpassword”, managehome => true, } } node /^.+.example.local/ inherits basenode { ... }‣ Node inheritance
    56. 56. Puppet manifests‣ Group together into a class
    57. 57. Puppet manifests class webserver { service { “apache”: ensure => running, require => Package[“apache”], } package { “apache” : ensure => installed, } }‣ Group together into a class
    58. 58. Puppet manifests class webserver { service { “apache”: ensure => running, require => Package[“apache”], } package { “apache” : ensure => installed, } file { “vhost_${webserver_name}” : path => “/etc/httpd/conf/10-vhost.conf”, content => template(“vhost.template.erb”), notify => Service[“httpd”], } }‣ Group together into a class
    59. 59. Puppet manifests vhost.template.erb <virtualHost <%= ipaddress %>:80> ServerName <%= webserver_name %> ServerAlias <%= webserver_alias %> DocumentRoot <%= webserver_docroot %> </virtualHost>‣ ERB Templates can use custom variables and facts
    60. 60. Puppet manifests node “web01.example.local” inherits base { $webserver_name = “web01.example.local” $webserver_alias = “www.example.local” $webserver_docroot = “/var/www/web01” import webserver } node “web02.example.local” inherits base { $webserver_name = “web02.example.local” $webserver_alias = “crm.example.local” $webserver_docroot = “/var/www/web02” import webserver }
    61. 61. Puppet modules ‣ A puppet module is a collection of resources, classes, templates. ‣ Used for easy distribution and code-reuse. ‣ Self-contained, run out-of-the-box
    62. 62. Puppet modules ‣ puppetforge / github ‣ Create your own (and share!). ‣ Use the ones from puppet enterprise edition. ‣ Use the standard layout / best practices
    63. 63. Puppet modules class ntp::install { package{"ntpd": ensure => latest } } class ntp::config { File{ require => Class["ntp::install"], notify => Class["ntp::service"], owner => "root", group => "root", mode => 644 } file{"/etc/ntp.conf": source => "puppet:///ntp/ntp.conf"; "/etc/ntp/step-tickers": source => "puppet:///ntp/step-tickers"; } } class ntp::service { service{"ntp": ensure => running, enable => true, require => Class["ntp::config"], } } class ntp { include ntp::install, ntp::config, ntp::service }
    64. 64. Test your modules ‣ (Unit)test your modules ‣ Test them with: puppet apply --noop ‣ More advanced testing: cucumber / cucumber-puppet (BDD)
    65. 65. What can puppet manage ‣ Almost everything. ‣ standard 48 different resource types ‣ Ranging from “file” to “cron” to “ssh_key” to “user” to “selinux”. ‣ Can control your Cisco routers and windows machines too (sortakinda)‣ http://docs.puppetlabs.com/references/stable/type.html
    66. 66. Confusing puppet things
    67. 67. Confusing puppet things ‣ Puppet went from v0.25 to v2.6. ‣ REST interface since 2.6. XMLRPC before that. ‣ One binary to rule them all (puppet). ‣ Puppet v2.7 switched from GPLv2 to apache2.0 license.
    68. 68. Confusing puppet things ‣ --test does not mean dry-run! (--noop does). ‣ It’s not object oriented. (puppet class != php class) ‣ It’s a declarative language.
    69. 69. Puppet dashboardshttp://media.techtarget.com/digitalguide/images/Misc/puppetDashboard.gif
    70. 70. Puppet dashboardshttp://media.techtarget.com/digitalguide/images/Misc/puppetDashboard.gif
    71. 71. Live demo | MCollective?
    72. 72. MCollective‣ Puppet agent “calls” the master every 30 minutes.‣ But what about realtime command & control?‣ “Puppet kick”... (meh)‣ MCollective (Marionette Collective)
    73. 73. MCollective ‣ Which systems running a database and have 16GB or less? ‣ Which systems are using <50% of available memory? ‣ Restart all apache services in timezone GMT+5.‣ How do we handle large number of nodes?
    74. 74. MCollective Client Middleware Node MCollective Server MCollective Client ACTIVEMQ Server MCollective Server Collective‣ Middleware takes care of distribution,‣ queued, broadcast etc..
    75. 75. MCollective ‣ The collectivehttp://docs.puppetlabs.com/mcollective/reference/basic/subcollectives.html
    76. 76. MCollective$ mc-facts operatingsystemReport for fact: operatingsystem CentOS found 3 times Debian found 14 times Solaris found 4 times$ mc-facts -W operatingsystem=Centos operatingsystemreleaseReport for fact: operatingsystemrelease 6.0 found 1 times 5.6 found 2 times‣ Filter out nodes based on facts
    77. 77. MCollective - cool stuff ‣ Display all running processes ‣ Run or deploy software ‣ Restart services ‣ Start puppet agent ‣ Upgrade your systems
    78. 78. Recap -ETOOMUCHINFO Let’s recap
    79. 79. Recap (1) ‣ Configuration management tool. ‣ Focusses on “what” instead of “how”. ‣ Scales from 1 to 100K+ systems. ‣ Uses descriptive manifests.
    80. 80. Recap (2) ‣ Useful for sysadmins and developers. ‣ Keeps your infrastructure in sync. ‣ Keeps your infrastructure versioned. ‣ MCollective controls your hosts based on facts, not names.
    81. 81. Any questions?http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg
    82. 82. to remove this comic sans font, please rate my talk on: http://joind.in/3781

    ×