Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Intro To Puppet.Key

4,395 views

Published on

Intro to Puppet - Making Administration Sexy

Published in: Technology
  • why don't download?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Intro To Puppet.Key

  1. 1. Intro to Puppet Making Administration Sexy Larry Ludwig Email: larry@brandorr.com Twitter: @lludwig Web: brandorr.com
  2. 2. Image from http://community.uaf.edu/~cde/wiki/uploads/ITSFuturama05/sexy-bill2.png
  3. 3. How many servers/VPS instances do you currently manage?
  4. 4. How many servers/VPS instances do you currently manage? • <25
  5. 5. How many servers/VPS instances do you currently manage? • <25 • >25 & <100
  6. 6. How many servers/VPS instances do you currently manage? • <25 • >25 & <100 • >100 & <250
  7. 7. How many servers/VPS instances do you currently manage? • <25 • >25 & <100 • >100 & <250 • 250+
  8. 8. The Evolution of Administration Image from http://www.wordinfo.info/words/images/evolution-man-computer.gif
  9. 9. The Evolution of Administration Image from http://www.wordinfo.info/words/images/evolution-man-computer.gif
  10. 10. From the Single Mainframe Computer Image from http://tvtropes.org/pmwiki/pub/images/monolith.jpg
  11. 11. To Today Many Virtual Servers Image from http://www.code-muse.com/blog/wp-content/uploads/2007/11/df20021001.jpg
  12. 12. What’s wrong with Administration Today?
  13. 13. What’s wrong with Administration Today? • Too many computers/services
  14. 14. What’s wrong with Administration Today? • Too many computers/services • Too many different operating systems
  15. 15. What’s wrong with Administration Today? • Too many computers/services • Too many different operating systems • Not enough time
  16. 16. What’s wrong with Administration Today? • Too many computers/services • Too many different operating systems • Not enough time • Mostly a manual process
  17. 17. What’s wrong with Administration Today? • Too many computers/services • Too many different operating systems • Not enough time • Mostly a manual process • No feedback loop - (work and boss)
  18. 18. What’s wrong with Administration Today? • Too many computers/services • Too many different operating systems • Not enough time • Mostly a manual process • No feedback loop - (work and boss) • Best practices are not shared
  19. 19. What’s wrong with Administration Today? • Too many computers/services • Too many different operating systems • Not enough time • Mostly a manual process • No feedback loop - (work and boss) • Best practices are not shared • Too much money lost when they fail
  20. 20. What’s wrong with Administration Today? • Too many computers/services • Too many different operating systems • Not enough time • Mostly a manual process • No feedback loop - (work and boss) • Best practices are not shared • Too much money lost when they fail
  21. 21. Not exactly modern Image from http://flickr.com/photos/silverwood/593965547/
  22. 22. In fact, they kinda suck Image from http://flickr.com/photos/jefframone/1426716646/
  23. 23. Sysadmin Programming Language Progression
  24. 24. Sysadmin Programming Language Progression • Assembly Language
  25. 25. Sysadmin Programming Language Progression • Assembly Language • High Level compiled languages (Cobol, C, C++)
  26. 26. Sysadmin Programming Language Progression • Assembly Language • High Level compiled languages (Cobol, C, C++) • Shell Scripting (bash, awk, sed, grep, etc.)
  27. 27. Sysadmin Programming Language Progression • Assembly Language • High Level compiled languages (Cobol, C, C++) • Shell Scripting (bash, awk, sed, grep, etc.) • High Level Interpreted (Perl and Python)
  28. 28. Sysadmin Programming Language Progression • Assembly Language • High Level compiled languages (Cobol, C, C++) • Shell Scripting (bash, awk, sed, grep, etc.) • High Level Interpreted (Perl and Python) • Administration Based Programming (CFEngine)
  29. 29. What’s Wrong With Existing Tools?
  30. 30. What’s Wrong With Existing Tools? • Monitoring is immature and requires far too much effort
  31. 31. What’s Wrong With Existing Tools? • Monitoring is immature and requires far too much effort • Few if any built-in feedback loops
  32. 32. What’s Wrong With Existing Tools? • Monitoring is immature and requires far too much effort • Few if any built-in feedback loops • Each tool is an independent
  33. 33. What’s Wrong With Existing Tools? • Monitoring is immature and requires far too much effort • Few if any built-in feedback loops • Each tool is an independent • All failures lead directly to human intervention
  34. 34. What’s Wrong With Existing Tools? • Monitoring is immature and requires far too much effort • Few if any built-in feedback loops • Each tool is an independent • All failures lead directly to human intervention • No sharing of best practices and a manual process prone to errors
  35. 35. What’s Wrong With Existing Tools? • Monitoring is immature and requires far too much effort • Few if any built-in feedback loops • Each tool is an independent • All failures lead directly to human intervention • No sharing of best practices and a manual process prone to errors • Security policies via documentation files
  36. 36. What is Puppet?
  37. 37. What is Puppet? • Puppet is a programming language that automates system administration
  38. 38. What is Puppet? • Puppet is a programming language that automates system administration • It’s the glue between resources and configuration files
  39. 39. What is Puppet? • Puppet is a programming language that automates system administration • It’s the glue between resources and configuration files • Allows for repeatable sysadmin best practices
  40. 40. Net Result • Servers are configured exactly how you specify • Code once, deploy many • Self documenting code • Allow for repeatable built machines • Allows for a constantly updated infrastructure
  41. 41. Puppet Features
  42. 42. Puppet Features • Open Source GPL license
  43. 43. Puppet Features • Open Source GPL license • Developed in Ruby language
  44. 44. Puppet Features • Open Source GPL license • Developed in Ruby language • Declarative language
  45. 45. Puppet Features • Open Source GPL license • Developed in Ruby language • Declarative language • Resource abstraction
  46. 46. Puppet Features • Open Source GPL license • Developed in Ruby language • Declarative language • Resource abstraction • client/server model - centralized management with downloadable resources
  47. 47. Puppet Features • Open Source GPL license • Developed in Ruby language • Declarative language • Resource abstraction • client/server model - centralized management with downloadable resources • Platform independent (supports many *nixes)
  48. 48. Puppet Features • Open Source GPL license • Developed in Ruby language • Declarative language • Resource abstraction • client/server model - centralized management with downloadable resources • Platform independent (supports many *nixes) • Relationships and execution order
  49. 49. Puppet Features • Open Source GPL license • Developed in Ruby language • Declarative language • Resource abstraction • client/server model - centralized management with downloadable resources • Platform independent (supports many *nixes) • Relationships and execution order • Can be used with servers and desktops
  50. 50. Puppet Features • Open Source GPL license • Developed in Ruby language • Declarative language • Resource abstraction • client/server model - centralized management with downloadable resources • Platform independent (supports many *nixes) • Relationships and execution order • Can be used with servers and desktops • Recipes “make it so” - ensures correctness and repeatable
  51. 51. Puppet Features • Open Source GPL license • Developed in Ruby language • Declarative language • Resource abstraction • client/server model - centralized management with downloadable resources • Platform independent (supports many *nixes) • Relationships and execution order • Can be used with servers and desktops • Recipes “make it so” - ensures correctness and repeatable • Not only for installs but to maintain and upgrade
  52. 52. O SSH
  53. 53. Net Effects
  54. 54. Your Infrastructure is a now a program
  55. 55. 10,000 ft Overview
  56. 56. Centralized Management
  57. 57. Each host gets a Resource Catalog
  58. 58. The Configuration Process
  59. 59. The Configuration Process 1. Retrieve resource catalog from central server
  60. 60. The Configuration Process 1. Retrieve resource catalog from central server 2. Determine resource order
  61. 61. The Configuration Process 1. Retrieve resource catalog from central server 2. Determine resource order 3. Check each resource in turn, fixing if necessary
  62. 62. The Configuration Process 1. Retrieve resource catalog from central server 2. Determine resource order 3. Check each resource in turn, fixing if necessary 4. Rinse and repeat, every 30 minutes
  63. 63. Transactions (for each resource)
  64. 64. Transactions (for each resource) 1. Retrieve current state (e.g., by querying dpkg db or doing a stat)
  65. 65. Transactions (for each resource) 1. Retrieve current state (e.g., by querying dpkg db or doing a stat) 2. Compare to desired state
  66. 66. Transactions (for each resource) 1. Retrieve current state (e.g., by querying dpkg db or doing a stat) 2. Compare to desired state 3. Fix, if necessary (or just log)
  67. 67. Configurations are idempotent
  68. 68. Configurations are idempotent
  69. 69. Idempotency allows management through the lifecycle
  70. 70. Resource sorting is done via dependencies Otherwise known as a ‘Resource Graph’
  71. 71. Abstraction
  72. 72. Portable Resources This:
  73. 73. Portable Resources This: Becomes:
  74. 74. Portable Resources This: Becomes:
  75. 75. Portable Resources This: Becomes:
  76. 76. Portable Resources This: Becomes:
  77. 77. Portable Resources This: Becomes:
  78. 78. What can you manage? • 40+ resource types • Users in NetInfo, useradd, pw and LDAP • Support for Debian, Ubuntu, Red Hat, Solaris, OS X, Gentoo, SuSE, FreeBSD, AIX, HP-UX and more (currently not Windows)
  79. 79. Built In Types • augeas • nagios_hostgroup • computer • nagios_service • cron • nagios_servicedependency • exec • nagios_serviceescalation • file • nagios_serviceextinfo • filebucket • nagios_servicegroup • group • nagios_timeperiod • host • notify • k5login • package • macauthorization • resources • mailalias • schedule • maillist • selmodule • mcx • service • mount • ssh_authorized_key • nagios_command • sshkey • nagios_contact • tidy • nagios_contactgroup • user • nagios_host • yumrepo • nagios_hostdependency • zfs • nagios_hostescalation • zone • nagios_hostextinfo • zpool
  80. 80. Reuse
  81. 81. Same concept, different code Debian
  82. 82. Same concept, different code Debian Red Hat
  83. 83. Portability and Naming
  84. 84. One solution per problem
  85. 85. Relationships
  86. 86. Relationships matter but are often implicit
  87. 87. Relationships matter but are often implicit Package
  88. 88. Relationships matter but are often implicit Configuration should get Package modifed after package installation Configuration
  89. 89. Relationships matter but are often implicit Configuration should get Package modifed after package installation Service should restart when Configuration configuration changes Service
  90. 90. Relationships matter
  91. 91. Classes provide Intent
  92. 92. Facter
  93. 93. What is Facter?
  94. 94. What is Facter? • Collects and display facts about the host
  95. 95. What is Facter? • Collects and display facts about the host • Integrated into puppet. Variables are inserted into Puppet recipes
  96. 96. What is Facter? • Collects and display facts about the host • Integrated into puppet. Variables are inserted into Puppet recipes • Can create custom facts
  97. 97. What is Facter? • Collects and display facts about the host • Integrated into puppet. Variables are inserted into Puppet recipes • Can create custom facts • Detects changes and updates variables
  98. 98. Sample Output
  99. 99. Configuration Files
  100. 100. How to manage configuration files?
  101. 101. How to manage configuration files? • Direct File
  102. 102. How to manage configuration files? • Direct File • Inline Template (erb - ruby template language)
  103. 103. How to manage configuration files? • Direct File • Inline Template (erb - ruby template language) • Augeas (new puppet type written by RedHat)
  104. 104. File Template source => [ “puppet:///nagios-nrpe/nrpe.${fqdn}.conf”, “puppet:///nagios-nrpe/nrpe.${hostname}.conf”, “puppet:///nagios-nrpe/nrpe.conf ],
  105. 105. erb config files content => $config_exim_setup ? { "antispam”=> template("directadmin-exim/exim.antispam.conf.erb"), "custom" => template("directadmin-exim/exim.${hostname}.conf.erb", default => template("directadmin-exim/exim.default.conf.erb"), }, exim.antispam.conf.erb ... <% if config_da_clamd == "true" -%> # enable clamav av_scanner = clamd:/var/run/clamav/clamd.sock <% end -%> ...
  106. 106. Augeas • New to Puppet (version 0.24.7) • Currently supports RH/CentOS? • Allows for line by line file editing augeas{"jboss_conf": context => "/files", changes => [ "set /etc/jbossas/jbossas.conf/JBOSS_IP $ipaddress", "set /etc/jbossas/jbossas.conf/JAVA_HOME /usr" ], load_path => "$/usr/share/jbossas/lenses", }
  107. 107. Node Classification
  108. 108. Puppet’s Internals
  109. 109. Puppet scales like HTTPS
  110. 110. All communication is via XMLRPC over HTTPS REST over HTTPS in 0.25.x
  111. 111. Uses SSL, and provides a Certificate Authority
  112. 112. Logs go to syslog (by default)
  113. 113. Pros and Cons
  114. 114. Pros • Forever changes the way you think about administration. Administration now follows a development lifecycle • Relationships • Make a consistent configuration that always works • External Node classification - (LDAP or external app) • Good open source community
  115. 115. Cons • Weak with complex configuration files (Augeas should help) • Scalability issues out of box (uses webrick by default) • Documentation is slightly lacking and wiki needs improvement • Memory pig (especially with 64 bit OSes) • Administration becomes system programming • Test test test! • Bad code, can have massive ripple effects
  116. 116. The Competition
  117. 117. Puppet vs. Capistrano • Primarily used for app deployment lifecycle (mostly RoR) • On top of SSH • No resource abstraction • similar to existing scripting
  118. 118. Puppet vs. Cfengine • Closed sourced • No resource abstraction • No ordering • No code reuse • Cfengine 3 is a much needed improvement
  119. 119. Puppet vs. Chef • Puppet uses an external DSL while Chef is Ruby based • Imperative (since is Ruby language) • Chef’s relationship ordering is top down (order of code matters) • No true dependency graph
  120. 120. Resources • http://puppet.reductivelabs.com/ • Source Code • Recipes • Wiki • Documentation • Bug Tracker • http://groups.google.com/group/puppet-users • “Pulling Strings with Puppet: Configuration Management Made Easy” - James Turnbull • ERB templates - http://www.ruby-doc.org/stdlib/libdoc/erb/rdoc/
  121. 121. NYC Puppet Group • http://groups.google.com/group/puppet-nyc • If demand supports it - monthly • A.D.D. Moment: Don’t forget about marketing research question
  122. 122. Questions?
  123. 123. After NYLUG TGI Fridays 8:30 PM 677 Lexington Avenue and 56th Street Second floor, Northeast corner.
  124. 124. Larry Ludwig Available for Puppet consulting services and best practices. In and out of the cloud Larry Ludwig Email: larry@brandorr.com Twitter: @lludwig Web: brandorr.com

×