1. Intro to Puppet
Making Administration Sexy
Larry Ludwig
Email: larry@brandorr.com
Twitter: @lludwig
Web: brandorr.com

2. Image from http://community.uaf.edu/~cde/wiki/uploads/ITSFuturama05/sexy-bill2.png

3. How many servers/VPS instances
do you currently manage?

4. How many servers/VPS instances
do you currently manage?
• <25

5. How many servers/VPS instances
do you currently manage?
• <25
• >25 & <100

6. How many servers/VPS instances
do you currently manage?
• <25
• >25 & <100
• >100 & <250

7. How many servers/VPS instances
do you currently manage?
• <25
• >25 & <100
• >100 & <250
• 250+

8. The Evolution of
Administration
Image from http://www.wordinfo.info/words/images/evolution-man-computer.gif

9. The Evolution of
Administration
Image from http://www.wordinfo.info/words/images/evolution-man-computer.gif

10. From the Single Mainframe Computer
Image from http://tvtropes.org/pmwiki/pub/images/monolith.jpg

11. To Today Many Virtual Servers
Image from http://www.code-muse.com/blog/wp-content/uploads/2007/11/df20021001.jpg

12. What’s wrong with
Administration Today?

13. What’s wrong with
Administration Today?
• Too many computers/services

14. What’s wrong with
Administration Today?
• Too many computers/services
• Too many different operating systems

15. What’s wrong with
Administration Today?
• Too many computers/services
• Too many different operating systems
• Not enough time

16. What’s wrong with
Administration Today?
• Too many computers/services
• Too many different operating systems
• Not enough time
• Mostly a manual process

17. What’s wrong with
Administration Today?
• Too many computers/services
• Too many different operating systems
• Not enough time
• Mostly a manual process
• No feedback loop - (work and boss)

18. What’s wrong with
Administration Today?
• Too many computers/services
• Too many different operating systems
• Not enough time
• Mostly a manual process
• No feedback loop - (work and boss)
• Best practices are not shared

19. What’s wrong with
Administration Today?
• Too many computers/services
• Too many different operating systems
• Not enough time
• Mostly a manual process
• No feedback loop - (work and boss)
• Best practices are not shared
• Too much money lost when they fail

20. What’s wrong with
Administration Today?
• Too many computers/services
• Too many different operating systems
• Not enough time
• Mostly a manual process
• No feedback loop - (work and boss)
• Best practices are not shared
• Too much money lost when they fail

21. Not exactly modern
Image from http://flickr.com/photos/silverwood/593965547/

22. In fact,
they kinda
suck
Image from http://flickr.com/photos/jefframone/1426716646/

23. Sysadmin Programming
Language Progression

24. Sysadmin Programming
Language Progression
• Assembly Language

25. Sysadmin Programming
Language Progression
• Assembly Language
• High Level compiled languages (Cobol, C, C++)

26. Sysadmin Programming
Language Progression
• Assembly Language
• High Level compiled languages (Cobol, C, C++)
• Shell Scripting (bash, awk, sed, grep, etc.)

27. Sysadmin Programming
Language Progression
• Assembly Language
• High Level compiled languages (Cobol, C, C++)
• Shell Scripting (bash, awk, sed, grep, etc.)
• High Level Interpreted (Perl and Python)

28. Sysadmin Programming
Language Progression
• Assembly Language
• High Level compiled languages (Cobol, C, C++)
• Shell Scripting (bash, awk, sed, grep, etc.)
• High Level Interpreted (Perl and Python)
• Administration Based Programming (CFEngine)

29. What’s Wrong With
Existing Tools?

30. What’s Wrong With
Existing Tools?
• Monitoring is immature and requires far too
much effort

31. What’s Wrong With
Existing Tools?
• Monitoring is immature and requires far too
much effort
• Few if any built-in feedback loops

32. What’s Wrong With
Existing Tools?
• Monitoring is immature and requires far too
much effort
• Few if any built-in feedback loops
• Each tool is an independent

33. What’s Wrong With
Existing Tools?
• Monitoring is immature and requires far too
much effort
• Few if any built-in feedback loops
• Each tool is an independent
• All failures lead directly to human
intervention

34. What’s Wrong With
Existing Tools?
• Monitoring is immature and requires far too
much effort
• Few if any built-in feedback loops
• Each tool is an independent
• All failures lead directly to human
intervention
• No sharing of best practices and a manual
process prone to errors

35. What’s Wrong With
Existing Tools?
• Monitoring is immature and requires far too
much effort
• Few if any built-in feedback loops
• Each tool is an independent
• All failures lead directly to human
intervention
• No sharing of best practices and a manual
process prone to errors
• Security policies via documentation files

36. What is Puppet?

37. What is Puppet?
• Puppet is a programming language that
automates system administration

38. What is Puppet?
• Puppet is a programming language that
automates system administration
• It’s the glue between resources and
configuration files

39. What is Puppet?
• Puppet is a programming language that
automates system administration
• It’s the glue between resources and
configuration files
• Allows for repeatable sysadmin best
practices

40. Net Result
• Servers are configured exactly how you
specify
• Code once, deploy many
• Self documenting code
• Allow for repeatable built machines
• Allows for a constantly updated
infrastructure

41. Puppet Features

42. Puppet Features
• Open Source GPL license

43. Puppet Features
• Open Source GPL license
• Developed in Ruby language

44. Puppet Features
• Open Source GPL license
• Developed in Ruby language
• Declarative language

45. Puppet Features
• Open Source GPL license
• Developed in Ruby language
• Declarative language
• Resource abstraction

46. Puppet Features
• Open Source GPL license
• Developed in Ruby language
• Declarative language
• Resource abstraction
• client/server model - centralized management with
downloadable resources

47. Puppet Features
• Open Source GPL license
• Developed in Ruby language
• Declarative language
• Resource abstraction
• client/server model - centralized management with
downloadable resources
• Platform independent (supports many *nixes)

48. Puppet Features
• Open Source GPL license
• Developed in Ruby language
• Declarative language
• Resource abstraction
• client/server model - centralized management with
downloadable resources
• Platform independent (supports many *nixes)
• Relationships and execution order

49. Puppet Features
• Open Source GPL license
• Developed in Ruby language
• Declarative language
• Resource abstraction
• client/server model - centralized management with
downloadable resources
• Platform independent (supports many *nixes)
• Relationships and execution order
• Can be used with servers and desktops

50. Puppet Features
• Open Source GPL license
• Developed in Ruby language
• Declarative language
• Resource abstraction
• client/server model - centralized management with
downloadable resources
• Platform independent (supports many *nixes)
• Relationships and execution order
• Can be used with servers and desktops
• Recipes “make it so” - ensures correctness and
repeatable

51. Puppet Features
• Open Source GPL license
• Developed in Ruby language
• Declarative language
• Resource abstraction
• client/server model - centralized management with
downloadable resources
• Platform independent (supports many *nixes)
• Relationships and execution order
• Can be used with servers and desktops
• Recipes “make it so” - ensures correctness and
repeatable
• Not only for installs but to maintain and upgrade

52. O
SSH

53. Net Effects

54. Your Infrastructure is a
now a program

55. 10,000 ft Overview

56. Centralized
Management

57. Each host gets a
Resource Catalog

58. The Configuration
Process

59. The Configuration
Process
1. Retrieve resource catalog from central
server

60. The Configuration
Process
1. Retrieve resource catalog from central
server
2. Determine resource order

61. The Configuration
Process
1. Retrieve resource catalog from central
server
2. Determine resource order
3. Check each resource in turn, fixing if
necessary

62. The Configuration
Process
1. Retrieve resource catalog from central
server
2. Determine resource order
3. Check each resource in turn, fixing if
necessary
4. Rinse and repeat, every 30 minutes

63. Transactions (for each
resource)

64. Transactions (for each
resource)
1. Retrieve current state (e.g., by querying dpkg
db or doing a stat)

65. Transactions (for each
resource)
1. Retrieve current state (e.g., by querying dpkg
db or doing a stat)
2. Compare to desired state

66. Transactions (for each
resource)
1. Retrieve current state (e.g., by querying dpkg
db or doing a stat)
2. Compare to desired state
3. Fix, if necessary (or just log)

67. Configurations are
idempotent

68. Configurations are
idempotent

69. Idempotency allows
management through
the lifecycle

70. Resource sorting is
done via dependencies
Otherwise known as a ‘Resource Graph’

71. Abstraction

72. Portable Resources
This:

73. Portable Resources
This:
Becomes:

74. Portable Resources
This:
Becomes:

75. Portable Resources
This:
Becomes:

76. Portable Resources
This:
Becomes:

77. Portable Resources
This:
Becomes:

78. What can you manage?
• 40+ resource types
• Users in NetInfo, useradd, pw and LDAP
• Support for Debian, Ubuntu, Red Hat,
Solaris, OS X, Gentoo, SuSE, FreeBSD,
AIX, HP-UX and more (currently not
Windows)

79. Built In Types
• augeas • nagios_hostgroup
• computer • nagios_service
• cron • nagios_servicedependency
• exec • nagios_serviceescalation
• file • nagios_serviceextinfo
• filebucket • nagios_servicegroup
• group • nagios_timeperiod
• host • notify
• k5login • package
• macauthorization • resources
• mailalias • schedule
• maillist • selmodule
• mcx • service
• mount • ssh_authorized_key
• nagios_command • sshkey
• nagios_contact • tidy
• nagios_contactgroup • user
• nagios_host • yumrepo
• nagios_hostdependency • zfs
• nagios_hostescalation • zone
• nagios_hostextinfo • zpool

80. Reuse

81. Same concept, different
code
Debian

82. Same concept, different
code
Debian
Red Hat

83. Portability and Naming

84. One solution per
problem

85. Relationships

86. Relationships matter
but are often implicit

87. Relationships matter
but are often implicit
Package

88. Relationships matter
but are often implicit
Configuration should get
Package modifed after package
installation
Configuration

89. Relationships matter
but are often implicit
Configuration should get
Package modifed after package
installation
Service should restart
when
Configuration
configuration changes
Service

90. Relationships matter

91. Classes provide Intent

92. Facter

93. What is Facter?

94. What is Facter?
• Collects and display facts about the
host

95. What is Facter?
• Collects and display facts about the
host
• Integrated into puppet. Variables are
inserted into Puppet recipes

96. What is Facter?
• Collects and display facts about the
host
• Integrated into puppet. Variables are
inserted into Puppet recipes
• Can create custom facts

97. What is Facter?
• Collects and display facts about the
host
• Integrated into puppet. Variables are
inserted into Puppet recipes
• Can create custom facts
• Detects changes and updates variables

98. Sample Output

99. Configuration Files

100. How to manage
configuration files?

101. How to manage
configuration files?
• Direct File

102. How to manage
configuration files?
• Direct File
• Inline Template (erb - ruby template language)

103. How to manage
configuration files?
• Direct File
• Inline Template (erb - ruby template language)
• Augeas (new puppet type written by RedHat)

104. File Template
source => [
“puppet:///nagios-nrpe/nrpe.${fqdn}.conf”,
“puppet:///nagios-nrpe/nrpe.${hostname}.conf”,
“puppet:///nagios-nrpe/nrpe.conf
],

105. erb config files
content => $config_exim_setup ? {
"antispam”=> template("directadmin-exim/exim.antispam.conf.erb"),
"custom" => template("directadmin-exim/exim.${hostname}.conf.erb",
default => template("directadmin-exim/exim.default.conf.erb"),
},
exim.antispam.conf.erb
...
<% if config_da_clamd == "true" -%>
# enable clamav
av_scanner = clamd:/var/run/clamav/clamd.sock
<% end -%>
...

106. Augeas
• New to Puppet (version 0.24.7)
• Currently supports RH/CentOS?
• Allows for line by line file editing
augeas{"jboss_conf":
context => "/files",
changes => [
"set /etc/jbossas/jbossas.conf/JBOSS_IP $ipaddress",
"set /etc/jbossas/jbossas.conf/JAVA_HOME /usr"
],
load_path => "$/usr/share/jbossas/lenses",
}

107. Node Classification

108. Puppet’s Internals

109. Puppet scales like
HTTPS

110. All communication is
via XMLRPC over
HTTPS
REST over HTTPS in 0.25.x

111. Uses SSL, and provides
a Certificate Authority

112. Logs go to syslog (by
default)

113. Pros and Cons

114. Pros
• Forever changes the way you think about
administration. Administration now follows a
development lifecycle
• Relationships
• Make a consistent configuration that always works
• External Node classification - (LDAP or external app)
• Good open source community

115. Cons
• Weak with complex configuration files (Augeas
should help)
• Scalability issues out of box (uses webrick by
default)
• Documentation is slightly lacking and wiki needs
improvement
• Memory pig (especially with 64 bit OSes)
• Administration becomes system programming
• Test test test!
• Bad code, can have massive ripple effects

116. The Competition

117. Puppet vs. Capistrano
• Primarily used for app deployment lifecycle
(mostly RoR)
• On top of SSH
• No resource abstraction
• similar to existing scripting

118. Puppet vs. Cfengine
• Closed sourced
• No resource abstraction
• No ordering
• No code reuse
• Cfengine 3 is a much needed improvement

119. Puppet vs. Chef
• Puppet uses an external DSL while Chef is
Ruby based
• Imperative (since is Ruby language)
• Chef’s relationship ordering is top down
(order of code matters)
• No true dependency graph

120. Resources
• http://puppet.reductivelabs.com/
• Source Code
• Recipes
• Wiki
• Documentation
• Bug Tracker
• http://groups.google.com/group/puppet-users
• “Pulling Strings with Puppet: Configuration Management Made
Easy” - James Turnbull
• ERB templates - http://www.ruby-doc.org/stdlib/libdoc/erb/rdoc/

121. NYC Puppet Group
• http://groups.google.com/group/puppet-nyc
• If demand supports it - monthly
• A.D.D. Moment: Don’t forget about
marketing research question

122. Questions?

123. After NYLUG
TGI Fridays 8:30 PM
677 Lexington Avenue and 56th Street
Second floor, Northeast corner.

124. Larry Ludwig
Available for Puppet consulting services and best
practices. In and out of the cloud
Larry Ludwig
Email: larry@brandorr.com
Twitter: @lludwig
Web: brandorr.com