Gary Larizza's 1.5 hour talk from Puppetconf 2014 about best practices for refactoring Puppet code into Roles & Profiles, Hiera, R10k, and more.

1. 2014
Presented by
The Refactor Dance
Gary Larizza
Professional Services | Puppet Labs
@glarizza

2. http://bit.ly/refactordance
Presented by

3. Presented by

4. Worst. Hands-on. Ever
Presented by

5. Presented by
• Abstraction and data separation
• Data Hierarchy
• Classification
• Workflow
• Sticky points and caveats

6. Presented by
Classification
Implementation Implementation

7. Presented by
Role
Hiera
Profile Profile
Component
Modules

8. Presented by
roles::application_server
Hiera
profiles::java profiles::tomcat
Component
Modules

9. Component
Modules
Presented by

10. Stop writing custom
goddamn component
Presented by
modules

11. Presented by
$httpd_root = "/opt/corp/data/http"
package
{
‘httpd’:
ensure => latest,
}
file {
“/opt/corp/data/http/conf.d”:
owner => "httpd",
ensure => directory;
“/opt/corp/data/http/conf.d/corp.conf”:
owner => "httpd",
ensure => file;
}

12. Stop writing component modules
Presented by
• Too many ‘okay’ modules
• Maintenance & upkeep
• You are not unique
• You are entirely too lazy

13. Parameterize
your classes
Presented by

14. Presented by
class apache {!
case $::osfamily {!
'RedHat': {!
$confdir = ‘/etc/httpd/conf‘!
$conffile = “${confdir}/httpd.conf”!
}!
'Debian': {!
$confdir = ‘/etc/apache2/conf‘!
$conffile = “${confdir}/apache2.conf”!
}!
}!
}

15. Presented by
class apache (!
$confdir = $apache::params::confdir,!
$conffile = $apache::params::conffile,!
) inherits apache::params {!
file { $confdir:!
ensure => directory,!
}!
file { $conffile:!
ensure => file,!
content => template(’apache/apache.conf.erb’),!
}!
}

16. Presented by
Parameterize classes
• Parameters = API
• Single-entry classes
• The ‘Forge test’

17. Presented by
Params &
shareable data

18. Presented by
class apache::params {!
case $::osfamily {!
'RedHat': {!
$confdir = ‘/etc/httpd/conf‘!
$conffile = “${confdir}/httpd.conf”!
}!
'Debian': {!
$confdir = ‘/etc/apache2/conf‘!
$conffile = “${confdir}/apache2.conf”!
}!
}!
}

19. Presented by
Shareable data
• OS-specific data != private data
• Sane defaults
• Validation…

20. Presented by
Validation

21. Presented by
class xinetd (!
$confdir = $apache::params::confdir,!
$conffile = $apache::params::conffile,!
) inherits xinetd::params {!
file { $confdir:!
ensure => directory,!
}!
file { $conffile:!
ensure => file,!
content => template(’apache/apache.conf.erb’),!
}!
}

22. Presented by
class xinetd (!
$confdir = $apache::params::confdir,!
$conffile = $apache::params::conffile,!
) inherits xinetd::params {!
validate_absolute_path($confdir)!
validate_absolute_path($conffile)!
file { $confdir:!
ensure => directory,!
}!
file { $conffile:!
ensure => file,!
content => template(’apache/apache.conf.erb’),!
}!
}

23. Presented by
Validation
• Functions in puppetlabs-stdlib
• Never pass unvalidated data to resources

24. Presented by
Class
containment

25. Presented by
class mysql::server (!
## params here!
) inherits mysql::params {!
!
include ::mysql::server::install!
include ::mysql::server::config!
include ::mysql::server::service!
!
}

26. Presented by
class mysql::server (!
## params here!
) inherits mysql::params {!
!
include ::mysql::server::install!
include ::mysql::server::config!
include ::mysql::server::service!
!
anchor { ‘mysql:start’: }!
-> Class[‘mysql::server::install’]!
-> Class[‘mysql::server::config’]!
-> Class[‘mysql::server::service’]!
-> anchor { ‘mysql:end’: }!
}

27. Presented by
class mysql::server (!
## params here!
) inherits mysql::params {!
!
contain ::mysql::server::install!
contain ::mysql::server::config!
contain ::mysql::server::service!
!
}
* Puppet ≥ 3.4.0

28. Presented by
Class containment
• Before Puppet 3.4.0 - use anchors
• After Puppet 3.4.0 - use contain

29. Presented by
Hiera

30. Presented by
class data_in_code {!
case $::application_tier {!
'dev': {!
$java_version = '6.0.3'!
$tomcat_version = '6.0'!
}!
!
'test': {!
$java_version = '7.0.1'!
$tomcat_version = '7.0'!
}!
}!
}

31. class data_in_code {!
$java_version = hiera(’java_version’)!
$tomcat_version = hiera(’tomcat_version’)!
}
Presented by

32. Presented by
hiera.yaml
---
:backends:
- yaml
:yaml:
:datadir: /etc/puppetlabs/puppet/hieradata
:hierarchy:
- “nodes/%{::clientcert}”
- “location/%{::location}"
- “tier/%{::application_tier}"
- common

33. Presented by
---
java_version: 7.0
tomcat_version: 8.0
dev.yaml
---
java_version: 6.0
tomcat_version: 7.0
prod.yaml

34. Where’s
$osfamily?!
Presented by

35. What’s an
Application
Presented by
Tier?

36. ‘Application tier’
• Long lived
• Data usually separate
• ‘The Data’
Presented by
!
!
!
‘Environment’
• Short lived
• Migration path to ‘production’
• ‘The Model’
!
!
!

37. Presented by
Hierarchy structure?
• How/where is data different?
• Most -> least specific
• Folders are your friends

38. Profiles
Presented by

39. But first…
a question:
Presented by

40. Presented by
include apache!
vs.!
class { ‘apache’: }!

41. Presented by
include apache!
include apache!
include apache!
include apache!
include apache!

42. Presented by
class { ‘apache’: }!
include apache!

43. Presented by
include apache!
class { ‘apache’: }!
include apache!

44. Namespacing
Presented by

45. class data_in_code {!
$java_version = hiera(’java_version’)!
$tomcat_version = hiera(’tomcat_version’)!
!
notify { “Java is: ${java_version}”: }!
}
Presented by

46. Presented by
class data_in_code {!
$java_version = hiera(’java_version’)!
$tomcat_version = hiera(’tomcat_version’)!
!
notify { “Java is: ${data_in_code::java_version}”: }!
}

47. class profiles::jenkins {!
include jenkins!
}
Presented by

48. class profiles::jenkins {!
include ???????!
}
Presented by

49. class profiles::jenkins {!
include ::jenkins!
}
Presented by

50. Presented by
Data
separation

51. Presented by
class data_in_code {!
case $::application_tier {!
'dev': {!
$java_version = '6.0.3'!
$tomcat_version = '6.0'!
}!
!
'test': {!
$java_version = '7.0.1'!
$tomcat_version = '7.0'!
}!
}!
}

52. Presented by
class profiles::tomcat {!
$java_version = hiera(’java_version’)!
$tomcat_version = hiera(’tomcat_version’)!
!
class { ’::tomcat’:!
version => $tomcat_version,!
}!
!
class { ’::java’:!
version => $java_version,!
}!
}

53. Presented by
class apache {!
file { ‘/opt/custom/key.pem’:!
ensure => file,!
source => ’puppet:///modules/apache/key.pem'!
}!
!
file { ‘/things/that/dont/belong/in/apache’:!
ensure => file,!
source => ’puppet:///modules/apache/blargh'!
} !
}

54. Presented by
class profiles::apache {!
include apache!
$keypath = hiera(’apache_keypath’)!
!
file { “${keypath}/key.pem”:!
ensure => file,!
source => ’puppet:///modules/profiles/key.pem'!
}!
!
file { ‘/things/that/dont/belong/in/apache’:!
ensure => file,!
source => ’puppet:///modules/profiles/blargh'!
}!
}

55. Dependencies
Presented by

56. Presented by
class tomcat {!
class { ‘java’:!
version => ‘6.0’,!
}!
!
Class[‘java’]!
-> Class[‘tomcat’]!
}

57. Presented by
class profiles::tomcat {!
$java_version = hiera(’java_version’)!
$tomcat_version = hiera(’tomcat_version’)!
!
class { ‘::java’:!
version => $java_version,!
}!
class { ‘::tomcat’:!
version => $tomcat_version,!
}!
!
Class[‘::java’]!
-> Class[‘::tomcat’]!
}

58. class profiles::tomcat {!
include profiles::java!
$tomcat_version = hiera(’tomcat_version’)!
!
class { ‘::tomcat’:!
Presented by
version => $tomcat_version,!
}!
!
Class[‘profiles::java’]!
-> Class[‘::tomcat’]!
}

59. Presented by
Profiles
• Hiera for business-specific data
• Proprietary resources
• Inter-class dependencies and containment
• Implementation ‘libraries’

60. Roles
Presented by

61. Classification
Presented by

62. denuatapp06p
Presented by
falcor

63. roles::app_server::pci
Presented by
roles::proxy

64. class roles {!
include profiles::security::base!
include profiles::mycorp::users!
include profiles::mycorp::os_base!
}
Presented by

65. class roles::app_server inherits roles {!
include profiles::tomcat!
include profiles::our_app!
include profiles::shibboleth!
Presented by
!
Class[‘profiles::tomcat’]!
-> Class[‘profiles::our_app’]!
-> Class[‘profiles::shibboleth’]!
}

66. class roles::app_server::pci inherits
roles::app_server {!
include profiles::pci!
}
Presented by

67. Presented by
class roles::app_server::pci {!
include profiles::security::base!
include profiles::mycorp::users!
include profiles::mycorp::os_base!
include profiles::pci!
include profiles::tomcat!
include profiles::our_app!
include profiles::shibboleth!
include profiles::pci!
!
Class[‘profiles::java’]!
-> Class[‘profiles::our_app’]!
-> Class[‘profiles::shibboleth’]!
}

68. Presented by
Roles
• Hostnames minus Hiera
• Technology-independent
• Inheritance makes sense (or not)

69. Workflow
Presented by

70. Presented by
Module
Pinning

71. forge "http://forge.puppetlabs.com"!
!
# Modules from the Puppet Forge!
mod "puppetlabs/apache"!
mod "puppetlabs/ntp"!
!
# Modules from Github using various references!
mod 'notifyme',!
:git => 'git://github.com/glarizza/puppet-notifyme',!
:ref => '50c01703b2e3e352520a9a2271ea4947fe17a51f'!
!
mod 'profiles',!
:git => 'git://github.com/glarizza/puppet-profiles',!
:ref => '3611ae4253ff01762f9bda1d93620edf8f9a3b22'
Presented by

72. R10k - Bad name, good robot
1. Ensuring modules based on a Puppetfile
2. Dynamically creating Puppet environments
Presented by

73. Presented by
Puppetfile
Manifest
Hieradata
Control Repository

74. Presented by
Puppetfile
Manifest
Hieradata
Puppetfile
Manifest
Hieradata
Puppetfile
Manifest
Hieradata

75. Presented by
Puppetfile
Manifest
Hieradata
Branch Branch
Puppetfile
Manifest
Hieradata
Puppetfile
Manifest
Hieradata
Puppet Environment Puppet Environment

76. Presented by
Demo

77. Presented by
Summary
• Simple, generic component modules
• Extract company-specific data with Hiera
• Layer implementation with Profiles
• Classification with Profiles
• R10k for module pinning/workflow