Presentation about Puppet, life and everything at Config Management Camp 2024

1. Strategies for Puppet code
upgrade and refactoring
Alessandro Franceschi
Config Management Camp 2024

2. About me
Alessandro Franceschi
@alvagante
• "World longest active Puppet
consultant"
• Started with version 0.20 in 2007 in
Bank of Italy
• Developed example42 modules
(psick, tp, puppi, nextgen modules)
• Playing with Puppet, IT,
AI (because we can’t miss the AI race, can we?),
life, universe and everything
Puppet Modules and Interoperability - PuppetCamp Europe (2010) - Ghent

3. About this
presentation
• Brief history of Puppet [breaking] changes
• Puppet 8
• Code refactoring obvious practices
• Refactoring techniques in Puppet
• Testing and rollout approaches
• and an example42 surprise

4. Brief history of
Puppet updates
• From 0.25 to 2.6 (eleven times better!)
• Pure Ruby manifests :-O
• Run Stages introduced
• Parameterised classes!
• Extlookup (Hiera’s parent)
• New relationship syntax ( ->, ~> )
• Basic Windows support
• Single binary (puppetd -> puppet apply)
2.6

5. Brief history of
Puppet updates
• From 2.x to 3.x
• Variables dynamic scoping removed
• Params in defines must be variables
• puppet:///modules/ required with modules
• Old puppet* commands removed
• Pure Ruby manifests deprecated
• Hiera in core!
• Automatic params lookup
3.x

6. Brief history of
Puppet updates
• From 3.x to 4.x
• Future parser!
• All-in-One packaging
• New file locations
• Application orchestration added
• Directory environments (control-repo!)
• Removed hiera_puppet backend
4.x

7. Brief history of
Puppet updates
• From 4.x to 7.x
• puppetserver ca replaces puppet cert (6.x)
• Many types (nagios, cron, mount, yumrepo…)
moved from core to modules (6.x)
• Resource API (6.x)
• Application orchestration removed:
application, site, consumes,
produces, export and consume (7.x)
7.x

8. Puppet
changes
• Legacy facts excluded by default
• Strict mode enabled by default
• Hiera 3 removed by default
• Thin reports with only changes
8

9. Puppet 8:
Legacy facts removed
• Legacy facts are no longer collected by the
agent and can't be used in code
• Use puppet-lint legacy-facts plugin to identify
and in some case fix legacy facts in your code
To preserve backwards compatibility:
Old behavior can be reestablished setting in puppet.conf:
include_legacy_facts = true

10. Common facts mapping
architecture => facts['os']['architecture']
domain => facts['networking']['domain']
fqdn => facts['networking']['fqdn']
hostname => facts['networking']['hostname']
ipaddress => facts['networking']['ip']
macaddress => facts['networking']['mac']
Netmask => facts['networking']['netmask']
network => facts['networking']['network']
operatingsystem => facts['os']['name']
operatingsystemmajrelease => facts['os']['release']['major’]
operatingsystemrelease => facts['os']['release']['full']
osfamily => facts['os']['family']
lsbdistid => facts['os']['distro']['id’]
selinux => facts['os']['selinux']['enabled']

11. Puppet 8:
Strict mode
Strict mode is enabled by default
Puppet compilation fails in these conditions:
• Accessing undefined variables
• Accessing a legacy fact:
notice($facts['osfamily'])
• Coercing a string into a numeric: "1" + 1
• Using the stdlib deprecation function

12. Puppet 8:
Strict mode
To preserve backwards compatibility:
Old behavior can be reestablished setting in puppet.conf:
strict_variables = false
strict=warning

13. Puppet 8:
Goodbye Hiera 3
• Hiera 3 is removed by default
• hiera.yaml must use v5 format
• Hiera 3 based backends must be converted
• Replace legacy hiera functions with lookup
To preserve backwards compatibility:
Install separately the hiera 3 gem.

14. Puppet 8:
Only events in reports
By default, Puppet reports include only events
Unchanged resources are not listed in reports
PRO: Huge reduction of reports length
To restore existing behaviour
To restore full reports on agents’ puppet.conf set:
exclude_unchanged_resources = false

15. Refactoring ?
• Why
• How
• Challenges
• Best practices

16. Refactoring: WHY?
Limitation of
technical debt
Enhancement of
code readability
Improved
maintainability
Increased
efficiency
Legacy
code update
Respond to new
requirements

17. Refactoring
CHALLENGES
• Tackling technical debt
• Define new and emerging needs
• Cope with current production
• Develop safely
• Test
• Control rollout

18. Refactoring
HOW?
Start
• Start small
and work
incrementally
Focus
• Focus on high-
impact changes
Keep
• Keep the codebase
maintainable
Understand
• Understand risks
and impact
Collaborate
• Collaborate with
stakeholders

19. Refactoring
in Puppet world
• Feature flags enable new code
• Parallel files creation (to diff)
• In place refactoring for safe changes

20. Refactoring:
feature flags
• Existing code is untouched
• Refactored code is added and lives side by side
• Refactored code is triggered via a param like:
class my_class (
Boolean $use_refactor = false,
) { }
PRO: No impact, by default, on current systems
PRO: Rollout can be controlled via Hiera

21. Refactoring: feature flags
class my_class (
Boolean $use_refactor = false,
) {
if $use_refactor {
# NEW CODE HERE
} else {
# Leave, as is, your existing code
}
}
Configure in Hiera
To enable refactored code set in Hiera data:
my_class::use_refactor: true

22. Refactoring: parallel files
Applies only to configuration files
1. A parallel file is created based on new code
2. Validate changes with scripts to diff the two files
3. Via Hiera class::use_refactor param manage content
PRO: Safe, verified and controlled, for most delicate cases
CON: Testing / validation times can be long

23. Refactoring: parallel files
class my_class (
Boolean $use_refactor = false,
) {
if $use_refactor {
$config_file_content => epp(‘my_class/my_class.conf.epp),
$diff_file_content => # How you used to manage config file,
} else {
$config_file_content => # Whaterver you have now
$diff_file_content => epp(‘my_class/my_class.conf.epp),
}
file { ‘/etc/my_class.conf’:
content => $config_file_content,
}
file { ‘/etc/my_class.conf.diff’:
content => $diff_file_content,
notify => undef,
}
}

24. In place refactoring
• On directly on existing code no parallel codebase
• Good for safe and limited changes
• Puppet lint fixes
• Code compatibility updates
• General improvements
• Safe addition of new parameters
• Cleanups of unused code

25. Testing and rolling out
changes
• Noop mode, on client and server
• Canary testing
• Diff review
• CI/CD

26. Puppet noop mode
See what Puppet would change without changing anything
Client side noop: Set on agent’s puppet.conf
[agent]
noop = true
Server side noop: Set in Puppet code
noop() function from trlinkin-noop module
noop(true)

27. Server side noop
class my_class (
Boolean $noop_manage = false,
Boolean $noop_mode = true,
) {
if $noop_manage {
noop($noop_mode)
}
# All the following resources in the class
# are applied in $noop_mode
# regardless of client’s noop setting
}

28. Canary testing
• Rollout changes to growing subsets of nodes
• Test and validate the result
• The rollout is expanded incrementally
• Allows for early detection and mitigation of issues
• Canaries should be a good representation of:
• All the OS supported
• All the different roles

29. Catalog diff review
• Module puppet/catalog_diff (alternatives exist)
• Setup:
• Create a dedicated certificate for the user running diff
• Grant the certificate access to PuppetDB
• Add options as needed
• Exclude resource which change with environment
• Visualise
• Catalog diff viewer
https://github.com/voxpupuli/puppet-catalog-diff-viewer

30. Catalog diff review
options="--filter_old_env --
old_catalog_from_puppetdb --certless --
show_resource_diff --content_diff --
ignore_parameters alias --changed_depth 1000 --
output_report ${HOME}/lastrun-$$.json --threads
10"
excludes="--
exclude_resource_types=File[/opt/puppetlabs/pe_pa
tch/pe_patch_fact_generation.sh]"
puppet catalog diff
$puppet_server/$default_branch
$puppet_server/${source_branch}
$options $excludes

31. Continuous
Integration
• All the previous tests should be automated
• A sane Puppet CI/CD pipeline should have:
• Syntax checks
• Syntax checks on Puppet target version
(pdk)
• Puppet-lint validation
• Catalog diff review on a subset of nodes
• Noop run on a subset of nodes
• Enforcing run on testing nodes with status
checks
• Canary based rollout with status checks

32. Thank you
ALL images by Tatlin
100% AI free

33. One last thing…

34. Live sessions streamed on
youtube.com/@example42
Everybody can join the live anytime:
https://bit.ly/example42live
Fluid agenda.
Mostly Puppet.
Free as beer and speech.
Puppettizing in Public
LIVE, REALTIME
NO SAFETY
NETS
@alvagante

35. When alone:
• Building in public
example42 OSS
• Tests and
developments
With others:
• Realtime
troubleshooting
• Quick Puppet support
• AMA sessions
• Tiny Data Requests
With many:
• Dynamic round tables
• Collective problem
solving
With experts:
• Product or modules
updates from authors
• Tiny Data applications
drill down
Puppettizing in Public @alvagante

36. See you thisFriday!
Friday, February 9 17:00 GMT
Puppettizing in Public
Episode 0
Watch:
youtube.com/@example42
Join live:
bit.ly/example42live
@alvagante