SlideShare a Scribd company logo
1 of 31
Enjoying the Journey from
Puppet 3.x to Puppet 4.x
Rob Nelson
Who Am I?
Puppet user since 2014 (3.3 era)
Vox Pupuli, puppet-lint contributor
@rnelson0, https://rnelson0.com/
Agenda
• Why upgrade?
• Refactor our codebase for Puppet 4
• Upgrade our Puppet master(s) and agents to Puppet 4.x
• Refactor our codebase to remove Puppet 2- and 3-isms
• Tips, tricks, and tools
• Enjoying ourselves
Why?
• Puppet 4 is old! First released March, 2015
• Puppet 3 is really old! End Of Support on December 31, 2016
• Puppet 4 only modules
• Puppet 4 language improvements
• Application Orchestration
• PE first, FOSS eventually; some free implementations (such as choria) appearing
• AIO Puppet and Puppetserver
• Better performance, security; same agent/puppetserver between FOSS and PE
• Puppet 5 is coming!
Who does this apply to?
• Puppet Enterprise users
• Puppet Opensource users
• Foreman (1.13+) users
• Master and Masterless
Blueprint
• Start with Puppet 3.x
• Read the release notes
• Plan the roadmap
• Validate / create tests
• Refactor until the new version passes all tests
• Upgrade/Replace the master(s)
• Upgrade the agents
• Repeat the Refactor / Upgrade steps until you get to 4.latest
Release Notes
• All of them - not just the latest version
• Identify potential issues, deprecated features, etc
• Determine the minimum version required to upgrade to target version
• Stay up to date
Define the Roadmap
• Determine the current version
• In-place upgrades or new infrastructure?
• Identify intermediate version steps
• Enable Future Parser [and Strict Variables] before you hit 4.x
• PE: Requires intermediate upgrades or fresh installs (check KB)
• FOSS: Go straight to 4.latest
• Determine how upgrades and interruptions affect ecosystem products – PE
Console/puppetboard, SEIMs, monitoring, etc.
FOSS Example Roadmap
• 3.6.0 -> 3.8.7
• 3.8.7 w/Future Parser [and Strict Variables]
• 3.8.7 -> 4.7.0
• Today’s example roadmap
PE Example Roadmap
• 3.7.2 -> 3.8.6
• 3.8.6 w/Future Parser [and Strict Variables]
• 3.8.6 -> 2015.3.3
• 2015.3.3 -> 2016.2.1
Validate/Create Tests
• Tests assure (mostly) predictable behavior
• Determine what kinds of tests you need - unit, acceptance, integration, other?
• Good testing setup in puppet-module-skeleton
• puppet-retrospec generates naive tests that need tuned
• Existing tests must pass before modifying code
• Turn on Future Parser [and Strict Variables] only at 3.8.x
• Use puppet-lint community plugins, esp for v4 transition
• Beyond tests: catalog diffs, personalized tests
Rspec Tests
$ cat spec/classes/apache_spec.rb
require 'spec_helper'
describe 'profile::apache', :type => :class do
let :facts do
{
facts_hash
}
end
context 'with defaults for all parameters' do
it { is_expected.to create_class('profile::apache') }
it { is_expected.to contain_package('httpd') }
it { is_expected.to contain_user("apache") }
end
end
Rspec Run
[rnelson0@build03 profile:production]$ bundle exec rspec
spec/classes/apache_spec.rb
profile::apache
with defaults for all parameters
should contain Class[profile::apache]
should contain Package[httpd]
should contain User[apache]
Finished in 7.82 seconds (files took 2.49 seconds to load)
3 examples, 0 failures
Refactor
• Create a new branch for the target version, e.g. 3.8.7
• Test against current and target versions, e.g. ~>3.6.0 and ~>3.8.0, with and
without Future Parser/Strict Variables
• Identify failing tests, refactor to fix
• Upgrade modules as early as possible. Be aware of the required Puppet
version for a module version, and look out for defunct or migrated modules,
such as those transferred to Vox Pupuli
• Move forward when tests are green for the next version – previous may be
red
Testing with particular Puppet versions
$ grep PUPPET Gemfile
gem "puppet", ENV['PUPPET_GEM_VERSION'] || '~> 4.0'
[rnelson0@build controlrepo]$ export PUPPET_GEM_VERSION='~>3.8.0'
[rnelson0@build controlrepo]$ bundle update
Installing puppet 3.8.7 (was 4.6.0)
[rnelson0@build controlrepo]$ bundle exec puppet --version
3.8.7
[rnelson0@build controlrepo]$ export PUPPET_GEM_VERSION='3.8.1'
[rnelson0@build controlrepo]$ bundle update
Installing puppet 3.8.1 (was 3.8.7)
[rnelson0@build controlrepo]$ bundle exec puppet --version
3.8.1
High level Master(s) upgrade process
• Prep for new master/in-place upgrade
• Deploy new/upgrade in testing
• Revert
• Deploy new/upgrade in production
• Start with Master of Masters or other “parent” nodes first
• Update separate PuppetDB node, puppetdb-termini on masters
Replace the Master
• Prepare a new operational environment
• Do not serve bad/incorrect catalogs to existing nodes
• Deploy a new master on the target puppet version
• Bootstrap configuration/code
• Test the master against itself, puppet agent -t
• Deploy and test canary nodes in the same operational environment
In-place Master upgrade
• Snapshot (or equivalent) the master(s) and canary nodes
• Restrict access to the master:
• Control access with firewall/load balancer
• Disable puppet agent on nodes with orchestration
• Revoke certificates for non-canary nodes
• Revoke the CA, generate a new CA and new agent certs for canary nodes only
• Upgrade the master
• Test the master then canary nodes with puppet agent -t
Troubleshooting
• Collect logs from the master and canaries
• Look for changed resources, number of resources in catalog
• Revert production environment
• Analyze cause(s)
• Refactor code and data to address issues
• Try again
• Learn from failures, prevent them in the future
Upgrade the Agents
• Can often skip on PATCH versions and some MINOR versions (see rel notes)
• puppetlabs/puppet_agent (requirements) updates agents on next check-in
• Orchestration
• Replace nodes with new instances running the new agent
• By hand
Repeat
• Relax, enjoy the success of an upgrade!
• Start working on the next version/feature flags
• Repeat the Refactor / Upgrade steps
Keeping Up
Keeping Up
Refactor to take advantage of Puppet 4 language improvements, new tools (ex:
r10k -> PE Code Manager), new file locations, etc.
• PE has quarterly upgrades, FOSS more frequent
• The less frequently you do something, the more painful it is. “Upgrade early
and upgrade often!”
• Try not to get more than 2 MINORs behind
• Test against puppet version ~>4.0 (latest v4) and run bundle update before
manual tests
Puppet 4 Language Improvements
• Replace create_resources() with iteration
• Replace validate_*() with data types (including a Sensitive type)
• There is a validate_legacy() helper function available in puppetlabs/stdlib to
assist with replacing validate_*() functions (blog)
• Simplified resource wrappers with * and + operators
• Improved default attributes are per-expression
• New template type EPP is available
• Puppet Lookup, Data In Modules, and other hiera improvements
• Use $facts[] instead of global variables to tidy up the namespace and remove
ambiguity
Tips & Tricks – Puppet Enterprise
• PE includes support, use it for planning/errors
• Puppet Enterprise Upgrade Service to engage Pro Services
• PE Classifier changes over time. Review Preconfigured Node Groups
documentation
• pe_puppetserver_gem is out, puppetserver_gem is in
• Do not use PE’s bundled Ruby for other Ruby tasks, conflicts between
bundled/downloaded gems. Recommend rbenv/rvm or SCL-equiv instead
• Do not ever do this on your master. EVER!
Tips & Tricks - Strings
Understand how string conversion works in puppet, hiera, rspec-puppet, and how it
has changed:
• rspec-puppet: 'undef' represents an undefined value
• Puppet DSL: it is the string undef! Try :undef, without quotes, instead
• If you have a file resource with a title or path of ${undefvar}/${populatedvar},
rspec will start failing because file { 'undef/etc/app.conf' :} is not valid
• Similar issue with 'true' vs true and 'false' vs false
• Other common issues: input from hiera/ENC, quoted numbers as strings, stringify
vs structured facts, unquoted strings in case selectors, etc
• May require acceptance tests/canary nodes to become apparent
Tips & Tricks - Hiera
• Hiera eyaml gem is lost during the upgrade to the 4.x puppetserver
• Enable the yaml backend and ensure that the master does not rely on eyaml data
• Run the agent on the master to redeploy the gem (with puppet/hiera or similar)
before agents check in
• %{}: used to prevent variable interpolation, as in %%{}{environment} to generate
the string %{environment}. In 3.x and in 4.5 resolves to an empty string, in 4.0-4.4
it returned the scope, giving strings like %<#Hiera:7329A802#>{environment}. Use
%{::} instead, as in %%{::}{environment}. Affects PE < 2016.2.0
• datadir: some versions expect :: prepends to variables and others do not.
Change %{environment} to %{::environment}. Likely PE < 2016.2.0 as well
Tips & Tricks - Other
• Review modules and their supported versions. May be incorrect or weak
assumptions (>= 3 but should also include < 4 – check tests)
• Upgrades across major versions mean additional troubleshooting
• Upgrade early – but with caveats
• Many tools to assist with automating version upgrades in your Puppetfile
• ERB scope: prepend most variables with @ (<%= var %> to <%= @var %>)
• Script to detect usage of hardcoded /etc/puppet paths, no longer correct in v4
• External fact weighting bug: FACT-1413
• Minimize coupled/entangled changes
• Ask for help! Colleagues, social media, etc.
Tools
• Puppet Community Slack / IRC and Mailing Lists
• puppet-ghostbuster helps you find "dead code" that you may want to prune
before you start on your refactoring journey.
• rspec-puppet, puppetlabs_spec_helper, and puppet-lint are improving their
Puppet 4 support
• A number of catalog diff tools exist (diff generators and a viewer) to inspect
the actual catalog differences from active nodes across different versions of
Puppet.
Links
Additional information on Puppet 4 and Migrations
• Official Puppet Upgrade Docs
• Whirlwind Tour of Puppet 4 by R.I. Pienaar
• The Power of Puppet 4 by Martin Alfke
• Puppet - our journey from Puppet 3.8 to Puppet 4 by Jonas Genannt
Summary
• Plan the upgrade roadmap
• Have working tests before upgrading
• Step through the new versions / feature flags
• Refactor code to take advantage of the language and tool improvements
• Keep mowing
• Enjoy the journey!

More Related Content

What's hot

Whirlwind Tour of Puppet 4
Whirlwind Tour of Puppet 4Whirlwind Tour of Puppet 4
Whirlwind Tour of Puppet 4ripienaar
 
Creating a mature puppet system
Creating a mature puppet systemCreating a mature puppet system
Creating a mature puppet systemrkhatibi
 
Pragmatic plone projects
Pragmatic plone projectsPragmatic plone projects
Pragmatic plone projectsAndreas Jung
 
Perl Dist::Surveyor 2011
Perl Dist::Surveyor 2011Perl Dist::Surveyor 2011
Perl Dist::Surveyor 2011Tim Bunce
 
Intro To Spring Python
Intro To Spring PythonIntro To Spring Python
Intro To Spring Pythongturnquist
 
PuppetConf 2017: Puppet Tasks: Taming ssh in a "for" loop- Alex Dreyer, Puppet
PuppetConf 2017: Puppet Tasks: Taming ssh in a "for" loop- Alex Dreyer, PuppetPuppetConf 2017: Puppet Tasks: Taming ssh in a "for" loop- Alex Dreyer, Puppet
PuppetConf 2017: Puppet Tasks: Taming ssh in a "for" loop- Alex Dreyer, PuppetPuppet
 
Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013
Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013
Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013Puppet
 
The Grand Puppet Sub-Systems Tour - Nicholas Fagerlund, Puppet Labs
The Grand Puppet Sub-Systems Tour - Nicholas Fagerlund, Puppet LabsThe Grand Puppet Sub-Systems Tour - Nicholas Fagerlund, Puppet Labs
The Grand Puppet Sub-Systems Tour - Nicholas Fagerlund, Puppet LabsPuppet
 
Keep your repo clean
Keep your repo cleanKeep your repo clean
Keep your repo cleanHector Canto
 
PECL Picks - Extensions to make your life better
PECL Picks - Extensions to make your life betterPECL Picks - Extensions to make your life better
PECL Picks - Extensions to make your life betterZendCon
 
Introduction to Programming in Go
Introduction to Programming in GoIntroduction to Programming in Go
Introduction to Programming in GoAmr Hassan
 
Bangpypers april-meetup-2012
Bangpypers april-meetup-2012Bangpypers april-meetup-2012
Bangpypers april-meetup-2012Deepak Garg
 
Advanced Perl Techniques
Advanced Perl TechniquesAdvanced Perl Techniques
Advanced Perl TechniquesDave Cross
 

What's hot (20)

Whirlwind Tour of Puppet 4
Whirlwind Tour of Puppet 4Whirlwind Tour of Puppet 4
Whirlwind Tour of Puppet 4
 
Creating a mature puppet system
Creating a mature puppet systemCreating a mature puppet system
Creating a mature puppet system
 
Pragmatic plone projects
Pragmatic plone projectsPragmatic plone projects
Pragmatic plone projects
 
Effective Benchmarks
Effective BenchmarksEffective Benchmarks
Effective Benchmarks
 
Perl Dist::Surveyor 2011
Perl Dist::Surveyor 2011Perl Dist::Surveyor 2011
Perl Dist::Surveyor 2011
 
Intro To Spring Python
Intro To Spring PythonIntro To Spring Python
Intro To Spring Python
 
PuppetConf 2017: Puppet Tasks: Taming ssh in a "for" loop- Alex Dreyer, Puppet
PuppetConf 2017: Puppet Tasks: Taming ssh in a "for" loop- Alex Dreyer, PuppetPuppetConf 2017: Puppet Tasks: Taming ssh in a "for" loop- Alex Dreyer, Puppet
PuppetConf 2017: Puppet Tasks: Taming ssh in a "for" loop- Alex Dreyer, Puppet
 
Python at Facebook
Python at FacebookPython at Facebook
Python at Facebook
 
Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013
Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013
Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013
 
Getting testy with Perl
Getting testy with PerlGetting testy with Perl
Getting testy with Perl
 
The Grand Puppet Sub-Systems Tour - Nicholas Fagerlund, Puppet Labs
The Grand Puppet Sub-Systems Tour - Nicholas Fagerlund, Puppet LabsThe Grand Puppet Sub-Systems Tour - Nicholas Fagerlund, Puppet Labs
The Grand Puppet Sub-Systems Tour - Nicholas Fagerlund, Puppet Labs
 
Keep your repo clean
Keep your repo cleanKeep your repo clean
Keep your repo clean
 
Getting Testy With Perl6
Getting Testy With Perl6Getting Testy With Perl6
Getting Testy With Perl6
 
PECL Picks - Extensions to make your life better
PECL Picks - Extensions to make your life betterPECL Picks - Extensions to make your life better
PECL Picks - Extensions to make your life better
 
Cross platform php
Cross platform phpCross platform php
Cross platform php
 
System Programming and Administration
System Programming and AdministrationSystem Programming and Administration
System Programming and Administration
 
Introduction to Programming in Go
Introduction to Programming in GoIntroduction to Programming in Go
Introduction to Programming in Go
 
Stacking Up Middleware
Stacking Up MiddlewareStacking Up Middleware
Stacking Up Middleware
 
Bangpypers april-meetup-2012
Bangpypers april-meetup-2012Bangpypers april-meetup-2012
Bangpypers april-meetup-2012
 
Advanced Perl Techniques
Advanced Perl TechniquesAdvanced Perl Techniques
Advanced Perl Techniques
 

Viewers also liked

Voxpupuli: a home for your puppet modules
Voxpupuli: a home for your puppet modulesVoxpupuli: a home for your puppet modules
Voxpupuli: a home for your puppet modulesJulien Pivotto
 
Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014Puppet
 
Monitoring Splunk: S.o.S, DMC, and Beyond Breakout Session
Monitoring Splunk: S.o.S, DMC, and Beyond Breakout SessionMonitoring Splunk: S.o.S, DMC, and Beyond Breakout Session
Monitoring Splunk: S.o.S, DMC, and Beyond Breakout SessionSplunk
 
PuppetConf 2016: The Challenges with Container Configuration – David Lutterko...
PuppetConf 2016: The Challenges with Container Configuration – David Lutterko...PuppetConf 2016: The Challenges with Container Configuration – David Lutterko...
PuppetConf 2016: The Challenges with Container Configuration – David Lutterko...Puppet
 
PuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, Puppet
PuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, PuppetPuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, Puppet
PuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, PuppetPuppet
 
Puppet Design Patterns - PuppetConf
Puppet Design Patterns - PuppetConfPuppet Design Patterns - PuppetConf
Puppet Design Patterns - PuppetConfDavid Danzilio
 
Managing a R&D Lab with Foreman
Managing a R&D Lab with ForemanManaging a R&D Lab with Foreman
Managing a R&D Lab with ForemanJulien Pivotto
 
Stop using Nagios (so it can die peacefully)
Stop using Nagios (so it can die peacefully)Stop using Nagios (so it can die peacefully)
Stop using Nagios (so it can die peacefully)Andy Sykes
 
Puppet DSL: back to the basics
Puppet DSL: back to the basicsPuppet DSL: back to the basics
Puppet DSL: back to the basicsJulien Pivotto
 
Continuous Integration (Jenkins/Hudson)
Continuous Integration (Jenkins/Hudson)Continuous Integration (Jenkins/Hudson)
Continuous Integration (Jenkins/Hudson)Dennys Hsieh
 
Jenkins - From Continuous Integration to Continuous Delivery
Jenkins - From Continuous Integration to Continuous DeliveryJenkins - From Continuous Integration to Continuous Delivery
Jenkins - From Continuous Integration to Continuous DeliveryVirendra Bhalothia
 

Viewers also liked (11)

Voxpupuli: a home for your puppet modules
Voxpupuli: a home for your puppet modulesVoxpupuli: a home for your puppet modules
Voxpupuli: a home for your puppet modules
 
Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014Test Driven Development with Puppet - PuppetConf 2014
Test Driven Development with Puppet - PuppetConf 2014
 
Monitoring Splunk: S.o.S, DMC, and Beyond Breakout Session
Monitoring Splunk: S.o.S, DMC, and Beyond Breakout SessionMonitoring Splunk: S.o.S, DMC, and Beyond Breakout Session
Monitoring Splunk: S.o.S, DMC, and Beyond Breakout Session
 
PuppetConf 2016: The Challenges with Container Configuration – David Lutterko...
PuppetConf 2016: The Challenges with Container Configuration – David Lutterko...PuppetConf 2016: The Challenges with Container Configuration – David Lutterko...
PuppetConf 2016: The Challenges with Container Configuration – David Lutterko...
 
PuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, Puppet
PuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, PuppetPuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, Puppet
PuppetConf. 2016: Puppet Best Practices: Roles & Profiles – Gary Larizza, Puppet
 
Puppet Design Patterns - PuppetConf
Puppet Design Patterns - PuppetConfPuppet Design Patterns - PuppetConf
Puppet Design Patterns - PuppetConf
 
Managing a R&D Lab with Foreman
Managing a R&D Lab with ForemanManaging a R&D Lab with Foreman
Managing a R&D Lab with Foreman
 
Stop using Nagios (so it can die peacefully)
Stop using Nagios (so it can die peacefully)Stop using Nagios (so it can die peacefully)
Stop using Nagios (so it can die peacefully)
 
Puppet DSL: back to the basics
Puppet DSL: back to the basicsPuppet DSL: back to the basics
Puppet DSL: back to the basics
 
Continuous Integration (Jenkins/Hudson)
Continuous Integration (Jenkins/Hudson)Continuous Integration (Jenkins/Hudson)
Continuous Integration (Jenkins/Hudson)
 
Jenkins - From Continuous Integration to Continuous Delivery
Jenkins - From Continuous Integration to Continuous DeliveryJenkins - From Continuous Integration to Continuous Delivery
Jenkins - From Continuous Integration to Continuous Delivery
 

Similar to Enjoying the Journey from Puppet 3.x to Puppet 4.x (PuppetConf 2016)

PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T
PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T
PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T Puppet
 
Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?
Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?
Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?NETWAYS
 
Can you upgrade to Puppet 4.x?
Can you upgrade to Puppet 4.x?Can you upgrade to Puppet 4.x?
Can you upgrade to Puppet 4.x?Martin Alfke
 
Continuing Evolution of Perl: Highlights of ActivePerl 5.14
Continuing Evolution of Perl: Highlights of ActivePerl 5.14Continuing Evolution of Perl: Highlights of ActivePerl 5.14
Continuing Evolution of Perl: Highlights of ActivePerl 5.14ActiveState
 
Bootstrapping Puppet and Application Deployment - PuppetConf 2013
Bootstrapping Puppet and Application Deployment - PuppetConf 2013Bootstrapping Puppet and Application Deployment - PuppetConf 2013
Bootstrapping Puppet and Application Deployment - PuppetConf 2013Puppet
 
Strategies for Puppet code upgrade and refactoring
Strategies for Puppet code upgrade and refactoringStrategies for Puppet code upgrade and refactoring
Strategies for Puppet code upgrade and refactoringAlessandro Franceschi
 
20111110 how puppet-fits_into_your_existing_infrastructure_and_change_managem...
20111110 how puppet-fits_into_your_existing_infrastructure_and_change_managem...20111110 how puppet-fits_into_your_existing_infrastructure_and_change_managem...
20111110 how puppet-fits_into_your_existing_infrastructure_and_change_managem...garrett honeycutt
 
Scaling to-5000-nodes
Scaling to-5000-nodesScaling to-5000-nodes
Scaling to-5000-nodesPhilip Watts
 
Puppet Camp New York 2015: Puppet Enterprise Scaling Lessons Learned (Interme...
Puppet Camp New York 2015: Puppet Enterprise Scaling Lessons Learned (Interme...Puppet Camp New York 2015: Puppet Enterprise Scaling Lessons Learned (Interme...
Puppet Camp New York 2015: Puppet Enterprise Scaling Lessons Learned (Interme...Puppet
 
Puppet getting started by Dirk Götz
Puppet getting started by Dirk GötzPuppet getting started by Dirk Götz
Puppet getting started by Dirk GötzNETWAYS
 
PHP North-East - Automated Deployment
PHP North-East - Automated DeploymentPHP North-East - Automated Deployment
PHP North-East - Automated DeploymentMichael Peacock
 
Automated Deployment
Automated DeploymentAutomated Deployment
Automated Deploymentphpne
 
Scalable Systems Management with Puppet
Scalable Systems Management with PuppetScalable Systems Management with Puppet
Scalable Systems Management with PuppetPuppet
 
Scalable systems management with puppet
Scalable systems management with puppetScalable systems management with puppet
Scalable systems management with puppetPuppet
 
Puppet slides for intelligrape
Puppet slides for intelligrapePuppet slides for intelligrape
Puppet slides for intelligrapeSharad Aggarwal
 
Puppet Camp Paris 2014: Test Driven Development
Puppet Camp Paris 2014: Test Driven DevelopmentPuppet Camp Paris 2014: Test Driven Development
Puppet Camp Paris 2014: Test Driven DevelopmentPuppet
 
20140408 tdd puppetcamp-paris
20140408 tdd puppetcamp-paris20140408 tdd puppetcamp-paris
20140408 tdd puppetcamp-parisJohan De Wit
 
Puppet Development Workflow
Puppet Development WorkflowPuppet Development Workflow
Puppet Development WorkflowJeffery Smith
 

Similar to Enjoying the Journey from Puppet 3.x to Puppet 4.x (PuppetConf 2016) (20)

PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T
PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T
PuppetConf 2016: Enjoying the Journey from Puppet 3.x to 4.x – Rob Nelson, AT&T
 
Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?
Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?
Puppet Camp Duesseldorf 2014: Martin Alfke - Can you upgrade to puppet 4.x?
 
Can you upgrade to Puppet 4.x?
Can you upgrade to Puppet 4.x?Can you upgrade to Puppet 4.x?
Can you upgrade to Puppet 4.x?
 
Continuing Evolution of Perl: Highlights of ActivePerl 5.14
Continuing Evolution of Perl: Highlights of ActivePerl 5.14Continuing Evolution of Perl: Highlights of ActivePerl 5.14
Continuing Evolution of Perl: Highlights of ActivePerl 5.14
 
Bootstrapping Puppet and Application Deployment - PuppetConf 2013
Bootstrapping Puppet and Application Deployment - PuppetConf 2013Bootstrapping Puppet and Application Deployment - PuppetConf 2013
Bootstrapping Puppet and Application Deployment - PuppetConf 2013
 
Strategies for Puppet code upgrade and refactoring
Strategies for Puppet code upgrade and refactoringStrategies for Puppet code upgrade and refactoring
Strategies for Puppet code upgrade and refactoring
 
20111110 how puppet-fits_into_your_existing_infrastructure_and_change_managem...
20111110 how puppet-fits_into_your_existing_infrastructure_and_change_managem...20111110 how puppet-fits_into_your_existing_infrastructure_and_change_managem...
20111110 how puppet-fits_into_your_existing_infrastructure_and_change_managem...
 
Automation using Puppet 3
Automation using Puppet 3 Automation using Puppet 3
Automation using Puppet 3
 
Scaling to-5000-nodes
Scaling to-5000-nodesScaling to-5000-nodes
Scaling to-5000-nodes
 
Puppet Camp New York 2015: Puppet Enterprise Scaling Lessons Learned (Interme...
Puppet Camp New York 2015: Puppet Enterprise Scaling Lessons Learned (Interme...Puppet Camp New York 2015: Puppet Enterprise Scaling Lessons Learned (Interme...
Puppet Camp New York 2015: Puppet Enterprise Scaling Lessons Learned (Interme...
 
Puppet getting started by Dirk Götz
Puppet getting started by Dirk GötzPuppet getting started by Dirk Götz
Puppet getting started by Dirk Götz
 
PHP North-East - Automated Deployment
PHP North-East - Automated DeploymentPHP North-East - Automated Deployment
PHP North-East - Automated Deployment
 
Automated Deployment
Automated DeploymentAutomated Deployment
Automated Deployment
 
Scalable Systems Management with Puppet
Scalable Systems Management with PuppetScalable Systems Management with Puppet
Scalable Systems Management with Puppet
 
Scalable systems management with puppet
Scalable systems management with puppetScalable systems management with puppet
Scalable systems management with puppet
 
Puppet slides for intelligrape
Puppet slides for intelligrapePuppet slides for intelligrape
Puppet slides for intelligrape
 
Puppet Camp Paris 2014: Test Driven Development
Puppet Camp Paris 2014: Test Driven DevelopmentPuppet Camp Paris 2014: Test Driven Development
Puppet Camp Paris 2014: Test Driven Development
 
20140408 tdd puppetcamp-paris
20140408 tdd puppetcamp-paris20140408 tdd puppetcamp-paris
20140408 tdd puppetcamp-paris
 
Puppet Development Workflow
Puppet Development WorkflowPuppet Development Workflow
Puppet Development Workflow
 
Puppet_training
Puppet_trainingPuppet_training
Puppet_training
 

Recently uploaded

2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dashnarutouzumaki53779
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demoHarshalMandlekar2
 

Recently uploaded (20)

2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Visualising and forecasting stocks using Dash
Visualising and forecasting stocks using DashVisualising and forecasting stocks using Dash
Visualising and forecasting stocks using Dash
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Sample pptx for embedding into website for demo
Sample pptx for embedding into website for demoSample pptx for embedding into website for demo
Sample pptx for embedding into website for demo
 

Enjoying the Journey from Puppet 3.x to Puppet 4.x (PuppetConf 2016)

  • 1. Enjoying the Journey from Puppet 3.x to Puppet 4.x Rob Nelson
  • 2. Who Am I? Puppet user since 2014 (3.3 era) Vox Pupuli, puppet-lint contributor @rnelson0, https://rnelson0.com/
  • 3. Agenda • Why upgrade? • Refactor our codebase for Puppet 4 • Upgrade our Puppet master(s) and agents to Puppet 4.x • Refactor our codebase to remove Puppet 2- and 3-isms • Tips, tricks, and tools • Enjoying ourselves
  • 4. Why? • Puppet 4 is old! First released March, 2015 • Puppet 3 is really old! End Of Support on December 31, 2016 • Puppet 4 only modules • Puppet 4 language improvements • Application Orchestration • PE first, FOSS eventually; some free implementations (such as choria) appearing • AIO Puppet and Puppetserver • Better performance, security; same agent/puppetserver between FOSS and PE • Puppet 5 is coming!
  • 5. Who does this apply to? • Puppet Enterprise users • Puppet Opensource users • Foreman (1.13+) users • Master and Masterless
  • 6. Blueprint • Start with Puppet 3.x • Read the release notes • Plan the roadmap • Validate / create tests • Refactor until the new version passes all tests • Upgrade/Replace the master(s) • Upgrade the agents • Repeat the Refactor / Upgrade steps until you get to 4.latest
  • 7. Release Notes • All of them - not just the latest version • Identify potential issues, deprecated features, etc • Determine the minimum version required to upgrade to target version • Stay up to date
  • 8. Define the Roadmap • Determine the current version • In-place upgrades or new infrastructure? • Identify intermediate version steps • Enable Future Parser [and Strict Variables] before you hit 4.x • PE: Requires intermediate upgrades or fresh installs (check KB) • FOSS: Go straight to 4.latest • Determine how upgrades and interruptions affect ecosystem products – PE Console/puppetboard, SEIMs, monitoring, etc.
  • 9. FOSS Example Roadmap • 3.6.0 -> 3.8.7 • 3.8.7 w/Future Parser [and Strict Variables] • 3.8.7 -> 4.7.0 • Today’s example roadmap
  • 10. PE Example Roadmap • 3.7.2 -> 3.8.6 • 3.8.6 w/Future Parser [and Strict Variables] • 3.8.6 -> 2015.3.3 • 2015.3.3 -> 2016.2.1
  • 11. Validate/Create Tests • Tests assure (mostly) predictable behavior • Determine what kinds of tests you need - unit, acceptance, integration, other? • Good testing setup in puppet-module-skeleton • puppet-retrospec generates naive tests that need tuned • Existing tests must pass before modifying code • Turn on Future Parser [and Strict Variables] only at 3.8.x • Use puppet-lint community plugins, esp for v4 transition • Beyond tests: catalog diffs, personalized tests
  • 12. Rspec Tests $ cat spec/classes/apache_spec.rb require 'spec_helper' describe 'profile::apache', :type => :class do let :facts do { facts_hash } end context 'with defaults for all parameters' do it { is_expected.to create_class('profile::apache') } it { is_expected.to contain_package('httpd') } it { is_expected.to contain_user("apache") } end end
  • 13. Rspec Run [rnelson0@build03 profile:production]$ bundle exec rspec spec/classes/apache_spec.rb profile::apache with defaults for all parameters should contain Class[profile::apache] should contain Package[httpd] should contain User[apache] Finished in 7.82 seconds (files took 2.49 seconds to load) 3 examples, 0 failures
  • 14. Refactor • Create a new branch for the target version, e.g. 3.8.7 • Test against current and target versions, e.g. ~>3.6.0 and ~>3.8.0, with and without Future Parser/Strict Variables • Identify failing tests, refactor to fix • Upgrade modules as early as possible. Be aware of the required Puppet version for a module version, and look out for defunct or migrated modules, such as those transferred to Vox Pupuli • Move forward when tests are green for the next version – previous may be red
  • 15. Testing with particular Puppet versions $ grep PUPPET Gemfile gem "puppet", ENV['PUPPET_GEM_VERSION'] || '~> 4.0' [rnelson0@build controlrepo]$ export PUPPET_GEM_VERSION='~>3.8.0' [rnelson0@build controlrepo]$ bundle update Installing puppet 3.8.7 (was 4.6.0) [rnelson0@build controlrepo]$ bundle exec puppet --version 3.8.7 [rnelson0@build controlrepo]$ export PUPPET_GEM_VERSION='3.8.1' [rnelson0@build controlrepo]$ bundle update Installing puppet 3.8.1 (was 3.8.7) [rnelson0@build controlrepo]$ bundle exec puppet --version 3.8.1
  • 16. High level Master(s) upgrade process • Prep for new master/in-place upgrade • Deploy new/upgrade in testing • Revert • Deploy new/upgrade in production • Start with Master of Masters or other “parent” nodes first • Update separate PuppetDB node, puppetdb-termini on masters
  • 17. Replace the Master • Prepare a new operational environment • Do not serve bad/incorrect catalogs to existing nodes • Deploy a new master on the target puppet version • Bootstrap configuration/code • Test the master against itself, puppet agent -t • Deploy and test canary nodes in the same operational environment
  • 18. In-place Master upgrade • Snapshot (or equivalent) the master(s) and canary nodes • Restrict access to the master: • Control access with firewall/load balancer • Disable puppet agent on nodes with orchestration • Revoke certificates for non-canary nodes • Revoke the CA, generate a new CA and new agent certs for canary nodes only • Upgrade the master • Test the master then canary nodes with puppet agent -t
  • 19. Troubleshooting • Collect logs from the master and canaries • Look for changed resources, number of resources in catalog • Revert production environment • Analyze cause(s) • Refactor code and data to address issues • Try again • Learn from failures, prevent them in the future
  • 20. Upgrade the Agents • Can often skip on PATCH versions and some MINOR versions (see rel notes) • puppetlabs/puppet_agent (requirements) updates agents on next check-in • Orchestration • Replace nodes with new instances running the new agent • By hand
  • 21. Repeat • Relax, enjoy the success of an upgrade! • Start working on the next version/feature flags • Repeat the Refactor / Upgrade steps
  • 23. Keeping Up Refactor to take advantage of Puppet 4 language improvements, new tools (ex: r10k -> PE Code Manager), new file locations, etc. • PE has quarterly upgrades, FOSS more frequent • The less frequently you do something, the more painful it is. “Upgrade early and upgrade often!” • Try not to get more than 2 MINORs behind • Test against puppet version ~>4.0 (latest v4) and run bundle update before manual tests
  • 24. Puppet 4 Language Improvements • Replace create_resources() with iteration • Replace validate_*() with data types (including a Sensitive type) • There is a validate_legacy() helper function available in puppetlabs/stdlib to assist with replacing validate_*() functions (blog) • Simplified resource wrappers with * and + operators • Improved default attributes are per-expression • New template type EPP is available • Puppet Lookup, Data In Modules, and other hiera improvements • Use $facts[] instead of global variables to tidy up the namespace and remove ambiguity
  • 25. Tips & Tricks – Puppet Enterprise • PE includes support, use it for planning/errors • Puppet Enterprise Upgrade Service to engage Pro Services • PE Classifier changes over time. Review Preconfigured Node Groups documentation • pe_puppetserver_gem is out, puppetserver_gem is in • Do not use PE’s bundled Ruby for other Ruby tasks, conflicts between bundled/downloaded gems. Recommend rbenv/rvm or SCL-equiv instead • Do not ever do this on your master. EVER!
  • 26. Tips & Tricks - Strings Understand how string conversion works in puppet, hiera, rspec-puppet, and how it has changed: • rspec-puppet: 'undef' represents an undefined value • Puppet DSL: it is the string undef! Try :undef, without quotes, instead • If you have a file resource with a title or path of ${undefvar}/${populatedvar}, rspec will start failing because file { 'undef/etc/app.conf' :} is not valid • Similar issue with 'true' vs true and 'false' vs false • Other common issues: input from hiera/ENC, quoted numbers as strings, stringify vs structured facts, unquoted strings in case selectors, etc • May require acceptance tests/canary nodes to become apparent
  • 27. Tips & Tricks - Hiera • Hiera eyaml gem is lost during the upgrade to the 4.x puppetserver • Enable the yaml backend and ensure that the master does not rely on eyaml data • Run the agent on the master to redeploy the gem (with puppet/hiera or similar) before agents check in • %{}: used to prevent variable interpolation, as in %%{}{environment} to generate the string %{environment}. In 3.x and in 4.5 resolves to an empty string, in 4.0-4.4 it returned the scope, giving strings like %<#Hiera:7329A802#>{environment}. Use %{::} instead, as in %%{::}{environment}. Affects PE < 2016.2.0 • datadir: some versions expect :: prepends to variables and others do not. Change %{environment} to %{::environment}. Likely PE < 2016.2.0 as well
  • 28. Tips & Tricks - Other • Review modules and their supported versions. May be incorrect or weak assumptions (>= 3 but should also include < 4 – check tests) • Upgrades across major versions mean additional troubleshooting • Upgrade early – but with caveats • Many tools to assist with automating version upgrades in your Puppetfile • ERB scope: prepend most variables with @ (<%= var %> to <%= @var %>) • Script to detect usage of hardcoded /etc/puppet paths, no longer correct in v4 • External fact weighting bug: FACT-1413 • Minimize coupled/entangled changes • Ask for help! Colleagues, social media, etc.
  • 29. Tools • Puppet Community Slack / IRC and Mailing Lists • puppet-ghostbuster helps you find "dead code" that you may want to prune before you start on your refactoring journey. • rspec-puppet, puppetlabs_spec_helper, and puppet-lint are improving their Puppet 4 support • A number of catalog diff tools exist (diff generators and a viewer) to inspect the actual catalog differences from active nodes across different versions of Puppet.
  • 30. Links Additional information on Puppet 4 and Migrations • Official Puppet Upgrade Docs • Whirlwind Tour of Puppet 4 by R.I. Pienaar • The Power of Puppet 4 by Martin Alfke • Puppet - our journey from Puppet 3.8 to Puppet 4 by Jonas Genannt
  • 31. Summary • Plan the upgrade roadmap • Have working tests before upgrading • Step through the new versions / feature flags • Refactor code to take advantage of the language and tool improvements • Keep mowing • Enjoy the journey!

Editor's Notes

  1. Tons of talks on Puppet 4 language here at PuppetConf, check them out! Puppetserver is the future and Ruby moves fast, take advantage of the AIO builds when possible – improved support and security, too Other reasons: better performance, packaging and security updates, same agent between FOSS and PE.
  2. If you are on Puppet 2.x, you need to get to 3.x first! But that is not this talk. There were some good talks about this at PuppetConf 2015. If you are actually starting new - start with Puppet 4!
  3. When you upgrade, you could affect data feeds into other systems, which could generate extra tickets or result in some data loss, at least during the upgrade. Add it to your external test regimen – is the SEIM still collecting the logs you would expect, is dynamic monitoring showing the new devices, etc?
  4. Testing talks: "Turning Pain Into Gain" today at 2:30PM and "The Future of Testing Puppet Code" tomorrow at 3:45PM. Tests do not guarantee success, but can identify many regressions and determine when failure is guaranteed. Future parser existed in earlier versions but had some bugs, wait till you hit 3.8. Catalog diffs can be useful for many people, but not all – see the talk "Getting to the Latest Puppet" tomorrow at 1PM for more information. Don’t go overboard, just find what works and improve it over time.
  5. This is a test on a profile module, not a component module, but this is absolutely needed! This is a majority of the code you are going to write and it helps with your design phase if you write them first.
  6. There’s a lot of different ways to upgrade – a new master with the same IP/DNS as the original, an in-place upgrade, a brand new master with a new CA structure. You’ll have to determine which is best for your architecture. As usual, there is no right answer, only what works for you. Remember that --noop and canary tests will send reports to puppetdb. If your puppetdb and puppet master services are on different nodes, work on them together.
  7. 'Environment' is such an overloaded term. What I mean by 'operational environment' is the set of proper network, host, etc. where the new master will reside. It should NOT be in the same operational environment as the existing master, so that there is no way existing agents could check in to the new master. Maybe use DNS to ensure the default server name puppet uses doesn’t get confused between the two. Of course, it is fairly privileged to assume you can do this. If you do not have that luxury, do your best to ensure old and new systems do not mix. There are many ways to bootstrap your master. If you do not have a process to bootstrap your master, we will talk about in-place upgrades shortly. You will eventually want the ability to bootstrap, though.
  8. There is no right way, just a way that works for you!
  9. Recently, Puppet changed their GPG signing key. Puppet has details to fix that at https://puppet.com/blog/updated-puppet-gpg-signing-key
  10. EPP = Embedded Puppet Template. In addition to writing templates in the puppet DSL that you already know, it addresses some of the scoping issues that were common problems in the ruby-based ERB. You can use both template types just fine, however.
  11. Now that the generalities are out of the way, we can talk specifics and lessons learned. I know many of us do not like to engage support, but if you do not use it for updates, when will you use it? And if you need someone to do the upgrade, not just assistance, that is an option now as well. Using the PE/AIO ruby can cause unexpected conflicts between bundled gems and downloaded gems. See PUP-6106 for an example.
  12. Truthiness is a difficult concept sometimes, it’s worth gaining a deeper understanding of it. One suggestion is to look for active resources and resource counts on reports to indicate where these may be issues with its interpretation. Stringify vs structured facts: Foreman only received the ability to natively process structured facts in its latest release, 1.13.
  13. Anecdata suggests that sometimes eyaml gem is lost on 4.x upgrades, though docs say it should not Be sure the hiera.yaml file gets put in the right location, $confdir or $codedir, depending on your puppet version – modules handle this for you
  14. "Trust but verify" support levels. You should try and upgrade modules ASAP, but there are good reasons not to. Newer versions may require Puppet 4 before you get there, or a newer OS, or deploy a newer version of an application you are not ready for. This is a VERY rough guideline. If you use external facts in /etc/facter/facts.d, you may expect them to override calculated facts. The facter that addresses the issue has not shipped as of puppet-agent 1.7.0. You are going to be making a lot of changes at once, but minimize coupled or entangled changes. A week of small changes beats trying to do it all at once, whcih will likely be longer when including troubleshooting time.
  15. Slack and IRC are great ways to engage the vendor as well as other community members – ask difficult questions or just get another pair of eyes on something before you commit it. Tools are constantly improving, check back frequently to see if your concerns are addressed.