Story of our migration to Puppet 5, as presented with Lander Van den Bulcke at the Open Source Camp on Puppet last week in Nuremberg

1. The road to Puppet 5 6
Kris Buytaert &
Lander Van den Bulcke
@krisbuytaert
@landervdb

2. Kris BuytaertKris Buytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● 10+ years of puppet10+ years of puppet
● CTO and Open Source ConsultantCTO and Open Source Consultant
@@inuits.euinuits.eu
● Everything is a freaking DNS ProblemEverything is a freaking DNS Problem
● Evangelizing devopsEvangelizing devops
● Organiser of #devopsdays,Organiser of #devopsdays,
#cfgmgmtcamp, #loadays, ….#cfgmgmtcamp, #loadays, ….

3. Lander Van den BulckeLander Van den Bulcke
● OpsOps
● Open Source Consultant @ Inuits.euOpen Source Consultant @ Inuits.eu
● Mainly focusing on infrastructure as code,Mainly focusing on infrastructure as code,
automation, monitoringautomation, monitoring

4. Why Migrate to Puppet n+1Why Migrate to Puppet n+1
● 201X : nope , there is no business value201X : nope , there is no business value

5. Why Migrate to Puppet n+1Why Migrate to Puppet n+1
● 2015 : nope , there is no business value2015 : nope , there is no business value
•
& we need foreman& we need foreman

6. Why Migrate to Puppet n+1Why Migrate to Puppet n+1
● 2015 : nope , there is no business value2015 : nope , there is no business value
● 2016 :2016 :
•
nope, still no business value, but 3.X is EOL ,nope, still no business value, but 3.X is EOL ,
people are thinkingpeople are thinking

7. Why Migrate to Puppet 4Why Migrate to Puppet 4
● 2015 : nope , there is no business value2015 : nope , there is no business value
● 2016 :2016 :
•
nope, still no business value, but 3.X is EOL ,nope, still no business value, but 3.X is EOL ,
people are thinkingpeople are thinking
•
1 customer considering1 customer considering
•
0 customers actually migrating0 customers actually migrating

8. Why Migrate to Puppet 4Why Migrate to Puppet 4
● 2016 :2016 :
•
Vox Pupuli movesVox Pupuli moves
forwardforward
•
@daenny writes@daenny writes
“Goodbye, Puppet”“Goodbye, Puppet”
Puppet consideredPuppet considered
Feature CompleteFeature Complete

9. Why Migrate to Puppet 5Why Migrate to Puppet 5
● 2017 :2017 :
•
fsck .. upstream breaks on Puppet 3,fsck .. upstream breaks on Puppet 3,
•
Both puppetlabs and VoxBoth puppetlabs and Vox
•
We’re pinning oour versionsWe’re pinning oour versions

10. Why Migrate to Puppet 5Why Migrate to Puppet 5
● 2017 :2017 :
•
fsck .. upstream breaks on Puppet 3,fsck .. upstream breaks on Puppet 3,
•
Both puppetlabs and VoxBoth puppetlabs and Vox
•
We’re pinning our versionsWe’re pinning our versions
•
Hmm.. this Choria thing looks coolHmm.. this Choria thing looks cool
•
Vault + Hiera integrationVault + Hiera integration

11. Why Migrate to Puppet 5Why Migrate to Puppet 5
● 2015 : nope , there is no business value2015 : nope , there is no business value
● 2016 : nope2016 : nope
● 2017 : nope2017 : nope
● 2018 : hmm. We need support for XYZ and2018 : hmm. We need support for XYZ and
that’s in the upstream module in a puppet 5that’s in the upstream module in a puppet 5
version.version.
Let’s migrate...Let’s migrate...

12. State in 2018-09State in 2018-09
● 75% of our consulting customer base is still on75% of our consulting customer base is still on
3.X with no intent of migrating their code base3.X with no intent of migrating their code base
● New projects are in Puppet 5New projects are in Puppet 5
● Maintaining 2 stacks.Maintaining 2 stacks.
● Some have moved to AnsibleSome have moved to Ansible
● This hot new Container thingie .. K8sThis hot new Container thingie .. K8s

13. Our internal EcoSystemOur internal EcoSystem
● 20+ platforms20+ platforms
● (dev)/(uat)/prod(a,b)(dev)/(uat)/prod(a,b)
● +/- 60 puppet environments+/- 60 puppet environments
● Continuous Delivery of IaC !!!Continuous Delivery of IaC !!!
● Pipeline as CodePipeline as Code
● 3.8 fte3.8 fte

14. Our Trees:Our Trees:
├──├── environment.confenvironment.conf
├──├── manifestsmanifests
│ ├──│ ├── nodesnodes
│ └──│ └── site.ppsite.pp
├──├── modulesmodules
├── ├── internalinternal
└── └── upstreamupstream
Git submodules for release managementGit submodules for release management

17. Continuous DeliveryContinuous Delivery

18. 2 years earlier2 years earlier
● puppet-tree-validate runs in a containerpuppet-tree-validate runs in a container
● Multiple versions get testedMultiple versions get tested
● Non breaking on the not used versionNon breaking on the not used version
● Testing on n, n+1 n+2Testing on n, n+1 n+2

19. Spinning up a P5 stackSpinning up a P5 stack
● Fresh Puppet MasterFresh Puppet Master
•
CA,CA,
•
ForemanForeman
•
PuppetDBPuppetDB
•
ChoriaChoria

20. Everything is aEverything is a
Freaking DNS problemFreaking DNS problem
Fun with srv recordsFun with srv records

21. A new PuppetDB ?A new PuppetDB ?
● But we heavily use(d) Exported ResourcesBut we heavily use(d) Exported Resources
•
IcingaIcinga
•
Reverse proxiesReverse proxies
•
Database CreationsDatabase Creations

22. Making Bernd HappyMaking Bernd Happy
● Puppet 5 stack => Icinga 2 migrationPuppet 5 stack => Icinga 2 migration

23. Clean TreeClean Tree
● For module in modules/upstream/For module in modules/upstream/
s/currenthash/master/g;s/currenthash/master/g;
● Break all the thingsBreak all the things
•
Bug in sudo : %%group vs %groupBug in sudo : %%group vs %group
•
......
•

24. Naming ThingsNaming Things
● How do you call the puppet5 version of theHow do you call the puppet5 version of the
environment currently known asenvironment currently known as
mgmtdev,mgmtprod ?mgmtdev,mgmtprod ?
mgmtdev5,mgmtprod5 ?mgmtdev5,mgmtprod5 ?

25. Puppet DB APIPuppet DB API
● Inventory views are based on puppetdbInventory views are based on puppetdb

26. Puppet DB APIPuppet DB API
● Inventory views are based on puppetdbInventory views are based on puppetdb

27. MVPMVP
● WebWeb
● DbDb
● LbLb
● Icinga2Icinga2

28. Hiera + VaultHiera + Vault
● We reverted our integration in P3 becaus it wasWe reverted our integration in P3 becaus it was
under performing (blocking)under performing (blocking)
● Jruby 9K issuesJruby 9K issues
● Performance issues still not solved :(Performance issues still not solved :(

29. Hiera + VaultHiera + Vault
Source: Joe Pinsonault – https://tickets.puppetlabs.com/browse/SERVER-1864

30. Hiera + VaultHiera + Vault
Source: Joe Pinsonault – https://tickets.puppetlabs.com/browse/SERVER-1864

31. Current stateCurrent state
● All new Roles/Profles we build are P5 onlyAll new Roles/Profles we build are P5 only
● Some stacks are mixed P5 , P3Some stacks are mixed P5 , P3
● We’re not actually migrating yet.We’re not actually migrating yet.
● We’re onboarding new stacks in the newWe’re onboarding new stacks in the new
ecosystemecosystem

32. Next StepsNext Steps
● Take Recent stack that ft in the MVP rolesTake Recent stack that ft in the MVP roles
•
Migrate them to P5Migrate them to P5
● No efort planned on EL6 puppet codeNo efort planned on EL6 puppet code
● Rebuild EL6 stacks on EL{7,8} with newRebuild EL6 stacks on EL{7,8} with new
Roles/ProflesRoles/Profles

33. One more thingOne more thing
Confg Management Camp 2019Confg Management Camp 2019
4-6 february 20194-6 february 2019
Gent , BelgiumGent , Belgium
https://cfp.cfgmgmtcamp.be/https://cfp.cfgmgmtcamp.be/

34. ContactContact
Kris Buytaert Kris.Buytaert@inuits.euKris Buytaert Kris.Buytaert@inuits.eu
Further ReadingFurther Reading
@krisbuytaert@krisbuytaert
http://www.krisbuytaert.be/blog/http://www.krisbuytaert.be/blog/
http://www.inuits.eu/http://www.inuits.eu/
Inuits.euInuits.eu
Essensteenweg 31Essensteenweg 31
BrasschaatBrasschaat
BelgiumBelgium
891.514.231891.514.231
+32 475 961221+32 475 961221