The document summarizes the process of rebuilding Puppet infrastructure at Adform, including moving from a monolithic setup to a more modular one using roles, profiles, and Hiera for data storage. Key changes included adopting version control and testing for Puppet code, implementing a CI/CD pipeline, and using tools like Vagrant and Puppet Explorer for local development and monitoring production nodes. The rebuilt infrastructure followed Puppet best practices for code organization, reuse, and separation of configuration data from logic.

1. The story of rebuilding
Puppet
October 2015
1
Artem Zhurbila
artemzhurbilo@gmail.com

2. 2
2013 ... Old Puppet infra
New Puppet infra
June 2015
May 2015
Joins Adform

3. Why new puppet infra?
3

4. Old Puppet infra New Puppet infra
One giant repository with all modules One core repository (puppet-control) + independant modules
Public modules are downloaded once and never updated Modules and theirs versions are controled in Puppetfile
Code duplication Profiles / Roles pattern encourage to reuse modules
No consistent style Lint, style checking is enforced
No possibility to test puppet manifests locally Vagrant box can be used to run and provision VMs easily on
a local machine
No unit tests Unit tests are supported (rake spec)
No acceptance tests Acceptance tests are supported (rake acceptance)
No clear branching model All new development starts by feature-branching which is
merged only after code review
No visibility Puppet Explorer dashboard show status of all nodes
4

5. Puppet 4 bonuses
❏ Puppet Server -> clojure (JVM)
❏ Environments approach by default
❏ More clear naming rules
❏ New language features
❏ Functions (each, map, filter, map, reduce)
❏ Types (Integer, Boolean, Enum, Collection, etc)
5

6. Vagrant boxes
6

7. Box versioning
7

8. Puppetexplorer
8

9. CI pipeline
.gitlab-ci.yml
localtest:
script:
- rake localtest
check-Puppetfile:
script:
- ruby -c Puppetfile
9
task :localtest => [
:syntax,
:lint,
:yamllint,
:jsonlint,
:metadata,
]

10. Acceptance tests
10

11. Puppet modules
11

12. Puppetfile
12
#!/usr/bin/env ruby
#^syntax detection
forge "https://forgeapi.puppetlabs.com"
# A module from the Puppet Forge
mod 'puppetlabs/stdlib', '4.6.0'
mod 'puppetlabs/java', '1.4.1'
mod 'cyberious/windows_java', '1.0.2'
mod 'puppetlabs/concat', '1.2.3'
mod 'stahnma/epel', '1.0.2'
mod 'puppetlabs/firewall', '1.6.0'
mod 'puppetlabs/ntp', '4.1.0'
...
...

13. Modules deploy
13
Advantage:
Optimized caching
Disadvantage:
No dependency resolution

14. Puppet-control repo
14

15. env VS tier
branch == environment
tier == [dev | preprod | prod | infra]
main branch is the master (trunk) branch which should always be in a stable state!
production branch is a default Puppet environment and it’s used only for initial Puppet agent
bootstrap.
15

16. Profiles / Roles pattern
16

17. Profiles / Roles pattern
17

18. Profiles / Roles pattern
1 node includes 1 role
1 role includes >= 1 profiles
18

19. Hieradata
19

20. hiera.yaml
20
...
:hierarchy:
- "nodes/%{::fqdn}"
- "tiers/%{::tier}/host_group/%{hostname_group}"
- "tiers/%{::tier}/%{calling_class_path}"
- "tiers/%{::tier}/common"
- "default/host_group/%{hostname_group}"
- "default/%{calling_class_path}"
- "default/common"
...

21. Hiera data
• Separate data from logic: move data to Hiera as much as
possible
• Attribute names should match your puppet class name.
E.g for 'profiles::elk::server' class, all attributes in hiera
should start with 'profiles::elk::server::
[ATTRIBUTE_NAME]'
21

22. Puppet-module-skeleton
repo
22

23. Puppet agent maintenance
mode
23

24. Maintenance mode
#puppet agent --disable 'FOOBAR maintenance for 6hrs;
contact @it.solution for any reason'
Puppet Explorer dashboard has a cell which shows node
count, which haven’t reported in last 24 hours
24

25. Puppet NOOP (dry-run) mode
25

26. NOOP
NOOP is enabled only in PROD env by default
$ puppet agent -t --no-noop.
This will run Puppet agent with disabled noop mode and
actually do the changes
26

27. The big picture - Puppet
27

28. 28

29. Recap:
- Focus on testing code changes early and often
- Reusing as much as we can
29

30. 30