A look at how to integrate Puppet into larger projects, co-ordinate with external systems, and develop end-to-end workflows and deployment pipelines.

1. Puppet in the Pipeline
Anna Kennedy

2. ● Basic workflow
● End-to-end workflows
● Example workflows
● Integration points
● Planning your workflow

3. Basic workflow:
deploy code from the master to a node

4. PuppetMaster
agent
VM
code
PuppetMaster
The very first setup you ever did

5. code
VC repo
PuppetMaster
agent
VM
Maybe we should use version control?

6. code
VC repo
PuppetMaster
agent
VM
● rsync
● post-commit hook
● r10k
How do we get the code onto the master?

7. r10k
Git repo
module1
production branch
development branch
module2
production branch
development branch
Puppet Master
$codedir/environments
/production
/module1
/module2
/development
/module1
/module2

8. r10k
SVN repo
module1
trunk
branches/development
module2
trunk
branches/development
Puppet Master
$codedir/environments
/production
/module1
/module2
/development
/module1
/module2

9. prod
staging
dev
VC repo
PuppetMaster
agent
VM
Using git branches as environments

10. prod
staging
dev
VC repo
PuppetMaster
agent
VM
● merge
● manual
● review
● testing
Deploying to environments sequentially
- should this be manual or automatic?

11. prod
staging
dev
VC repo
PuppetMaster
agent
VM
The Maintenance Workflow

12. End-to-end workflow:
set up, install, and configure the node

13. code
git repo
VM
request
PuppetMaster
VM created
OS +
puppet
installed
Connected to
PuppetMaster
Installed and
running
VM ready
End-to-end workflow

14. End-to-end workflow: technical options

15. VM
request
code
git repo
PuppetMaster
● Verbal
● Email
● Form
● Web page

16. VM
request
VM created
code
git repo
PuppetMaster
● VirtualBox
● Vagrant
● Docker
● VMware
● OpenStack

17. VM
request
VM created
OS +
puppet
installed
code
git repo
PuppetMaster
● manual
● golden image
● PXE boot
● kickstart /
preseed /
jumpstart /
...

18. code
git repo
VM
request
PuppetMaster
VM created
OS +
puppet
installed
Connected to
PuppetMaster
Certificate exchange
● Command line
● API
● Autosign
● Presigned

19. Certificate exchange
VM
PuppetMaster
puppet agent run
generates CSR
puppet cert sign <vm>
signs certificate
● command line
Signed certificate

20. Certificate exchange
VMPuppetMaster
puppet agent run
generates CSRcurl --cert my-cert.pem
--key my-private.pem
--cacert ca.pem
-X PUT
-H "Content-Type: text/pson"
--data
'{"desired_state":"signed"}'
https://puppetmaster.vm:8140/pro
duction/certificate_status/<vm>
● api
Signed certificate

21. Certificate exchange
VM
PuppetMaster
puppet agent run
generates CSR
● autosign
autosign
whitelist or policy based
Signed certificate

22. Certificate exchange
VM
PuppetMaster
● pre-signed
puppetca --generate <vm>
ssl/certs/<vm>.pem
ssl/certs/ca.pem
ssl/private_keys/<vm>.pem
Signed certificate

23. code
git repo
VM
request
PuppetMaster
VM created
OS +
puppet
installed
Connected to
PuppetMaster
Installed and
running
Node classification
● site.pp
● PE console
● Hiera
● ENC
● Node classifier API

24. ● site.pp
Node classification
node ‘web01.example.com’ {
include role::web
}
node ‘*.example.com’ {
include role::basic
}

25. Node classification
Classify based on facts:
- node name
- os
- kernel
- custom
- etc etc
● PE console

26. ● Hiera
Node classification
site.pp
hiera_include(‘classes’)
hiera.yaml
:hierarchy:
- “node/%{::fqdn}”
- “osfamily/%{::osfamily}
”
- common
debian.yaml
---
classes:
- repos::apt
common.yaml
---
classes:
- base

27. ● ENC
(external node
classifier)
Node classification
= an executable that
can be called by the
PuppetMaster
Returns yaml hash
- classes
- parameters
- environment
Takes one
parameter:
node FQDN

28. Node classification
curl --cert myserver-cert.pem
--key myserver-private.pem
--cacert ca.pem
-H "Content-Type: application/json"
https://puppetmaster.vm:4433/classifier-api/v1/groups -d
'
{
"name": "testnode.vm",
"environment": "testing",
"parent": "00000000-0000-4000-8000-000000000000",
"classes": {},
"rule": [
"or",
[
"=",
"name",
"testnode.vm"
]
]
}'
● Node
classifier API

29. An API interlude
APIs exist for:
● Puppet Master (Server)
● Certificate Authority
● PuppetDB
● Console

30. An API interlude
Step 1: Authentication of the requesting server
Need:
an ssl certificate for myserver (--cert)
the private key for the certificate (--key)
the CA certificate of the master (--cacert)
Do:
puppet cert generate myserver.vm
Copy from master to myserver:
ssl/certs/myserver.pem
ssl/private_keys/myserver.pem
ssl/certs/ca.pem

31. An API interlude
Step 2: Configuration of API service
● Puppet Master / Server / CA:
Edit auth.conf:
path /puppet/v3/status
method find
allow *
path /puppet-ca/v1/certificate_status
method find, save
auth any
allow myserver
Edit ca.conf:
certificate-authority: {
certificate-status: {
client-whitelist:
[myserver]
}
}
}

32. An API interlude
Step 2: Configuration of API service
● PuppetDB:
Edit: certificate-whitelist (optional)
myserver

33. An API interlude
Step 2: Configuration of API service
● Puppet console:
Edit: rbac-certificate-whitelist
myserver

34. An API interlude
Step 3:

35. Don’t mix classification types!

36. Don’t mix classification types!
If you mix classification types
MAKE SURE YOU KNOW WHAT YOU’RE DOING

37. Example workflows

38. code
git repo
VM
request
PuppetMaster
VM created
OS +
puppet
installed
Connected to
PuppetMaster
Installed and
running
VM ready
End-to-end
workflow

39. code
git repo
VM
request
PuppetMaster
VM created
OS +
puppet
installed
Connected to
PuppetMaster
Installed and
running
VM ready
unit
tests
acceptanc
e tests
Workflow
with testing

40. code
git repo
PuppetMaster
VM created
OS +
puppet
installed
Connected to
PuppetMaster
Installed and
running
Output
test
results
unit
tests
acceptanc
e tests
Workflow
for testing
VM created
VM created
OS +
puppet
installed
OS +
puppet
installed
Connected to
PuppetMaster
Connected to
PuppetMaster
Installed and
running
Installed and
running
Workflow
FOR testing
Different OSs

41. code
git repo
Load
increas
e -> VM
request
PuppetMaster
Connected to
PuppetMaster
Installed and
running
VM put into
production
VM created
from golden
image
Rapid-scaling
workflow

42. Planning your workflow

43. Planning your workflow
Need to integrate
with legacy systems?

44. Planning your workflow
What timescales are normal for you?
“ we need rapid
scalability
to cope with load - we
create and destroy
tens of VMs a day”
“ we spin up new
machines less than
once a month, and they
remain in service for
years “

45. Planning your workflow
What do your end users look like?

46. Planning your workflow
What do your internal users look like?

47. Planning your workflow
What does your development process look like?

48. Summing up

49. Node classification
● site.pp
● PE console
● Hiera
● ENC
● APIs
Certificate exchange
● Command line
● API
● Autosign
● Presigned
● manual
● golden image
● PXE boot
● kickstart / preseed /
jumpstart
● rsync
● post-commit hook
● r10kcode
git repo
PuppetMaster
VM created
OS +
puppet
installed
Connected to
PuppetMaster
Installed and
running
VM ready
● Verbal
● Email
● Form
● Web page
VM
request
Integration points

50. Consideration points:
●uncontrollable factors
●legacy systems
●timescales
●internal and external users
●development process