The document discusses securing secrets for Puppet infrastructure without slowing down automation. It acknowledges that many organizations manually copy secrets or store them insecurely due to time pressures. However, exploits are increasingly targeting insecure secrets. The document proposes using the open source CyberArk Conjur tool to authenticate and authorize machine identities to retrieve secrets in a secure workflow. It provides an example of updating Puppet manifests to use Conjur for dynamic secrets retrieval without overhauling existing code. Overall, the document advocates aligning security and velocity through identity-based secrets management to reduce risks and costs from security incidents.

1. Securing Secrets for Puppet,
without interrupting flow
Jody Hunt, Global SE Director
Ryan Prior, Software Engineer
October 11, 2017

2. How’s the automation going for you?
▪ How many of you are currently figuring out how
to properly secure your infrastructure?
▪ Are you concerned that making things more
secure will slow everything down?
▪ Be honest: have you ever committed any
security sins in the name of DevOps?

3. Confession time. Haven’t we all, at least once…
▪ Manually copied SSH keys to servers to provide access?
▪ Manually configured or changed production servers?
▪ Stored secrets in config files in S3?
▪ Stored secrets in source control?
▪ Embedded passwords in applications?
▪ Who thinks this might lead to problems?

4. Exploits are targeting the application supply chain
Database containing driver PII was compromised after
storing keys in a publicly available repository (May 2014)
XcodeGhost hack uses compiler backdoor to inject 3rd party
code into developed applications (September 2015)
Hacker accessed a Docker registry that contained the entire
Vine source code, API keys, and other secrets (July 2016)
Hackers have exploited known MongoDB vulnerabilities to
plant ransomware into high-profile clients such as Emory
Healthcare (January 2017)

5. Perspectives Matter
▪ Traditional view of security:
￭ A bunch of suits preventing my killer app
from going to prod
▪ An essential view of security:
￭ A thing that helps prevent the destruction of
my employer, my job, my information

6. Velocity vs. Security success criteria
▪ “We pushed 40% more times to prod this month!”
▪ “We….umm….didn’t get hacked?”

7. Why “we” care about security and automation
DevSecOps business benefits:
￭ Increase the velocity of a team (better efficiency)
￭ Reduce workflow blockers (better efficiency)
￭ Reduce operational workload (better efficiency)
￭ Reduce probability of a security event (reduce potential organization cost)
￭ Reduce impact if/when a network is exploited (reduce potential organization cost)

8. Some not so good practices (anti patterns)
▪ Storing secrets in source code (embedded secrets)
▪ Giving applications access to more secrets than they need
(overprivilege)
▪ Ignoring auditing!
￭ No audit trail, no proof it did, or didn’t, happen
￭ That admin you trust so much is as susceptible as anything
▪ Doing things manually [... but it’s Friday night and I’m ready for the weekend :’( ]
￭ Manually configuring/changing production servers
￭ Manually copying SSH keys to servers to provide access
￭ Not reproducible, not reliable at scale
S

9. Some good practices
• Encrypt secrets
• Rotate secrets
• Make secrets ephemeral
• Authenticate all requests (Zero Trust)
• Authorize minimally (Least Privilege)
• Audit everything
• Automate all of the above

10. Some better practices (Nirvana state)
▪ Workflows that support velocity and enhance security at the same time
▪ Security configuration is part of the development flow - i.e. in the same
code repos as the applications - otherwise security and code get out of sync
and deploys fail
▪ Security tools run separately from developer tools – to isolate secrets
management from application code

11. How do these ideas apply to Puppet?
Puppet
Master
Secret
Store
Staging
App 1
App 2
App 3
Production
App 1
App 2
App 3

12. Example CI workflow w/ GitHub, Jenkins & Docker
SECRETS
REQUIRED!!
SECRETS
REQUIRED!!
SECRETS
REQUIRED!!
SECRETS
REQUIRED!!

13. The Solution
+
Human Identity

14. If it can be identified, its access can be managed
▪ Build on a chain of trust
▪ Authenticate all requests
▪ Authorize w/ least amount of privilege
▪ Audit everything
▪ ...and do it with code!

15. CyberArk-Conjur: Open Source Secrets Management

16. CyberArk-Conjur Puppet Module

17. Machine Identity with Conjur
PUPPET
MASTER
Puppet Admin
Node Node
Configuration
Puppet
Agent
Puppet
Agent
Node
Puppet
Agent
TEAM 1 TEAM NTEAM 2
CyberArk-Conjur
Conjur Module
Node obtains a
Conjur access
token, encrypts it,
and places it in the
“facts”
• The Puppet master uses a Node’s
identity to fetch secrets via the Conjur
module.
• A Node’s access to Secrets is defined
in declarative Policies

18. Secure workflow w/ Machine Identity

19. Puppet Manifests before
################## Site.pp ##################
node 'default' {
require secrets
file { '/tmp/dbpass':
ensure => file,
content => "${secrets::postgres_password.unwrap}",
show_diff => false, # don't log file content
}
file { '/tmp/token':
ensure => file,
content => "${secrets::vendor_oauth_token.unwrap}",
show_diff => false,
}
################## Secrets.pp ##################
class secrets {
$vendor_oauth_token = Sensitive('5262e7a7-4cfd-479d-
833c-cbace96a32d3')
$postgres_password = Sensitive('wake bad operator pure')
}

20. Puppet Manifests using Conjur
################## Site.pp ##################
node 'default' {
require secrets
file { '/tmp/dbpass':
ensure => file,
content => "${secrets::postgres_password.unwrap}",
show_diff => false, # don't log file content
}
file { '/tmp/token':
ensure => file,
content => "${secrets::vendor_oauth_token.unwrap}",
show_diff => false,
}
################## Secrets.pp ##################
class { 'conjur':
account => 'demo',
appliance_url => 'http://conjur',
authn_login => "host/app-${::trusted['hostname']}",
host_factory_token => Sensitive('placeholder-for-HF-token'),
version => 5,
}
class secrets {
$vendor_oauth_token = conjur::secret('app/vendor-oauth-
token')
$postgres_password = conjur::secret('app/postgres-
password')
}

21. Conjur Policy that controls Secrets access
- !policy
id: app
annotations:
description: Conjur Puppet demo app policy
body:
# Roles
- !layer app
- !host-factory
annotations:
description: factory for new app node identities
layers: [ !layer app ]
# Secrets
- !variable
id: vendor-oauth-token
annotations:
description: authenticate vendor access to service endpoint
- !variable
id: postgres-password
annotations:
description: login credential for app database
# Entitlements
- !group users
- !permit
role: !group users
privileges: [ read, execute ]
resources: [ !variable vendor-oauth-token, !variable postgres-password ]
- !grant
role: !group users
members:
- !layer app

22. Demo

23. Tradeoffs to using identity to retrieve secrets...
▪ Pros:
￭ Eliminates overprivileged central attack target
￭ Enables fine grained control of secret retrieval & updating
￭ Secrets are dynamically retrieved without writing a lot of code
￭ Enables teams to self manage their application secrets (ex. staging)
￭ All access is authenticated, authorized and audited (makes Security happy)
▪ Cons:
￭ Requires a one-time manifest change

24. Truths we hold to be self-evident
• It is possible to align Velocity and Security
• If security is a bad UX, everybody loses
• Established Security principles still apply, but must acknowledge new realities
• Security policies should declaratively model applications, users and envs
• You can easily add secure, dynamic secrets retrieval to your manifests

25. Questions?

26. Sound Interesting?
▪ Come talk to us at the CyberArk booth and who knows
you may already be a CyberArk customer and not even
know it!
▪ Unlike every company, here, we’re hiring!! :)
▪ Get paid to build and release open source tooling to
help our community!

27. Resources
Conjur Download Details:
▪ https://www.conjur.org/
Conjur Puppet Module:
▪ https://forge.puppet.com/cyberark/conjur
Conjur Slack channel:
▪ https://conjur.slack.com

28. Thank You!!