The 5 Stages of Secrets Management Grief, And How to Prevail

The 5 Stages of Secrets Management Grief
(And How to Prevail)
Josh Bregman
Conjur

Josh has 20 years experience successfully architecting, evangelizing, and delivering
innovative identity management and security products to customers. Prior to joining
Conjur , Josh spent a decade as a solutions and pre-sales leader in the Oracle ecosystem. A
developer at heart, early in his career Josh worked as a software engineer at IBM, GTE
Labs, and Netegrity. He has 3 U.S. patents and received a B.A. in Math from the University
of Rochester in 1995.

Denial - We don’t have a problem
Anger - Why is this my problem?
Bargaining - A series of trade-offs?
Depression - This isn’t fixed?
Acceptance - We have a problem

You’re at Puppet Camp, so your
infrastructure is coded, and
your code is in source control.

class { 'wordpress':
db_user => 'wordpress',
db_password => 'hvyH(S%t("0"16',
db_host => 'db.example.com',
create_db => false,
create_db_user => false,
}

If you put your secrets in
source control, then anyone
who has access to the repo can
access all the secrets.

“Searching GitHub for AWS and
Azure credentials reveals that many
people are making the same
mistake as Ashley Madison, Uber
and D-Link.”

“Ashley Madison’s leaked code
included hard-coded AWS tokens,
database credentials, certificate
private keys and other credentials.

“Uber had a database containing
personal information about drivers
compromised in 2014, after storing
the key in a publicly available
repo”

“...and D-Link recently published its
private code signing keys in the
open source code for a firmware
update.”

“Your cloud credentials are likely to
end up subsidizing Bitcoin miners, who
scan GitHub for keys and use them to
run up hundreds or thousands of
dollars of bills.”

(Sound of everyone
making sure that repo
is private)

Should everyone at your
company who has read access
to the repo have access to the
database?

Anger - Why is it my problem?
Bargaining - A Series of Trade-offs?

Typical conversation BEFORE
something bad happens…
20

Question to Information Security:
I’m concerned that storing secrets in
source control isn’t safe. Is there a
recommended approach that I should
be following?
21

Answer from Information Security:
We’re super busy right now
protecting the company from APT,
passing our ISO 27001 audit, and
assessing our compliance NIST CSF…
22

Question to Information Security:
Well, this initiative is super important
to the business. Is there anything that
you can recommend?
23

Answer from Information Security:
We’ll we have an existing system that
we use to manage privileged
accounts. You just open a
ServiceNow ticket and…
24

That’s OK….we’ll just figure it out
25

Typical conversation AFTER
something bad happens…
26

“I’m going to automate those *@!
out of a job.” – Anonymous DevOps
“Those *@! are running with
scissors” – Anonymous InfoSec
27

Few organizations practice blame
free post-mortems, if they are on the
front page of the Wall Street Journal.
This is a huge cultural change for
Information Security.
28

SecDevOps or DevSecOps or
RuggedDevOps are all terms for the
inclusion of information security in
the DevOps workflow
29

Automated testing that includes
security tests like code scanning,
application security testing,
automated patching of vulnerabilities
are all pretty easy…
30

…because they can be added without
the direct cooperation of information
security teams.
31

Question: How many people here
have information security
professionals as part of their DevOps
teams?
32

Question: How many people here
ACTIVELY seek out the information
security professionals in their
organization?
33

NIST CyberSecurity Framework - The
Framework Implementation Tiers
(“Tiers”) provide context on how an
organization views
cybersecurity risk and the processes
34

NIST CSF Tiers:
Tier 1 -> Partial
Tier 2 -> Risk Informed
Tier 3 -> Repeatable
Tier 4 -> Adaptive
35

Go Find your Security Engineering
Team. This is the team that owns
and operates security solutions. Tell
them you can help them with
automation.
36

Anger - It’s their fault!
Bargaining - A Series of Trade-offs?

Two main camps on secrets
management with Puppet: Masterful
and Masterless

Masterful: I’m OK if all of my secrets
are on the master; my master is a
hardened command bunker.
Masterless: Secrets are ‘need to
know’ and my master doesn’t.

The “Masterful” approaches can be
accomplished with little commitment
from information security.

If you don’t engage them, and
something goes wrong, it’s all on you.

If you use E-YAML, secrets are
encrypted in source control, and in
the catalog.

If you use E-YAML, you have to figure
out how to secure the keys, and
rotate the keys, and work with
encrypted files...

Another “Masterful” approach that
some customers use is to deploy
separate Production and Non-
Production Puppet Masters

This keeps production information
limited to only those that need
access to production. By design, this
pattern makes Continuous Delivery
hard.

If you use DSL extensions or Hiera
backend, then secrets are in the
catalog, but not in source control.

Retrieves a Secret from Conjur
programmatically
https://github.com/dgrstl/puppet_conjurdemo/blob/master/lib/
puppet/functions/conjur_secret.rb

See it in action
https://github.com/dgrstl/puppet_conjurdemo/
blob/master/tests/notify.pp

Masterless approach - a.k.a - Node
Side Secrets - secrets aren’t in source
control and they are not on the
master

Scenario 1 - Write a configuration file
on the node that has a list of secrets

Puppet::conjur_demo {‘/opt/foo.conf’:
secrets =>
[‘/production/db/user’,
’/production/db/password’],
}

Scenario 2 - Control the value of an
attribute via Conjur

Puppet::conjurdemo_secert_value { ‘foo’:
secret_key_name => ‘bar’,
resource => Wordpress[‘server1’],
field => ‘db_password’,
}

datacat_collector { "$title Conjur secret":
template_body =>
template('puppet_conjurdemo/conjur_simple_secret.erb'),
target_resource => $resource,
target_field => $field,
notify => $resource,
}

<%% require 'conjur/cli'
require 'yaml'
Conjur::Config.load
Conjur::Config.apply
api = Conjur::Authn.connect
-%>
<% @secrets.each do |secret_key| %>
<%% secret = api.variable "<%=secret_key%>" %><%=secret_key%> =
<%%=secret.value-%>
<%end%>

Want to learn more?
Request a Conjur Demo

In both the masterful and masterless
models, the secrets wind up on the
nodes.

This represents a whole new threat
surface - a way that your secret
information can be compromised

Applications and services, not just
infrastructure also need access to
credentials. And applications are
stored in source control.

Summon is an open-source project
that allows for the retrieval of secrets
safely without checking the secrets
into source control

Summon works well with 12 factor
apps - those that expect to get their
configuration from the environment -
e.g. Java Application

secrets.yaml
MYSECRET: !var secret/path
MYSECRET2: !var secret/path2

USAGE:
summon [global options] command
[command options] [arguments...]
Ex: summon -f /opt/secrets.yaml
printenv

Supports simple provider interface
variable = sys.argv[1]
value = keyring.get_password(
os.environ.get('SUMMON_KEYRING_SERVICE', 'summon'),
variable
)

https://github.com/jbregman/puppet-
summon/tree/master/jbregman-summon/tests
Test 1 - Simple secrets.yaml
Test 1a - Parameterized secrets.yaml
Test 2a - Creating Config
Test 3 - facter

This is an evolution from the Conjur Puppet
integration that I presented at Puppet Conf in
2015

Integration with Puppet is an
important but emerging area. These
modules and repos are works in
progress.

Customers are very interested in
additional capabilities such as
rotation, versioning, secure service
lifecycle

Resources:
https://github.com/dgrstl/puppet_conjurdemo
https://puppetlabs.com/blog/using-node-side-
secrets-with-puppet
https://conjurinc.github.io/summon/

Call to action:
• Have a discussion “Are we a
masterless or masterful shop?”
• Make friends with information
security

The 5 Stages of Secrets Management Grief, And How to Prevail

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to The 5 Stages of Secrets Management Grief, And How to Prevail

Similar to The 5 Stages of Secrets Management Grief, And How to Prevail (20)

Recently uploaded

Recently uploaded (20)

The 5 Stages of Secrets Management Grief, And How to Prevail