Chef Vault: A Deep Dive on Securing Secrets in Chef Infra

Chef Vault: A Deep Dive
Nell Shamrell-Harrington
@nellshamrell
Chef Conf 2017

What do I do when I need
to share static information
among my nodes?

Chef Vault: A Deep Dive @nellshamrell
App Node 1 App Node 2

Database

Load Balancer
Database

Load Balancer
Database
App Node 1 App Node 2App Node 3

What if that information
needs to be encrypted?

It’s a little more
complicated…

Secrets management systems
must ﬁnd a balance between
security and usability

Preventing unauthorized
access is only half of
managing security

Security must also enable
authorized users to access
what they need when
they need it

A system that is secure
but unusable is
inherently insecure

When choosing a secret
management tool, you
must understand…

1. Your own environment
2. The capabilities and
limits of the tool

No tool is one  
size ﬁts all.

We will focus on Chef
Vault, but we will also
explore alternatives

• Sr. Software Engineer at Chef
• Core maintainer of Supermarket and Habitat
• Co-host of Foodﬁght Podcast
• @nellshamrell
• nshamrell@chef.io
Nell Shamrell-Harrington

Encrypted Data bags
$ cat my_item.json
{“db_password”:“some_password”}
Workstation
JSON ﬁle with data

$ openssl rand -base64 512 | tr -d 'rn' > my_key
Workstation
Generates a key
Encrypted Data bags

$
Workstation
my_key
Encrypted Data bags

my_databag
$ knife data bag from ﬁle my_databag my_item.json
Workstation
my_key
my_item_info
my_item
Creates data bag
Encrypted Data bags

$ knife data bag from ﬁle my_databag my_item.json
--secret-ﬁle /path/to/my_key
Workstation
my_key
my_databag
my_item
my_key
Encrypted Data bags
Key to
encrypt
data bag

$
my_key
Workstation
my_key
my_databag
my_item
Encrypted Data bags

$ knife data bag show my_databage my_item
—secret-ﬁle /path/to/my_key
Workstation
my_key
my_key
my_databag
my_item
Encrypted Data bags
Shows data bag

Workstation
my_key
my_key
my_databag
my_item
Encrypted Data bags
Key to
decrypt
data bag

Workstation
my_key
my_databag
my_item
my_item_info
Encrypted Data bags

workstation_1 workstation_2 workstation_3
my_databag
my_item
Encrypted Data bags

my_key my_key my_key
my_databag
my_item
Encrypted Data bags

my_key my_key my_key
node_1
my_key
node_2
my_key
node_3
my_key
my_databag
my_item
Encrypted Data bags

• Limited by need to distribute and
share a key
Encrypted Data bags

• Limited by need to distribute and
share a key
• What if someone leaves?
Encrypted Data bags

• Limited by need to distribute and share
a key
• What if someone leaves?
• If the key is compromised on one
node/workstation, data is compromised
for all nodes/workstations
Encrypted Data bags

Chef Vault is one
approach to solving
this problem

• Originally created by Nordstrom Chefs!
Chef Vault

• Ownership transferred to Chef in 2015
Chef Vault

• Ownership transferred to Chef in 2015
• Completely open source
• https://github.com/chef/chef-vault
Chef Vault

Chef Vault
Chef Server

Chef Server
node_2
node_1
Chef Vault

Chef Server
node_1
node_2
node_1_client
node_2_client
Chef Vault

Chef Server
node_1
node_2
key_1
key_2
Chef Vault
node_1_client
node_2_client

Chef Server
node_1
node_2
key_1.pub
key_2.pub
key_1
key_2
Chef Vault
node_1_client
node_2_client

Chef Server
user_1
user_2
Chef Vault

Chef Server
user_1
user_2
user_1
user_2
Chef Vault

Chef Server
user_1
user_2
user_1
user_2
key_3
key_4
Chef Vault

Chef Server
user_1
user_2
user_1
user_2
key_3.pub
key_4.pub
key_3
key_4
Chef Vault

my_vault
my_item
Chef Vault

my_vault
my_item
shared_key
Chef Vault

my_vault
my_item
user_1 node_1
shared_key
Chef Vault

my_vault
my_item
user_1
key_3
node_1
key_1
shared_key
Chef Vault

my_vault
my_item
user_1
key_3
node_1
shared_key
copy
shared_key
copy
key_1
shared_key
Chef Vault

my_vault
my_item
user_1
key_3
node_1
shared_key
copy
shared_key
copy
key_1key_3
key_1
shared_key
Chef Vault

my_vault
my_item
user_1
key_3
node_1
shared_key
copy
shared_key
copy
key_1
key_1
shared_key
Decrypts
Chef Vault

my_vault
my_item
user_1
key_3
node_1
shared_key
copy
shared_key
copy
key_1
key_1
Chef Vault
Decrypts

my_vault
my_item
user_1
key_3
node_1
shared_key
copy
shared_key
copy
key_1
Chef Vault
Decrypts

Chef Vault uses layers of
encryption to balance
security and usability

Chef Server
node_1
node_2
user_1
user_2
Chef Vault

Creating a Vault
Workstation
$ gem install chef-vault
Installs Chef Vault

Creating a Vault
Workstation
$ knife vault create my_vault my_item
Creates vault

Creating a Vault
-A “user_1,user_2”
Workstation
Users with access to vault

Creating a Vault
-A “user_1,user_2,node_1,node_2”
Workstation
Nodes with access to vault

Creating a Vault
-A “user_1,user_2,node_1,node_2” -M client
Workstation
Mode (client or solo)

Creating a Vault
-A “user_1,user_2,node_1,node_2” -M client -J ./
my_item.json
Workstation
File with data

Creating a Vault
Workstation
my_vault
my_item
my_item.json

Creating a Vault
Workstation
my_vault
my_item my_item_keys
Authorized keys
my_item.json

Creating a Vault
$ knife vault show my_vault my_item
Workstation
my_vault
Shows vault

Viewing a Vault
db_password: some_password
id: my_item
Workstation

Viewing a Vault
db_password:
cipher: aes-256-cbc
encrypted_data:
dsiBtNHX8Sbis42yKuYBvbdNXPpu8bQfJrS20op7zoys
fR8roFlzp
VHyoaG2
4yb3
Unauthorized Workstation

Using a Vault in a Recipe
metadata.rb
gem ‘chef-vault’

recipe.rb
chef_gem ‘chef-vault’ do
compile_time true if respond_to?(:compile_time)
end

end
require ‘chef-vault’
recipe.rb

end
vault = chef_vault_item(“my_vault”, “my_item”)
recipe.rb

end
vault = chef_vault_item(“my_vault”, “my_item”)
node.set[‘database’][‘password’] = vault[‘password’]
recipe.rb

Editing a Vault
$ knife vault edit my_vault my_item
Workstation
my_vault
Edits vault

Adding a User or Node
$ knife vault update my_vault my_keys -A “new-
username”
Workstation
my_vault
Adds user/node

Removing a User or Node
$ knife vault remove my_vault my_item -A
“some_user”
Workstation
my_vault
Remove user/node

Refreshing Keys
$ knife vault refresh my_vault my_item
Workstation
my_vault
Refresh user/node keys

Destroying a Vault
$ knife vault delete my_vault my_item
Workstation
my_vault
Destroys vault item

Destroying a Vault
$ knife vault delete my_vault my_item
Workstation
my_vault

Destroying a Vault
$ knife data bag delete my_vault
Workstation
my_vault
Destroys vault

Destroying a Vault
$ knife data bag delete my_vault
Workstation

Real Life Example:
Supermarket

Load Balancer
Postgres
DB
App Node App Node App Node
Redis
Cache
Supermarket Production

Load Balancer
Postgres
DB
App Node App Node App Node
Redis
Cache
Supermarket Staging

apps

apps
supermarket_staging
supermarket_prod

apps
supermarket_staging
supermarket_prod
supermarket_staging_keys
supermarket_prod_keys

• Adding new nodes requires human
intervention
• Not compatible with autoscaling
groups
Chef Vault

• Adding new nodes requires human
intervention
• Not compatible with autoscaling groups
• Could theoretically have one node
keep all keys and distribute those - but
that is a major security risk
Chef Vault

• No audit logs of access
Chef Vault

• Alternatives
• Hashicorp Vault (https://
www.vaultproject.io)
Chef Vault

• Alternatives
• Hashicorp Vault (https://
www.vaultproject.io)
• Citadel (https://github.com/poise/
citadel)
Chef Vault

• Where is your infrastructure?
(Citadel is AWS only)
Considerations

• Where is your infrastructure? (Citadel
is AWS only)
• Do you need autoscaling groups?
(Chef Vault is not a good ﬁt)
Considerations

• Where is your infrastructure? (Citadel
is AWS only)
• Do you need autoscaling groups?
(Chef Vault is not a good ﬁt)
• Do you want dynamic secrets?
(Hashicorp vault is your best bet)
Considerations

Encrypted information systems
must ﬁnd a balance between
security and accessibility

Chef Vault: A Deep Dive on Securing Secrets in Chef Infra

Recommended

Recommended

More Related Content

Similar to Chef Vault: A Deep Dive on Securing Secrets in Chef Infra

Similar to Chef Vault: A Deep Dive on Securing Secrets in Chef Infra (7)

More from Nell Shamrell-Harrington

More from Nell Shamrell-Harrington (20)

Recently uploaded

Recently uploaded (20)

Chef Vault: A Deep Dive on Securing Secrets in Chef Infra