Security,
Microservices
& Vault
Nicki Watt
@techiewatt
1
http://www.microservicesmanchester.com
About Me
• Hands on Lead consultant at OpenCredo

• Co-author Neo4j In Action

• Twitter: @techiewatt
2
Agenda
• Introduction
• Framework for assessing challenges
• Vault
• Conclusion
3
4
Introduction
5
You’ve already heard the stories 

of how …
6
from the monolith …
image credit: http://lovealwaysbear.blogspot.co.uk/2011_01_01_archive.html
Applications
7
to microservices
image credit: http://www.guinnessworldrecords.com/world-records/most-tennis-
balls-held-in-the-mouth-dog
Applications
8
to microservices
image credit: http://www.guinnessworldrecords.com/world-records/most-tennis-
balls-held-in-the-mouth-dog
Not every
problem
needs
m
icroservices!
Applications
9
from Silo’d teams with manual
release processes
image credit: http://kittypluscoco.blogspot.co.uk/
2011/04/day-at-dog-park.html
Teams
10
image credit: http://www.notey.com/@coolshitibuy/external/10054533/
ruffwear-approach-dog-backpack.html
to agile teams with fast, automated
software delivery
DevOps!
Teams
11
But …
12
What do you mean
“It’s going live today” ?
image credit: https://www.facebook.com/EarltheGrump/photos
Security ?
13
image credit: https://www.facebook.com/EarltheGrump/photos
SECURITY BOLTED
ON AT THE END!
#
FAIL!
Security ?
What do you mean
“It’s going live today” ?
15
image credit: http://www.beauswish.org/wp-content/uploads/2016/04/arianna.jpg
DevSecOps!
agile teams (with security as a 1st
class citizen) practicing fast, secure,
automated software delivery
Delivery Pipeline
17
http://www.devsecops.org/blog/2016/5/20/-security
<— Shifting Security to the Left
Shannon Lietz
DEV
TEST
OPS
SECURITY
Delivery Pipeline
17
http://www.devsecops.org/blog/2016/5/20/-security
<— Shifting Security to the Left
Shannon Lietz
DEV
TEST
OPS
SECURITY
“secure reasoning” 

should be 

in the forefront of every 

engineers minds
18
Microservice example:
Big retail store selling goods 

which includes a typical “web store”
19
20
user
service
product service
Example: web store
21
user
service
product service
Example: web store
external system XXX
22
user
service
product service
Example: web store
external system XXX
sensitive
data
passwords,
keys
23
Example: web store
external system XXX
store api
store front
user
service
product service
sensitive
data
passwords,
keys
24
sensitive
data
store api
store front
user
service
product service
external system XXX
passwords,
keys
Example: web store
Where do we start ?
25
Know thy playground!
• What infrastructure?
• What tech stacks?
• What databases?
• What type of delivery channels?
26
27
sensitive
data
store api
store front
user
service
product service
external system XXX
passwords,
keys
Example: web store
28
sensitive
data
store api
store front
user
service
product service
external system XXX
passwords,
keys
Example: web store
29
sensitive
data
store api
store front
user
service
product service
external system XXX
passwords,
keys
Example: web store
30
A framework for
thinking about
security …
31
NIST Cyber Security Framework
32
NIST Cyber Security Framework
33
IDENTIFY
PROTECT
DETECT
RESPOND
RECOVER
What stuff needs protecting?
What can I do to protect it?
How will I know if bad stuff happens?
What should I do when bad stuff happens?
How can I get my system back up and
running after bad stuff has happened?
34
IDENTIFY What stuff needs protecting?
35
IDENTIFY What stuff needs protecting?
Threat Modelling
36
IDENTIFY What stuff needs protecting?
Attack Trees
https://www.schneier.com/academic/archives/1999/12/attack_trees.html
38
IDENTIFY
sensitive
data
external system XXX
store api
store front
passwords,
keys
user
service
product service
steal
sensitive user
data
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
39
IDENTIFY
gain access
to internal
network
steal
sensitive user
data
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
social
engineering
sniff non
encrypted
traffic
external system XXX
sensitive
data
passwords,
keys
user
service
product service
40
IDENTIFY
store api
store front
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
steal
sensitive user
data
modify
data in DB
external system XXX
41
IDENTIFY
store api
store front
sensitive
data
passwords,
keys
user
service
product service
gain access
to internal
network
steal
sensitive user
data
social
engineering
sniff non
encrypted
traffic
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
42
IDENTIFY
gain access
to internal
network
steal
sensitive user
data
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
social
engineering
sniff non
encrypted
traffic
Security, and actually
being able to do things,
always requires a trade off!
43
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
44
PROTECT
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS Use prepared statements
build web app vuln
verification into CI/CD
gain access
to internal
network
social
engineering
sniff non
encrypted
traffic
steal
sensitive user
data
HTTPS
Firewall
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
45
PROTECT
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS Use prepared statements
build web app vuln
verification into CI/CD
gain access
to internal
network
social
engineering
sniff non
encrypted
traffic
steal
sensitive user
data
HTTPS
Firewall
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
46
PROTECT
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS Use prepared statements
build web app vuln
verification into CI/CD
gain access
to internal
network
social
engineering
sniff non
encrypted
traffic
steal
sensitive user
data
HTTPS
Firewall
cfssl
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
47
PROTECT
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS Use prepared statements
build web app vuln
verification into CI/CD
gain access
to internal
network
social
engineering
sniff non
encrypted
traffic
steal
sensitive user
data
HTTPS
Firewall
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
48
DETECT
Log suspicious queries
Log HTTP requests
Log HTTP
requests
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS Use prepared statements
build web app vuln
verification into CI/CD
gain access
to internal
network
social
engineering
sniff non
encrypted
traffic
steal
sensitive user
data
HTTPS
Firewall
IDS
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
49
gain access
to internal
network
infect
employee
computer
install
malware via
email
sniff non
encrypted
traffic
compromise
user data
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS
HTTPS
Firewall
antivirus
Use prepared statements
IDS
Log suspicious queries
Log HTTP requests
Log HTTP
requests
build web app vuln
verification into CI/CD
DETECT
Distributed
logging
capability
Container
level
logging
Alerting
capability
Infrastructure
level
logging
Serverless
logging
???
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
50
gain access
to internal
network
infect
employee
computer
install
malware via
email
sniff non
encrypted
traffic
compromise
user data
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS
HTTPS
Firewall
antivirus
Use prepared statements
IDS
Log suspicious queries
Log HTTP requests
Log HTTP
requests
build web app vuln
verification into CI/CD
DETECT
Distributed
logging
capability
Container
level
logging
Alerting
capability
Infrastructure
level
logging
Serverless
logging
???
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
52
RESPOND
Redirect to
HTTPS
Block consistent offenders
Adjust firewall rules
Block attackers
Log suspicious queries
Log HTTP requests
Log HTTP
requests
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS Use prepared statements
build web app vuln
verification into CI/CD
gain access
to internal
network
social
engineering
sniff non
encrypted
traffic
steal
sensitive user
data
HTTPS
Firewall
IDS
Change DB Password
Reset users passwords
Inform users
Redirect to HTTPS
store api
store front
sensitive
data
passwords,
keys
user
service
product service
external system XXX
53
Log suspicious queries
Block consistent offenders
RECOVER
Redirect to
HTTPS
Block consistent offenders
Adjust firewall rules
Block attackers
Log suspicious queries
Log HTTP requests
Log HTTP
requests
attack
store front
/ API
sniff non
encrypted
traffic
SQL
Injection
Alter
query to
get data
modify
data in DB
HTTPS Use prepared statements
build web app vuln
verification into CI/CD
gain access
to internal
network
social
engineering
sniff non
encrypted
traffic
steal
sensitive user
data
HTTPS
Firewall
IDS
Change DB Password
Reset users passwords
Inform users
Redirect to HTTPS
Restore from backup
Fix Code,
Blue/Green deploys:
redeploy microservice(s)
redeploy infrastructure
54
RECOVER
Trash & burn!
is your friend
• Due diligence: know thy playground
• Think holistically: identify, protect, detect,
respond, recover
Summary
55
Make security a
1st class citizen
in your thinking process!
• Multiple, diverse, interconnected services
• More varied attack surfaces
• Harder to track what’s going on 

(distributed, multi facetted logging capabilities)

• Transient components
• Dynamic transport level encryption (HTTPS)
• Authentication & Authorisation (see David’s talk :)
• Trash & burn recovery strategies
Microservice security challenges
56
Onto
the practical bit …
58
59
A tool for managing secrets
and other sensitive content
60
Deployment Tools
Application Component /
Microservices
service 1 service 2
Human Users
61
• Unified API to access multiple backends
• ACL policies - who can access what
• Audit Logs
62
Unseal
Init
service 1
service 2
Allow token
to be used by tools
to access secrets
Acquire policy
constrained
token
Create microservice
mount or area, add
secrets
System X
63
$ vault init -key-shares=3 -key-threshold=2
Key 1: 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba
Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c
Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91
Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b
Vault initialized with 3 keys and a key threshold of 2. Please
securely distribute the above keys. When the Vault is re-sealed,
restarted, or stopped, you must provide at least 2 of these keys
to unseal it again.
Vault does not store the master key. Without at least 2 keys,
your Vault will remain permanently sealed.
$ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba
Sealed: true
Key Shares: 3
Key Threshold: 2
Unseal Progress: 1
Vault init & unseal
$ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91
Sealed: true
Key Shares: 3
Key Threshold: 2
Unseal Progress: 0
64
$ vault init -key-shares=3 -key-threshold=2
Key 1: 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba
Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c
Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91
Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b
Vault initialized with 3 keys and a key threshold of 2. Please
securely distribute the above keys. When the Vault is re-sealed,
restarted, or stopped, you must provide at least 2 of these keys
to unseal it again.
Vault does not store the master key. Without at least 2 keys,
your Vault will remain permanently sealed.
$ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba
Sealed: true
Key Shares: 3
Key Threshold: 2
Unseal Progress: 1
Vault init & unseal
$ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91
Sealed: true
Key Shares: 3
Key Threshold: 2
Unseal Progress: 0
65
$ vault init -key-shares=3 -key-threshold=2
Key 1: 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba
Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c
Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91
Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b
Vault initialized with 3 keys and a key threshold of 2. Please
securely distribute the above keys. When the Vault is re-sealed,
restarted, or stopped, you must provide at least 2 of these keys
to unseal it again.
Vault does not store the master key. Without at least 2 keys,
your Vault will remain permanently sealed.
$ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba
Sealed: true
Key Shares: 3
Key Threshold: 2
Unseal Progress: 1
Vault init & unseal
$ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91
Sealed: true
Key Shares: 3
Key Threshold: 2
Unseal Progress: 0
66
Success! Ready for use
67
Unseal
Create segregated
area, policies,
add secrets
Init
Acquire policy
constrained
token
Allow token
to be used by tools
to access secrets
service 1
service 2
System X
68
$ vault mount -path=usersvc generic
Successfully mounted 'generic' at ‘usersvc'!
$ vault mounts
Path Type Default TTL Max TTL Description
cubbyhole/ cubbyhole n/a n/a per-token private secr ...
secret/ generic system system generic secret storage
sys/ system n/a n/a system endpoints used f...
usersvc/ generic system system
Vault create new mount
69
$ vault write usersvc/db-password value=ASDKJ234SF*2
Success! Data written to: usersvc/db-password
$ vault read usersvc/db-password
Key Value
lease_duration 2592000
value ASDKJ234SF*2
Vault write, then read back secret
70
$ cat usersvc.policy
path "usersvc/*" {
policy = "read"
}
$ vault policy-write usersvc usersvc.policy
Policy 'usersvc' written.
Vault create custom policy
71
Unseal
Allow token
to be used by tools
to access secrets
Init
Acquire policy
constrained
token
service 1
service 2
Create segregated
area, add secrets
System X
72
Basics of Vault complete!
Getting sensitive data
into microservices …
73
74
# Embedded Config
spring.datasource.url=jdbc:mysql://localhost/test
spring.datasource.username=dbuser
spring.datasource.password=dbpass
spring.datasource.driver-class-name=
com.mysql.jdbc.Driver
Java Code
@Component
public class MyBean {
private final JdbcTemplate jdbcTemplate;
@Autowired
public MyBean(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
// ...
}
Starting point …
user
service
db1
75
# Embedded Config
spring.datasource.url=jdbc:mysql://localhost/test
spring.datasource.username=dbuser
spring.datasource.password=dbpass
spring.datasource.driver-class-name=
com.mysql.jdbc.Driver
Java Code
@Component
public class MyBean {
private final JdbcTemplate jdbcTemplate;
@Autowired
public MyBean(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
// ...
}
Starting point …
user
service
db1
Separate
Code
and
Config
-
Especially
Secrets!!
76
# Embedded Config
spring.datasource.url=jdbc:mysql://localhost/test
spring.datasource.username=dbuser
spring.datasource.password=dbpass
spring.datasource.driver-class-name=
com.mysql.jdbc.Driver
Java Code
@Component
public class MyBean {
private final JdbcTemplate jdbcTemplate;
@Autowired
public MyBean(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
// ...
}
Starting point …
user
service
db1
Separate
Code
and
Config
-
Especially
Secrets!!DETECT
https://github.com/michenriksen/gitrob
https://github.com/awslabs/git-secrets
77
Options
• Push secrets in

• Pull secrets out

• Variations of the above …
78
Push secrets in …
user
service
db1
1authenticate
2
orchestration /
deployment platform
3
provide value as
environment
variables
read secret/db-password
79
user
service
db1
1authenticate
2
read secret/db-password
orchestration /
deployment platform
3
provide value as
environment
variables
$ vault auth e2d0a065-xxxx-yyyy-zzzz
Successfully authenticated! You are…
token_policies: [default, usersvc]
$ vault read usersvc/db-password
Key Value
--- -----
refresh_interval 2592000
value MyClearTextPassword
1
2
80
user
service
db1
1authenticate
2
read secret/db1
orchestration /
deployment platform
3
provide value as
environment
variables
$ # Start docker container,pass in vars
docker run
--name usersvc
-e DB_USER="MyDBName"
-e DB_PASSWORD="MyClearTextPassword"
-d usersvc:v1
3
81
Steal Sensitive User DataIDENTIFY
steal
sensitive user
data
steal
sensitive user
data
gain access to
internal network
gain access to
user DB
gain access to
running user
microservice(s)
dump startup
config
steal plaintext
password
social
engineering
the-machine$ docker ps
CONTAINER ID IMAGE ... CREATED STATUS NAMES
9950ea8e3c59 product-service:v1 ... 4 days ago Up 4 days prodsvc
29b9ebca6dab user-service:v2 ... 5 days ago Up 5 days usersvc
82
gain access to
running user
microservice(s)
83
gain access to
internal network
find a
disgruntled
employee
dump startup
config
the-machine$ docker inspect 29b9ebca6dab
[
{
"Id": “29b9ebca6dab147991201a5c61b72d3d546885ca8fd…”,
"Created": "2016-06-27T21:26:16.126414991Z",
"Args": [
"-jar",
"UserService"
],
"Config": {
"Hostname": "29b9ebca6dab",
"Env": [
“DB_USER=MyUserName",
“DB_PASSWORD=MyClearTextPassword",
“VAR1=something-else“
],
"Cmd": [
"java",
"-jar",
"UserService"
],
...
}
]
84
gain access to
internal network
find a
disgruntled
employee
the-machine$ docker inspect 29b9ebca6dab
[
{
"Id": “29b9ebca6dab147991201a5c61b72d3d546885ca8fd…”,
"Created": "2016-06-27T21:26:16.126414991Z",
"Args": [
"-jar",
"UserService"
],
"Config": {
"Hostname": "29b9ebca6dab",
"Env": [
“DB_USER=MyUserName",
“DB_PASSWORD=MyClearTextPassword",
“VAR1=something-else“
],
"Cmd": [
"java",
"-jar",
"UserService"
],
...
}
]
steal plaintext
password
85
gain access to
internal network
gain access to
user DB
gain access to
running user
microservice(s)
dump startup
config
social
engineering
PROTECT
don’t expose as
plain text
steal
sensitive user
data
steal plaintext
password
limit user access
Vault
Response Wrapping
86
Push secrets in … (take 2)
87
Push secrets in …
user
service
db1
1authenticate
2
orchestration /
deployment platform
3
provide value as
environment
variables
read secret/db-password
87
user
service
db1
1authenticate
2
read wrapped secret
orchestration /
deployment platform
3
provide wrapped
value as environment
variables
4
unwrap
Push wrapped secrets in …
88
user
service
db1
1authenticate
2
read wrapped secret
orchestration /
deployment platform
3
provide wrapped
value as environment
variables
4
unwrap
$ vault read -wrap-ttl=60s usersvc/db-password
Key Value
--- -----
wrapping_token: 57ccef32-471d-869
wrapping_token_ttl: 60
wrapping_token_creation_time: 2016-06-28 22:..
2
89
user
service
db1
1authenticate
2
read wrapped secret
orchestration /
deployment platform
3
provide wrapped
value as environment
variables
4
unwrap
$ # Start docker container,pass in vars
docker run
--name usersvc
-e DB_USER="MyDBName"
-e DB_PASSWORD="57ccef32-471d-869"
-d usersvc:v1
3
90
user
service
db1
1authenticate
2
read wrapped secret
orchestration /
deployment platform
3
provide wrapped
value as environment
variables
4
unwrap
$ vault unwrap 57ccef32-471d-869
Key Value
--- -----
refresh_interval 2592000
value MyClearTextPassword
4
91
dump startup
config
the-machine$ docker inspect 29b9ebca6dab
[
{
"Id": “29b9ebca6dab147991201a5c61b72d3d546885ca8fd…”,
"Created": "2016-06-27T21:26:16.126414991Z",
"Args": [
"-jar",
"UserService"
],
"Config": {
"Hostname": "29b9ebca6dab",
"Env": [
“DB_USER=MyUserName",
“DB_PASSWORD=57ccef32-471d-869",
“VAR1=something-else“
],
"Cmd": [
"java",
"-jar",
"UserService"
],
...
}
]
92
gain access to
internal network
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
steal plaintext
password
don’t expose as
plain text
gain access to
user DB
limit user access
93
gain access to
internal network
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
steal plaintext
password
don’t expose as
plain text
gain access to
user DB
limit user access
94
gain access to
internal network
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
don’t expose as
plain text
gain access to
user DB
steal wrapped
password
get real
password
limit user access
95
user
service
db1
1authenticate
2
read wrapped secret
orchestration /
deployment platform
3
provide wrapped
value as environment
variables
4
unwrap
$ vault unwrap 57ccef32-471d-869
error reading cubbyhole/response: Error making
API request.
URL: GET https://vault:8200/v1/cubbyhole/response
Code: 400. Errors:
* permission denied
4
96
gain access to
internal network
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
DETECT
don’t expose as
plain text
gain access to
user DB
steal wrapped
password
get real
password
Raise TOFU
alarm
Audit access
limit user access
97
gain access to
internal network
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
DETECT
don’t expose as
plain text
RESPOND
gain access to
user DB
steal wrapped
password
get real
password
Raise TOFU
alarm
Audit access
change
DB password
limit user access
98
gain access to
internal network
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
DETECT
don’t expose as
plain text
RESPOND
gain access to
user DB
steal wrapped
password
get real
password
Raise TOFU
alarm
Audit access
change
DB password
Expect secrets to change.
Make a habit of changing them regularly.
It will naturally force you to put
measures in place. limit user access
• Dynamic Secrets: Auto generate
credentials on the fly
Other handy options
99
100
user
service
db1
1authenticate
2
read dynamic password
orchestration /
deployment platform
3
provide value as
environment
variables
0
Human /
Other System
Users
101
user
service
db1
1authenticate
2
orchestration /
deployment platform
3
provide value as
environment
variables
$ vault mount postgresql
Successfully mounted 'postgresql' at
'postgresql'!
$ vault write postgresql/config/connection
connection_url="postgresql://
vault:somepassword@yourhost:5432/postgres"
$ vault write postgresql/roles/usersvc-ro 
sql="CREATE ROLE "{{name}}"
WITH LOGIN PASSWORD ‘{{password}}'
VALID UNTIL '{{expiration}}';
GRANT SELECT ON ALL TABLES
IN SCHEMA users TO "{{name}}";"
Success! Data written to: postgresql/roles/
read dynamic password
Human /
Other System
Users
0
0
102
user
service
db1
1authenticate
2
orchestration /
deployment platform
3
provide value as
environment
variables
$ vault mount postgresql
Successfully mounted 'postgresql' at
'postgresql'!
$ vault write postgresql/config/connection
connection_url="postgresql://
vault:somepassword@yourhost:5432/postgres"
$ vault write postgresql/roles/usersvc-ro 
sql="CREATE ROLE "{{name}}"
WITH LOGIN PASSWORD ‘{{password}}'
VALID UNTIL '{{expiration}}';
GRANT SELECT ON ALL TABLES
IN SCHEMA users TO "{{name}}";"
Success! Data written to: postgresql/roles/
read dynamic password
Human /
Other System
Users
0
0
103
user
service
db1
1authenticate
2
orchestration /
deployment platform
3
provide value as
environment
variables
$ vault mount postgresql
Successfully mounted 'postgresql' at
'postgresql'!
$ vault write postgresql/config/connection
connection_url="postgresql://
vault:somepassword@yourhost:5432/postgres"
$ vault write postgresql/roles/usersvc-ro 
sql="CREATE ROLE "{{name}}"
WITH LOGIN PASSWORD ‘{{password}}'
VALID UNTIL '{{expiration}}';
GRANT SELECT ON ALL TABLES
IN SCHEMA users TO "{{name}}";"
Success! Data written to: postgresql/roles/
read dynamic password
Human /
Other System
Users
0
0
104
user
service
db1
1authenticate
2
orchestration /
deployment platform
3
provide value as
environment
variables
$ vault read postgresql/creds/usersvc-ro
Key Value
lease_id postgresql/creds/usersvc-ro/
c888a097-b0e2-26a8-b306-fc7c84b98f07
lease_duration 3600
password 34205e88-0de1-68b7…
username vault-14301-usersvc-ro
read dynamic password
Human /
Other System
Users
0
2
105
user
service
db1
1authenticate
2
orchestration /
deployment platform
3
provide value as
environment
variables
$ # Start docker container,pass in vars
docker run
--name usersvc
-e DB_USER="vault-14301-usersvc-ro"
-e DB_PASSWORD="34205e88-0de1-68b7"
-d usersvc:v1
read dynamic password
• Dynamic Secrets: Auto generate
creds on the fly
• Ability to combine security primitives

dynamic secrets + resource wrapping
Other handy options
106
107
gain access to
internal network
gain access to
user DB
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
DETECT
steal wrapped
password
don’t expose as
plain text
get real
password
Raise TOFU
alarm
Audit access
RESPOND
change
DB password
108
gain access to
internal network
gain access to
user DB
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
DETECT
steal wrapped
password
don’t expose as
plain text
get real
password
Raise TOFU
alarm
Audit access
RESPOND
change
DB password
use time limited
dynamic creds
109
gain access to
internal network
gain access to
user DB
gain access to
running user
microservice(s)
dump startup
config
find a
disgruntled
employee
PROTECT
steal
sensitive user
data
DETECT
steal wrapped
password
don’t expose as
plain text
get real
password
Raise TOFU
alarm
Audit access
RESPOND
change
DB password
use time limited
dynamic creds
compromise
orchestration
platform
Turtles all the way down!
111
gain access to
internal network
gain access to
user DB
gain access to
running user
microservice(s)
dump startup
config
compromise
orchestration
platform
find a
disgruntled
employee
steal
sensitive user
data
steal vault
token
get db
password
1
2
34
Defense in Depth
Put enough hurdles in the way
of attackers for you to stop
when you can, but if not, to be
able to …
- realise what’s going on 

- react before too much damage
is done
112
• Centralised Secrets Management
• API - helps with automation
• Tries to address concerns across full
security lifecycle
• But still very new & maturing
Vault Summary
113
• Encryption as a service: offload
responsibility to Vault
• PKI: Generates X.509 certificates
dynamically based on configured
roles
• SSH: Dynamically generates SSH
credentials for remote hosts
Other Handy Features
114
Conclusion
115
116
Make security 

a first class citizen!
Don’t try and just bolt it on at the end!
117
Think holistically 

about security
Don’t stop at the protect stage!
118
Choose the right tech 

for the job
Microservice architectures add
complexity
119
Do your best!

but don’t do nothing!
Questions?
Nicki Watt
@techiewatt
120

Microservices Manchester: Security, Microservces and Vault by Nicki Watt

  • 1.
  • 2.
    About Me • Handson Lead consultant at OpenCredo
 • Co-author Neo4j In Action
 • Twitter: @techiewatt 2
  • 3.
    Agenda • Introduction • Frameworkfor assessing challenges • Vault • Conclusion 3
  • 4.
  • 5.
    5 You’ve already heardthe stories of how …
  • 6.
    6 from the monolith… image credit: http://lovealwaysbear.blogspot.co.uk/2011_01_01_archive.html Applications
  • 7.
    7 to microservices image credit:http://www.guinnessworldrecords.com/world-records/most-tennis- balls-held-in-the-mouth-dog Applications
  • 8.
    8 to microservices image credit:http://www.guinnessworldrecords.com/world-records/most-tennis- balls-held-in-the-mouth-dog Not every problem needs m icroservices! Applications
  • 9.
    9 from Silo’d teamswith manual release processes image credit: http://kittypluscoco.blogspot.co.uk/ 2011/04/day-at-dog-park.html Teams
  • 10.
  • 11.
  • 12.
    12 What do youmean “It’s going live today” ? image credit: https://www.facebook.com/EarltheGrump/photos Security ?
  • 13.
    13 image credit: https://www.facebook.com/EarltheGrump/photos SECURITYBOLTED ON AT THE END! # FAIL! Security ? What do you mean “It’s going live today” ?
  • 14.
    15 image credit: http://www.beauswish.org/wp-content/uploads/2016/04/arianna.jpg DevSecOps! agileteams (with security as a 1st class citizen) practicing fast, secure, automated software delivery
  • 15.
    Delivery Pipeline 17 http://www.devsecops.org/blog/2016/5/20/-security <— ShiftingSecurity to the Left Shannon Lietz DEV TEST OPS SECURITY
  • 16.
    Delivery Pipeline 17 http://www.devsecops.org/blog/2016/5/20/-security <— ShiftingSecurity to the Left Shannon Lietz DEV TEST OPS SECURITY
  • 17.
    “secure reasoning” shouldbe in the forefront of every engineers minds 18
  • 18.
    Microservice example: Big retailstore selling goods which includes a typical “web store” 19
  • 19.
  • 20.
  • 21.
    22 user service product service Example: webstore external system XXX sensitive data passwords, keys
  • 22.
    23 Example: web store externalsystem XXX store api store front user service product service sensitive data passwords, keys
  • 23.
    24 sensitive data store api store front user service productservice external system XXX passwords, keys Example: web store
  • 24.
    Where do westart ? 25
  • 25.
    Know thy playground! •What infrastructure? • What tech stacks? • What databases? • What type of delivery channels? 26
  • 26.
    27 sensitive data store api store front user service productservice external system XXX passwords, keys Example: web store
  • 27.
    28 sensitive data store api store front user service productservice external system XXX passwords, keys Example: web store
  • 28.
    29 sensitive data store api store front user service productservice external system XXX passwords, keys Example: web store
  • 29.
    30 A framework for thinkingabout security …
  • 30.
  • 31.
  • 32.
    33 IDENTIFY PROTECT DETECT RESPOND RECOVER What stuff needsprotecting? What can I do to protect it? How will I know if bad stuff happens? What should I do when bad stuff happens? How can I get my system back up and running after bad stuff has happened?
  • 33.
    34 IDENTIFY What stuffneeds protecting?
  • 34.
    35 IDENTIFY What stuffneeds protecting? Threat Modelling
  • 35.
    36 IDENTIFY What stuffneeds protecting? Attack Trees https://www.schneier.com/academic/archives/1999/12/attack_trees.html
  • 36.
    38 IDENTIFY sensitive data external system XXX storeapi store front passwords, keys user service product service steal sensitive user data
  • 37.
    store api store front sensitive data passwords, keys user service productservice external system XXX 39 IDENTIFY gain access to internal network steal sensitive user data attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB social engineering sniff non encrypted traffic
  • 38.
    external system XXX sensitive data passwords, keys user service productservice 40 IDENTIFY store api store front attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data steal sensitive user data modify data in DB
  • 39.
    external system XXX 41 IDENTIFY storeapi store front sensitive data passwords, keys user service product service gain access to internal network steal sensitive user data social engineering sniff non encrypted traffic
  • 40.
    store api store front sensitive data passwords, keys user service productservice external system XXX 42 IDENTIFY gain access to internal network steal sensitive user data attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB social engineering sniff non encrypted traffic
  • 41.
    Security, and actually beingable to do things, always requires a trade off! 43
  • 42.
    store api store front sensitive data passwords, keys user service productservice external system XXX 44 PROTECT attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS Use prepared statements build web app vuln verification into CI/CD gain access to internal network social engineering sniff non encrypted traffic steal sensitive user data HTTPS Firewall
  • 43.
    store api store front sensitive data passwords, keys user service productservice external system XXX 45 PROTECT attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS Use prepared statements build web app vuln verification into CI/CD gain access to internal network social engineering sniff non encrypted traffic steal sensitive user data HTTPS Firewall
  • 44.
    store api store front sensitive data passwords, keys user service productservice external system XXX 46 PROTECT attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS Use prepared statements build web app vuln verification into CI/CD gain access to internal network social engineering sniff non encrypted traffic steal sensitive user data HTTPS Firewall cfssl
  • 45.
    store api store front sensitive data passwords, keys user service productservice external system XXX 47 PROTECT attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS Use prepared statements build web app vuln verification into CI/CD gain access to internal network social engineering sniff non encrypted traffic steal sensitive user data HTTPS Firewall
  • 46.
    store api store front sensitive data passwords, keys user service productservice external system XXX 48 DETECT Log suspicious queries Log HTTP requests Log HTTP requests attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS Use prepared statements build web app vuln verification into CI/CD gain access to internal network social engineering sniff non encrypted traffic steal sensitive user data HTTPS Firewall IDS
  • 47.
    store api store front sensitive data passwords, keys user service productservice external system XXX 49 gain access to internal network infect employee computer install malware via email sniff non encrypted traffic compromise user data attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS HTTPS Firewall antivirus Use prepared statements IDS Log suspicious queries Log HTTP requests Log HTTP requests build web app vuln verification into CI/CD DETECT Distributed logging capability Container level logging Alerting capability Infrastructure level logging Serverless logging ???
  • 48.
    store api store front sensitive data passwords, keys user service productservice external system XXX 50 gain access to internal network infect employee computer install malware via email sniff non encrypted traffic compromise user data attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS HTTPS Firewall antivirus Use prepared statements IDS Log suspicious queries Log HTTP requests Log HTTP requests build web app vuln verification into CI/CD DETECT Distributed logging capability Container level logging Alerting capability Infrastructure level logging Serverless logging ???
  • 49.
    store api store front sensitive data passwords, keys user service productservice external system XXX 52 RESPOND Redirect to HTTPS Block consistent offenders Adjust firewall rules Block attackers Log suspicious queries Log HTTP requests Log HTTP requests attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS Use prepared statements build web app vuln verification into CI/CD gain access to internal network social engineering sniff non encrypted traffic steal sensitive user data HTTPS Firewall IDS Change DB Password Reset users passwords Inform users Redirect to HTTPS
  • 50.
    store api store front sensitive data passwords, keys user service productservice external system XXX 53 Log suspicious queries Block consistent offenders RECOVER Redirect to HTTPS Block consistent offenders Adjust firewall rules Block attackers Log suspicious queries Log HTTP requests Log HTTP requests attack store front / API sniff non encrypted traffic SQL Injection Alter query to get data modify data in DB HTTPS Use prepared statements build web app vuln verification into CI/CD gain access to internal network social engineering sniff non encrypted traffic steal sensitive user data HTTPS Firewall IDS Change DB Password Reset users passwords Inform users Redirect to HTTPS Restore from backup Fix Code, Blue/Green deploys: redeploy microservice(s) redeploy infrastructure
  • 51.
  • 52.
    • Due diligence:know thy playground • Think holistically: identify, protect, detect, respond, recover Summary 55 Make security a 1st class citizen in your thinking process!
  • 53.
    • Multiple, diverse,interconnected services • More varied attack surfaces • Harder to track what’s going on 
 (distributed, multi facetted logging capabilities)
 • Transient components • Dynamic transport level encryption (HTTPS) • Authentication & Authorisation (see David’s talk :) • Trash & burn recovery strategies Microservice security challenges 56
  • 54.
  • 55.
    59 A tool formanaging secrets and other sensitive content
  • 56.
    60 Deployment Tools Application Component/ Microservices service 1 service 2 Human Users
  • 57.
    61 • Unified APIto access multiple backends • ACL policies - who can access what • Audit Logs
  • 58.
    62 Unseal Init service 1 service 2 Allowtoken to be used by tools to access secrets Acquire policy constrained token Create microservice mount or area, add secrets System X
  • 59.
    63 $ vault init-key-shares=3 -key-threshold=2 Key 1: 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b Vault initialized with 3 keys and a key threshold of 2. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 2 of these keys to unseal it again. Vault does not store the master key. Without at least 2 keys, your Vault will remain permanently sealed. $ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Sealed: true Key Shares: 3 Key Threshold: 2 Unseal Progress: 1 Vault init & unseal $ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Sealed: true Key Shares: 3 Key Threshold: 2 Unseal Progress: 0
  • 60.
    64 $ vault init-key-shares=3 -key-threshold=2 Key 1: 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b Vault initialized with 3 keys and a key threshold of 2. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 2 of these keys to unseal it again. Vault does not store the master key. Without at least 2 keys, your Vault will remain permanently sealed. $ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Sealed: true Key Shares: 3 Key Threshold: 2 Unseal Progress: 1 Vault init & unseal $ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Sealed: true Key Shares: 3 Key Threshold: 2 Unseal Progress: 0
  • 61.
    65 $ vault init-key-shares=3 -key-threshold=2 Key 1: 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Key 2: 3fd762583cc9755222fa20f2a78770aca5ecba3b16d8c Key 3: a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Initial Root Token: 57dfce17-08c4-d042-91a3-68082965367b Vault initialized with 3 keys and a key threshold of 2. Please securely distribute the above keys. When the Vault is re-sealed, restarted, or stopped, you must provide at least 2 of these keys to unseal it again. Vault does not store the master key. Without at least 2 keys, your Vault will remain permanently sealed. $ vault unseal 8573410dd211cc9b5ea6b426b19d6d668e0184c39d4ba Sealed: true Key Shares: 3 Key Threshold: 2 Unseal Progress: 1 Vault init & unseal $ vault unseal a0428a6b6681eb15ffcea5be5c787beabcb7599a6fa91 Sealed: true Key Shares: 3 Key Threshold: 2 Unseal Progress: 0
  • 62.
  • 63.
    67 Unseal Create segregated area, policies, addsecrets Init Acquire policy constrained token Allow token to be used by tools to access secrets service 1 service 2 System X
  • 64.
    68 $ vault mount-path=usersvc generic Successfully mounted 'generic' at ‘usersvc'! $ vault mounts Path Type Default TTL Max TTL Description cubbyhole/ cubbyhole n/a n/a per-token private secr ... secret/ generic system system generic secret storage sys/ system n/a n/a system endpoints used f... usersvc/ generic system system Vault create new mount
  • 65.
    69 $ vault writeusersvc/db-password value=ASDKJ234SF*2 Success! Data written to: usersvc/db-password $ vault read usersvc/db-password Key Value lease_duration 2592000 value ASDKJ234SF*2 Vault write, then read back secret
  • 66.
    70 $ cat usersvc.policy path"usersvc/*" { policy = "read" } $ vault policy-write usersvc usersvc.policy Policy 'usersvc' written. Vault create custom policy
  • 67.
    71 Unseal Allow token to beused by tools to access secrets Init Acquire policy constrained token service 1 service 2 Create segregated area, add secrets System X
  • 68.
  • 69.
    Getting sensitive data intomicroservices … 73
  • 70.
    74 # Embedded Config spring.datasource.url=jdbc:mysql://localhost/test spring.datasource.username=dbuser spring.datasource.password=dbpass spring.datasource.driver-class-name= com.mysql.jdbc.Driver JavaCode @Component public class MyBean { private final JdbcTemplate jdbcTemplate; @Autowired public MyBean(JdbcTemplate jdbcTemplate) { this.jdbcTemplate = jdbcTemplate; } // ... } Starting point … user service db1
  • 71.
    75 # Embedded Config spring.datasource.url=jdbc:mysql://localhost/test spring.datasource.username=dbuser spring.datasource.password=dbpass spring.datasource.driver-class-name= com.mysql.jdbc.Driver JavaCode @Component public class MyBean { private final JdbcTemplate jdbcTemplate; @Autowired public MyBean(JdbcTemplate jdbcTemplate) { this.jdbcTemplate = jdbcTemplate; } // ... } Starting point … user service db1 Separate Code and Config - Especially Secrets!!
  • 72.
    76 # Embedded Config spring.datasource.url=jdbc:mysql://localhost/test spring.datasource.username=dbuser spring.datasource.password=dbpass spring.datasource.driver-class-name= com.mysql.jdbc.Driver JavaCode @Component public class MyBean { private final JdbcTemplate jdbcTemplate; @Autowired public MyBean(JdbcTemplate jdbcTemplate) { this.jdbcTemplate = jdbcTemplate; } // ... } Starting point … user service db1 Separate Code and Config - Especially Secrets!!DETECT https://github.com/michenriksen/gitrob https://github.com/awslabs/git-secrets
  • 73.
    77 Options • Push secretsin
 • Pull secrets out
 • Variations of the above …
  • 74.
    78 Push secrets in… user service db1 1authenticate 2 orchestration / deployment platform 3 provide value as environment variables read secret/db-password
  • 75.
    79 user service db1 1authenticate 2 read secret/db-password orchestration / deploymentplatform 3 provide value as environment variables $ vault auth e2d0a065-xxxx-yyyy-zzzz Successfully authenticated! You are… token_policies: [default, usersvc] $ vault read usersvc/db-password Key Value --- ----- refresh_interval 2592000 value MyClearTextPassword 1 2
  • 76.
    80 user service db1 1authenticate 2 read secret/db1 orchestration / deploymentplatform 3 provide value as environment variables $ # Start docker container,pass in vars docker run --name usersvc -e DB_USER="MyDBName" -e DB_PASSWORD="MyClearTextPassword" -d usersvc:v1 3
  • 77.
    81 Steal Sensitive UserDataIDENTIFY steal sensitive user data steal sensitive user data gain access to internal network gain access to user DB gain access to running user microservice(s) dump startup config steal plaintext password social engineering
  • 78.
    the-machine$ docker ps CONTAINERID IMAGE ... CREATED STATUS NAMES 9950ea8e3c59 product-service:v1 ... 4 days ago Up 4 days prodsvc 29b9ebca6dab user-service:v2 ... 5 days ago Up 5 days usersvc 82 gain access to running user microservice(s)
  • 79.
    83 gain access to internalnetwork find a disgruntled employee dump startup config the-machine$ docker inspect 29b9ebca6dab [ { "Id": “29b9ebca6dab147991201a5c61b72d3d546885ca8fd…”, "Created": "2016-06-27T21:26:16.126414991Z", "Args": [ "-jar", "UserService" ], "Config": { "Hostname": "29b9ebca6dab", "Env": [ “DB_USER=MyUserName", “DB_PASSWORD=MyClearTextPassword", “VAR1=something-else“ ], "Cmd": [ "java", "-jar", "UserService" ], ... } ]
  • 80.
    84 gain access to internalnetwork find a disgruntled employee the-machine$ docker inspect 29b9ebca6dab [ { "Id": “29b9ebca6dab147991201a5c61b72d3d546885ca8fd…”, "Created": "2016-06-27T21:26:16.126414991Z", "Args": [ "-jar", "UserService" ], "Config": { "Hostname": "29b9ebca6dab", "Env": [ “DB_USER=MyUserName", “DB_PASSWORD=MyClearTextPassword", “VAR1=something-else“ ], "Cmd": [ "java", "-jar", "UserService" ], ... } ] steal plaintext password
  • 81.
    85 gain access to internalnetwork gain access to user DB gain access to running user microservice(s) dump startup config social engineering PROTECT don’t expose as plain text steal sensitive user data steal plaintext password limit user access
  • 82.
  • 83.
    87 Push secrets in… user service db1 1authenticate 2 orchestration / deployment platform 3 provide value as environment variables read secret/db-password
  • 84.
    87 user service db1 1authenticate 2 read wrapped secret orchestration/ deployment platform 3 provide wrapped value as environment variables 4 unwrap Push wrapped secrets in …
  • 85.
    88 user service db1 1authenticate 2 read wrapped secret orchestration/ deployment platform 3 provide wrapped value as environment variables 4 unwrap $ vault read -wrap-ttl=60s usersvc/db-password Key Value --- ----- wrapping_token: 57ccef32-471d-869 wrapping_token_ttl: 60 wrapping_token_creation_time: 2016-06-28 22:.. 2
  • 86.
    89 user service db1 1authenticate 2 read wrapped secret orchestration/ deployment platform 3 provide wrapped value as environment variables 4 unwrap $ # Start docker container,pass in vars docker run --name usersvc -e DB_USER="MyDBName" -e DB_PASSWORD="57ccef32-471d-869" -d usersvc:v1 3
  • 87.
    90 user service db1 1authenticate 2 read wrapped secret orchestration/ deployment platform 3 provide wrapped value as environment variables 4 unwrap $ vault unwrap 57ccef32-471d-869 Key Value --- ----- refresh_interval 2592000 value MyClearTextPassword 4
  • 88.
    91 dump startup config the-machine$ dockerinspect 29b9ebca6dab [ { "Id": “29b9ebca6dab147991201a5c61b72d3d546885ca8fd…”, "Created": "2016-06-27T21:26:16.126414991Z", "Args": [ "-jar", "UserService" ], "Config": { "Hostname": "29b9ebca6dab", "Env": [ “DB_USER=MyUserName", “DB_PASSWORD=57ccef32-471d-869", “VAR1=something-else“ ], "Cmd": [ "java", "-jar", "UserService" ], ... } ]
  • 89.
    92 gain access to internalnetwork gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data steal plaintext password don’t expose as plain text gain access to user DB limit user access
  • 90.
    93 gain access to internalnetwork gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data steal plaintext password don’t expose as plain text gain access to user DB limit user access
  • 91.
    94 gain access to internalnetwork gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data don’t expose as plain text gain access to user DB steal wrapped password get real password limit user access
  • 92.
    95 user service db1 1authenticate 2 read wrapped secret orchestration/ deployment platform 3 provide wrapped value as environment variables 4 unwrap $ vault unwrap 57ccef32-471d-869 error reading cubbyhole/response: Error making API request. URL: GET https://vault:8200/v1/cubbyhole/response Code: 400. Errors: * permission denied 4
  • 93.
    96 gain access to internalnetwork gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data DETECT don’t expose as plain text gain access to user DB steal wrapped password get real password Raise TOFU alarm Audit access limit user access
  • 94.
    97 gain access to internalnetwork gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data DETECT don’t expose as plain text RESPOND gain access to user DB steal wrapped password get real password Raise TOFU alarm Audit access change DB password limit user access
  • 95.
    98 gain access to internalnetwork gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data DETECT don’t expose as plain text RESPOND gain access to user DB steal wrapped password get real password Raise TOFU alarm Audit access change DB password Expect secrets to change. Make a habit of changing them regularly. It will naturally force you to put measures in place. limit user access
  • 96.
    • Dynamic Secrets:Auto generate credentials on the fly Other handy options 99
  • 97.
    100 user service db1 1authenticate 2 read dynamic password orchestration/ deployment platform 3 provide value as environment variables 0 Human / Other System Users
  • 98.
    101 user service db1 1authenticate 2 orchestration / deployment platform 3 providevalue as environment variables $ vault mount postgresql Successfully mounted 'postgresql' at 'postgresql'! $ vault write postgresql/config/connection connection_url="postgresql:// vault:somepassword@yourhost:5432/postgres" $ vault write postgresql/roles/usersvc-ro sql="CREATE ROLE "{{name}}" WITH LOGIN PASSWORD ‘{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA users TO "{{name}}";" Success! Data written to: postgresql/roles/ read dynamic password Human / Other System Users 0 0
  • 99.
    102 user service db1 1authenticate 2 orchestration / deployment platform 3 providevalue as environment variables $ vault mount postgresql Successfully mounted 'postgresql' at 'postgresql'! $ vault write postgresql/config/connection connection_url="postgresql:// vault:somepassword@yourhost:5432/postgres" $ vault write postgresql/roles/usersvc-ro sql="CREATE ROLE "{{name}}" WITH LOGIN PASSWORD ‘{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA users TO "{{name}}";" Success! Data written to: postgresql/roles/ read dynamic password Human / Other System Users 0 0
  • 100.
    103 user service db1 1authenticate 2 orchestration / deployment platform 3 providevalue as environment variables $ vault mount postgresql Successfully mounted 'postgresql' at 'postgresql'! $ vault write postgresql/config/connection connection_url="postgresql:// vault:somepassword@yourhost:5432/postgres" $ vault write postgresql/roles/usersvc-ro sql="CREATE ROLE "{{name}}" WITH LOGIN PASSWORD ‘{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA users TO "{{name}}";" Success! Data written to: postgresql/roles/ read dynamic password Human / Other System Users 0 0
  • 101.
    104 user service db1 1authenticate 2 orchestration / deployment platform 3 providevalue as environment variables $ vault read postgresql/creds/usersvc-ro Key Value lease_id postgresql/creds/usersvc-ro/ c888a097-b0e2-26a8-b306-fc7c84b98f07 lease_duration 3600 password 34205e88-0de1-68b7… username vault-14301-usersvc-ro read dynamic password Human / Other System Users 0 2
  • 102.
    105 user service db1 1authenticate 2 orchestration / deployment platform 3 providevalue as environment variables $ # Start docker container,pass in vars docker run --name usersvc -e DB_USER="vault-14301-usersvc-ro" -e DB_PASSWORD="34205e88-0de1-68b7" -d usersvc:v1 read dynamic password
  • 103.
    • Dynamic Secrets:Auto generate creds on the fly • Ability to combine security primitives
 dynamic secrets + resource wrapping Other handy options 106
  • 104.
    107 gain access to internalnetwork gain access to user DB gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data DETECT steal wrapped password don’t expose as plain text get real password Raise TOFU alarm Audit access RESPOND change DB password
  • 105.
    108 gain access to internalnetwork gain access to user DB gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data DETECT steal wrapped password don’t expose as plain text get real password Raise TOFU alarm Audit access RESPOND change DB password use time limited dynamic creds
  • 106.
    109 gain access to internalnetwork gain access to user DB gain access to running user microservice(s) dump startup config find a disgruntled employee PROTECT steal sensitive user data DETECT steal wrapped password don’t expose as plain text get real password Raise TOFU alarm Audit access RESPOND change DB password use time limited dynamic creds compromise orchestration platform
  • 107.
    Turtles all theway down!
  • 108.
    111 gain access to internalnetwork gain access to user DB gain access to running user microservice(s) dump startup config compromise orchestration platform find a disgruntled employee steal sensitive user data steal vault token get db password 1 2 34 Defense in Depth
  • 109.
    Put enough hurdlesin the way of attackers for you to stop when you can, but if not, to be able to … - realise what’s going on - react before too much damage is done 112
  • 110.
    • Centralised SecretsManagement • API - helps with automation • Tries to address concerns across full security lifecycle • But still very new & maturing Vault Summary 113
  • 111.
    • Encryption asa service: offload responsibility to Vault • PKI: Generates X.509 certificates dynamically based on configured roles • SSH: Dynamically generates SSH credentials for remote hosts Other Handy Features 114
  • 112.
  • 113.
    116 Make security afirst class citizen! Don’t try and just bolt it on at the end!
  • 114.
    117 Think holistically aboutsecurity Don’t stop at the protect stage!
  • 115.
    118 Choose the righttech for the job Microservice architectures add complexity
  • 116.
    119 Do your best! butdon’t do nothing!
  • 117.