Docker is two years old. While security has always been at the core of the questions revolving around Docker, the nature of those questions has changed. Last year, the main concern was "can I safely colocate containers on the same machine?" and it elicited various responses. Dan Walsh, SELinux expert, notoriously said: "containers do not contain!", and at last year's LinuxCon, Jérôme delivered a presentation detailing how to harden Docker and containers to isolate them better. Today, people have new concerns. They include image transport, vulnerability mitigation, and more. After a recap about the current state of container security, Jérôme will explain why those new questions showed up, and most importantly, how to address them and safely deploy containers in general, and Docker in particular.

1. Containers, Docker,
and Security:
State ofthe Union
1 / 43

2. Who am I?
Jérôme Petazzoni (@jpetazzo)
French software engineer living in California
Joined Docker (dotCloud) almost 5 years ago
(I was at Docker before it was cool!)
I built and scaled the dotCloud PaaS
(millions of containers, no known security issues)
I learned a few things about running containers
(in production)
2 / 43

3. Outline
Yesterday
Today
Tomorrow
3 / 43

4. Yesterday
4 / 43

5. Containers and Security yesterday
Yesterday = Summer 2014.
At LinuxCon and OSCON, I gave a talk to answer the question:
"Is it safe to run applications in containers?"
5 / 43

6. Containers and Security yesterday
Yesterday = Summer 2014.
At LinuxCon and OSCON, I gave a talk to answer the question:
"Is it safe to run applications in containers?"
this really meant
"Can one container break out, and into another?"
6 / 43

7. Containers and Security yesterday
Yesterday = Summer 2014.
At LinuxCon and OSCON, I gave a talk to answer the question:
"Is it safe to run applications in containers?"
this really meant
"Can one container break out, and into another?"
Main concern: isolation
7 / 43

8. What was the answer?
8 / 43

9. What was the answer?
9 / 43

10. What was the answer?
"It's complicated"
Long list of recommendations
some were easy
(and automatically enforced by Docker)
some were not obvious
(and had to be enabled manually)
some were hard to deploy
(or required missing kernel features)
Video and Slides
10 / 43

11. How is this different today?
People still ask about container isolation
Much more frequently, they ask about
image security and provenance
They want to know:
if dockerpulldebianis what it claims to be
if jpetazzo/dindhas vulnerabilities
if a given image has been vetted by their sec team
11 / 43

12. Why has it changed?
Who cares about container isolation?
hosting providers (more density = more $$$)
PAAS (for rapid deployment; on-demand activation)
→ early adopters
Who doesn't care about container isolation?
people who use VMs only because autoscaling
people who would put multiple components per
machine anyway
→ second wave of users
12 / 43

13. Today
13 / 43

14. Docker and Containers Security Today
Improving what we had yesterday
(fine-grained permissions, immutable containers)
Addressing new challenges
(provenance, content verification, notary)
Defense in depth
(containers + VM)
The infosec mindset
(better upgrades, security benchmarks, policies)
14 / 43

15. Finer-grained permissions
Per-container ulimit
Capability reduction with --cap-drop/ --cap-add
allow network config: --cap-addnet_admin
forbid everything: --cap-dropall
Device access restrictions with --device
(better than --privileged!)
Improved handling of LSM (SELinux, AppArmor)
example: @frazelledazzell's bane
(AppArmor profile generator for docker containers)
15 / 43

16. Smaller attack surface
Hardware management done on the host
(no kernel, drivers, device handling... in containers)
Package management is optional
(once a container is built, it doesn't need to be changed)
Minimal distros can be used
(e.g. buildroot, Alpine Linux...)
Less software = less risk
16 / 43

17. Immutable containers
dockerrun--read-only
(makes it impossible to entrench in a container)
Helps with vulnerability detection
(audit can be performed on offline images)
Even without --read-onlyflag:
copy-on-write prevents changes from being permanent
break a container when hacking it → it gets recycled
dockerdiffallows easy audit of changes
17 / 43

18. Image provenance
How can I trust dockerpulldebian?
I must trust upstream
(i.e. Debian and whoever maintains the image)
I must trust Docker Inc.
(operator of the Hub)
I must trust the transport
(between the Hub and my Docker host)
18 / 43

19. Image provenance
How can I trust dockerpulldebian?
I must trust upstream
(i.e. Debian and whoever maintains the image)
I must trust Docker Inc.
(operator of the Hub)
I must trust the transport
(between the Hub and my Docker host)
That's a lot of trust
19 / 43

20. "I don't want to trust anybody!"
If you don't trust upstream, you have to ...
stop using apt-getand yumwith public repos
rebuild everything from source
verify source integrity
(full audit + review all changes)
If you don't trust Docker Inc., you probably should ...
audit the whole Docker Engine code
audit every single patch that goes into Docker
(if you can do that ... we're looking for reviewers)
20 / 43

21. "I'll never trust people who do curl|sh!"
21 / 43

22. "I'll never trust people who do curl|sh!"
22 / 43

23. "I'll never trust people who do curl|sh!"
23 / 43

24. Revisiting curl|sh
It's auditable
curlhttps://...>review.sh
^
lessreview.sh
sh-exreview.sh
Do you install deb/rpm/npm/etc packages?
Do you review their postinstall scripts?
(which, by the way, run as root)
Have you ever uploaded an npm package?
Did you have to disclose your identity,
sign your upload, anything like that?
(no)
24 / 43

25. Think twice
npminstallfoois way more dangerous than curl|sh
If you're on OS X, you're probably using Brew
ruby-e"$(curl-fsSL
https://raw.githubusercontent.com/Homebrew/install/master/install)"
See also: Curl | bash a victimless crime?
25 / 43

26. Security reminder
It's OK to be paranoid, but beware of:
Bumps in the carpet
(moving a problem rather than solving it)
Usability
(if security makes it hard/impossible to work,
people will work around it!)
Tinfoil hats
26 / 43

27. Can we trust the transport?
Registry v1 protocol had serious issues:
arbitrary layer IDs
no integrity check
(other than TLS transport integrity)
Registry v2 protocol has:
content-based layer IDs
signed image manifests
Is that enough?
27 / 43

28. Notary:
a better trust
framework
28 / 43

29. What are we trying to address?
Distributed content should be signed
Stealing a key should be hard
Stealing a key shouldn't have dire consequences
Replay attacks should be hard
(=can't serve you yesterday's vulnerable version)
Should use known models and research
Existing distribution infrastructure should be used
(=HTTP, HTTPS, FTP… are good)
Trusting Docker Hub should not be mandatory
29 / 43

30. Notary
Based on TUF (The Update Framework)
Sign content with offline keys
Trust thresholds (require K out of N keys)
(Stealing a key reduces signing requirements,
but doesn't break the whole model)
Guarantee freshness
Distribute signed content on (potentially insecure) servers
(leverage existing (insecure) transport and mirrors)
Enabled in Docker 1.8 by setting DOCKER_CONTENT_TRUST
30 / 43

31. Defense in depth
So, VM or containers?
31 / 43

32. Defense in depth
So, VM or containers?
VM and containers!
32 / 43

33. Defense in depth
So, VM or containers?
VM and containers!
Reduce number of VMs
(when security perimeter allows it)
Colocated containers are safer than colocated processes
Malicious code has to escape both layers
Docker provides an extra layer of isolation
Applications are safer with containers than without
33 / 43

34. The infosec mindset
Better upgrades
Accurate, actionable security benchmarks
Clear, sensible security policies
34 / 43

35. Better upgrades
Dockerfile= easy, fast, reliable builds and rebuilds
"But now I have 1000s of container images to upgrade!"
Yes, but that's way better than the 100s of server images
that you had before
The organizational risk is lower
(because if something goes wrong,
you have reliable rollbacks)
35 / 43

36. Security benchmarks
CIS (Center of Internet Security) Docker Benchmark
Docker Bench (https://dockerbench.com) :
automated assessment tool to check compliance
36 / 43

37. Policies
Docker Inc. (the company) and the Docker Project (open
source) have clear security guidelines
Mandatory code reviews (see CONTRIBUTING.md) to
ensure quality of code base
Quarterly security audits and pen tests of our
infrastructure
We support responsible disclosure
37 / 43

38. Tomorrow
38 / 43

39. Container security in the future
Personal predictions - not Docker Inc.'s roadmap!
39 / 43

40. Container security in the future
Personal predictions - not Docker Inc.'s roadmap!
Offline image audit
Hardening of immutable containers (noexec, nosuid)
Better GRSEC, PAX, LSM integration
User namespaces (on experimental.docker.com)
Better default seccomp profiles
40 / 43

41. Last words
David Mortman at DEFCON this year:
“ A year ago, [Docker and security] was pretty horrible,
six months ago it wasn't so bad, and now it's pretty usable.
41 / 43

42. Resources
Docker security page
Docker security presentation at DockerCon 2015 SF
Docker Security CheatSheet
Notary on GitHub
Docker Bench for Security
42 / 43

43. Thanks!
Questions?
@jpetazzo
@docker
43 / 43