Container security involves securing the host, container content, orchestration, and applications. The document discusses how container isolation evolved over time through namespaces, cgroups, capabilities, and other Linux kernel features. It also covers securing container images, orchestrators, and applications themselves. Emerging technologies like LinuxKit, Katacontainers, and MirageOS aim to provide more lightweight and secure container environments.

1. Container Security
Salman A. Baset
@salman_baset, sabaset@us.ibm.com

2. What is a container?
2

3. What is a container?
According to NIST (National Institute of Standards Technology)
• Virtualization: the simulation of the software and/or hardware upon which
other software runs. (800-125)
• System Virtual Machine: A System Virtual Machine (VM) is a software
implementation of a complete system platform that supports the execution of
a complete operating system and corresponding applications in a cloud. (800-
180 draft)
• Operating System Virtualization (aka OS Container): Provide multiple
virtualized OSes above a single shared kernel (800-190). E.g., Solaris Zone,
FreeBSD Jails, LXC
• Application Virtualization (aka Application Containers): Same shared kernel
is exposed to multiple discrete instances (800-180 draft). E.g., Docker
(containerd), rkt
3

4. “Container” Security
Orchestrator
Security
Content
Security
Host
Security= + +
2002 – to-date2015 – to-date2016 - date
Implement defense in depth
App
Security +
4
* - date

5. Container Runtime Stack on a Single Host
Host kernel
Container
runtime
VM kernel
Host kernel
Container
runtime
Application
containers
App
Bins/libs
App
Bins/libs
App
Bins/libs
App
Bins/libs
Application
containers
1. Is host isolated from application container?
2. Is one application container isolated from another application container?
5

6. Shared kervel vs. separate kernel debate…
CVEs inside Linux kernel
https://www.cvedetails.com/product/47/Linux-Linux-Kernel.html?vendor_id=33 6

7. The previous chart can be misleading…
• Gain privilege exploits in 2017: 36
• Includes CVEs in previous kernel versions
• 1 eCryptfs 3.18 or earlier
• Drivers: 5 Qualcomm drives, 2 NVIDIA, 1 USB, 1 BROADCOM, 1 GPU,
1 TTY
• KVM: 1 (that is, CVE found in kvm also)
• Not all CVEs are [easily] exploitable, but nevertheless…
Host kernel
Container
runtime
App
Bins/libs
App
Bins/libs
Application
containers
Focus of this talk
7

8. Usage Model
Host kernel
Container
runtime
App
Bins/libs
App
Bins/libs
Application
containers
Host kernel
Container
runtime
App
Bins/libs
App
Bins/libs
Application
containers
Host kernel
Container
runtime
App
Bins/libs
App
Bins/libs
Application
containers
Multiple apps of same user running on same/different hosts
Multiple apps of distinct users running on same/different hosts
Image
Registry
Orchestrator + +
8

9. What does host security mean? Informally…
• Isolation from host
• Can never be root
• Should not “see” host processes
• Shout not interfere with host operation
• Isolation from other containers
• Should not “see” other containers
• Should not impact performance of other containers
9

10. Container on Linux
• is just a process
• wrapper in bunch of “isolation gear”
• to isolate from host and other processes
• the isolation gear was developed independently over time
• docker engine (on a host) + containerd + runC
10

11. Namespaces
• Linux kernel namespaces provide the isolation
(hence “container”) in which we place one or more
processes
• Introduced ~2002 – inspired from Plan B
• Ok, what about devices?
• docker options
• --userns
• --pid
• --uts
pid mount
ipc
user net
uts
HOST SECURITY
11

12. Resource isolation - cgroups
• What is a
resource?
• CPU, memory,
disk, network
• PID, file
descriptors
• Devices
• cgroups v2 support introduced in Kernel 4.5.
• Docker has many options for tuning cpu, mem, disk
• cpu: 10
• device: 8
• memory: 5
• PID: configure the max number of PID descriptors
• Device: maximum device bandwidth, in, out
HOST SECURITY
12

13. Are namespaces and cgroups enough?
• No
• Linux capabilities:
• Fine-grained access capabilities besides root/non-root
• E.g., load a module, mount a file
• Docker container drops most capabilities by default
• chown, dac_override, fsetid, fowner,
mknod, net_raw, setgid, setuid, setfcap,
setpcap, net_bind_service, sys_chroot,
kill, audit_write
• Seccomp
• Restrict the system calls that a system is allowed to execute
• Often, security issues found in system calls pertaining to legacy
devices
• AppArmor / SELINUX
• Mandatory access control (MAC)
Default Docker capabilities
HOST SECURITY
13

14. Is that enough for host security?
• No
• Other host security best practices still apply!
• Patch management
• Monitoring
• Antivirus
• Malware
• Logging
• Privileged user monitoring
HOST SECURITY
14

15. Container Images
• Container images are the root of software supply chain
• Containers images are supposed to provide an immutable source
• In reality: secrets needed for running the container are stored outside the
container image, and can impact its run-time
• Different secrets for different run-time environments (stage, prod)
• Container images are often pulled through open source
• Important to vet the base image, malware, anti-virus
• Container images contain OS distro packages + application
packages + application code
• Ubuntu, Node.js
• Check vulnerabilities of all content that goes inside the image, including
scanning application source code
CONTENT SECURITY
App
Bins/libs
secrets
15

16. Orchestrator Security
• Containers are often deployed through orchestrators
• Docker Machine, Kubernetes
• Sane defaults, TLSv1.2 etc
• Who has access to those APIs just equally as important as other
aspects
• determine what volumes, devices are passed inside container
• who can docker exec into a running container
• Moreover, secrets are typically stored outside container registry
and container run-time. Access to these secrets also important
• Your Jenkins server may be leaking secrets!
ORCHESTRATOR SECURITY
16

17. App Security
• A developer still needs to configure their app securely.
• However it is still much better to run app in a container than inside a host
APP SECURITY
17
Configurepartitions
Runasnon-root,
utilizinguserand
kernelnamespaces
Configurelog,
monitoring,audit
Configurenetwork,
relatedservices,
disableIPforwarding
Configurepatching
andanti-virusagents
Configuremandatory
accesscontrol
Preventapplication
fromDoSingahost
Furtherdeprivilege
applicationthrough
subsetofLinux
capabilities
FurtherDeprivilege
applicationbylimiting
allowedsystemcalls
Configureapplication
security
Host
Container
Host
Manual
Manual
Manual
per app
One time
(in Docker)
Manual
per app
Manual
One time
Step 1 Step 2 Step 3
Manual
Manual
Step 4
Manual
Manual
Step 5 Step 6
Manual
per app
Out of box
(in Docker)
Out of box
(in Docker)
Step 7
Manual
per app
Step 8
Manual
per app
Out of box
(in Docker)
Manual
per app
Out of box
(in Docker)
Step 9 Step 10
Manual
per app
Manual

18. Kernel and Container Security Evolution
• 2015
• Content Addressability (image spec, phase 1) – Docker 1.6
• Default ulimits for all containers – Docker 1.6
• Docker Content Trust – (notary) image provenance/signing – Docker 1.8
• 2016
• Full migration to content addressability for images/layers – Docker 1.10
• User namespaces – Docker 1.10
• Secure computing (libseccomp) – Docker 1.10
• --pids-limit (cgroups pid limitation) – Docker 1.11 (kernel 4.3 +)
• cgroups “v2” – (kernel 4.5+)
• --no-new-privileges (limit process escalation) – Docker 1.11
• Storage driver quotas (limited) – Docker 1.12
• Secure by default multi-node orchestration (mutual TLS) – Docker 1.12
• 2017
• Improved resource isolation features in Linux Kernel and in Docker engine
18

19. Whats Next in Container Security?
• Host Security
• Linux Kit
• Lightweight virtualization (Katacontainers (formerly Intel clear
containers) )
• Unikernels (MirageOS)
• Content Security
• Distribution Specific Project (OCI)
• Project Grafeas
19

20. Linux Kit
• Secure, portable OSes for Linux containers
• Support Kubernetes, AWS, GCP, Azure etc
• https://github.com/linuxkit/linuxkit
20

21. Katacontainers
• Based on Intel Clear Containers
• Perform like containers but provide the workload isolation and
security advantages of VMs
• OCI compliant
• https://github.com/kata-containers/
21

22. MirageOS
• Library operating system for unikernels
• MirageOS3.0 released in Feb 2017
• https://mirage.io/
22

23. Distribution Specific Project
• Image and run-time formats through OCI
• Image distribution is now defacto Docker Image Registry API
• https://www.opencontainers.org/announcement/2018/04/09/oci-
announces-dist-spec-project
23

24. Project Grafeas
• Standardize container vulnerability format
• Standardize attestations
• https://grafeas.io/
24

25. Conclusion
• Container security has significantly evolved from early days of
Docker
• Secure-by-default in all major platforms
• For users, security focus is on securing software supply chain
• New features promise standardization, light-weight, VM
isolation
25