Containers have had an incredibly large adoption rate since Docker was launched, especially from the developer community, as it provides an easy way to package, ship, and run applications. Securing your container-based application is now becoming a critical issue as applications move from development into production. In this session, you learn ways to implement storing secrets, distributing AWS privileges using IAM roles, protecting your container-based applications with vulnerability scans of container images, and incorporating automated checks into your continuous delivery workflow.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
December 2016
CON402
Securing
Container-Based Applications
Henrik Johansson, Security Solutions Architect, AWS
Michael Capicotto, Solutions Architect, AWS

2. What to Expect from the Session
OS & Container Security
• Intro to container security and best practices
• Container lifecycle security (including vulnerability analysis)
• Managing secrets
Infrastructure security
• Enforce governance with Amazon ECS
• Using containers to automate security deployments

3. Intro to Container Security

4. Container vs VM isolation

5. Isolation
• Implemented via Linux namespaces
• Weaker than VM isolation
• Containers run on single kernel
• Containers share files, sockets, memory areas,
devices, etc.

6. Secure by default
pid namespace
nmt namespace
net namespace
uts namespace
user namespace
pivot_root
uid/gid drop
cap drop
all cgroups
selinux
apparmor
seccomp
Out-of-the-box default
settings and profiles
Granular controls to
customize settings
No SSH to containers
No system users
Immutable infrastructure
Secure
by
default

7. Defense through segmentation
• Run containers on top of virtual instances
• Keep instance and container up to date
• IAM roles for tasks to restrict host permissions
• Segregate containers
• Role/customer
• Risk/trust/exposure

8. Reduce attack surface area
• Container attack surface much smaller than VM
• Service, not system
• Emulate drivers
• VENOM attack / CVE-2015-3456
(http://venom.crowdstrike.com/)
• Best practice: run small images
• Only contain static binary
• Harden the cluster instance
• NIST, SANS, etc.
• Use many but smaller instances to limit blast radius

9. Best practices
• Limit memory, CPU allocation and resource limits
• Set filesystems to be read-only
• Limit container networking
• Remove setuid/setgid binaries from images
• Set containers to run as non-root user
• Leverage Linux kernel security features

10. Setting resource limits in ECS
Configured in the ECS task definition
• CPU
• Minimum number of CPU units to reserve for container
• RAM
• The number of MiB of memory reserved for container
• Ulimits
• Reduce for example filesize locks and memlocks allowed
"ulimits": [
{
"name": "core"|"cpu"|"data"|"fsize"|"locks"|"memlock"|"msgqueue"|"nice"|"nprock"…
"softLimit": integer,
"hardLimit": integer
}
...

11. • Configured in the ECS task definition
• Setting root file system to be read-only
• Maps to Docker ReadOnlyRootfs and --read-only
• Set volumes to be read-only
Read-only filesystem access
"readonlyRootFileSystem": true
"mountPoints": [
{
"sourceVolume": "string",
"containerPath": "string",
"readOnly": true
}
]

12. Securing container networking
• Containers should only open ports it needs
• Governance on task definition
• Pre deploy assessment
• Know expected traffic patterns
• North/South, East/West
• Disable inter-container communication
• Set Docker option flag --icc=false and --iptables=true
• Use --link to connect containers

13. Securing container networking -
• Detects threats from external and internal networks
• Attacks such as DDOS and XSS on containers
• Protects your container network from L3 to L7
• Detects application threats
• Has built-in security policies for 30+ applications and
protocols
• Prevents unauthorized connections between containers
• Monitors all ‘east-west’ container traffic
• Integrate logs with SIEM/Log aggregator

14. Remove setuid/setgid binaries
• Most apps don’t need access to setuid/setgid binaries
• Remove to avoid privilege escalation attacks
• Example Debian “defanged ” image Dockerfile:
FROM debian:wheezy
RUN find / -perm +6000 –type f -exec chmod a-s {} ; || true

15. Set a non-root user
Multiple methods
• Create the USER in Dockerfile
• Change to the user via USER command or sudo/gosu/su
• Set USER in Docker command or ECS task def
Don’t use root as default!

16. Seccomp
• Secure computing mode (Seccomp) is a Linux kernel
feature
• Supported from Docker 1.10
• Docker has default profile limiting many system calls (44
system calls out of 300+)
• Can also be customized for more blocks
• Can provide custom seccomp profile to containers via
Docker security options

17. SELinux
• Implementation of Mandatory Access Control (MAC)
• Default SELinux policy for Docker designed to protect host
from containers and containers from each other
• Uses tag to restrict usage
• Containers assigned default process type: svirt_lxc_net_t
and files accessible to container: svirt_sandbox_file_t.
• Policy enforces only read/execute container permissions to
/usr directory
• Assigns unique MCS category number to prevent cross
container access to files or resources

18. Security options in ECS

19. Security options in ECS
ECS_SELINUX_CAPABLE=true

20. Container Lifecycle Security

22. Lifecycle risks
• Are my developers
introducing new
vulnerabilities
during
development?
• Are my container
images in my
repository
vulnerable?
• Do I have a runtime
defence if containers are
compromised?
• How do I ensure by
Docker platform is
secure over time?

23. Mitigation approaches
• Do image
vulnerability analysis
in the CI/CD pipeline
• Only allow
developers to use
“approved” images
• Configuration
governance
• Regularly analyse
images in your
private repository
• Only allow compliant
images to run in
production
• Have runtime defence
in place
• Regular checks of
Docker platform
against industry
benchmarks
Own your repo!

24. Secure Host via Docker Bench
• Recommendations provided by Center for Internet
Security’s Benchmark for Docker Engine
• Best practice: use Docker Bench to validate host
against recommendations
• www.dockerbench.com

25. Run DockerBench via EC2 Run Cmd
Output report to
CloudWatch logs
Create CloudWatch alarm
on non-compliance
Automating Docker Bench
Invoke Lambda on schedule

27. Scan running containers for vulnerabilities in dev, test, staging, and
production.
• Public CVE vulnerabilities
• Some application specific vulnerabilities which may or may not have
CVE
• Scan the host file system, not just containers
• Can to tell whether the host is safe or not to load application
containers.
Visualize services, containers, and network behavior easily in staging
and production
Protects running containers against violations, threats, and
vulnerabilities

28. Continuous integration

29. CI/CD pipeline / Container lifecycle
CodeCommit
Source Build CodePipeline
Jenkins
1. Build Docker image
2. Check image for vulnerabilities
3. Push image to ECR
4. Update ECS service via CloudFormation
Run
Live scan

30. Managing Secrets

31. Bake into image

32. Bake into image

33. Environment variables
• Suggested by 12-factor apps
• Environment variables can be seen in
too many places
• linked containers, ECS API calls, docker
inspect
• Can’t be deleted

34. S3-based secrets storage
• Secrets stored in S3 bucket
• Accessed via IAM roles for EC2
• Enforce encryption at rest and flight via IAM
policies and KMS
• Use VPC endpoint for S3 to lock down access
from certain VPCs
https://blogs.aws.amazon.com/security/post/Tx2B3QUWAA7KOU/

35. Third-party secrets management solutions
• Vault from HashiCorp
• https://www.vaultproject.io/
• Keywhiz from Square
• https://square.github.io/keywhiz/
• Secrets accessed via leases
• Good for Dynamic Secrets generation

36. Temporary credentials for RDS databases
Vault for RDS credentials

37. Enforce Governance with ECS

38. ECS primer
ECS cluster
…
Task 2
Task N
Task 1
Task definition 1 Amazon ECR
Docker images

39. Enforcing governance… is simple!
Task definition controls
• Which Docker images to use (app_latest)
• CPU & memory allocation
• Container links
• Host-to-container port mappings
• **NEW** IAM roles for tasks

40. IAM roles for tasks
Benefits
• Simplify usage of AWS SDKs in containers
• Credential isolation between tasks
• Authorization per task
• Auditability in CloudTrail with taskArn

41. IAM roles for tasks - example
Web UI
Container
Cluster Instance
Data Insights
Container
ECS Task 2ECS Task 1
IAM role
The old way…
Amazon
DynamoDB
Amazon S3
Undesired
permission
Undesired
permission

42. IAM roles for tasks - example
Web UI
Container
Cluster Instance
Data Insights
Container
ECS Task 2ECS Task 1
IAM Role 1
The new way!
IAM Role 2
Amazon
DynamoDB
Amazon S3

43. IAM roles for tasks – explained
1. ECS agent periodically queries ECS control plane
2. Control plane generates ID token
 Auto-rotated
3. ECS agent:
 Constructs HTTP URL for each container
 Sets AWS_CREDENTIALS_ENDPOINT in HostConfig
4. AWS SDK extracts URL

44. IAM roles for tasks - demo

45. Automate Security Deployments

46. Governance during continuous integration
Docker image
Developers Security Engineers Ops Engineers
Amazon ECR
Task definition

47. Governance during continuous deployment
Developers Ops Engineers Security Engineers
App Team
Developers Ops Engineers Security Engineers
InfoSec Team

48. Docker Images
Web Application Host-Based Intrusion Detection
Securely merge…
…and deploy
Governance during continuous deployment
App Team InfoSec Team
Task Definition Docker Images Task Definition

49. Deploying security functionality - example
Rate limiting proxy
Web Application
Container
Cluster instance
Reverse-Proxy
Container
Outbound
network
access
ECS Task

50. Deploying security functionality - demo

51. Why should I care?
• Remove accidental conflicts
• Make security processes continuous and automatic
• Encapsulate software artifacts and implement controls
one level up
• Control changes to this framework via IAM

52. Not just containers!
Amazon Machine
Images (AMIs)
Docker Images
OS Packages
Amazon EC2
Container Service
AWS
CloudFormation
AWS CodeDeploy

53. Amazon Machine
Images (AMIs)
Docker Images
OS Packages
Amazon EC2
Container Service
AWS
CloudFormation
AWS CodeDeploy
Software Artifacts Deployment Services
Not just containers!

54. Not just containers!
CloudFormation
Template
Task Definition
Application Specification
File (AppSpec.yml)
…applies to any
deployment
process.

55. Thank you!

56. Remember to complete
your evaluations!