Docker for HPC in a Nutshell

© 2013 Cassini Consulting
Andreas Schmidt | @aschmidt75
Docker: User Friendly Application & Service
Containers for HPC Environments
Docker In A Nutshell

4 16.07.15 Cassini Consulting
vs.Pets Cattle

16.07.15 Cassini Consulting6
Technical Perspective
Developer Perspective
Security
Ecosystem & Partners

Security

Docker Daemon
Docker Engine core components
Containers
Images
Containers

Docker Daemon
Docker Engine core components
Containers
Images
Containers
API
Docker
Client
Docker
Client

The Docker workflow & commands
Image

Registry
Image
pull

Registry
Image
Docker-
file
pull
build

Registry
Image
Docker-
file
push
pull
build

Registry
Image
Container
Docker-
file
push
pull
run
build

Registry
Image
Container
Docker-
file
push
pull
run
commit
build

Registry
Image
Container
Docker-
file
push
pull
run
commit
start,
stop,
...
build

Registry
Image
Container
Docker-
file
✖
push
pull
run
commit
rm
start,
stop,
...
build

Registry
Image
Container
Docker-
file
✖ ✖
push
pull
run
commit
rm
rmi
start,
stop,
...
build

Registry
Image
Container
Docker-
file
External
Tarball
✖ ✖
push
pull
run
commit
rm
rmi
start,
stop,
...
build
export

Docker Engine technology foundation
Isolation through Kernel Namespaces
Linux Capabilities
Resource limitation through Linux control groups
Filesystem isolation, Copy-On-Write & Union FS

Docker Engine technology foundation
21 16.07.15 (*) https://www.docker.com/whatisdocker

Virtual Machines vs. Containers
22 16.07.15 (*) https://www.docker.com/whatisdocker

Security

From Development to Production: Challenges ahead!

What Docker gives to developers
Easy sandbox approach
Easy way to deliver software
Dev / Prod parity

Easy Sandbox approach
§  Create build and test environments
§  Choose Libraries and Framework Dependencies, per application
§  Deal with incompatible version mismatches
(Ruby 1.9 vs. Ruby 2.1, including libs, bundler, version switchers, ...)
§  Lightweight alternative to using virtual machines

Easy Sandbox approach
A Docker Image contains its own userland libraries and binaries
§  separated from other images
§  take exactly the versions of libraries you need
§  leave out things you do not need
§  reproducible, lightweight, easily testable
§  look at it as a unit of delivery

Code Deliverables (i.e. RPM
packages) are not runnable.
An installed instance (i.e. in a
VM) is hard to transport.

https://twitter.com/joyent/status/565243828718678016

Dockerfile as a Contract between Development and Operations
what to base from
set environment params
prepare the image, i.e.
install something,
configure it
describe the interface
what to run

Dockerfile as the blueprint for reusable building blocks
what to base from
Redis is an official repository
at dockerhub

Pave the way for Dev/Prod parity
§  From Applications to Systems
§  Describe not only compute, but also storage and networking.
§  Example: docker-compose

Security

Is Docker secure?

Let's ask this question more specific.
Is the docker daemon secure?
Are images transported securely?
Are images built in a secure fashion?
Are containers as secure as virtual machines?
Are application processes more or less secure
when containerized?

Are containers as secure as virtual machines?
Control group-separated,
chroot-like,
namespaced resources,
running on a shared kernel.
Application Containers Virtual Machines
Virtualized pieces of hardware,
running its own kernel
with process/user/network
spaces separated on
hypervisor level.

Are application processes more or less secure
when containerized?
Definitely more secure, if "used properly(*)"
Docker Container ~ Application process,
ideally a single process, without management daemons
Smaller attack surface
Namespaced process, network, FS mounts, ...:
~ Application cannot see "the outside OS world"
Reduced Linux Capabilities, can be fine tuned
Additional isolation mechanisms at hand:
SELinux Type Enforcement, AppArmor profiles, Libseccomp System Call Filtering
(*) http://container-solutions.com/is-docker-safe-for-production

Docker Hardening – Docker Security Benchmark
§  Extensive Guide on
hardening docker hosts,
images and containers,
including checks
§  Automated tools are in
development
§  benchmarks.cisecurity.org

Security

Tooling around Docker
Where to run Specialised Operating systems
Where to pull images from Registries (Private, On Premise, ...)
How to operate it Orchestration, Scheduling,
Management, Monitoring
From Infrastructure to
Applications
Platform-As-a-Service
How to build containers Config Management, Developer
Tools
Technical topics Networking, Security, Storage

Competitors & Container Runtime Alternatives
LXC + LXD
CoreOS Rocket (rkt), Application Container Spec (appc)
Open Container Project (www.opencontainers.org)

Technical PerspectiveDeveloper Perspective

Technical PerspectiveDeveloper PerspectiveSecurity

Technical PerspectiveDeveloper PerspectiveSecurityEcosystem & Partners

Technical PerspectiveDeveloper PerspectiveSecurityEcosystem & CompetitorsThank you!

Docker for HPC in a Nutshell

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (13)

Similar to Docker for HPC in a Nutshell

Similar to Docker for HPC in a Nutshell (20)

More from inside-BigData.com

More from inside-BigData.com (20)

Recently uploaded

Recently uploaded (20)

Docker for HPC in a Nutshell