Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker for HPC in a Nutshell

2,802 views

Published on

In this deck from the Docker Workshop at ISC 2015, Andreas Schmidt from Cassini Consulting describes Docker in a Nutshell

"As the newest flavor of Linux Containers, Docker gained a lot of momentum in the last 12 months. With a very convenient and open API-driven architecture Docker is able to help decrease the complexity of operations and increase the productivity of computation. During the last two years Andreas, Christian, and Wolfgang gained a lot of experience with Docker and were thrilled by its possible impact early on. Andreas started working with Docker in mid-2013 and is interested in developing tools for solving Enterprise IT requirements on networking and security. In 2014 he held talks and workshops about these topics. Christian started using Docker in 2013 to virtualize a complete HPC cluster stack and since then held multiple talks about how Docker might impact HPC. Wolfgang and his partner Burak Yenier introduced Docker as a corner-stone of the UberCloud Marketplace to drastically improve and simplify access to HPC cloud resources. UberCloud just announced their new containers for computational fluid dynamics software like Fluent, STAR-CCM+ and OpenFOAM."

Watch the video presentation: http://wp.me/p3RLHQ-enP

Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Published in: Technology

Docker for HPC in a Nutshell

  1. 1. © 2013 Cassini Consulting Andreas Schmidt | @aschmidt75 Docker: User Friendly Application & Service Containers for HPC Environments Docker In A Nutshell
  2. 2. 2 16.07.15 Cassini Consulting
  3. 3. 3 16.07.15 Cassini Consulting
  4. 4. 4 16.07.15 Cassini Consulting vs.Pets Cattle
  5. 5. 5 16.07.15 Cassini Consulting
  6. 6. 16.07.15 Cassini Consulting6 Technical Perspective Developer Perspective Security Ecosystem & Partners
  7. 7. 16.07.15 Cassini Consulting7 Technical Perspective Developer Perspective Security Ecosystem & Partners
  8. 8. Docker Daemon Docker Engine core components 8 16.07.15 Cassini Consulting Containers Images Containers
  9. 9. Docker Daemon Docker Engine core components 9 16.07.15 Cassini Consulting Containers Images Containers API Docker Client Docker Client
  10. 10. The Docker workflow & commands 10 16.07.15 Cassini Consulting Image
  11. 11. The Docker workflow & commands 11 16.07.15 Cassini Consulting Registry Image pull
  12. 12. The Docker workflow & commands 12 16.07.15 Cassini Consulting Registry Image Docker- file pull build
  13. 13. The Docker workflow & commands 13 16.07.15 Cassini Consulting Registry Image Docker- file push pull build
  14. 14. The Docker workflow & commands 14 16.07.15 Cassini Consulting Registry Image Container Docker- file push pull run build
  15. 15. The Docker workflow & commands 15 16.07.15 Cassini Consulting Registry Image Container Docker- file push pull run commit build
  16. 16. The Docker workflow & commands 16 16.07.15 Cassini Consulting Registry Image Container Docker- file push pull run commit start, stop, ... build
  17. 17. The Docker workflow & commands 17 16.07.15 Cassini Consulting Registry Image Container Docker- file ✖ push pull run commit rm start, stop, ... build
  18. 18. The Docker workflow & commands 18 16.07.15 Cassini Consulting Registry Image Container Docker- file ✖ ✖ push pull run commit rm rmi start, stop, ... build
  19. 19. The Docker workflow & commands 19 16.07.15 Cassini Consulting Registry Image Container Docker- file External Tarball ✖ ✖ push pull run commit rm rmi start, stop, ... build export
  20. 20. Docker Engine technology foundation Isolation through Kernel Namespaces Linux Capabilities Resource limitation through Linux control groups Filesystem isolation, Copy-On-Write & Union FS 20 16.07.15 Cassini Consulting
  21. 21. Docker Engine technology foundation 21 16.07.15 (*) https://www.docker.com/whatisdocker
  22. 22. Virtual Machines vs. Containers 22 16.07.15 (*) https://www.docker.com/whatisdocker
  23. 23. 16.07.15 Cassini Consulting23 Technical Perspective Developer Perspective Security Ecosystem & Partners
  24. 24. From Development to Production: Challenges ahead! 24 16.07.15 Cassini Consulting
  25. 25. What Docker gives to developers 25 16.07.15 Cassini Consulting Easy sandbox approach Easy way to deliver software Dev / Prod parity
  26. 26. Easy Sandbox approach §  Create build and test environments §  Choose Libraries and Framework Dependencies, per application §  Deal with incompatible version mismatches (Ruby 1.9 vs. Ruby 2.1, including libs, bundler, version switchers, ...) §  Lightweight alternative to using virtual machines 26 16.07.15 Cassini Consulting
  27. 27. Easy Sandbox approach A Docker Image contains its own userland libraries and binaries §  separated from other images §  take exactly the versions of libraries you need §  leave out things you do not need §  reproducible, lightweight, easily testable §  look at it as a unit of delivery 27 16.07.15 Cassini Consulting
  28. 28. Easy way to deliver software 28 16.07.15 Cassini Consulting Code Deliverables (i.e. RPM packages) are not runnable. An installed instance (i.e. in a VM) is hard to transport.
  29. 29. 29 16.07.15 Cassini Consulting https://twitter.com/joyent/status/565243828718678016
  30. 30. Easy way to deliver software Dockerfile as a Contract between Development and Operations 30 16.07.15 Cassini Consulting what to base from set environment params prepare the image, i.e. install something, configure it describe the interface what to run
  31. 31. Easy way to deliver software Dockerfile as the blueprint for reusable building blocks 31 16.07.15 Cassini Consulting what to base from Redis is an official repository at dockerhub
  32. 32. Easy way to deliver software Dockerfile as the blueprint for reusable building blocks 32 16.07.15 Cassini Consulting what to base from Redis is an official repository at dockerhub
  33. 33. Pave the way for Dev/Prod parity 33 16.07.15 Cassini Consulting §  From Applications to Systems §  Describe not only compute, but also storage and networking. §  Example: docker-compose
  34. 34. 16.07.15 Cassini Consulting34 Technical Perspective Developer Perspective Security Ecosystem & Partners
  35. 35. 35 16.07.15 Cassini Consulting Is Docker secure?
  36. 36. Let's ask this question more specific. Is the docker daemon secure? Are images transported securely? Are images built in a secure fashion? Are containers as secure as virtual machines? Are application processes more or less secure when containerized? 36 16.07.15 Cassini Consulting
  37. 37. Are containers as secure as virtual machines? Control group-separated, chroot-like, namespaced resources, running on a shared kernel. 37 16.07.15 Cassini Consulting Application Containers Virtual Machines Virtualized pieces of hardware, running its own kernel with process/user/network spaces separated on hypervisor level.
  38. 38. Are application processes more or less secure when containerized? Definitely more secure, if "used properly(*)" Docker Container ~ Application process, ideally a single process, without management daemons Smaller attack surface Namespaced process, network, FS mounts, ...: ~ Application cannot see "the outside OS world" Reduced Linux Capabilities, can be fine tuned Additional isolation mechanisms at hand: SELinux Type Enforcement, AppArmor profiles, Libseccomp System Call Filtering 38 16.07.15 Cassini Consulting (*) http://container-solutions.com/is-docker-safe-for-production
  39. 39. Docker Hardening – Docker Security Benchmark 39 16.07.15 Cassini Consulting §  Extensive Guide on hardening docker hosts, images and containers, including checks §  Automated tools are in development §  benchmarks.cisecurity.org
  40. 40. 16.07.15 Cassini Consulting40 Technical Perspective Developer Perspective Security Ecosystem & Partners
  41. 41. Tooling around Docker 41 16.07.15 Cassini Consulting Where to run Specialised Operating systems Where to pull images from Registries (Private, On Premise, ...) How to operate it Orchestration, Scheduling, Management, Monitoring From Infrastructure to Applications Platform-As-a-Service How to build containers Config Management, Developer Tools Technical topics Networking, Security, Storage
  42. 42. 42 16.07.15 Cassini Consulting
  43. 43. Competitors & Container Runtime Alternatives LXC + LXD CoreOS Rocket (rkt), Application Container Spec (appc) Open Container Project (www.opencontainers.org) 43 16.07.15 Cassini Consulting
  44. 44. 16.07.15 Cassini Consulting44 Technical Perspective
  45. 45. 16.07.15 Cassini Consulting45 Technical PerspectiveDeveloper Perspective
  46. 46. 16.07.15 Cassini Consulting46 Technical PerspectiveDeveloper PerspectiveSecurity
  47. 47. 16.07.15 Cassini Consulting47 Technical PerspectiveDeveloper PerspectiveSecurityEcosystem & Partners
  48. 48. 16.07.15 Cassini Consulting48 Technical PerspectiveDeveloper PerspectiveSecurityEcosystem & CompetitorsThank you!

×