11. 11
What are containers?
Containers offer an isolated environment – as close as possible
to a VM – without the overhead of running a separate kernel
and simulating all the hardware
15. 15
Docker
• Open Source
• Builds on top of LXC
• Provides portable format and
runtime environment to deploy
application inside containers
• Containers optimized by layers
• Manages dependencies
20. 20
Container Orchestration
• Manage/Provision multi-container
applications
• Application blueprint from a
schema (JSON or YAML)
• Monitoring
• Upgrades (rolling) and Rollback
• Policy management
• Service discovery
• Ease of use
• Integration with existing systems
21. 21
Docker Swarm
$ docker swarm init
$ docker swarm join
• Cluster management within the Docker engine
• No additional software to manage a Swarm
• Easy to setup
• Scaling
• Built-In
• Multi-host networking
• Service discovery
• Load balancing (L4)
• Security
• Rolling updates
• But… Not a real orchestration tool (not yet!)
Agenda – Just like meetings should have agenda. So you can decide if you want to stay
Why containers?
Before we answer that question, lets take a look at the hardware installed on various production systems.
Virt – tech that separates an os from physical resources.
Reduce energy, vendor lock in, faster provisioning of servers, reduce data center footprint
Hypervisor presents the the host os a complete set of cpu memory and disk fooling it to believe that its running on a real hardware.
So this solves the hardware resources problem by abstracting the hardware from the OS and provides the ability of allocating chunks of compute when necessary
Modern software systems are very complex.
A simple app can have a RDBMS, a document DB, quques, middleware systems, static website etc.
And all of these systems can be written in a plethora of languages as per the component needs.
So you have many components with different runtimes, and you code that runs on them.
Consider CI/CD env and things get really messy
Multiple targets
So complex software systems which requires multiple runtimes and are being distributed to variety of physical machines gives us a the Matrix hell
Before we go further, lets look at a similar problem in the physical world. The shipping industry
Until 1950s shipping was messy labor intensive and a drain
Malcom Mclean invented the container which standardized the movement of cargo
1966 first international ship voyage
So just like the shipping industry intermodal containers, our Linux container helps overcoming the matrix of hell
Agenda – Just like meetings should have agenda. So you can decide if you want to stay
Sort of the definition from the linuxcontainer org
Before we talk more about container, lets take a step back and understand the diff between virt and containerization
So virt is a tech that separates the OS from the physical resources. The hypervisor provides a complete set of hardware, cpu, mem disk etc to the guest OS fooling it to believe that it is running on a real hardware
Containerization, host and guests share the same kernel.
Eliminates waste. Free of OS overhead
So its considerably smaler, easier to move and most imp faster to provision
I/O and mem are also close to native perf
But all containers need to share the same base linux kernel
Containers are not something that Docker invented.
The bases of containers have been available in the linux kernel in some form since many year now.
Chroot – or change root. Changes root dir of process and child process. So provides isolation of some form to install virtualized copies of software
Cgroups – limits and isolates resource usage of a collection of processes
LXC – is an OS level virtualization that is built using Linux kernel features like chroot, cgroups, namespaces etc
LXC is a userspace interface for linux kernel containment features
LXC creates a env as close to a standard linux kernel but without the need for a separate kernel
Its between a chroot and a full fledged VM
Kernel namespaces – ipc, uts, mount, pid, network and user
Just as chroot allows processes to see any arbitrary directory as the root of the system (independent of the rest of the processes), Linux namespaces allow other aspects of the operating system to be independently modified as well. This includes the process tree, networking interfaces, mount points, inter-process communication resources and more
Apparmor and SELinux profiles
Linux kernel security
allow for easy control of access between processes and objects, processes and other processes, and objects and other objects
Seccomp policies
restrict system calls that can be made from a process. Key component in sandboxing
Chroots
arbitory directory
Cgroups
allocates resources such as system mem, network bw, CPU time to user defined groups of process. They can controlled, monitored and changed. This is how you can limit cpu and mem. All linux process are children of a common parent: init process. Cgroups are also hierarchical but multiple hierarchy can exist at the same time. Unconnected trees of processes
Docker is open source software to build ship and run any application as a container. Specifically LXC container.
It extends LXC. Make it more usable by application developers.
Docker containers are very light weight. Optimized with layers so easy to transport
Cross platform dependencies
Conflicting dependencies
Custom dependencies
Dockerd process runs as a daemon on a Linux system
Core process that manages building of images, publishing them to registries and run them
Docker d exposes a http api.
Client talks to the daemon using the http api.
Registry is where docker images are stored. Which can be later pulled
What is a docker file. Set of linux commands that describes you images.
Docker defines a build as running a sequence of Unix commands, one after the other, in the same container. Build commands modify the contents of the container (usually by installing new files on the filesystem), the next command modifies it some more, etc. Since each build command inherits the result of the previous commands, the order in which the commands are executed expresses dependencies.
Show a sample from out git repo
Docker build – sends the build context to the daemon which process the dockerfile and creates an image
Docker run – runs a container for that image
Container before docker
We have used this sometimes to describe Cars.com arch
With Docker creating deploying and managing linux container is very simple
Orchestration
So we can run containers.
But what about multiple containers that for an application?
What about 100s of container instances across multiple server spread over a large geographic area?
That when orchestration tools come in. They should have the benefits listed hear
Many players in this space
What is docker swarm.
We want to talk about swarm because since docker 1.12, cluster management and routing capabilities are built in
Not a complete orchestration tool
But provides a lot of cluster management tools with in the docker engine itself
Lets pray to the demo gods
Add a swarm worker using terraform.
Export DOCKER_HOST to one of the mangers
Show docker node ls
Docker network ls
Etc
Take a clutch based app and add PUBLISHED_PORT and INSTANCES in the mk file
And run powertrain service create