Docker
DevOps with Containers
Agenda
Monday Tuesday Thursday Friday
Morning VMs vs Containers
Docker Overview
Install Docker
Docker commands
Running containers
Q&A
Recap
Docker Images
Docker Networks
Q&A
Dockerfiles review
Docker Storage
Q&A
Recap
Docker Compose
Kubernetes
Afternoon Videos
Labs
Reading
Videos
Labs
Reading
Videos
Labs
Reading
Videos
Labs
Reading
Topics to review before we start
● Linux command line
○ ls, ls -R
○ date
○ grep, cat
○ type
○ man
● Filesystem
○ mkdir, chmod, chown, pwd
○ permissions
○ cd, touch
● Pipes
● SSH
● Shell alias
● Editor: vi
● Superuser
● nginx
● git
Why Containers?
Sharing Compute Resources
Physical Server
Also: bare-metal server
Fixed processor, memory, network,
storage
Operating system hosts applications
with their dependencies
Single tenant (owner)
Virtual Server
Also: virtual machine (VM)
Emulation of a physical computer
Hardware is shared by multiple
VMs
Each VM runs an operating system
Multiple tenants (owners)
Container
Operating system is shared
Applications are packaged as
container images with their
dependencies and isolated at
runtime (Docker)
Lower overhead than VMs
Virtual Machines vs. Containers
State of the Art
● Applications
○ run as multiple instances of small containers
○ managed by “orchestrators” like Kubernetes
● Databases
○ special performance and storage requirements
○ run as virtual machines
Docker Overview
Docker
● Open Source Container engine
● A ”Docker image” is an application containerized using
docker
● Docker is also the Company behind docker
● A Docker image is pushed to a Registry, and pulled to
start a container
From Docker image to container
Image
Dockerfile Container Container
Developer Operator
writes
docker build
docker push
Docker Registry
docker pull
docker run docker run
Docker Overview (13 min)
DevOps and Containers
Containers
Containers
Containers
Containers
What is DevOps?
Articles to read
What is a container
Containers as the foundation for DevOps collaboration
Docker and the Three Ways of DevOps
Exercise
Write down three benefits of DevOps and how containers make it easier to
achieve them.
Docker Commands
● Follow instructions at https://docs.docker.com/get-docker/
○ Install Docker using the convenience script
● Add your user to the docker group
○ sudo adduser <username> docker
● Log out and log in again
● Check Docker version
○ docker version
● Run a hello world command
○ docker run hello-world
Install Docker on your Virtual Machine
The anatomy of a Linux process
● The executable - the binary file located in the file system
○ which docker; which vim; file $(which cat)
● The process ID and hierarchy
○ ps; ps -elf; pstree <user>; pstree -pa; docker ps; docker -as
● STDIN, STDOUT, STDERR and redirections
○ {STDIN} → Process → STDOUT + STDERR
○ [process] < [file] (stdin_input)
○ [process] > [file] (stdout_output)
○ [process] 2> [file] (stderr_output)
● Environment variables - Executables can read them.
○ printenv; export
Common Binaries to use
● Nginx: web server to serve static sites or to load balance HTTP requests to
applications
● Redis: in-memory database, for storing simple data structures as key/value
pairs
● MySQL: relational database, frequently used as backend for websites (e.g.
Wordpress)
● PostgreSQL: relational database like MySQL, older, feature rich
● Ansible, Chef, Puppet: you write configuration "as code" and the tool applies
it automatically on servers
● Ubuntu, CentOS, Fedora, Debian, Alpine: Linux distributions
Docker commands
● docker run <image>
○ If image not found locally, downloads the latest image from Registry.
○ Starts container with a Container ID != Image name
○ Can start the container in detached mode
○ Can setup port mapping to access the Container
● docker ps [-as]
○ Lists running containers
● docker images
● docker exec <container> <command>
○ Runs a command inside a container
● ...
Docker Commands (12 min)
Katacoda Scenario; KodeKloud Lab [CouseLink]
● Start and stop containers
● Pull and remove images
● Inspect what an image is made of
Docker CLI Reference
Lab 1: Docker Basic Commands
Docker Run (9 min)
Lab 2: Docker Run Commands
Katacoda Scenario; KodeKloud Lab
● Inspect running containers
● Configure port mappings
● Lookup image usage details on Docker Hub
Docker CLI Reference
Environment Variables (2 min)
Recap Day 1
Why to use containers
● Containerization provides consistent environments. Applications behave the
same on the developer's workstation and in production deployments.
● Containerized applications are isolated from other applications also when
containers of different applications run on the same server.
● Container images include software dependencies of the application like
programming language runtimes or software libraries.
● Containers are more lightweight than physical servers and virtual machines.
This reduces costs and enables faster updates of applications.
● Many tools are available to build and run containerized applications. This
week, we get to know Docker, Docker Compose, and Kubernetes.
What happens when Docker starts a container
1. Look for the image in the local cache
2. If not found, look for the image in the remote image repository
3. When needed, pull the image from the repository to the local cache
4. Extract all the layers of the container image
5. Create a layer on top which will be used for changes by the new container
6. Give the container its own internal IP address and connect it to the network
7. If requested, establish port mappings from the host to the container
8. Start the command specified in Dockerfile within the container environment
Docker Commands Cheat Sheet
● docker help ps|run|exec|stop|rm|images|rmi|inspect # get quick help
● docker ps [-q] [-a] # list containers, also stopped with -a, only IDs with -q
● docker run [-d] [--name name] [-e name=value] [-p hostport:containerport] image[:tag]
● docker exec -it container /bin/bash # open shell in running container
● docker stop container… # stop running containers
● docker rm container… # delete stopped containers
● docker rm $(docker ps -q -a) # delete all existing containers, all have to be stopped
● docker images [-q] # list local container images, only IDs with -q
● docker rmi image… # delete local container images
● docker inspect container|image # inspect details about containers or images
Docker Images
Docker Images (7 min)
Lab 4: Docker Images
Katacoda Scenario; KodeKloud Lab
● Check the size of images
● Build images with Dockerfiles
CMD vs. ENTRYPOINT (7 min)
Lab 5: CMD vs. ENTRYPOINT
KodeKloud Lab
● Inspect Dockerfiles
Build Your Own Image
Short Quiz to Warm Up
https://forms.gle/X38MDMw3PXYZcq398
Extensive documentation about Dockerfile is available on the Docker site:
https://docs.docker.com/engine/reference/builder/. You may need this reference
also for the assignment on the next slide.
How to build container images
1. Create a new Dockerfile in your application source tree
2. Specify the base image (FROM)
3. Install dependencies with the Linux distro's package manager (RUN)
4. Define the directory where the application will run (WORKDIR)
5. Copy the dependency manifest, e.g. package.json, into the image (COPY)
6. Install direct dependencies of your application e.g. with npm (RUN)
7. Copy the source code into the image (COPY)
8. Define on which port the application will be listening (EXPOSE)
9. Specify the command to be run when the container starts (CMD)
Assignment: Build Your Own Image
Take an existing Node.js app and dockerize it
Download the source code of the app to your instance with this command:
curl -s https://lanfranchi.ch/fabio/dockerfile-assignment-1.tar.gz | tar -xzvf -
● The code is extracted to a subdirectory named dockerfile-assignment-1.
● Start by reading the detailed instructions in the Dockerfile.
● Flesh out the Dockerfile. Build it. Test it. Run it.
● Expect this to be iterative. Rarely do I get it right the first time.
● Use the Alpine version of the official 'node' 16.x image.
● Expected result is a web site.
Container Registry
● Where to store Docker Images
● Can be public or private
● Hosted or Self-hosted
● Organized by Repositories ( == app name), ordered by tags ( == versions)
Docker Registry (5 mins)
A Hoster Docker Registry
Docker Hub Account and upload
● Sign up with Docker Hub: https://hub.docker.com
● Use e.g. powercoder<your name> as Docker ID
● Log in with your Docker Hub credentials on your workstation: docker login
● Create a repository in Docker Hub
● Upload your image
Recap Day 2
Show your Docker image in DockerHub
Docker Networking
A few useful Linux commands
● ip address; ip link
● ping <ip>
● telnet <ip> <port>
● netstat -nat ; netstat -nat | grep LISTEN ; netstat -nat | grep :<port>
● curl http://<ip>:<port>
● curl [-v] http://<url>
● touch <file>; cat <file>; nano <file>
● cat <file> | grep [-i] <text> | wc
● echo << EOF
Docker networks
‘docker networks ls’
● bridge (default) = Containers get an IP and can communicate with other
containers in the bridge.
○ Expose ports externally via mapping.
● none = Containers do not get any IP and cannot communicate with
others.
● host = Containers shares the network of the server. All containers share
the same IP address of the host.
○ All ports are reachable on the server IP.
● custom bridge = As the default bridge, but controls the IP subnet and has
built-in DNS.
Multiple ports on Containers
nginx :80 nginx :80
host_ip: 10.10.1.1
bridge 172.16.0.12
172.16.0.11
-p 8080:80
-p 8090:80
nginx :8080 nginx :8090
host_ip: 10.10.1.1
host
Docker Networking (5 min)
Lab 6: Docker Networking
Katacoda Scenario; KodeKloud Lab
● Configure the network of containers
● Connect two containers on a user-defined network
Docker Storage
Docker Storage
● Writable layer = Changes are deleted after container is stopped. Copy-
on-Write.
● Volume = Changes are permanent after container is stopped. In
/var/lib/docker/volumes. Managed by Docker.
● Bind Mount= Changes are permanent after container is stopped.
Managed by the host Server. Can be in any mount point.
● Tempfs = Changes are deleted after container is stopped. Paths outside
of the Writable layer (i.e. /app/data)
Docker Storage
● Volumes and Bind Mounts for
○ Share data between containers
○ Share configuration files from host to container
○ Write logs to external storage
Docker Storage (12 min)
Lab 7: Docker Storage
Katacoda Scenario; Katacoda Scenario
● Use persistent storage for a database container
Recap Day 3
Quiz Time
Docker Compose
Introduction to YAML (8 min)
Lab 8: Introduction to YAML
YAML Quiz
About Docker compose
Deploy application stacks in a single run
Application stack = Application made of several containers
docker build -t admin-web -f admin-web/Dockerfile
docker build -t users-web -f users-web/Dockerfile
docker volume create sql_vol
docker run -d -p 8080:80 -name admin-web admin-web
docker run -d -p 80:80 -name users-web users-web
docker run -d -v sql_vol:/var/mysql -name db mysql
version:2
services:
admin-web:
build: admin-web/
ports:
- “8080:80”
users-web:
build: users-web/
ports:
- “80:80”
db:
image: db
volumes:
- sql_vol:/var/mysql
volumes:
sql_vol: {}
docker-compose up
docker-compose down
Docker Compose (18 min)
Lab 9: Docker Compose
Katacoda Scenario
● Run an application consisting of two containers
● Create a docker-compose.yml file (in format version 1)
Kubernetes
What Kubernetes does
Manages a group of servers that run containers = Nodes
Deploys application stacks on the cluster = Pod
Manages and keeps the Pods healthy
Cluster = Master + Nodes
An Introduction to Kubernetes
https://www.jeremyjordan.me/kubernetes/
Container Orchestration (4 min)
Kubernetes (6 min)

Powercoders · Docker · Fall 2021.pptx

  • 1.
  • 2.
    Agenda Monday Tuesday ThursdayFriday Morning VMs vs Containers Docker Overview Install Docker Docker commands Running containers Q&A Recap Docker Images Docker Networks Q&A Dockerfiles review Docker Storage Q&A Recap Docker Compose Kubernetes Afternoon Videos Labs Reading Videos Labs Reading Videos Labs Reading Videos Labs Reading
  • 3.
    Topics to reviewbefore we start ● Linux command line ○ ls, ls -R ○ date ○ grep, cat ○ type ○ man ● Filesystem ○ mkdir, chmod, chown, pwd ○ permissions ○ cd, touch ● Pipes ● SSH ● Shell alias ● Editor: vi ● Superuser ● nginx ● git
  • 4.
  • 5.
    Physical Server Also: bare-metalserver Fixed processor, memory, network, storage Operating system hosts applications with their dependencies Single tenant (owner)
  • 6.
    Virtual Server Also: virtualmachine (VM) Emulation of a physical computer Hardware is shared by multiple VMs Each VM runs an operating system Multiple tenants (owners)
  • 7.
    Container Operating system isshared Applications are packaged as container images with their dependencies and isolated at runtime (Docker) Lower overhead than VMs
  • 8.
  • 9.
    State of theArt ● Applications ○ run as multiple instances of small containers ○ managed by “orchestrators” like Kubernetes ● Databases ○ special performance and storage requirements ○ run as virtual machines
  • 10.
  • 11.
    Docker ● Open SourceContainer engine ● A ”Docker image” is an application containerized using docker ● Docker is also the Company behind docker ● A Docker image is pushed to a Registry, and pulled to start a container
  • 12.
    From Docker imageto container Image Dockerfile Container Container Developer Operator writes docker build docker push Docker Registry docker pull docker run docker run
  • 13.
  • 14.
  • 15.
  • 16.
    Articles to read Whatis a container Containers as the foundation for DevOps collaboration Docker and the Three Ways of DevOps
  • 17.
    Exercise Write down threebenefits of DevOps and how containers make it easier to achieve them.
  • 18.
  • 19.
    ● Follow instructionsat https://docs.docker.com/get-docker/ ○ Install Docker using the convenience script ● Add your user to the docker group ○ sudo adduser <username> docker ● Log out and log in again ● Check Docker version ○ docker version ● Run a hello world command ○ docker run hello-world Install Docker on your Virtual Machine
  • 20.
    The anatomy ofa Linux process ● The executable - the binary file located in the file system ○ which docker; which vim; file $(which cat) ● The process ID and hierarchy ○ ps; ps -elf; pstree <user>; pstree -pa; docker ps; docker -as ● STDIN, STDOUT, STDERR and redirections ○ {STDIN} → Process → STDOUT + STDERR ○ [process] < [file] (stdin_input) ○ [process] > [file] (stdout_output) ○ [process] 2> [file] (stderr_output) ● Environment variables - Executables can read them. ○ printenv; export
  • 21.
    Common Binaries touse ● Nginx: web server to serve static sites or to load balance HTTP requests to applications ● Redis: in-memory database, for storing simple data structures as key/value pairs ● MySQL: relational database, frequently used as backend for websites (e.g. Wordpress) ● PostgreSQL: relational database like MySQL, older, feature rich ● Ansible, Chef, Puppet: you write configuration "as code" and the tool applies it automatically on servers ● Ubuntu, CentOS, Fedora, Debian, Alpine: Linux distributions
  • 22.
    Docker commands ● dockerrun <image> ○ If image not found locally, downloads the latest image from Registry. ○ Starts container with a Container ID != Image name ○ Can start the container in detached mode ○ Can setup port mapping to access the Container ● docker ps [-as] ○ Lists running containers ● docker images ● docker exec <container> <command> ○ Runs a command inside a container ● ...
  • 23.
  • 24.
    Katacoda Scenario; KodeKloudLab [CouseLink] ● Start and stop containers ● Pull and remove images ● Inspect what an image is made of Docker CLI Reference Lab 1: Docker Basic Commands
  • 25.
  • 26.
    Lab 2: DockerRun Commands Katacoda Scenario; KodeKloud Lab ● Inspect running containers ● Configure port mappings ● Lookup image usage details on Docker Hub Docker CLI Reference
  • 27.
  • 28.
  • 29.
    Why to usecontainers ● Containerization provides consistent environments. Applications behave the same on the developer's workstation and in production deployments. ● Containerized applications are isolated from other applications also when containers of different applications run on the same server. ● Container images include software dependencies of the application like programming language runtimes or software libraries. ● Containers are more lightweight than physical servers and virtual machines. This reduces costs and enables faster updates of applications. ● Many tools are available to build and run containerized applications. This week, we get to know Docker, Docker Compose, and Kubernetes.
  • 30.
    What happens whenDocker starts a container 1. Look for the image in the local cache 2. If not found, look for the image in the remote image repository 3. When needed, pull the image from the repository to the local cache 4. Extract all the layers of the container image 5. Create a layer on top which will be used for changes by the new container 6. Give the container its own internal IP address and connect it to the network 7. If requested, establish port mappings from the host to the container 8. Start the command specified in Dockerfile within the container environment
  • 31.
    Docker Commands CheatSheet ● docker help ps|run|exec|stop|rm|images|rmi|inspect # get quick help ● docker ps [-q] [-a] # list containers, also stopped with -a, only IDs with -q ● docker run [-d] [--name name] [-e name=value] [-p hostport:containerport] image[:tag] ● docker exec -it container /bin/bash # open shell in running container ● docker stop container… # stop running containers ● docker rm container… # delete stopped containers ● docker rm $(docker ps -q -a) # delete all existing containers, all have to be stopped ● docker images [-q] # list local container images, only IDs with -q ● docker rmi image… # delete local container images ● docker inspect container|image # inspect details about containers or images
  • 32.
  • 33.
  • 34.
    Lab 4: DockerImages Katacoda Scenario; KodeKloud Lab ● Check the size of images ● Build images with Dockerfiles
  • 35.
  • 36.
    Lab 5: CMDvs. ENTRYPOINT KodeKloud Lab ● Inspect Dockerfiles
  • 37.
  • 38.
    Short Quiz toWarm Up https://forms.gle/X38MDMw3PXYZcq398 Extensive documentation about Dockerfile is available on the Docker site: https://docs.docker.com/engine/reference/builder/. You may need this reference also for the assignment on the next slide.
  • 39.
    How to buildcontainer images 1. Create a new Dockerfile in your application source tree 2. Specify the base image (FROM) 3. Install dependencies with the Linux distro's package manager (RUN) 4. Define the directory where the application will run (WORKDIR) 5. Copy the dependency manifest, e.g. package.json, into the image (COPY) 6. Install direct dependencies of your application e.g. with npm (RUN) 7. Copy the source code into the image (COPY) 8. Define on which port the application will be listening (EXPOSE) 9. Specify the command to be run when the container starts (CMD)
  • 40.
    Assignment: Build YourOwn Image Take an existing Node.js app and dockerize it Download the source code of the app to your instance with this command: curl -s https://lanfranchi.ch/fabio/dockerfile-assignment-1.tar.gz | tar -xzvf - ● The code is extracted to a subdirectory named dockerfile-assignment-1. ● Start by reading the detailed instructions in the Dockerfile. ● Flesh out the Dockerfile. Build it. Test it. Run it. ● Expect this to be iterative. Rarely do I get it right the first time. ● Use the Alpine version of the official 'node' 16.x image. ● Expected result is a web site.
  • 41.
    Container Registry ● Whereto store Docker Images ● Can be public or private ● Hosted or Self-hosted ● Organized by Repositories ( == app name), ordered by tags ( == versions)
  • 42.
  • 43.
  • 44.
    Docker Hub Accountand upload ● Sign up with Docker Hub: https://hub.docker.com ● Use e.g. powercoder<your name> as Docker ID ● Log in with your Docker Hub credentials on your workstation: docker login ● Create a repository in Docker Hub ● Upload your image
  • 45.
    Recap Day 2 Showyour Docker image in DockerHub
  • 46.
  • 47.
    A few usefulLinux commands ● ip address; ip link ● ping <ip> ● telnet <ip> <port> ● netstat -nat ; netstat -nat | grep LISTEN ; netstat -nat | grep :<port> ● curl http://<ip>:<port> ● curl [-v] http://<url> ● touch <file>; cat <file>; nano <file> ● cat <file> | grep [-i] <text> | wc ● echo << EOF
  • 48.
    Docker networks ‘docker networksls’ ● bridge (default) = Containers get an IP and can communicate with other containers in the bridge. ○ Expose ports externally via mapping. ● none = Containers do not get any IP and cannot communicate with others. ● host = Containers shares the network of the server. All containers share the same IP address of the host. ○ All ports are reachable on the server IP. ● custom bridge = As the default bridge, but controls the IP subnet and has built-in DNS.
  • 49.
    Multiple ports onContainers nginx :80 nginx :80 host_ip: 10.10.1.1 bridge 172.16.0.12 172.16.0.11 -p 8080:80 -p 8090:80 nginx :8080 nginx :8090 host_ip: 10.10.1.1 host
  • 50.
  • 51.
    Lab 6: DockerNetworking Katacoda Scenario; KodeKloud Lab ● Configure the network of containers ● Connect two containers on a user-defined network
  • 52.
  • 53.
    Docker Storage ● Writablelayer = Changes are deleted after container is stopped. Copy- on-Write. ● Volume = Changes are permanent after container is stopped. In /var/lib/docker/volumes. Managed by Docker. ● Bind Mount= Changes are permanent after container is stopped. Managed by the host Server. Can be in any mount point. ● Tempfs = Changes are deleted after container is stopped. Paths outside of the Writable layer (i.e. /app/data)
  • 54.
    Docker Storage ● Volumesand Bind Mounts for ○ Share data between containers ○ Share configuration files from host to container ○ Write logs to external storage
  • 55.
  • 56.
    Lab 7: DockerStorage Katacoda Scenario; Katacoda Scenario ● Use persistent storage for a database container
  • 57.
  • 58.
  • 59.
  • 60.
    Lab 8: Introductionto YAML YAML Quiz
  • 61.
    About Docker compose Deployapplication stacks in a single run Application stack = Application made of several containers docker build -t admin-web -f admin-web/Dockerfile docker build -t users-web -f users-web/Dockerfile docker volume create sql_vol docker run -d -p 8080:80 -name admin-web admin-web docker run -d -p 80:80 -name users-web users-web docker run -d -v sql_vol:/var/mysql -name db mysql version:2 services: admin-web: build: admin-web/ ports: - “8080:80” users-web: build: users-web/ ports: - “80:80” db: image: db volumes: - sql_vol:/var/mysql volumes: sql_vol: {} docker-compose up docker-compose down
  • 62.
  • 63.
    Lab 9: DockerCompose Katacoda Scenario ● Run an application consisting of two containers ● Create a docker-compose.yml file (in format version 1)
  • 64.
  • 65.
    What Kubernetes does Managesa group of servers that run containers = Nodes Deploys application stacks on the cluster = Pod Manages and keeps the Pods healthy Cluster = Master + Nodes
  • 66.
    An Introduction toKubernetes https://www.jeremyjordan.me/kubernetes/
  • 67.
  • 68.

Editor's Notes

  • #6 Image a new application needs running: slow to deploy new servers During the night: resources are not used optimally Imagine a Server breaks: Maintenance and operations are slow (fix, repair)
  • #7 Image a new application needs running: fast to deploy new servers During the night: The HW is used more, because it is shared. During the day: peak use of resources cause contention Imagine a Server breaks: Easy migration (?) Problem: Operation, still each VM is unique. Hard to track dependencies (app needs to build the whole stack up to the OS) Compute resources wasted on running the same OS multiple times. Multiple versions (?) OSs to patch slow (fix, repair
  • #8 Docker is not the only one, there are other Container engines, like lxd.
  • #9 All applications are based on the same OS: Linux! No dependencies are installed on the Host OS, they are packaged in the application The Application cannot require tuning the OS - No customization VMs are good for isolation Containers are smaller, easier to move
  • #10 Micro Services architecture
  • #13 Why do we need an Orchestrator Applications made of several Containers Monitor and restart containers that failed Determine where to run containers The Developer and the Operator can be the same person
  • #20 Why do we need to log out and in again? > Changes to the user membership apply on new login
  • #21 EnvVars = A way to pass configuration information
  • #23 Why do we have a Containerd ID and not the Image name? What are they different?
  • #42 Why do we have a Containerd ID and not the Image name? What are they different?