Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker Concepts for Oracle/MySQL DBAs and DevOps

1,265 views

Published on

Oracle Week 2017 Slides
Agenda:
Docker overview – why do we even need containers?
Installing Docker and getting started
Images and Containers
Docker Networks
Docker Storage and Volumes
Oracle and Docker
Docker tools, GUI and Swarm

Published in: Technology

Docker Concepts for Oracle/MySQL DBAs and DevOps

  1. 1. Docker Concepts for Oracle/MySQL DBAs and DevOps Zohar Elkayam CTO, Brillix Zohar@Brillix.co.il www.realdbamagic.com Twitter: @realmgic
  2. 2. 2 • Zohar Elkayam, CTO at Brillix • Programmer, DBA, team leader, database trainer, public speaker, and a senior consultant for over 19 years • Oracle ACE Since 2014 • ilOUG Board Member – Israel Oracle User Group • Blogger – www.realdbamagic.com and www.ilDBA.co.il Who am I?
  3. 3. 3 About Brillix • We offer complete, integrated end-to-end solutions based on best-of- breed innovations in database, security and big data technologies • We provide complete end-to-end 24x7 expert remote database services • We offer professional customized on-site trainings, delivered by our top-notch world recognized instructors
  4. 4. 4 Some of Our Customers
  5. 5. 5 • Founded in 1999, over 7000 members at its peak • Divided into Technologies (Databases) and Applications groups • Free Bi-monthly meetups for DBAs and Developers at Oracle Israel offices • Yearly conference with guests speakers from all over the world: • January 22-23, 2018 • Over 30 Speakers from all over the world: Ask TOM team, Oracle product mangers and Oracle ACEs from 9 different countries! • You can join our community: meetup group, facebook group or follow us on twitter (@ilOUG_org)! • Our website: www.ilOUG.org ilOUG – Israel Oracle User Group
  6. 6. 6 Agenda • Docker overview – why do we even need containers? • Installing Docker and getting started • Images and Containers • Docker Networks • Docker Storage and Volumes • Oracle and Docker • Docker tools, GUI and Swarm
  7. 7. 7 Our Goal Today • Learn what Docker is and how to use it • See how Docker containers might be helpful with Oracle • We will not expert everything • Did you hear about Oracle 18c? • This is a starting point – don’t be afraid to try!
  8. 8. 8 •‫דעתכם‬ ‫חוות‬ ‫את‬ ‫לקבל‬ ‫ונשמח‬ ‫משוב‬ ‫טופס‬ ‫יחולק‬ ‫הסמינר‬ ‫יום‬ ‫בסיום‬. •‫חשובה‬ ‫דעתכם‬:‫יותר‬ ‫טובים‬ ‫הסמינרים‬ ‫את‬ ‫לעשות‬ ‫לנו‬ ‫המאפשרת‬ ‫זו‬ ‫היא‬! Our REAL Agenda 10:30-10:45‫הפסקה‬ 12:30-13:30‫משתתפ‬ ‫לכל‬ ‫צהריים‬ ‫ארוחת‬‫המלון‬ ‫בגן‬ ‫הכנס‬ ‫י‬ 15:00-15:15‫הפנים‬ ‫קבלת‬ ‫במתחם‬ ‫מתוקה‬ ‫הפסקה‬ 16:30‫הביתה‬ ‫הולכים‬
  9. 9. Docker
  10. 10. 10 What is Docker? • Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating system–level virtualization on Linux
  11. 11. 11 What’s the Buzz Around Containers? • The software industry has changed dramatically • 5-10 years ago: • Monolithic applications • Long development cycles • Slowly scaling up • Now: • Decoupled services, Microservices • Fast, iterative improvements • Quickly scaling out • Google claims all of its services now run in containers
  12. 12. 12 It’s All About CI/CD • CI/CD: Continuous Integration and Continuous Deployment • We want to always integrate new things into our applications and have a fast deployment • The challenge: development and deployment became complex: multiple stacks, multiple targets. Keeping everything in sync and deploying a reliable environment in short spans to production is hard • Docker Goal: Provide a uniformed wrapper around a software package: «Build, Ship and Run Any App, Anywhere» • Using Docker to achieve CI/CD is the most common usage for Docker today
  13. 13. 13 The Analogy: Cargo Transport
  14. 14. 14 The Analogy: Cargo Transport Solution
  15. 15. 15 Docker Solution: Containers
  16. 16. 16 Docker Containers • The concept: One Application per Container • Self contained - everything you need is in an isolated environment • Isolation for security - containers not aware of each other • Isolation for performance – hard limits on CPU, RAM and IO per container
  17. 17. 17 VMs vs. Containers • Less overhead but It’s not a lightweight VM – it’s a Software Delivery Platform • It's about isolation - Containers run on Linux kernel of host and are visible on host
  18. 18. 18 Why Use Docker Containers • It’s highly portable • It Runs everywhere: bare metal, VM, and even the cloud (AWS, GCP, Oracle Cloud, Microsoft Azure) • Small, lightweight and highly scalable • Better performance than VMs, smaller foot print • Very cost effective
  19. 19. 19 Why Does IT Cares About Docker? Developers care because: • Quickly create ready-to-run packaged applications • A clean, safe, hygienic, portable runtime environment • No missing/conflicting dependencies or packages • Each app runs in an isolated container • Automate testing, integration, packaging • Reduce/eliminate platform compatibility issues • Cheap/zero cost deployment, with instant replay and reset
  20. 20. 20 Why Does IT Cares About Docker? (2) Administrators care because: • Configure once, run many times • Makes app lifecycle efficient, consistent and repeatable • Eliminate environment inconsistencies between development, test, production • Supports segregation of duties • Improve speed and reliability of continuous integration and deployment • Lightweight containers address performance, costs, deployment and portability issues
  21. 21. 21 Common Use Cases • Development and Test Environment • Micro-services • Application Deployment without “dependency hell” of multiple applications • It Solves the “It Worked For Me!” issue • Continuous Integration • Platform-as-a-Service (PaaS)
  22. 22. 22 Docker Inc. • Founded in 2009 • Primary sponsor of the Docker Project • Hires maintainers and contributors • Provides infrastructure for the project • Runs the Docker Hub and Store • How does Docker Inc. make money? • Docker Hub has free and paid services • DTR (Docker Trusted Registry) • Enterprise support for Engine and other products • Training and professional services
  23. 23. 23 Docker Adoption Graph
  24. 24. 24 Docker Adoption
  25. 25. Installing Docker
  26. 26. 26 Let’s Begin: Installing Docker • Docker is easy to install • Runs on practically every Linux distribution (Kernel must be 2.6.32+ or 3.10+) • Runs on OS X via a virtual machine • Microsoft Windows (10) via virtual machine • Community Edition (CE) and Enterprise Edition are available • EE has support, and some certifications to hardware/software • CE has the community support and is way more common • Latest version is 17.09 (released on September, 2017), I still use 17.06.2…
  27. 27. 27 Docker on Windows 10 • Make sure your virtualization is on:
  28. 28. 28 Docker on Windows • Download the Docker Toolbox from https://www.docker.com/products/docker-toolbox • That will install: • Docker Client for Windows • Docker Toolbox management tool and ISO • Oracle VM VirtualBox • Git MSYS-git UNIX tools • Click the “QuickStart Terminal” on your desktop and enable the User Account Control permission
  29. 29. 29 Installing Docker: Oracle Linux 7 • Unlike other distros, Oracle Linux doesn’t enable the yum repository holding Docker by default, so we need to first enable it: • Use yum to install Docker engine: [root@lnx7-oracle-1 ~]# vi /etc/yum.repos.d/public-yum-ol7.repo ... [ol7_addons] name=Oracle Linux $releasever Add ons ($basearch) baseurl=http://yum.oracle.com/repo/OracleLinux/OL7/addons/$basearch/ gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle gpgcheck=1 enabled=1 [root@lnx7-oracle-1 ~]# yum install docker-engine
  30. 30. 30 Installing Docker: Oracle Linux 7 • Create a btrfs mount point for the docker file system • This filesystem will provide better performance and copy-on-write support • More on copy-on-write and btrfs - later mkfs.btrfs -L btrfs1 /dev/sdb echo “LABEL=btrfs1 /var/lib/docker btrfs defaults 0 1” >>/etc/fstab mkdir -p /var/lib/docker mount /var/lib/docker
  31. 31. 31 Installing Docker: Oracle Linux 7 • Start the Docker daemon and enable it to start automatically: [root@lnx7-oracle-1 ~]# systemctl start docker [root@lnx7-oracle-1 ~]# systemctl enable docker Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service. [root@lnx7-oracle-1 ~]# systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Drop-In: /etc/systemd/system/docker.service.d └─docker-sysconfig.conf Active: active (running) since Thu 2017-11-16 21:30:31 IST; 13min ago Docs: https://docs.docker.com Main PID: 881 (dockerd) Memory: 18.6M CGroup: /system.slice/docker.service ├─881 /usr/bin/dockerd --selinux-enabled
  32. 32. 32 Check Docker Version • After install, verify the docker version and docker info [root@lnx7-oracle-1 ~]# docker version Client: Version: 17.06.2-ol API version: 1.30 Go version: go1.8.3 Git commit: d02b7ab Built: Fri Oct 6 00:02:23 2017 OS/Arch: linux/amd64 Server: Version: 17.06.2-ol API version: 1.30 (minimum version 1.12) Go version: go1.8.3 Git commit: d02b7ab Built: Fri Oct 6 00:03:48 2017 OS/Arch: linux/amd64 Experimental: false
  33. 33. 33 The docker Group • Add a docker group • Add a user to the docker group • Restart the docker deamon • Relogin to the use the user, without using sudo to run things from root $ sudo groupadd docker $ sudo gpasswd -a $USER docker $ sudo systemctl restart docker.service
  34. 34. 34 Docker and Security • The docker user is root equivalent • It provides root-level access to the host • You should restrict access to it like you would protect root • If you give somebody the ability to access the Docker API, you are giving them full access on the machine • Therefore, the Docker control socket is (by default) owned by the docker group, to avoid unauthorized access on multi-user machines
  35. 35. Getting Started: Working With Docker
  36. 36. 36 Docker Components
  37. 37. 37 Docker Architecture • The Docker daemon (or "Engine"): receives and processes incoming Docker API requests • The Docker client talks to the Docker daemon via the Docker API • We'll use mostly the CLI embedded within the docker binary • GUI tools available, we will talk about them later • Docker Hub Registry (and Docker Store): Collection of public images • The Docker daemon talks to it via the registry API
  38. 38. 38 Hello World: Busybox • Let’s run a container that echo “hello world” • This started a very small (>1 MB) container called Busybox, and ran the single command we provided • Busybox is usually used for embedded system (routers, switches, etc.) [zohar@lnx7-oracle-1 ~]$ docker run --rm busybox echo hello world Unable to find image 'busybox:latest' locally latest: Pulling from library/busybox 0ffadd58f2a6: Pull complete Digest: sha256:bbc3a03235220b170ba48a157dd097dd1379299370e1ed99ce976df0355d24f0 Status: Downloaded newer image for busybox:latest hello world
  39. 39. 39 Using Shell to Busybox • Let’s connect to that container and have a look around • We used -it option to indicate “interactive” and “tty” (terminal) [zohar@lnx7-oracle-1 ~]$ docker run --rm -it busybox sh / # ps -ef PID USER TIME COMMAND 1 root 0:00 sh 5 root 0:00 ps -ef / # uname -a Linux a451342a146a 4.1.12-103.9.2.el7uek.x86_64 #2 SMP Tue Oct 31 16:43:46 PDT 2017 x86_64 GNU/Linux / #
  40. 40. 40 Which Containers are Up? • We can check which containers are up using docker ps • We can check which container was the last to start by using docker ps -l
  41. 41. 41 Detaching an Interactive Container • We can detach the interactive container by using CTRL+P,Q • When checking on the host what containers are up, we can see it’s running in the background • We can re-attach back to it by using docker attach command • We can use the container name or ID in order to re-attach [zohar@lnx7-oracle-1 ~]$ docker attach vibrant_borg / #
  42. 42. 42 Running Ubuntu in Container • Let’s try a more useful container: Ubuntu Linux • Here we can actually install things on it (it’s connected to the internet!) [zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu root@502f2202f0bc:/# [zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu root@502f2202f0bc:/# apt-get update Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB] Get:3 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [52.0 kB] Get:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB] [...]
  43. 43. 43 Working in the Container • Let’s have some fun: let’s install figlet • This only install the program on our current container, it does not change our image or our host system root@502f2202f0bc:/# apt-get install figlet Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: figlet […] root@502f2202f0bc:/# figlet hello _ _ _ | |__ ___| | | ___ | '_ / _ | |/ _ | | | | __/ | | (_) | |_| |_|___|_|_|___/
  44. 44. 44 Running Containers in the Background • We can start containers and run them in the background using the run -d flag – but what happens to their console output? • Console output is being logged – we can attach to the full output by using the docker logs command [zohar@lnx7-oracle-1 ~]$ docker run -d jpetazzo/clock 70e4be1a0c7ece5831632d169f21974ef9945234c6f48223a24ac92f8e73517f [zohar@lnx7-oracle-1 ~]$ docker ps -l CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 70e4be1a0c7e jpetazzo/clock "/bin/sh -c 'while..." 12 seconds ago Up 10 seconds tender_murdock [zohar@lnx7-oracle-1 ~]$ docker logs 70e Fri Nov 17 11:23:50 UTC 2017 Fri Nov 17 11:23:51 UTC 2017 Fri Nov 17 11:23:52 UTC 2017 Fri Nov 17 11:23:53 UTC 2017
  45. 45. 45 View Tail of the Log • To view only the tail of the log, we use docker logs --tail command (with or without –f) [zohar@lnx7-oracle-1 ~]$ docker logs 70e --tail 3 Fri Nov 17 11:27:30 UTC 2017 Fri Nov 17 11:27:31 UTC 2017 Fri Nov 17 11:27:32 UTC 2017
  46. 46. 46 Stopping the Container • Stopping our container gracefully: • We can also use the kill command which will not wait for the container to stop gracefully for 10 seconds • When using interactive mode, CTRL+C will send SIGINIT and close the container [zohar@lnx7-oracle-1 ~]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 502f2202f0bc ubuntu "/bin/bash" 5 minutes ago Up 5 minutes tender_keller [zohar@lnx7-oracle-1 ~]$ docker stop 502 502 [zohar@lnx7-oracle-1 ~]$
  47. 47. 47 One-Off Containers • Using the --rm flag when running the container, will cause the container to be removed completely when it is taken down • This is useful for “one-off” containers that will do an action and then disappear (like the “hello world” we did before) • We can also use the docker rm command to delete unused containers
  48. 48. 48 Showing Stopped Containers • Stopped containers still exist – they are not automatically removed • We can list all containers (running and stopped) by using docker ps –a [zohar@lnx7-oracle-1 ~]$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS 70e4be1a0c7e jpetazzo/clock "/bin/sh -c 'while..." 16 minutes ago Exited (137) 1 second ago 502f2202f0bc ubuntu "/bin/bash" 2 hours ago Exited (0) 16 minutes ago 262ebcab83c8 ubuntu "/bin/bash" 2 hours ago Exited (0) 2 hours ago
  49. 49. 49 Restarting and Attaching to Our Container • If we restart our container, the figlet command will be available (but we still can’t duplicate our container) • If we start another container (using run), the figlet will not be available because it’s not in our base image [zohar@lnx7-oracle-1 ~]$ docker start 502 502 [zohar@lnx7-oracle-1 ~]$ docker attach 502 root@502f2202f0bc:/# figlet hello again _ _ _ _ | |__ ___| | | ___ __ _ __ _ __ _(_)_ __ | '_ / _ | |/ _ / _` |/ _` |/ _` | | '_ | | | | __/ | | (_) | | (_| | (_| | (_| | | | | | |_| |_|___|_|_|___/ __,_|__, |__,_|_|_| |_| |___/
  50. 50. 50 docker exec Command • In some cases, attaching to a container will not allow us to interact with it (we’ll talk about CMD and ENTRYPOINT later) • In those cases, we will want to execute a command in the container to interact with it: • We can run whatever command we want, but bash is one of the more common… [zohar@lnx7-oracle-1 ~]$ docker exec -ti plugpdb bash
  51. 51. 51 Summary for the Basic Commands • docker ps [-a] [-l] [-q] • docker run [-it|-d] [--rm] • docker attach • docker start • docker stop • docker kill • docker rm (docker rmi)
  52. 52. Under the Hood: Images and Layers
  53. 53. 53 Images and Layers • An image is a collection of files + some meta data (technically: those files form the root filesystem of a container) • Images are made of layers, conceptually stacked on top of each other • Each layer can add, change, and remove files • Images can share layers to optimize disk usage, transfer times, and memory use • Layer default location: /var/lib/docker [Source: https://docs.docker.com/terms/layer/]
  54. 54. 54 Differences Between Containers and Images • An image is a read-only filesystem - we can’t change them after they are created • A container is an encapsulated set of processes running in a read- write copy of that filesystem • Images are like templates/stencils we create Containers from • To optimize container boot time, copy-on-write is used instead of regular copy • The command docker run starts a container from a given image
  55. 55. 55 Copy-On-Write • Copy-on-write, sometimes referred to as implicit sharing or shadowing is a resource-management technique to efficiently implement a "duplicate" or "copy" operation on modifiable resources • If a resource is duplicated but not modified, it is not necessary to create a new resource • Modifications must still create a copy, but the copy operation is deferred to the first write • By sharing resources in this way, it is possible to significantly reduce the resource consumption of unmodified copies, while adding a small overhead to resource-modifying operations
  56. 56. 56 UnionFS and BTRFS • AUFS is a union filesystem. The aufs storage driver was previously the default storage driver used for managing images and layers on Docker • Btrfs is a next generation copy-on-write filesystem that supports many advanced storage technologies that make it a good fit for Docker • Btrfs is included in the mainline Linux kernel • Docker’s btrfs storage driver leverages many btrfs features for image and container management: block-level operations, thin provisioning, copy-on-write snapshots, and ease of administration
  57. 57. 57 Where Do We Get a Base Image? • Build our own from “Scratch” • Docker Hub and Docker Hub Store • Some images might require approve in the store before pulling it • Get recipes off GitHub or BitBucket • Remember: Images are read-only, we can’t change them after they are created but we can evolve them
  58. 58. 58 Wait! Images Are Read-Only?! • Question: If images are read-only, how do we change them? • Answer: We don’t! • We create a new container from that image • We make changes to that container • When we are satisfied with those changes, we evolve them into a new layer • A new image is created by stacking the new layer on top of the old image
  59. 59. 59 The Chicken and Egg Problem
  60. 60. 60 Creating the First Image • There is a special empty image called scratch which means the first image is built from “scratch” • The docker import command loads a tarball into Docker • The imported tarball becomes a standalone image • That new image has a single layer • We probably don’t need this – we will use more “common” base images
  61. 61. 61 The Docker Hub Repository • Docker Hub is a public repository of pre-built images • We can upload (push) images, or download (pull) other people/organizations images to base our containers/layers on • If we don’t want to use the public Hub, we can create our own local repositories (and namespaces)
  62. 62. 62 Downloading an Image • Explicitly: we can download an image without initiating a container using the docker pull command • Download the exact image only • Image kept on the Docker host ready to use • Implicitly: when executing docker run command, it will download all the images needed that are not locally stored • Download the entire dependency layers • Only being done once, on the first run of that image
  63. 63. 63 Pulling Image Example • Using the run command, we tried to start a container from the Ubuntu image. Some layers did not exist so they were automatically pulled and downloaded: [zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu bash Unable to find image 'ubuntu:latest' locally latest: Pulling from library/ubuntu ae79f2514705: Downloading [===================> ] 18.19MB/47.62MB c59d01a7e4ca: Download complete 41ba73a9054d: Download complete f1bbfd495cc1: Download complete 0c346f7223e2: Download complete
  64. 64. 64 Listing The Local Docker Images • List all the local images by using the docker images command [zohar@lnx7-oracle-1 ~]$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE figlet latest 564ca4364896 5 days ago 162MB mysql latest 5709795eeffa 12 days ago 408MB ubuntu latest dd6f76d9cc90 13 days ago 122MB busybox latest 6ad733544a63 13 days ago 1.13MB jpetazzo/namer master f1557ed35146 2 years ago 723MB jpetazzo/clock latest 12068b93616f 2 years ago 2.43MB
  65. 65. 65 But How Do We Find Images? • Finding an image in the repository is easy : • Find it online using hub.docker.com or store.docker.com • Use the shell interface: $ docker search mysql
  66. 66. 66 Finding an Image: Shell • Name is the name and namespace of the image we are searching for • Root-like (i.e. mysql or mariadb) • User/organizaion (i.e. zabbix or realmgic) • Self hosted (e.g. registry.example.com:5000/my-private- image) • Description is an optional description for your images
  67. 67. 67 Namespaces • Root: official images. They are put there by Docker Inc., but they are generally authored and maintained by third parties • Small, "swiss-army-knife" images like busybox • Distro images to be used as bases for your builds, like ubuntu, fedora • Ready-to-use components and services, like redis, postgresql, mysql • User: The user namespace holds images for Docker Hub users and organizations • Self-Hosted: Images which are not hosted on Docker Hub, but on third party registries. They contain the hostname (or IP address), and optionally the port, of the registry server.
  68. 68. 68 Finding an Image (cont.) • Stars indicate how poplar an image is • Official indicate it is part of the root namespace • Automated images are built automatically by the Docker Hub using submitted Dockerfiles in GitHub - this means that their build recipe is always available
  69. 69. 69 A Word of Warning! • Images on the Hub are not being verified and might be harmful • Use trusted images from the store to make sure legitimate source • You can also use images with a known Dockerfile – for example, Automated Docker Images where you can verify their build process
  70. 70. 70 Image Tags • Images can have tags to define image versions or variants • docker pull ubuntu will refer to ubuntu:latest • The :latest tag is generally updated often • We only specify tags when: • We want to ensure that the same version will be used everywhere • Recording a procedure into a script • Going to production • Ensure repeatability later
  71. 71. 71 Saving a New Image • docker commit • Saves all the changes made to a container into a new layer • Creates a new image (effectively a copy of the container) • Be careful, it saves everything you did to the container! • docker build • Performs a repeatable build sequence in a Dockerfile (recipe) • This is the preferred method!
  72. 72. 72 Pushing an Image to the Hub • Once the image is ready, pushing it to the hub is simple - using the docker push command • Make sure your image is tagged before uploading [zohar@lnx7-oracle-1 ~]$ export DOCKER_ID_USER=“realmgic“ [zohar@lnx7-oracle-1 ~]$ docker login Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one. Username (realmgic): Password: Login Succeeded [zohar@lnx7-oracle-1 ~]$ docker push $DOCKER_ID_USER/my_image
  73. 73. 73 Building a New Image Example • Let’s use the example from earlier and build an image that has the figlet tool in it • Our base image will be the Ubuntu Image • The base package might not be updated, so let’s update its Linux packages and install figlet [zohar@lnx7-oracle-1 ~]$ docker run -it ubuntu root@502f2202f0bc:/# root@502f2202f0bc:/# apt-get update && apt-get install figlet Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB] Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB] [...]
  74. 74. 74 Check Container Differences • We can check what changed between this container and the base image using the docker diff command • Since Docker uses copy-on-write, these are the only files kept for this container on disk. Multiple containers share the base image files (due to UnionFS/btrfs for example) [zohar@lnx7-oracle-1 ~]$ docker diff 502 C /root A /root/.bash_history C /etc C /etc/alternatives A /etc/alternatives/figlet A /etc/alternatives/figlet.6.gz [...]
  75. 75. 75 Commit The Changes • Create a new image from the newly built container using the docker commit command • The new image will look like this since we didn’t tag it with a name: • Tag the new image: [zohar@lnx7-oracle-1 ~]$ docker commit 502 sha256:8cc5680434bb94b272e59624cd44667eaab191a55a08c38a90c472f9f9b45b3f [zohar@lnx7-oracle-1 ~]$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> 8cc5680434bb About a minute ago 162MB [zohar@lnx7-oracle-1 ~]$ docker tag 8cc5680434bb figlet [zohar@lnx7-oracle-1 ~]$ docker images REPOSITORY TAG IMAGE ID CREATED SIZE figlet latest 8cc5680434bb 5 minutes ago 162MB
  76. 76. 76 Manual Bad, Automatic Good
  77. 77. 77 Automated Docker Images: Dockerfile • Dockerfile is a build recipe to customize the base (r/o) image and make it into a new image Base Docker Image Instruction • Run commands • Add files/directories • Define ENV variables • Define processes to run • Set Entrypoint New Docker Image
  78. 78. 78 Automated Docker Images on Hub • Docker Hub can automatically build images based on Dockerfiles placed on Github • These Images can be built automatically by the Hub or we can download the recipe and build it ourselves • Advantages: • Images built this way are built exactly as specified • The Dockerfile is available to anyone with access to your Docker Hub repository • Your repository is kept up-to-date with code changes automatically
  79. 79. 79 Creating a Dockerfile • Our new Dockerfile must be in an empty directory • In the Dockerfile, we specify the base Image we start from using the FROM command • The RUN commands are never interactive, so we’ll need to sort it out when we build the script [zohar@lnx7-oracle-1 ~]$ mkdir figlet [zohar@lnx7-oracle-1 ~]$ cd figlet/ [zohar@lnx7-oracle-1 figlet]$ vi Dockerfile FROM ubuntu RUN apt-get update -y RUN apt-get install –y figlet
  80. 80. 80 Build The Image • Create a new image from the Dockerfile we createdusing the docker build command [zohar@lnx7-oracle-1 figlet]$ docker build -t figlet_auto . Sending build context to Docker daemon 2.048kB Step 1/3 : FROM ubuntu ---> dd6f76d9cc90 Step 2/3 : RUN apt-get update -y ---> Running in 174cec778baa [...] ---> 035ad14c7312 Removing intermediate container 174cec778baa Step 3/3 : RUN apt-get install figlet -y ---> Running in e9d2c2870c5d [...] ---> e6e0b459c15c Removing intermediate container e9d2c2870c5d Successfully built e6e0b459c15c Successfully tagged figlet_auto:latest
  81. 81. 81 Reading the Output: Step 1 • Our base image is Ubuntu – so we start a new container for it (dd6f76d9cc90) and make it our base image • The -t indicates the tag for the name of our new image [zohar@lnx7-oracle-1 figlet]$ docker build -t figlet_auto . Sending build context to Docker daemon 2.048kB Step 1/3 : FROM ubuntu ---> dd6f76d9cc90
  82. 82. 82 Reading the Output: Step 2 • We start a new container from our base image (174cec778baa) and modify it using our RUN command • At the end of this step, we’ll have a new base image: 035ad14c7312 • The intermediate base image will be removed Step 2/3 : RUN apt-get update -y ---> Running in 174cec778baa [...] ---> 035ad14c7312 Removing intermediate container 174cec778baa
  83. 83. 83 Reading the Output: Step 3 • We repeat the process for step 3 • The final container becomes our base image (e6e0b459c15c) and it is tagged to the name we chose (figlet_auto:latest) Step 3/3 : RUN apt-get install figlet -y ---> Running in e9d2c2870c5d [...] ---> e6e0b459c15c Removing intermediate container e9d2c2870c5d Successfully built e6e0b459c15c Successfully tagged figlet_auto:latest
  84. 84. 84 Image History • The history command lists all the layers composing an image • For each layer, it shows its creation time, size, and creation command • When using Dockerfile, each line indicates a single command [zohar@lnx7-oracle-1 figlet]$ docker history figlet_auto IMAGE CREATED CREATED BY SIZE COMMENT 4cbdf48e2076 About a minute ago /bin/sh -c apt-get install figlet -y 1.02MB c12b1ebe20b8 2 minutes ago /bin/sh -c apt-get update -y 39.2MB dd6f76d9cc90 13 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B <missing> 13 days ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B <missing> 13 days ago /bin/sh -c sed -i 's/^#s*(deb.*universe... 2.76kB <missing> 13 days ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B <missing> 13 days ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B <missing> 13 days ago /bin/sh -c #(nop) ADD file:5b334adf9d9a225... 122MB
  85. 85. 85 Image History (cont.) • When NOT using Dockerfile, some lines will not explain what really happened: [zohar@lnx7-oracle-1 figlet]$ docker history figlet IMAGE CREATED CREATED BY SIZE COMMENT 8cc5680434bb 43 minutes ago /bin/bash 40.3MB dd6f76d9cc90 13 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0B <missing> 13 days ago /bin/sh -c mkdir -p /run/systemd && echo '... 7B <missing> 13 days ago /bin/sh -c sed -i 's/^#s*(deb.*universe... 2.76kB <missing> 13 days ago /bin/sh -c rm -rf /var/lib/apt/lists/* 0B <missing> 13 days ago /bin/sh -c set -xe && echo '#!/bin/sh' >... 745B <missing> 13 days ago /bin/sh -c #(nop) ADD file:5b334adf9d9a225... 122MB
  86. 86. 86 The Caching System • Building the same image again will be instantaneous • After each step docker takes a snapshot of the resulting image • Before executing the step, Docker checks if it has already built that sequence (based on the text in the Dockerfile) • Example: • Changing the apt-get install figlet and add a space, it will run again • The apt-get update will not run again even though the apt mirrors updated (same command) • We can force a rebuild with docker build --no-cache
  87. 87. 87 CMD or ENTRYPOINT? • Both ENTRYPOINT and CMD give you a way to identify which command should be run when a container is started from your image • For example, most Linux container will use /bin/bash or /bin/sh as their default CMD to allow shell access • Running a container without a CMD or ENTRYPOINT will result with an error (either built in or via docker run command) • Each image can have only one CMD and one ENTRYPOINT. Specifying more than one in the Dockerfile will override the previous command
  88. 88. 88 Using CMD • Example: let’s say we want to change or figlet image to display “hello” message. • We need to modify the Dockerfile and build it: • When running a new image: FROM ubuntu RUN apt-get update -y RUN apt-get install figlet -y CMD figlet hello [zohar@lnx7-oracle [zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto _ _ _ | |__ ___| | | ___ | '_ / _ | |/ _ | | | | __/ | | (_) | |_| |_|___|_|_|___/
  89. 89. 89 Override CMD • If we want to override the CMD, we just send in a new command in the run command • The bash canceled our figlet command – but what happens if we don’t want it to override? What if we want to send in a parameter? [zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto bash root@388ac4ed1d28:/#
  90. 90. 90 Using ENTRYPOINT • We want to be able to specify a different message on the command line (parameter) • When running a new image: FROM ubuntu RUN apt-get update -y RUN apt-get install figlet -y ENTRYPOINT [“figlet”, “-f”, “banner”] [zohar@lnx7-oracle [zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto banner ##### ## # # # # ###### ##### # # # # ## # ## # # # # ##### # # # # # # # # ##### # # # # ###### # # # # # # # ##### # # # # # ## # ## # # # ##### # # # # # # ###### # #
  91. 91. 91 Override ENTRYPOINT • If we want to override the ENTRYPOINT, we can’t just send in a new command in the run command because it will be printed… • We need to override it with the --entrypoint flag: [zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto bash ##### ## #### # # # # # # # # # ##### # # #### ###### # # ###### # # # # # # # # # # # ##### # # #### # # [zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti --entrypoint bash figlet_auto root@5dd5da9a0825:/#
  92. 92. 92 Using CMD and ENTRYPOINT Together • In our example, figlet needs and extra parameter to work (the message). How can we specify a default value for it? • We can combine using CMD and ENTRYPOINT • ENTRYPOINT will specify the command to run • CMD will specify the default parameter FROM ubuntu RUN apt-get update -y RUN apt-get install figlet -y ENTRYPOINT ["figlet", "-f", "banner"] CMD ["CMD"]
  93. 93. 93 Using ENTRYPOINT and CMD Example [zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto ##### # # ###### # # ## ## # # # # # # # # # # # # # # # # # # # # # # # # # # ##### # # ###### [zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti figlet_auto test ##### ###### #### ##### # # # # # ##### #### # # # # # # # # # # # ###### #### #
  94. 94. 94 Copying Files From Host to Image • We can copy files from our host to the image when we build it • We can see the file on our image’s container • Changing the file will invalidate the cached layer – so rebuilding the image with a new file will generate a new image FROM ubuntu RUN apt-get update -y RUN apt-get install figlet -y COPY zohar.txt / ENTRYPOINT ["figlet", "-f", "banner"] CMD ["CMD"] [zohar@lnx7-oracle-1 figlet]$ docker run --rm -ti --entrypoint bash figlet_auto root@b65dd0df7597:/# cat /zohar.txt Testing a copy file...
  95. 95. 95 The ADD Command • Since the COPY command does not support copying file from a URL, we can use the ADD command for that • ADD also allows us to copy tar files and automatically open them • It is best practice to use COPY unless you must use ADD
  96. 96. 96 The WORKDIR Instruction • The WORKDIR instruction sets the working directory for subsequent instructions • It also affects CMD and ENTRYPOINT, since it sets the working directory used when starting the container • You can specify WORKDIR again to change the working directory for further operations WORKDIR /src
  97. 97. 97 The ENV Instruction • The ENV instruction specifies environment variables that should be set in any container launched from the image • This is the same as exporting in the OS level • ENV variables can be overridden by the -e flag in the docker run command ENV ORACLE_SID orclcdb docker run -p 1521:1521 -e ORACLE_SID=orcl store/oracle/database-enterprise:12.2.0.1
  98. 98. 98 The USER Instruction • The USER instruction sets the user name or UID to use when running the image • It can be used multiple times to change back to root or to another user
  99. 99. Docker Networks
  100. 100. 100 Docker Network Basics • Containers cannot have public IPv4 addresses • They do have private addresses • Services have to be exposed port by port • Ports have to be mapped to different ports on the host to avoid conflicts • Containers can see the network by default – it uses our default bridge network • Docker --link only worked on a single host – does not work in swarm/cluster and considered deprecated
  101. 101. 101 Example: NGINX Server • Starting a NGINX (web) server • The -P flag (publish all) indicates every exposed port to be mapped automatically to a port in the host server. • In this case, port 80 will be mapped to port 9000 • We can also ask the container for his mapped ports: [zohar@lnx7-oracle-1 ~]$ docker run -d -P nginx [zohar@lnx7-oracle-1 ~]$ docker ps -l CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 47733cd2d419 nginx "nginx -g 'daemon ..." About a minute ago Up About a minute 0.0.0.0:9000->80/tcp gallant_poincare [zohar@lnx7-oracle-1 ~]$ docker port 477 80/tcp -> 0.0.0.0:9000
  102. 102. 102 Example: NGINX Server (cont.) • When we try to reach the container port, we use the localhost port for it: [zohar@lnx7-oracle-1 ~]$ curl -l localhost:9000 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title>
  103. 103. 103 Manual Binding of Ports • Setting the ports manually can be done during the run command • The convention is host-port:container-port • Trying to start another container with the same port will result with an error [zohar@lnx7-oracle-1 ~]$ docker run --rm -d -p 80:80 nginx [zohar@lnx7-oracle-1 ~]$ docker run --rm -d -p 80:80 nginx 6abc743b8dbaa46b0af1d6f4d72f6313bd41baa9c74524649ac7fe489e387709 docker: Error response from daemon: driver failed programming external connectivity on endpoint friendly_bohr (dde052b96b19300988e0c4ea389fe5d0a91eab3e91bafd0dfd9d1d454869dc27): Bind for 0.0.0.0:80 failed: port is already allocated.
  104. 104. 104 Exposing Ports in the Image using Dockerfile • The EXPOSE instruction tells Docker what ports are to be published in this image • The Dockerfile doesn’t actually map the port to a port on the host – we still need to make it public in our run command EXPOSE 1521
  105. 105. 105 Finding Container Private IP • We can use docker inspect command to find the private IP of a container • The docker inspect is an advanced command, that can retrieve tons of information about our containers • We can ping our container [zohar@lnx7-oracle-1 ~]$ docker inspect --format '{{ .NetworkSettings.IPAddress }}' keen_neumann 172.17.0.3 [zohar@lnx7-oracle-1 ~]$ ping 172.17.0.3 PING 172.17.0.3 (172.17.0.3) 56(84) bytes of data. 64 bytes from 172.17.0.3: icmp_seq=1 ttl=64 time=0.109 ms 64 bytes from 172.17.0.3: icmp_seq=2 ttl=64 time=0.148 ms
  106. 106. 106 Docker Networks • Conceptually, a Docker network is a virtual switch • It can be local (to a single Engine) or global (across multiple hosts) • Each network has a dedicated subnet • Containers with explicit names are discoverable via DNS • A network is managed by drivers – some of them are built in and some can be added by plugins [zohar@lnx7-oracle-1 ~]$ docker network ls NETWORK ID NAME DRIVER SCOPE 03d9803daff0 bridge bridge local 3eddac377534 host host local e733f2a48ed2 none null local
  107. 107. 107 The Default Bridge Driver • By default, the container gets a virtual eth0 interface • It is connected to the Docker bridge network • Addresses are allocated on a private, internal subnet • Outbound traffic goes through an iptables MASQUERADE rule • Inbound traffic goes through an iptables DNAT rule • The container can have its own routes, iptables rules, etc.
  108. 108. 108 The Host Driver • The host network is internal network between the host and the container • A container with a host driver sees (and can access) the network interfaces of the host • Network traffic doesn't have to go through NAT, bridge, or veth • Performance = native!
  109. 109. 109 The null Driver • It only gets the lo loopback interface. No eth0 • It can't send or receive network traffic • Useful for isolated/untrusted workloads
  110. 110. 110 Creating a Network • Create a network • Use network for a container [zohar@lnx7-oracle-1 ~]$ docker network create OracleWeek 72635c667b4c3e54e350925339cef73cf480528cef6368d6f59954d2358c210d [zohar@lnx7-oracle-1 ~]$ docker network ls NETWORK ID NAME DRIVER SCOPE 72635c667b4c OracleWeek bridge local 03d9803daff0 bridge bridge local 3eddac377534 host host local e733f2a48ed2 none null local [zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont1 --net OracleWeek busybox / # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 144: eth0@if145: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff inet 172.18.0.2/16 scope global eth0 valid_lft forever preferred_lft forever
  111. 111. 111 Containers DNS • All containers with specific name in the same network share the same dynamic DNS (placed at 127.0.0.11): [zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont2 --net OracleWeek busybox / # ping cont1 PING cont1 (172.18.0.2): 56 data bytes 64 bytes from 172.18.0.2: seq=0 ttl=64 time=0.176 ms 64 bytes from 172.18.0.2: seq=1 ttl=64 time=0.332 ms / # nslookup cont1 Server: 127.0.0.11 Address 1: 127.0.0.11 Name: cont1 Address 1: 172.18.0.2 cont1.OracleWeek / # nslookup cont2 Server: 127.0.0.11 Address 1: 127.0.0.11 Name: cont2 Address 1: 172.18.0.3 05efb3eb1863
  112. 112. 112 Connecting Multiple Networks • Networks are independent from each other but a container can be connected to multiple networks • We can attach a network to a running container • Example: • Create a new network and a new container in that network [zohar@lnx7-oracle-1 ~]$ docker network create ilOUG 41244791088e6970dca741b54fa54a8e0f0a08eeb1b184a78220a9cf03d2528a [zohar@lnx7-oracle-1 ~]$ docker run -it --rm --name cont3 --net ilOUG busybox / # ping cont1 ping: bad address 'cont1‘
  113. 113. 113 Connecting Multiple Networks (cont.) [zohar@lnx7-oracle-1 docker network connect ilOUG cont1 [zohar@lnx7-oracle-1 ~]$ docker attach cont1 / # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 144: eth0@if145: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue link/ether 02:42:ac:12:00:02 brd ff:ff:ff:ff:ff:ff inet 172.18.0.2/16 scope global eth0 valid_lft forever preferred_lft forever 153: eth1@if154: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue link/ether 02:42:ac:13:00:03 brd ff:ff:ff:ff:ff:ff inet 172.19.0.3/16 scope global eth1 valid_lft forever preferred_lft forever / # ping cont2 PING cont2 (172.18.0.3): 56 data bytes 64 bytes from 172.18.0.3: seq=0 ttl=64 time=0.203 ms / # ping cont3 PING cont3 (172.19.0.2): 56 data bytes 64 bytes from 172.19.0.2: seq=0 ttl=64 time=0.108 ms
  114. 114. 114 Multi-host Networking: Overlay Driver • When using cluster, overlay network will interact between nodes
  115. 115. 115 Overlay Driver • The overlay network driver can support a clustered (swarm mode) environment • When you create a service that uses the overlay network, the manager node automatically extends the overlay network to nodes that run service tasks • Only swarm services can connect to overlay networks, not standalone containers • The overlay network without swarm requires a valid key-value store service (Consul, Etcd, and ZooKeeper (Distributed store)) and might be deprecated in the future
  116. 116. Docker Storage Volumes, Bind Mounts and tmpfs
  117. 117. 117 Docker “External” Storage • Our containers are independent – they can’t see the host filesystem , but sometimes we want to share files/directories with the host or between containers • The solution: map a volume to the container so it will be available to both the host and all the using containers
  118. 118. 118 Named Volume • Created and managed by Docker • We can create a volume explicitly using the docker volume create command, or Docker can create a volume during container or service creation • When creating a volume, it is stored within a directory on the Docker host (empty) • When you mount the volume into a container, this directory is what is mounted into the container • A given volume can be mounted into multiple containers simultaneously
  119. 119. 119 Named Volumes Example • We can create volumes, list them and inspect for actual location: [zohar@lnx7-oracle-1 ~]$ docker volume create test Test [zohar@lnx7-oracle-1 ~]$ docker volume ls DRIVER VOLUME NAME local fc7d0d5897f2090f51a3ee7589ba360841344dc34c605626337debff4dfa9860 local test [zohar@lnx7-oracle-1 ~]$ docker inspect test [ { "Driver": "local", "Labels": {}, "Mountpoint": "/var/lib/docker/volumes/test/_data", "Name": "test", "Options": {}, "Scope": "local" } ]
  120. 120. 120 Volumes Functionality • Volumes act as passthroughs to the host filesystem • Volumes are easier to back up or migrate than bind mounts • Volumes can be more safely shared among multiple containers • When you docker commit, the content of volumes is not brought into the resulting image • Volume drivers allow you to store volumes on remote hosts or cloud providers, to encrypt the contents of volumes, or to add other functionality
  121. 121. 121 Bind Mounts • Bind mounts have limited functionality compared to volumes • When we use a bind mount, a file or directory on the host machine is mounted into a container • The file or directory is referenced by its full path on the host machine • The file or directory does not need to exist on the Docker host already, if it does not exist – it will be automatically created (when using –v) • We use bind mounts when we want to mount a non-empty directory from the host to the container
  122. 122. 122 Binding the Mount Volumes • Volumes can be mounted in two ways: • on the docker run command • Within a Dockerfile, with a VOLUME instruction docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox VOLUME /zohar
  123. 123. 123 Mounting a Volume Manually • We can map volumes on the docker run command • The –v flag mounts a directory from your host into your Docker container. The flag structure is: • If [host-path] or [container-path] doesn't exist it is created • You can control the write status of the volume with the ro and rw options • If you don't specify rw or ro, it will be rw by default docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox [host-path]:[container-path]:[rw|ro|z|Z]
  124. 124. 124 Volumes and SElinux • The permission on the volume will be the same as on the host • That might be a problem if the files are owned by a user the container doesn’t know • We can “help” Docker and SElinux play nice together by specifying the “z” (r/o) or “Z” (r/w) flag: [zohar@lnx7-oracle-1 ~]$ docker run -it -v $(pwd):/zohar --rm --name cont1 --net ilOUG busybox / # ls -l /zohar/ ls: can't open '/zohar/': Permission denied / # [zohar@lnx7-oracle-1 docker run -it -v $(pwd):/zohar:Z --rm --name cont1 --net ilOUG busybox / # ls /zohar/ Desktop Documents Downloads Music Pictures Public Templates Videos figlet myimage namer oradata
  125. 125. 125 tmpfs Mounts • A tmpfs mount is not persisted on disk, either on the Docker host or within a container • It can be used by a container during the lifetime of the container, to store non-persistent state or sensitive information [zohar@lnx7-oracle-1 ~]$ docker run -it --rm --tmpfs /zohar/tmpfs busybox / # df -h /zohar/tmpfs/ Filesystem Size Used Available Use% Mounted on tmpfs 1.8G 0 1.8G 0% /zohar/tmpfs
  126. 126. 126 Orphaned Volumes • What happens when you remove containers with volumes? • They stay on disk – even if no one will ever use them again • Use the –v flag to remove the container with the volumes attached • We can prune the unused volumes using the docker volume prune command • Ultimately, you are the one responsible for logging, monitoring, and backup of your volumes!
  127. 127. Oracle RDBMS, MySQL and Docker
  128. 128. 128 Oracle RDBMS Official Image • Oracle provides their own Oracle RDBMS official image with database (12.2.0.1 EE) included (free): • https://store.docker.com/images/oracle-database-enterprise-edition • In order to pull this image, we need a Docker repository account • We also need to accept the conditions on the store itself • Official Image from Oracle Store: • https://container-registry.oracle.com/ docker login docker pull store/oracle/database-enterprise:12.2.0.1
  129. 129. 129 Oracle Official Image Limitations • Supports a single instance database, • No multiple homes • No Grid Infrastructure/ASM • No multiple instances on the same container • No AMM • Dataguard is not supported • Database options and patching are not supported
  130. 130. 130 Oracle Official Slim Image • Since the official image is quite big (~4GB) there is also a slim version: • This image does not support the following features - Analytics, Oracle R, Oracle Label Security, Oracle Text, Oracle Application Express and Oracle DataVault. • The slim version is around 1.4GB • Oracle support does not require you to use the provided Docker files – we can build our own images! docker pull store/oracle/database-enterprise:12.2.0.1-slim
  131. 131. 131 Building Our Own Images • Oracle official Docker team github account with different Dockerfiles and samples: https://github.com/oracle/docker-images • On the repository: • Oracle 11.2.0.2 (XE) Dockerfile • Oracle 12.1.0.2 Dockerfile and scripts (SE2, EE) • Oracle 12.2.0.1 Dockerfile and scripts (SE2, EE) • Lots of other Oracle stuff: Oracle Java, Weblogic, GoldenGate and NoSQL… • The build recipes build our own images, so we can customize it • Recipes require ZIP files of the software for deployment on the image
  132. 132. 132 How It’s Going to Work • The official image based on Oracle Linux 7 • The image contains an instance running inside the container • The ORACLE_HOME and datafiles are kept on a volume (host machine) • The image also runs the listener on port 1521 and exposes this port • The instance runs inside the container is a root container (CDB) in our multitenant database. It also has a single plugged database • Persistence: • There is a volume plug we can use to map our local pluggable database files to the database and then we can plug our database to the instance
  133. 133. 133 Creating the Container • Creating the container (might take some time): [zohar@lnx7-oracle-1 ~]$ docker volume create OracleDBData [zohar@lnx7-oracle-1 ~]$ docker run --name plugpdb -p 1521:1521 -p 5500:5500 -e ORACLE_SID=MYCDB -v OracleDBData:/ORCL /home/zohar/oradata:/opt/oracle/oradata store/oracle/database-enterprise:12.2.0.1 Setup Oracle Database Oracle Database 12.2.0.1 Setup Sat Nov 18 16:09:47 UTC 2017 Check parameters ...... log file is : /home/oracle/setup/log/paramChk.log paramChk.sh is done at 0 sec untar DB bits ...... log file is : /home/oracle/setup/log/untarDB.log untarDB.sh is done at 166 sec config DB ...... log file is : /home/oracle/setup/log/configDB.log […]
  134. 134. 134 Inside the Container • The entire database home and database file are stored in the same location: /ORCL, divided into u01-u04 for different usages • We can mount these areas into our own disks or volume • In our example, the entire /ORCL is mounted into one volume: OracleDBData • We can mount other mount points or volumes to keep our pluggable databases
  135. 135. 135 Connecting to the Database • Connecting to the database, via the container: • The initial password for SYS is “Oradoc_db1” • We can also connect using bash command and work like any other “server” docker exec -it plugdb bash -c "source /home/oracle/.bashrc; sqlplus /nolog"
  136. 136. 136 Building Our Own Image • Download the appropriate directory from and unzip it on the server • https://github.com/oracle/docker- images/tree/master/OracleDatabase/dockerfiles • Download the Oracle Software from the Oracle site and place them in the same directory • Modify whatever you need – scripts, Dockerfile, keep some of the options manually removed, etc. • Build the image docker build --force-rm=true --no-cache=true -t oracle/database:12.2.0.1-ee -f Dockerfile.ee .
  137. 137. 137 Start the Container • Running the container is just like before – we can pass some env variables but most of them already has default values • This database doesn’t have to use containers (like the official image): ORACLE_CONTAINERDB=false docker run --name orcl1 -p 1522:1521 -p 5502:5500 -e ORACLE_SID=orclcdb -e ORACLE_PWD=manager1 -e ORACLE_CHARACTERSET=WE8MSWIN1252 -e ORACLE_CONTAINERDB=false oracle/database:12.2.0.1-ee
  138. 138. 138 MySQL • MySQL database is ideal for Docker Containers – its containers exist since forever • There are actually two MySQL main images: • mysql (also available on the store) • mysql/mysql-server (automated, connected to mysql GitHub) • Both images are tagged with multiple version: 5.5, 5.6, 5.7 (latest) and 8.0 (preview). They also run sub version (e.g. 5.6.38, 5.7.20, etc.)
  139. 139. 139 Starting the MySQL Container • Starting the container • Connecting to the database using root: docker run --name my-container-name -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:tag docker exec -it my-container-name mysql -uroot -p
  140. 140. 140 Where to Store Data • There are several ways to store data used by applications that run in Docker containers: • Let Docker manage the storage of your database data by writing the database files to disk on the host system using its own internal volume management. This is the default and is easy and fairly transparent to the user • Create a data directory on the host system (outside the container) and mount this to a directory visible from inside the container. This places the database files in a known location on the host system, and makes it easy for tools and applications on the host system to access the files
  141. 141. 141 Connecting to Existing Database • If you start your mysql container instance with a data directory that already contains a database (specifically, a mysql subdirectory), it will use that as its database • Note: the $MYSQL_ROOT_PASSWORD variable should be omitted from the run command line; it will in any case be ignored, and the pre-existing database will not be changed in any way • If there is no database initialized when the container starts, then a default database will be created.
  142. 142. 142 Creating Database Dumps • Most of the normal tools will work, although their usage might be a little convoluted in some cases to ensure they have access to the mysqld server. A simple way to ensure this is to use docker exec and run the tool from the same container, similar to the following $ docker exec some-mysql sh -c 'exec mysqldump --all-databases -uroot -p"$MYSQL_ROOT_PASSWORD"' > /some/path/on/your/host/all-databases.sql
  143. 143. 143 Overriding Options and MySQL.cnf • We can override options by using the --option flag • To provide a different mysql.cnf, use the –v flag: • Remember SELinux? We need to allow the file for the container: docker run --name my-container-name -d mysql/mysql-server --option1=value -- option2=value docker run --name my-container-name -d mysql/mysql-server --character-set- server=utf8 --collation-server=utf8_general_ci docker exec -it my-container-name cat /etc/my.cnf > /my/custom/config-file docker run --name my-new-container-name -v /my/custom/config-file:/etc/my.cnf -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:tag chcon -Rt svirt_sandbox_file_t /my/custom/config-file
  144. 144. Docker Compose
  145. 145. 145 Docker Compose Tool • Dockerfiles are great to build a single container • But when you want to start a complex stack made of multiple containers, you need a different tool. This tool is Docker Compose • Docker Compose is optional (needs to be installed separately) • The general idea of Compose is to enable a very simple, powerful onboarding workflow: • Clone your code • Run docker-compose up • Your app is up and running!
  146. 146. 146 Compose overview • You describe a set (or stack) of containers in a YAML file called docker-compose.yml • You run docker-compose up • Compose automatically pulls images, builds containers, and starts them • Compose can set up networks, volumes, and other Docker options • Compose can run the containers in the background, or in the foreground • When containers are running in the foreground, their aggregated output is shown
  147. 147. 147 Installing Compose • We can download and install Docker Compose right off its github project: https://github.com/docker/compose/releases • Make sure to use the latest version (current version: 1.17.1): curl -L https://github.com/docker/compose/releases/download/1.17.1/docker-compose- `uname -s`-`uname -m` -o /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose
  148. 148. 148 docker-compose.yml • Example for a Docker compose yaml file: • This file will start two containers: one based on the local Dockerfile (www) and one based on the redis image. It will also mount a volume and map a port. version: '2' services: web: build: . ports: - "5000:5000" volumes: - .:/code redis: image: redis
  149. 149. 149 docker-compose.yml for Oracle DBMS • Another example for a Docker compose yaml file: • In this example, we will start a database server using the 12.2.0.1-ee image, map some ports. Our Oracle SID is also predefined. version: '2' services: database: image: oracle/database:12.2.0.1-ee volumes: - /home/oracle/oradata:/opt/oracle/oradata # persistent oracle database data. ports: - 1521:1521 - 8080:8080 - 5500:5500 environment: -ORACLE_SID=DEVOPSUATCDB Example by Gerald Venzl
  150. 150. 150 Docker Compose Usages • Starting a Docker Compose environment: • A network and two containers were created • Check what is up (in our work directory only): [zohar@lnx7-oracle-1 wordpress]$ docker-compose up -d Creating network "wordpress_default" with the default driver Creating wordpress_db_1 ... Creating wordpress_db_1 ... done Creating wordpress_wordpress_1 ... Creating wordpress_wordpress_1 ... done [zohar@lnx7-oracle-1 wordpress]$ docker-compose ps Name Command State Ports ------------------------------------------------------------------------------------- wordpress_db_1 docker-entrypoint.sh mysqld Up 3306/tcp wordpress_wordpress_1 docker-entrypoint.sh apach ... Up 0.0.0.0:8000->80/tcp
  151. 151. 151 Docker and Docker Compose • We can still use our original docker commands [zohar@lnx7-oracle-1 wordpress]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ce8847b28d67 wordpress:latest "docker-entrypoint..." 2 minutes ago Up 2 minutes 0.0.0.0:8000->80/tcp wordpress_wordpress_1 daff10f9d8b9 mysql:5.7 "docker-entrypoint..." 2 minutes ago Up 2 minutes 3306/tcp wordpress_db_1 [zohar@lnx7-oracle-1 wordpress]$ docker network ls NETWORK ID NAME DRIVER SCOPE [...] 4f395cc553ad wordpress_default bridge local
  152. 152. 152 Stopping the Service • We can stop the entire service by using docker-compose stop • We can remove the service by using the docker-compose down command. This will delete all containers as well (but not the volumes) [zohar@lnx7-oracle-1 wordpress]$ docker-compose stop Stopping wordpress_wordpress_1 ... done Stopping wordpress_db_1 ... done [zohar@lnx7-oracle-1 wordpress]$ docker-compose down Stopping wordpress_wordpress_1 ... done Stopping wordpress_db_1 ... done Removing wordpress_wordpress_1 ... done Removing wordpress_db_1 ... done Removing network wordpress_default
  153. 153. Misc.
  154. 154. 154 Docker Containers After Restart • When the host shut down, all the containers will automatically stop • When the host comes back up, all containers will be DOWN • In order to automatically start the container after boot, we can use the --restart option on the run command $ docker run -dit --restart unless-stopped redis Flag Description no Do not automatically restart the container. (the default) on-failure Restart the container if it exits due to an error, which manifests as a non-zero exit code. unless-stopped Restart the container unless it is explicitly stopped or Docker itself is stopped or restarted. always Always restart the container if it stops.
  155. 155. 155 Limiting Container Performance • We can limit the performance on a container: • CPU: number of cpu (which cpus to use), number of cycles, and different quotas • Memory: amount of memory, amount of virtual memory docker run --rm --cpuset-cpus=0 busybox echo hello world docker run --rm --cpuset-cpus=0 --cpu-period=50000 busybox echo hello world
  156. 156. 156 HEALTHCHECK • HEALTHCHECK instruction in DockerFile • Tells Docker how to test a container to check that it is still working properly (not just “up”) – i.e. the database/listener are up • New status added to container lists • Adds “(healthy)” to Status column in a “docker ps response” [zohar@lnx7-oracle-1 ~]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS d2926bc4b490 portainer/portainer "/portainer" 39 minutes ago Up 39 minutes 6a204203d9e7 store/oracle/database-ente... "/bin/sh -c '/bin/..." 3 hours ago Up 3 hours (healthy)
  157. 157. 157 Docker Secrets • Stored encrypted in swarm mode only • Exposed only to nodes that run services that need them • Presented in container via RAM only tmpfs files • never persisted to disk in encrypted format • when container stops, secret is no longer present • All communications between swam nodes via TLS, so secret never in the clear on the wire either • Different secret values per environment using tags
  158. 158. 158 Docker Security • Use trusted images / with a known Dockerfile • Think (twice) about pulling images from public repos / Docker hub • Docker images can be signed • Docker can use TLS (client to daemon) • Do NOT disable SELinux or set enforce off (avoid setenforce 0) • Do not keep secrets in images • Remember: The host can always access container
  159. 159. GUI, Orchestration and Clusters Swarm, GUI Interfaces
  160. 160. 160 Docker GUI • Kitematic – the default (now obsolete) interface from Docker themselves • Portainer - very simple, very robust GUI interface for Docker • Shipyard – Built on Docker Swarm, Shipyard gives you the ability to manage Docker resources including containers, images, private registries and more • Rancher – high end solution that interfaces nicely with cloud solutions • There are others – but I suggest going with Portainer (for beginners) or Rancher or Shipyard (more advanced)
  161. 161. 161 Portainer • Portainer is a simple but robust GUI interface for Docker • It runs from within a Docker container • It’s easier to see what is going on: • Containers • Volumes • Networks • Logs • Etc. docker run -d --privileged -p 9000:9000 -v /var/run/docker.sock:/var/run/docker.sock -v /opt/portainer:/data portainer/portainer
  162. 162. 162 Portainer Deshboard
  163. 163. 163 Portainer Containers
  164. 164. 164 Using Console From Web
  165. 165. 165 Monitoring Performance
  166. 166. 166 Kubernetes: Container Orchestration • Kubernetes is an open source container orchestration tool developed by Google to allow a wider usage of Docker with more control and better scaling and availability • It is one of the most commonly used container orchestration tool and has a very large community • Kubernetes allows you to completely control your container deployment across the cluster with custom storage and network solutions and well defined deployment rules • Kubernetes allows for easy integration with public (AWS, GCE, Azure) and private (VMWare) cloud solutions
  167. 167. 167 Kubernetes Concepts • While it provides a useful, scalable, and powerful tool, Kubernetes also gets a lot of clout because it’s an intricate and sometimes overly-complicated system • Installing Kubernetes is a slightly more complex endeavor since you will need additional tools to provide network (i.e. flannel) and store the cluster data (i.e. etcd). • Basic Concepts: • Pod: The basic deployment unit, One or more containers (usually one) that share a common function. • ReplicaSet: Controls the replication and availability of a pod. • Deployment: Controls the pod’s deployment by creating and managing ReplicaSets. Allows for rolling updates. • Service: Advertises a set of pods to other pods or to the outside world. Allows easy discovery with the DNS add-on. • StatefulSet: Manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods.
  168. 168. 168 Kubernetes Source: wikipedia
  169. 169. 169 Docker Swarm • Native Docker cluster  same API as a single engine • Fast provisioning, about 500 msec • Scheduling Algo: spread, binpack, rand • Features are optional, you can continue use Kubernetes etc. • There is NO insecure mode
  170. 170. 170 Docker Swarm Overview
  171. 171. 171 Docker Swarm • Since Docker 1.12 • Swarm is merged with Docker engine: • Load balancer included • Service discovery • Cluster scheduler • Swarm has many features like Google's Kubernetes • Easier to get started • Better performance
  172. 172. 172 Swarm Services • To deploy an application image when Docker Engine is in swarm mode, you create a service • Frequently a service will be the image for a microservice within the context of some larger application • Examples of services might include an HTTP server, a database, or any other type of executable program that you wish to run in a distributed environment
  173. 173. 173 Setup Swarm and Machine • Create Swarm ID • Create Machine with Swarm master • Create Machine with Swarm agent01 / 02 • Set Docker env for Swarm master • Deploy service to the swarm • If needed, scale the service to run more container on more nodes
  174. 174. 174 Learn More More information: • https://docs.docker.com/ • Reference, information, how-to • https://training.docker.com/ • Training, self paced and trainer • Blog: https://blog.docker.com/ • Videos: https://www.youtube.com/user/dockerrun
  175. 175. 175 Oracle Support • Support for Docker Running on Oracle Linux (Doc ID 1921163.1) • https://support.oracle.com/epmos/faces/DocumentDisplay?id=1921163.1 • Oracle Support Document 2216342.1 (Oracle Support for Database Running on Docker): • https://support.oracle.com/epmos/faces/DocumentDisplay?id=2216342.1 • Gerald Venzl (will be here in ilOUG Tech Days 2018 (January 2018), speaking about Docker!) • https://geraldonit.com/2017/08/21/creating-an-oracle-database-docker- image/
  176. 176. Did You Hear About Oracle 18c?
  177. 177. 177 • Oracle is changing the way they number products • Products will be release more often • Version names will be based on last two digits of the year and a subversion • There is no more Release 1-Release 2 for major versions, we can’t tell when we jumped to the next major version and when for the next minor • Examples: • 12.2.0.1 will be followed by 18.1 (and not 12.2.0.2) • SQL Developer 4.2.1 was followed by 17.3 • Read more at the updated MOS Note 742060.1 – Release Schedule of Current Database Releases Wait! 18c?! What happened to 13c?
  178. 178. 178 Summary • We talked about Docker and Docker containers use cases • We learned how to use Docker (slides will be available soon) • We saw how Docker works with Oracle and MySQL • We overviewed swarm mode (not for databases) • We talked about Docker in the cloud environment
  179. 179. Q&A Any Questions? Now will be the time!
  180. 180. Zohar Elkayam twitter: @realmgic Zohar@Brillix.co.il www.ilDBA.co.il www.realdbamagic.com

×