Docker is becoming more mainstream and adopted by users as a method to package and deploy self-sufficient applications in primarily stateless Linux containers. It's a great toolset on top of OS-level virtualization (LXC, a.k.a containers) and plays well in the world of micro services.
However, Docker containers are transient by default. If a container is destroyed, all data created is also lost. For a stateful service like a database, this is a major headache to say the least.
There are a number ways to provide persistent storage in Docker containers. In this presentation, we will talk about how to setup a persistence data service with Docker that can be torn down and brought up across hosts and containers.
We will touch upon orchestration tools, shared volumes, data-only-containers, security and configuration management, multi-host networking, service discovery and implications on monitoring when we move from host-centric to role-centric services with shorter life cycles.
5. What is Docker?
Copyright 2012 Severalnines AB
● Lightweight application container platform.
● Each container has its own:
○ Filesystem
○ Process space
○ Network stack
● Use pre-built image.
● Advantages:
○ Rapid deployment
○ Greater density per host
○ Isolation
Docker Architecture
8. Docker Image - Introduction
Copyright 2012 Severalnines AB
● A template for Docker container.
● Create an image:
○ Commit the container's changes
○ Build from Dockerfile
● Share the image:
○ Save as tarball.
○ Push to registry:
■ Public - Docker Hub, Docker Store
■ Private, hosted - Quay, ECR, GCR, Bintray,
Artifactory
■ Private, self-hosted - Your own registry
# Save image to tarball
$ docker save -o mysql.tar abc
# Start your own Docker registry service
$ docker run -d -p 5000:5000 --name registry
registry:2
# Push image to Docker Hub
$ docker push myimage/mysql:5.7
9. Docker Image - Commit a Container
Copyright 2012 Severalnines AB
Host
# Save image to tarball
$ docker run -d --name=abc ubuntu:16.04
# Commit the changes (in another window)
$ docker commit abc myimage/mysql:5.7
# Push image to Docker Hub
$ docker exec -it abc /bin/bash
root@abc:/# apt-get update
root@abc:/# DEBIAN_FRONTEND=noninteractive apt-get
install -y mysql-server
root@abc:/# vi /etc/mysql/my.cnf
# Ubuntu 16.04 host
$ apt-get update
$ DEBIAN_FRONTEND=noninteractive apt-get install -y
mysql-server
$ vi /etc/mysql/my.cnf
# Run MySQL server as a process
$ mysqld --socket=/var/lib/mysql/mysqld.sock
--pid-file=/var/lib/mysql/mysqld.pid
Docker
# Run MySQL server as a container
$ docker run -d myimage/mysql:5.7 mysqld
--socket=/var/lib/mysql/mysqld.sock
--pid-file=/var/lib/mysql/mysqld.pid
10. Docker Image - Using Dockerfile
Copyright 2012 Severalnines AB
Host
# Build as image
$ docker build -t myimage/mysql:5.7 .
# vi Dockerfile
FROM ubuntu:16.04
RUN apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y
mysql-server
COPY my.cnf /etc/mysql/my.cnf
EXPOSE 3306
# Ubuntu 16.04 host
$ apt-get update
$ DEBIAN_FRONTEND=noninteractive apt-get install -y
mysql-server
$ vi /etc/mysql/my.cnf
# Run MySQL server as a process
$ mysqld --socket=/var/lib/mysql/mysqld.sock
--pid-file=/var/lib/mysql/mysqld.pid
Docker
# Run MySQL server as a container
$ docker run -d myimage/mysql:5.7 mysqld
--socket=/var/lib/mysql/mysqld.sock
--pid-file=/var/lib/mysql/mysqld.pid
# Create MySQL config file
$ vi my.cnf
11. Docker Image - Build an Image
Copyright 2012 Severalnines AB
# Get image size
$ docker images
REPOSITORY TAG SIZE
myimage/mysql 5.7 529MB
# Show created layers
$ docker history myimage/mysql:5.7
CREATED BY SIZE
/bin/sh -c #(nop) COPY file:e8163ef656b6da... 101B
/bin/sh -c DEBIAN_FRONTEND=noninteractive ... 370MB
/bin/sh -c apt-get update 39.1MB
/bin/sh -c #(nop) CMD ["/bin/bash"] 0B
/bin/sh -c mkdir -p /run/systemd && echo '... 7B
/bin/sh -c sed -i 's/^#s*(deb.*universe... 2.76kB
/bin/sh -c rm -rf /var/lib/apt/lists/* 0B
/bin/sh -c set -xe && echo '#!/bin/sh' >... 745B
/bin/sh -c #(nop) ADD file:39d3593ea220e68... 120MB
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
724179e94999
f6016aab25f1
d51f459239fb
Image: myimage/mysql:5.7
ubuntu:16.04
ad74af05f5a2
Dockerfile
12. Docker Image - Best Practice for MySQL
Copyright 2012 Severalnines AB
# Get image size
$ docker images
REPOSITORY TAG SIZE
mysql 5.6 299MB
mysql 5.7 412MB
# tail entrypoint.sh
...
...
exec mysqld "$@"
● Extend an existing image available in the registry,
instead of building a new one from scratch.
● The smaller the image size, the faster the deployment.
(Hint: alpine).
● If using an entrypoint script, ensure mysqld command
runs with "exec", indicating it's the primary process
that holds PID 1 inside container.
● General guidelines:
https://docs.docker.com/engine/userguide/eng-image/
dockerfile_best-practices/
# vi Dockerfile
FROM mysql:5.7
# vi Dockerfile
FROM ubuntu:16.04
RUN apt-get install {all MySQL stuff}
14. Docker Container - Run a Single
Container
Copyright 2012 Severalnines AB
# Run a MySQL container
$ docker run -d
--name my-test
--env MYSQL_ROOT_PASSWORD=mypassword
mysql
# Get the container layer size
$ docker ps -s
CONTAINER ID IMAGE NAMES SIZE
d51f459239fb mysql my-test 4B (virtual 412MB)
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
d51f459239fb
0639788facc8
35369f9634e1
937bbdd4305a
my-test
Image layers
(RO)
412 MB
Container layers
(RW)
4 B
15. Docker Container - Run another
Container
Copyright 2012 Severalnines AB
# Run a MySQL container
$ docker run -d
--name my-test2
--env MYSQL_ROOT_PASSWORD=mypassword
mysql
# Get the container layer size
$ docker ps -s
CONTAINER ID IMAGE NAMES SIZE
d51f459239fb mysql my-test 4B (virtual 412MB)
e9de9ed50ced mysql my-test2 4B (virtual 412MB)
Image: docker.io/mysql
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
d51f459239fb
0639788facc8
35369f9634e1
937bbdd4305a
my-test
Image layers
(RO)
412 MB
Container
layers (RW)
4 B + 4 B
e9de9ed50ced
my-test2
16. Docker Container - Container Layer
Changes
Copyright 2012 Severalnines AB
# Operation 2 - Create a schema and table
$ docker exec -it my-test
mysql -uroot -pmypassword -e
'CREATE SCHEMA testdb; CREATE TABLE t1 (id INT PRIMARY
KEY AUTO_INCREMENT, data TEXT);'
Image: docker.io/mysql
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
d51f459239fb
0639788facc8
35369f9634e1
937bbdd4305a
my-test
c33a2b88395d
Copy-on-Write
(CoW)
416f2a97355f
Supported CoW file system & devices:
AUFS, Btrfs, Device Mapper, OverlayFS/Overlay2, ZFS, VFS
# Operation 1 - Update /etc/mysql/my.cnf
$ docker exec -it my-test vi /etc/mysql/my.cnf
1
2New
Branch
17. Docker Container - Application vs Data
Copyright 2012 Severalnines AB
Application Data
Stateless Stateful
Ephemeral Persistent
Independant Dependant
Frontend Backend
Web, API Database, File Storage
19. Docker Volume - Type
Copyright 2012 Severalnines AB
# Mount a named volume into container
-v mysql1-data:/var/lib/mysql
# Mount tmpfs into container
--tmpfs /tmp
● Bind mount:
○ A file or directory on the host machine is
mounted into a container.
○ You manage the directory’s contents.
● Named volume:
○ A new directory is created within Docker’s
storage directory.
○ Docker manages the directory’s contents.
○ Extensible via volume plugin.
● tmpfs:
○ Stored on host machine’s memory (or swap).
○ Linux only.
# Bind mount a directory into container
-v /data/mysql1/data:/var/lib/mysql
# Bind mount a file into container
-v /data/mysql1/my.cnf:/etc/mysql/my.cnf
20. Docker Volume - Persistent
Volumes
Copyright 2012 Severalnines AB
# Run with persistent named volume, using -v or --volume
$ docker run -d
--name mysql-local
--env MYSQL_ROOT_PASSWORD=mypassword
--volume local-datadir:/var/lib/mysql
mysql:5.7
Image: docker.io/mysql:5.7
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-local
416f2a92355f
/var/lib/mysql
local-datadir
# Run with persistent named volume, using --mount
$ docker run -d
--name mysql-local
--env MYSQL_ROOT_PASSWORD=mypassword
--mount source=local-datadir,target=/var/lib/mysql
mysql:5.7
21. Docker Volume - Changes on
Volume
Copyright 2012 Severalnines AB
# Run a container with port 3308
$ docker run -d
--name mysql-local
-p 3308:3306
-e MYSQL_ROOT_PASSWORD=mypassword
-v local-datadir:/var/lib/mysql
mysql
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-local
416f2a92355f
/var/lib/mysql
local-datadir
# Run with persistent named volume, using --mount
$ mysql -uroot -p -h127.0.0.1 -P3308 < dump.sql
dump
.sql
3308
22. Docker Volume - Remote Volume
Copyright 2012 Severalnines AB
# Run a container with NFS bind mount
$ docker run -d
--name mysql-nfs
-p 3308:3306
-e MYSQL_ROOT_PASSWORD=mypassword
-v /nfs/mysql-nfs:/var/lib/mysql
mysql
Image: docker.io/mysql
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-nfs
416f2a92355f
/var/lib/mysql
192.168.1.100
/storage/docker
# Mount NFS with extra options for MySQL
$ mount 192.168.1.100:/storage/docker /nfs -o
noatime,nodiratime,hard,intr
3308
/nfs/mysql-nfs
Further Reading - Using NFS with MySQL:
https://dev.mysql.com/doc/refman/5.7/en/disk-issues.htm
l#disk-issues-nfs
23. Docker Volume - MySQL Persistency
Copyright 2012 Severalnines AB
Description Variable Name Default Value (5.7)
MySQL data directory datadir /var/lib/mysql
MySQL plugin directory plugin_dir {basedir}/lib/plugin
MySQL configuration directory config_dir /etc/my.cnf.d
MySQL binary log log_bin {datadir}/{hostname}-bin
Slow query log slow_query_log_file {datadir}/{hostname}-slow.log
Error log log_error {datadir}/{hostname}.err
General log general_log_file {datadir}/{hostname}.log
24. Docker Volume - Non-Persistent
Volume
Copyright 2012 Severalnines AB
# Run a container with NFS bind mount and tmpfs volumes
$ docker run -d
--name mysql-nfs-tmp
-p 3308:3306
-e MYSQL_ROOT_PASSWORD=mypassword
-v /nfs/mysql-nfs:/var/lib/mysql
--tmpfs /tmp:rw,size=1g,mode=177
mysql
Image: docker.io/mysql:latest
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-nfs-tmp
416f2a92355f
/var/lib/mysql
192.168.1.100
/storage/docker
3308
/nfs/mysql-nfs
/tmp
25. Docker Volume - Drivers
Copyright 2012 Severalnines AB
● Default is local.
● You can use other different volume driver (or plugin).
○ Install the plugin.
○ Use --volume-driver option.
● Volume drivers allow you to extend Docker's volume
capabilities:
○ Snapshot
○ Backup
○ Encryption
# Install Rexray EBS docker plugin
$ docker plugin install rexray/ebs EBS_ACCESSKEY=XXXX
EBS_SECRETKEY=YYYY
# Verify if the plugin is loaded
$ docker info -f '{{json .Plugins.Volume}}' | jq
[
"local",
"rexray"
]
# Run a container with volume plugin
$ docker run -d
--name=mysql-ebs
--volume-driver=rexray/ebs
-v ebs-datadir:/var/lib/mysql
mysql:5.7
# List volumes
$ docker volume ls
DRIVER VOLUME NAME
local local-datadir1
local local-datadir2
rexray ebs-datadir
29. Docker Network - Types
Copyright 2012 Severalnines AB
● Single-host:
○ Host
○ None
○ Bridge
■ Default bridge (docker0)
■ User-defined bridge
● Multi-host:
○ Default overlay
○ User-defined overlay
● Extensible via Docker network plugins.
# List all networks
$ docker network ls
NAME DRIVER SCOPE
bridge bridge local
db_multi overlay swarm
db_single bridge local
docker_gwbridge bridge local
host host local
ingress overlay swarm
none null local
30. Docker Network - Host Network
Copyright 2012 Severalnines AB
# Run a container on host network
$ docker run -d
--name mysql-host
--net host
-e MYSQL_ROOT_PASSWORD=mypassword
mysql:5.7
Container 1
eth0
eth0
Container 2
eth0
Docker
Host
# Display active TCP connections
$ netstat -tulpn | grep mysql
tcp6 0 0 :::3306 :::* LISTEN 24601/mysqld
● Container’s network interfaces will be identical with
the machine host.
● Only one host network per machine host.
● Container linking, --link mysql-test:mysql is not
supported.
● Port mapping, --publish 3307:3306 is not
supported.
31. Docker Network - Default Bridge
Network
Copyright 2012 Severalnines AB
# Run a container on docker0 bridge network
$ docker run -d
--name mysql-bridge
-p 3308:3306
-e MYSQL_ROOT_PASSWORD=mypassword
mysql:5.7
Container 1
eth0
eth0
Container 2
eth0
Docker
Host
# Display active TCP connections
$ netstat -tulpn | grep 3308
tcp6 0 0 :::3308 :::* LISTEN 24601/docker-proxy
● Packet forwarding by iptables to the bridge network.
● --publish is supported. Run multiple containers with
same image through different ports.
● --link is supported. Link with other containers to
expose environment variables.
● docker-proxy redirects connection to the correct
container through NAT.
docker0
(virtual Ethernet bridge)
vethxxx vethyyy
32. Docker Network - User-Defined
Bridge Network
Copyright 2012 Severalnines AB
# Run a container on user bridge network
$ docker run -d
--name mysql1
--net db_single
-p 3308:3306
--ip 192.168.10.10
--hostname mysql1
-e MYSQL_ROOT_PASSWORD=mypassword
mysql:5.7
Container 1
eth0
eth0
Container 2
eth0
Docker
Host
# Create a new bridge network
$ docker network create subnet=192.168.10.0/24 db_single
Similar to the default docker0 bridge, plus:
● --hostname is supported for sticky hostname.
● --ip is supported for sticky IP address.
● --link is not supported. Use embedded DNS to
resolve container's name in the same network.
db_single
(virtual Ethernet bridge)
vethxxx vethyyy
DNS
33. Docker Network - Plugins
Copyright 2012 Severalnines AB
● Similar to volume, Docker network can be extended
through plugins:
○ Multi-host networking (must use outside of
Swarm)
○ Name resolver and service discovery.
○ Encryption.
● Popular network plugins:
○ Contiv
○ Calico
○ Weave
○ Flannel
Further Reading - Multi-host Networking with Calico:
https://severalnines.com/blog/mysql-docker-multi-host-networkin
g-mysql-containers-part-2-calico
34. Docker Network - Default Overlay
Network
Copyright 2012 Severalnines AB
# Run a service on default overlay network
$ docker service create
--name mysql-service
--replicas 1
-p 3308:3306
-e MYSQL_ROOT_PASSWORD=mypassword
-v nfs-datadir:/var/lib/mysql
mysql:5.7
Container 1
eth0
eth0 eth1
Docker
Host
# Initialize Docker Swarm
$ docker swarm init
● Docker Swarm must be enabled.
● Each task (container) has:
○ eth0 - Interface to external network via
docker_gwbridge.
○ eth1 - Interface to overlay network via VXLAN.
docker_gw
bridge
vethxxx vethyyy
br0
External
Container 2
eth0 eth1
Docker
docker_gw
bridge
vethxxx vethyyy
br0
eth0
Host
VXLAN
udp/4789
35. Docker Network - User-Defined
Overlay Network
Copyright 2012 Severalnines AB
# Run a service on default overlay network
$ docker service create
--name mysql-service
--replicas 1
--net db_multi
-p 3308:3306
-e MYSQL_ROOT_PASSWORD=mypassword
-v nfs-datadir:/var/lib/mysql
mysql:5.7
Container 1
eth0
eth0 eth1
Docker
Host
# Create an overlay network
$ docker network create -d overlay db_multi
Similar with the default overlay, plus:
● Embedded DNS resolver:
○ {service_name} → Service VIP
○ tasks.{service_name} → Container's IP
docker_gw
bridge
vethxxx vethyyy
br0
External
Container 2
eth0 eth1
Docker
docker_gw
bridge
vethxxx vethyyy
br0
eth0
Host
VXLAN
udp/4789
DNS
39. Docker Orchestration - Run with
Docker Compose
Copyright 2012 Severalnines AB
# Bring up the production MySQL container in background
$ docker-compose up -d
Creating network "compose_db-prod" with driver "bridge"
Creating compose_mysql-prod_1
Image: docker.io/mysql:5.7.18
de70fa77eb2b
f6016aab25f1
0639788facc8
35369f9634e1
937bbdd4305a
ad74af05f5a2
0639788facc8
35369f9634e1
937bbdd4305a
mysql-prod
416f2a92355f
/var/lib/mysql
mysql-datadir
/tmp
mysql-tmp
3307
# Stop the container
$ docker-compose stop
# Destroy everything
$ docker-compose down
# Start the container
$ docker-compose start
Scaling is supported via docker-compose scale, however
conflicting options must be omitted:
hostname, ports, ipv4_address, volume
40. Docker Orchestration - Docker Swarm
Copyright 2012 Severalnines AB
● Multi-host deployment, through overlay network.
● Concepts:
○ Node
■ Manager - Dispatches tasks
■ Worker - Executes tasks by manager
○ Service
■ Replicated - Number of replicas
■ Global - One task per available node
○ Task
■ Container
○ Mesh routing
■ All Swarm nodes route to the running tasks
■ Ingress load balancing (round-robin only)
○ Scheduler
■ Resources availability
■ Label and constraints
41. Docker Orchestration - Simple Auto
Failover using Swarm
Copyright 2012 Severalnines AB
# Run a replica of MySQL service
$ docker service create
--name mysql-service
--replicas 1
--net db_multi
-p 3307:3306
-e MYSQL_ROOT_PASSWORD=mypassword
--mount
type=volume,source=ebs-datadir,destination=/var/lib/mysql
,volume-driver=rexray
--mount
type=volume,source=ebs-conf,destination=/etc/my.cnf.d,vol
ume-driver=rexray
mysql:5.7
mysql-
service
Docker
Host 1
ebs-datadir
mysql-
service
Docker
Host 2
3307
ebs-conf
44. Docker Orchestration - Best Practice for
MySQL
Copyright 2012 Severalnines AB
● Add label to node and use placement constraints:
○ Avoid overcommit resources.
○ Manageability and predictability.
● Use Service's name and virtual IP address to distinguish
container's role.
● Understand the scheduling strategy:
○ Spread
○ Binpack
○ Random
○ Custom
● Rolling update:
○ Only perform update one container at a time.
○ Add delay before move to the next container.
46. MySQL Container - Service Control
Copyright 2012 Severalnines AB
# Get container's log
$ docker logs -f mysql-staging
...
[Note] mysqld: Shutdown complete
# List processes inside container
$ docker exec -it mysql-staging ps -e
PID TTY TIME CMD
1 ? 00:00:00 mysqld
62 pts/0 00:00:00 bash
86 pts/0 00:00:00 ps
● mysqld must hold PID 1 inside container to receive the
signal.
● docker stop might be dangerous. Default is 10s grace
period. If still running, SIGKILL.
● Use docker kill with proper signal (SIGTERM). No
grace period.
● Verify the shutdown process through docker log.
● Stopping won’t delete the container.
# Stop MySQL container using kill
$ docker kill --signal=TERM mysql-staging
# Stop MySQL container using stop
$ docker stop mysql-staging
47. MySQL Container - Resource Control
Copyright 2012 Severalnines AB
# Limit memory and open file descriptors
$ docker run -d
--name mysql-staging
--memory 4g
--memory-swap 4g
--ulimit nofile=16824:16824
mysql:5.7.6
● By default, no resource constraints, use as much of a
given resource as the host’s kernel will allow.
● Important options!
○ --memory & --memory-swap
■ Must be equal to disable swap.
■ If --memory-swap is not set, container
swap is default to --memory * 2
○ --ulimit nofile={soft:hard}
■ Maximum number of files MySQL server
can open simultaneously.
● Some of the container resources can be controlled
dynamically using docker update:
○ Tune my.cnf accordingly.
# Increase container's memory.
$ docker update
--memory 6g
--memory-swap 6g
mysql-staging
48. MySQL Container - Resource Monitoring
Copyright 2012 Severalnines AB
# Monitor containers
$ docker stats --format "table
{{.Name}}t{{.CPUPerc}}t{{.MemUsage}}"
NAME CPU % MEM USAGE / LIMIT
minio1 0.00% 6.078MiB / 7.78GiB
mysql2 0.12% 194.1MiB / 7.78GiB
mysql-local 0.14% 220MiB / 7.78GiB
● Use docker stats.
● Inside container, free and top commands are not
accurate:
○ Memory - /sys/fs/cgroup/memory/memory.*
○ CPU - /sys/fs/cgroup/cpu/cpu.*
○ IO - /sys/fs/cgroup/blkio/blkio.*
● External open-source monitoring tools:
○ CLI: sysdig, dockviz
○ UI: Portainer, Shipyard, Rancher, Prometheus
Portainer
49. MySQL Container - Configuration
Management
Copyright 2012 Severalnines AB
# Bind mount my.cnf
$ docker run -d
--name mysql1
-e MYSQL_ROOT_PASSWORD=mypassword
-v /storage/mysql1/my.cnf:/etc/mysql/my.cnf
mysql:5.7.6
● Most MySQL configuration parameters can be changed
during runtime. (Hint: SET GLOBAL)
● You can either:
○ Use persistent storage:
■ Bind mount your my.cnf file.
■ Create a named volume for /etc/my.cnf.d
○ Append the configuration as a "CMD" (right after
image name), passes as flags to mysqld inside
container.
● Standard MySQL and InnoDB optimization applies.
# Named volume for MySQL configuration directory
$ docker run -d
--name mysql2
-e MYSQL_ROOT_PASSWORD=mypassword
-v mysql2-conf:/etc/my.cnf.d
mysql:5.7.6
# Append configuration options
$ docker run -d
--name mysql3
-e MYSQL_ROOT_PASSWORD=mypassword
mysql:5.7.6
--innodb_buffer_pool_size=1G
--max_connections=100
50. MySQL Container - Security
Copyright 2012 Severalnines AB
# Create a secret
$ echo 'MyP2s$w0rD' | docker secret mysql_root_passwd -
● Docker Secrets
○ Only for Swarm service.
○ Accessible under /run/secrets/ inside
container.
● Runtime privileges:
○ Unprivileged is default and recommended.
Container is not allowed to access other
devices.
○ Use --privileged only if you really need to
modify kernel parameters (sysctl, /proc,/sys)
or you want to run container inside
container.
# Create a service and use the secret as root password
$ docker service create
--name mysql-container
--publish 4406:3306
--secret mysql_root_passwd
-e MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root_passwd
-v local-datadir:/var/lib/mysql
mysql:5.7
51. MySQL Container - Backup
Copyright 2012 Severalnines AB
# mysqldump and store inside physical host
$ docker exec -it mysql-prod mysqldump -uroot -p
--single-transaction > /path/in/physical/host/dump.sql
● Logical backup:
○ Mysqldump
○ mysqlpump
● Xtrabackup (hot-backup):
○ Install inside machine host.
○ Install inside the MySQL container.
○ Use perconalab/percona-xtrabackup image.
● Snapshot:
○ MyISAM only - FTWRL + copy over
○ Stop container + copy over
# FTWRL
$ docker exec -it mysql-prod /bin/bash
root@mysql-prod:# mysql -uroot -p
mysql> FLUSH TABLE WITH READ LOCK;
# Copy volume to another location (another terminal)
$ cp /var/lib/docker/volumes/mysql-datadir/_data
/destination/in/physical/host
# Release lock (same terminal as FTWRL)
mysql> UNLOCK TABLES;
# Use perconalab/percona-xtrabackup image to backup
$ docker run --rm -it
--net db-prod
-v mysql-datadir:/var/lib/mysql
-v /tmp/backup:/xtrabackup_backupfiles
perconalab/percona-xtrabackup
--backup --host=mysql-prod --user=root
--password=mypassword
52. MySQL Container - Restore
Copyright 2012 Severalnines AB
# Restore a MySQL dump
$ docker exec -it mysql-staging mysql -uroot -p <
/path/in/physical/host/dump.sql
● Logical backup:
○ Fairly straightforward. Use standard procedure.
● Xtrabackup (hot-backup):
○ Have to be prepared first
○ Once prepared, simply starts a new container by
mounting the prepared directory as datadir.
● Snapshot & Copy:
○ Bind mount:
i. Direct mount the directory as a volume.
○ Named volume:
i. Create a volume.
ii. Copy the content to the volume's
directory.
iii. Start a new container by mounting the
volume.
# Run a test container from a copied datadir
(snapshot) or from a prepared xtrabackup directory.
$ docker run -d
--name mysql-restored
-v /tmp/backup:/var/lib/mysql
mysql:5.7.16
# Prepare the backup created by xtrabackup
$ docker run --rm -it
-v mysql-datadir:/var/lib/mysql
-v /tmp/backup:/xtrabackup_backupfiles
perconalab/percona-xtrabackup
--prepare --target-dir /xtrabackup_backupfiles
53. MySQL Container - Logical Upgrade
Copyright 2012 Severalnines AB
1. Pull the new image, M.
2. Start new container, B with new image, M.
3. Export MySQL on A.
4. Stop A.
5. Import into B.
6. Run mysql_upgrade inside B.
7. OK? Remove A.
8. Fallback? Stop B, start A.
# 1
$ docker pull mysql:5.7.18
# 2
$ docker run -d
--name=mysql-new
-e MYSQL_ROOT_PASSWORD=mypassword
-v local-datadir:/var/lib/mysql
mysql:5.7.18
# 3
$ docker exec -it mysql-old mysqldump -uroot -p > dump.sql
# 4
$ docker kill --signal=TERM mysql-old
# 5
$ docker exec -it mysql-new mysql -uroot -p < dump.sql
# 6
$ docker exec -it mysql-new mysql_upgrade -uroot -p
# 7
$ docker rm -f mysqld-old
# 8
$ docker stop mysqld-new
$ docker start mysqld-old
54. MySQL Container - In-Place Upgrade
Copyright 2012 Severalnines AB
# 1
$ docker pull mysql:5.7.18
# 2
$ docker exec -it mysql-old mysql -uroot -p -e 'SET GLOBAL
innodb_fast_shutdown = 0'
# 3
$ docker kill --signal=TERM mysqld-old
# 4
$ docker run -d
--name mysql-new
--publish 3307:3306
-e MYSQL_ROOT_PASSWORD=mypassword
-v local-datadir:/var/lib/mysql
mysql:5.7.18
# 5
$ docker exec -it mysql-new mysql_upgrade -uroot -p
# 6
$ docker rm -f mysqld-old
# 7
$ docker stop mysqld-new
$ docker start mysqld-old
1. Pull the new image, M.
2. Set innodb_fast_shutdown=0 inside container.
3. Stop the container, A.
4. Start a new container, B with new image, M.
5. Run mysql_upgrade inside B.
6. OK? Remove A.
7. Fallback? Stop B, start A.
56. Copyright 2017 Severalnines AB
● Visit our booth for a free T-shirt and other goodies. While
stock lasts!
● Follow our MySQL on Docker blog series at
severalnines.com/blog
● Attend the MySQL on Docker webinar, Wednesday Sept
27th, 2017:
○ 09:00 BST / 10:00 CEST
○ 09:00 PST (US) / 12:00 EST (US)
● Check out ClusterControl Docker image.
What's Next