MariaDB Roadshow Presentation 2018

1. Running MariaDB on
Containers &
Kubernetes
Modern Database + Modern
Infrastructure Management

2. Agenda • The Promise of Containers
• Orchestration and Management
– Kubernetes
• Databases, MariaDB & Containers
• Q&A

3. The Promise of Containers
Consistency and Control

4. Existing Deployment Models Are Broken
Version
control
1. Development 2. Test 3. Stage / Production
Developer QA / QE Sysadmin

5. Architectures Are Complex
Challenges
• Orchestration
• Complexity
• Maintainability
• Durability
• Consistency
• Scalability
• Cost ($)
• (Hybrid) CloudEnterprise Environment
Legacy Mainframe
Operational Database
Caching Layer
Pricing /
Inventory /
Billing
Real-time
Decisioning
Real-time
Consumer
facing
Streaming
Data
Data Warehouse
Data Lake
RDBMS Transactional
Systems

6. What do Containers give me?
Encapsulation of Dependencies
• O/S packages & Patches
• Execution environment (e.g. Python 2.7)
• Application Code & Dependencies
Process Isolation
• Isolate the process from anything else running
Faster, Lightweight virtualization

7. Virtual Machines vs. Containers
App 1 App 2 App 3
Bins/Libs Bins/Libs Bins/Libs
Guest OS Guest OS Guest OS
Hypervisor
Host Operating System
Infrastructure
Docker Engine
Operating System
Infrastructure
App 1 App 2 App 3
Bins/Libs Bins/Libs Bins/Libs

8. Dockerfile - Example
FROM centos:7
...
COPY cnf/ /etc/my.cnf.d/
...
RUN mkdir -p /var/lib/mariadb-socket /var/lib/mariadb-load-data /var/lib/mysql/mysql /home/mariadb
&& yum -y update
&& yum -y install hostname
...
&& usermod -d /home/mariadb mysql
...
USER mysql
WORKDIR /home/mariadb
VOLUME /var/lib/mysql /var/lib/mariadb-socket /var/lib/mariadb-load-data
EXPOSE 3306
...
ENTRYPOINT ["mariadb-server-entry"]

9. Orchestration and Management
Coordination of Resources

10. What about orchestration and Management?
Orchestration and Management of Containers and higher-level
constructs (services, deployments, etc….) is evolving
Amazon ECS
Google Container
Engine
Azure Container
Service

11. docker
docker
docker
node
node
node
kubernetes
master(s)
kubectl
REST
Kubernetes Components
kubernetes
interfaces
volumes
containers
volumes
containers
volumes
containers
Pod

12. mysql:
3306
mysql:
3306
port:
3306
Kubernetes Objects
pods
services
spec:
controllers
mysql:
3306
spec:
deployments
provide access to
manage
manage

13. Databases, MariaDB & Containers
Managing Persistent Distributed Systems

14. Direction
Containers with Persisted Storage +
Databases = Happy Developers
Ephemeral Containers + Databases =
DevOps headaches
4 Things you must use to evaluate
• Data Redundancy
• Dynamic Self Discovery & Cluster formation
• Self Healing (as containers enter and leave)
• Application Tier discovery of Database Cluster

15. Headless Service: clust
Pod-0:
Master
MariaDB Clusters in Kubernetes
StatefulSet:
MariaDB Server instance
replicas: 3
StatefulSet:
MaxScale instance
replicas: 1
Service:
(maxs)
4006 (W)
4008 (R)
Pod-1:
Slave
Client
Application
Pod-0:
MaxScale
jdbc:mariadb://
maxs:4006/DB maxscaled
prometheus
fluentd
mysqld
prometheus
fluentd
mysqld
prometheus
fluentd
maxs-0.clust mariadb-0.clust mariadb-1.clust
PersistentVolumes
mariadb-vol-0 mariadb-vol-1
Pod-2:
Slave
mysqld
prometheus
fluentd

16. Stateful Sets

17. Master is ready to accept
connections!
Logs

18. Services

19. Persistent
Volumes

20. Why Use StatefulSets?
Stability
• Stable DNS of Pods
• Persistent volumes
• Defined order of instantiation
Scaling, Self-healing
• Failed pods are replaced automagically
• The number of pods is governed by a single parameter
Identical Configuration on Pods
• Main service (e.g. mysqld)
• Supporting services (e.g. prometheus export for monitoring, fluentd for logging)

21. Challenges for DB Containers
StatefulSet
• Identical pods: additional scripting required to instantiate different roles
• Complete enterprise cluster requires more StatefulSets (MaxScale+Master/Slave)
Persistence
• PersistentVolumes add new layers of abstraction (and latency)
• e.g. GlusterFS: volume on top of a DFS on top of a FS on top of a block device
• Provisioning of IOPS or throughput is vendor-specific
Resource Management
• DB instances contend with other apps for CPU, RAM, network and I/O resources
• CPU and memory requests/limits available but do not provide guarantees
• Throttling network and IOPS not available yet
• Container restarted when over memory limits (DB engine configuraion needed to
ensure memory limits are not exceeded)

22. Direction
YAML Templates Library
• Capturing Main MariaDB Topologies
• Based on standard Docker Hub images
• Development/Evaluation Environments
• Simple Scalability using Kubectl
• Easy to Customize
Kubernetes Operator
• Capturing Best-Practice Enterprise Topologies
• Simple Declarative Definition of DB in App Definition
• Advanced Operations (Backup/Recovery/Load Testing)
• Advanced Scenarios (Disaster Recovery, Auto scaling)

23. MariaDB Corp
Docker Hub
https://hub.docker.com/u/mariadb/

24. Other MariaDB Docker Resources
• Community Single Instance Docker Image
github.com/docker-library/mariadb
• Bitnami curated Master->Slave
github.com/bitnami/bitnami-docker-mariadb
• Galera Cluster (with Docker Swarm Support)
github.com/colinmollenhour/mariadb-galera-swarm
• Sample App using Galera on Docker
mariadb.com/resources/webinars/how-build-and-scale-application-mariadb-docker
• Columnstore Single Instance Docker for Dev & Evaluations
github.com/mariadb-corporation/mariadb-columnstore-docker

25. Q&A

26. Thank you
james.mclaurin@mariadb.com.

27. Appendix
Backup Slides and More

28. Open Container Initiative (OCI) – Polyglot Vendors
Coalition of industry leaders join forces to eliminate fragmentation
• Form a vendor-neutral, open source governance model under the Linux Foundation
• Establish common standards for container format and runtime
• Docker donated its container format, runtime and associated specifications
• Appoint maintainers for the libcontainer project

32. Docker Toolchain
Machine provisions
Docker Engines
Swarm clusters
Docker Engines
Compose orchestrates
Container deployment
Containers instantiate an
image and are run by
Docker Engine
Docker Machine Docker Compose
Docker Swarm
Docker Engine
Image
Images encapsulates your
code, dependencies…

33. MariaDB AX / ColumnStore
• Multi-node on Kubernetes
–mariadb.com/resources/webinars/get-started-analytics-easy-way-
mariadb-ax-kubernetes