Downloaded 67 times
![Dockerfile
FROM python:2.7
RUN apt-get update && apt-get -y install python-dev libssl-dev
WORKDIR /app
RUN pip install Flask MySQL-python
ADD . /app
EXPOSE 5000
CMD ["python", "app.py"]](https://image.slidesharecdn.com/346593457-scaling-mariadb-with-docker-webinar-170428145741/85/MariaDB-on-Docker-26-320.jpg)
![haproxy & web services
services:
haproxy:
image: dockercloud/haproxy
networks:
- front
- back
volumes:
- /var/run/docker.sock:/var/run/docker.sock
ports:
- 80:80
deploy:
placement:
constraints: [node.role == manager]
web:
image: alvinr/demo-webapp-vote:mariadb
environment:
SERVICE_PORTS: "5000"
VIRTUAL_HOST: "prod.myapp.com"
APP_MARIADB_HOST: "maxscale"
APP_USER: "app"
APP_PASSWORD_FILE: "/run/secrets/app_password"
APP_DATABASE: "test"
networks:
- back
deploy:
placement:
constraints: [node.role != manger]
secrets:
- app_password](https://image.slidesharecdn.com/346593457-scaling-mariadb-with-docker-webinar-170428145741/85/MariaDB-on-Docker-35-320.jpg)
![Container Placement
mariadb_cluster:
image: alvinr/mariadb-galera-swarm
...
labels:
com.mariadb.cluster: "myapp-prod-cluster"
...
deploy:
replicas: 1
placement:
constraints: [engine.labels.com.mariadb.cluster != myapp-prod-cluster]](https://image.slidesharecdn.com/346593457-scaling-mariadb-with-docker-webinar-170428145741/85/MariaDB-on-Docker-39-320.jpg)
![Restarting on Failure
maxscale:
image: alvinr/maxscale-swarm
...
labels:
com.mariadb.cluster: "myapp-maxscale"
networks:
- back
deploy:
replicas: 1
restart_policy:
condition: on-failure
delay: 10s
placement:
constraints: [engine.labels.com.mariadb.cluster != myapp-maxscale]
secrets:
- app_password](https://image.slidesharecdn.com/346593457-scaling-mariadb-with-docker-webinar-170428145741/85/MariaDB-on-Docker-40-320.jpg)
Uploaded byMariaDB plc
MariaDB on Docker
The document discusses the integration of MariaDB with Docker to enhance scalability and maintainability in database deployment. It highlights the importance of data redundancy, dynamic cluster formation, and self-healing in containerized environments. Additionally, it features the capabilities of MariaDB MaxScale for managing security and high availability, and provides development to production use cases leveraging Docker's orchestration tools.
More Related Content
MariaDB on Docker
- 1. MariaDB + Docker ScalingMariaDB
- 2. Conclusion Containers + Databases= Happy Developers Ephemeral Containers + Databases = DevOps headaches 4 Things you must use to evaluate • Data Redundancy • Dynamic Self Discovery & Cluster formation • Self Healing (as containers enter and leave) • Application Tier discovery of Database Cluster
- 3. Part One “Here’s anothernice mess you have got me into” – Laurel & Hardy circa 1929
- 4. Existing Deployment ModelsAre Broken Version control 1. Development 2. Test 3. Stage / Production Developer QA / QE Sysadmin
- 5. Architectures Are Complex& Static Challenges • Orchestration • Velocity • Maintainability • Durability • Consistency • Scalability • Cost ($) • (Hybrid) CloudEnterprise Environment Legacy Mainframe Operational Database Caching Layer Pricing / Inventory / Billing Real-time Decisioning Real-time Consumer facing Streaming Data Data Warehouse Data Lake RDBMS Transactional Systems
- 6.
- 7. What do Containersgive me? Encapsulation of Dependencies • O/S packages & Patches • Execution environment (e.g. Python 2.7) • Application Code & Dependencies Process Isolation • Isolate the process from anything else running Faster, Lightweight virtualization
- 8. Containers vs. VirtualMachines App 1 App 2 App 3 Bins/Libs Bins/Libs Bins/Libs Guest OS Guest OS Guest OS Hypervisor Host Operating System Infrastructure Docker Engine Operating System Infrastructure App 1 App 2 App 3 Bins/Libs Bins/Libs Bins/Libs
- 9. Deployment Simplicity Build ShipRun Open Standards Plumbing Platform Clustering Distribution Image spec Container run-time spec Runtime Trust
- 10. Dockerfile - Example FROMpython:2.7 ADD . /code WORKDIR /code RUN apt-get update && apt-get -y install python-dev libssl-dev RUN pip install Flask MySQL-python EXPOSE 5000 CMD python app.py
- 11. Open Container Initiative(OCI) – Polyglot Vendors Coalition of industry leaders join forces to eliminate fragmentation • Form a vendor-neutral, open source governance model under the Linux Foundation • Establish common standards for container format and runtime • Docker donated its container format, runtime and associated specifications • Appoint maintainers for the libcontainer project
- 12. Docker Toolchain inpictures Machine provisions Docker Engines Swarm clusters Docker Engines Compose orchestrates Container deployment Containers are run by Docker Engine Docker Machine Docker Compose Docker Swarm Docker Engine Container Containers encapsulates your code, dependencies…
- 13. But… Docker evolves:1.13 / 17.3 GA features Docker Engine in “Swarm” mode • Engine based clustering versus Container based clustering • Master based, RAFT for consensus Docker Stacks • Bundles of Services • “Sort of” compatible with docker-compose Docker Services • Image + Configuration • Replicable across the cluster • Scale Up & Down
- 14. Part Three Databases &Docker
- 15. Requirements Data Redundancy • Containersare Ephemeral – Need more than one copy of the data Dynamic Self Discovery & Cluster formation • Need to start and stop Containers when needed • Clusters needs to grow and shrink dynamically Self Healing • Loss of nodes must not be fatal to the cluster integrity • Addition of nodes must scale capacity Application Tier discovery of Database Cluster • Automatic discovery of nodes • Automatic routing of requests to the correct nodes
- 16.
- 17. MARIADB SERVER Enterprise-grade secure, highlyavailable and scalable relational database with a modern, extensible architecture MARIADB MAXSCALE MARIADB CLUSTER Next-generation database proxy that manages security, scalability and high availability in scale-out deployments Multi-Master, synchronous replication - improves availability and scales reads and writes MariaDB Portfolio
- 18. MariaDB MaxScale High Availability Ensureuptime with no single point of failure and minimize downtime during upgrade Data Streaming Stream transactional data to data lake for real-time analytics Scalability Manage your scaled-out infrastructure without changing application code Security Secure database firewall to prevent cyber attacks like SQL injection and DDoS MariaDB MaxScale is a next-generation database proxy that manages security, scalability and high availability in a scale out deployment.
- 19. MariaDB Cluster Multi-Master • Synchronousreplication Faster Failover • All nodes synchronized, therefore equal Scale reads and writes MariaDB MariaDB MariaDB Load Balancing and Failover Application / App Server
- 20. MaxScale + Galera UseCase Each application server uses only 1 connection MaxScale selects one node as “master” and the other nodes as “slaves” If the “master” node fails, a new one can be elected immediately Galera Cluster + R/W split routing Max Scale
- 21.
- 22. Demo: Development ThroughProduction Development • Build & Run an App in Development – Python + MariaDB Production • Deploy to a Swarm cluster in Production • Scale Web nodes – Add more Web containers behind HAProxy • Database High Availability – Deploy 3 nodes Galera cluster – Deploy 2 node MaxScale
- 23. Python / Flask Let’sBuild An App! Development app
- 24. Production Virtual IP Virtual IP MaxScale 1 MaxScale2 HAProxy 1 2 3 Then Scale in Production... Development app app1 app2 app3 app4 appN
- 25. Demo 1 Build anApplication
- 26. Dockerfile FROM python:2.7 RUN apt-getupdate && apt-get -y install python-dev libssl-dev WORKDIR /app RUN pip install Flask MySQL-python ADD . /app EXPOSE 5000 CMD ["python", "app.py"]
- 27. docker-compose.yml services: web: build: . ports: - "5000:5000" links: -mariadb hostname: dev.myapp.com environment: APP_MARIADB_HOST: dev_mariadb_1 APP_PASSWORD: foo mariadb: image: mariadb:10.1 environment: MYSQL_ROOT_PASSWORD: foo
- 28. Roll The ApplicationBehind Haproxy Development app Production Virtual IP Virtual IP MaxScale 1 HAProxy 1 app1
- 29. Scale the ApplicationTier Development app Production Virtual IP Virtual IP MaxScale 1 MaxScale 2 HAProxy 1 2 3 app1 app2 app3 app4 appN
- 30. Docker Networking Docker Host(swarm-2) MaxScale Container Endpoint Docker Host (swarm-3) MariaDB Container Endpoint “front” Network “back” Overlay Network Docker Host (swarm-1) App Container Endpoint Docker Host (swarm-0) HAProxy Container Endpoint Endpoint
- 31. $ docker networkcreate -d overlay --attachable --opt encrypted myapp_back $ cat docker-compose.stack.yml ... networks: front: back: external: name: myapp_back Docker Networking
- 32. Demo 2 Scale WebTier
- 33. Secrets… Opppsss services: web: build: . ports: -"5000:5000" links: - mariadb hostname: dev.myapp.com environment: APP_MARIADB_HOST: dev_mariadb_1 APP_PASSWORD: foo mariadb: image: mariadb:10.1 environment: MYSQL_ROOT_PASSWORD: foo
- 34. Docker secrets $ more./app_password.txt appfoo $ cat docker-compose.stack.yml ... secrets: app_password: file: ./app_password.txt mariadb_root_password: file: ./mariadb_password.txt xtrabackup_password: file: ./xtrabackup_password.txt
- 35. haproxy & webservices services: haproxy: image: dockercloud/haproxy networks: - front - back volumes: - /var/run/docker.sock:/var/run/docker.sock ports: - 80:80 deploy: placement: constraints: [node.role == manager] web: image: alvinr/demo-webapp-vote:mariadb environment: SERVICE_PORTS: "5000" VIRTUAL_HOST: "prod.myapp.com" APP_MARIADB_HOST: "maxscale" APP_USER: "app" APP_PASSWORD_FILE: "/run/secrets/app_password" APP_DATABASE: "test" networks: - back deploy: placement: constraints: [node.role != manger] secrets: - app_password
- 36.
- 37. Container Placement Docker Host(swarm-2) MaxScale Container Endpoint Docker Host (swarm-3) MariaDB Container Endpoint “front” Network “back” Overlay Network Docker Host (swarm-1) App Container Endpoint Docker Host (swarm-0) HAProxy Container Endpoint Endpoint MariaDB Container Endpoint
- 38. Container Placement Docker Host(swarm-2) MaxScale Container Endpoint Docker Host (swarm-3) MariaDB Container Endpoint “front” Network “back” Overlay Network Docker Host (swarm-1) App Container Endpoint Docker Host (swarm-0) HAProxy Container Endpoint Endpoint MariaDB Container Endpoint
- 39. Container Placement mariadb_cluster: image: alvinr/mariadb-galera-swarm ... labels: com.mariadb.cluster:"myapp-prod-cluster" ... deploy: replicas: 1 placement: constraints: [engine.labels.com.mariadb.cluster != myapp-prod-cluster]
- 40. Restarting on Failure maxscale: image:alvinr/maxscale-swarm ... labels: com.mariadb.cluster: "myapp-maxscale" networks: - back deploy: replicas: 1 restart_policy: condition: on-failure delay: 10s placement: constraints: [engine.labels.com.mariadb.cluster != myapp-maxscale] secrets: - app_password
- 41.
- 42. DNS RESOLUTION • Dockerassigns VIP to Service, each Task has own IP • nslookup, dig, getent etc. 3rd PARTY • consul, etcd, zookeeper etc. DOCKER EVENTS • https://docs.docker.com/engine/ reference/api/docker_remote_api/ • Interlock - ttps://github.com/ehazlett/interlock Service Discovery - How to mesh nodes?
- 43. Storage: Inside orOutside the Container? Inside • Encapsulation of Concerns Outside • Separation of Concerns • Storage features (e.g. Snapshots) • 3rd Party options – NetApp, Google Compute Engine, Rancher Convoy – Flocker Host Docker Daemon Container Docker Daemon Container /dev/xvdb /mnt/xx:/var/lib/mysql Networked e.g. EBS Volume Local Disk e.g. SSD / NVMe
- 44. Storage: Data Container? Inside •Managed like other containers • Special rule for Destruction • TBD: Performance Host Docker Daemon Container Docker Daemon Container --volumes-from {container name} Host
- 45. And... • Swarm locking •Image verification (trusted Images) • AppArmor / Seccomp profiles • Monitoring • Heathchecks • Rolling Upgrades
- 46. Summary One Solution Development-> Production • Define Images & Orchestration once • Reuse when needed, inject required behaviours MariaDB in Production with Docker • Ops define the whitelisted images, security policies • Dev approve images to build upon • Eliminate complexity (and cost) of Deployment • Scale easily, maintain SLA requirements of component
- 47. Thanks and Q&A Code •https://github.com/alvinr/docker-demo/tree/master/mariadb/vote Docker Images • https://hub.docker.com/_/mariadb/ MariaDB & Docker deployment guide • https://mariadb.com/kb/en/mariadb/installing-and-using-mariadb-via-docker/ Contact me! • alvin@mariadb.com • @jonnyeight