1.
Disaster Recovery
Planning for MySQL &
MariaDB
Bart Oles
Severalnines

2.
Copyright 2017 Severalnines AB

3.
Copyright 2017 Severalnines AB
Free to download
Initial 30 days Enterprise trial
Converts into free Community Edition
Enterprise / paid versions available

4.
Copyright 2017 Severalnines AB
Automation & Management
Deployment (Free Community)
● Deploy a Cluster in Minutes
○ On-Prem
○ Cloud (AWS/Azure/Google) - paid
Monitoring (Free Community)
● Systems View with 1 sec Resolution
● DB / OS stats & Performance Advisors
● Configurable Dashboards
● Query Analyzer
● Real-time / historical
Management (Paid Features)
● Backup Management
● Upgrades & Patching
● Security & Compliance
● Operational Reports
● Automatic Recovery & Repair
● Performance Management
● Automatic Performance Advisors

5.
Copyright 2017 Severalnines AB
Supported Databases

6.
Copyright 2017 Severalnines AB
Our Customers

7.
Agenda
- How it is implemented?
- What is encrypted:
- Tablespaces?
- General tablespace?
- Double write buffer/parallel double write buffer?
- Temporary tablespaces? (KEY BLOCKS)
- Binlogs?
- Slow/general/error logs?
- MyISAM? MyRocks? X?
- Performance overhead.
- Backups?
- Transportable tablespaces. Transfer key.
- Plugins
- Keyrings in general
- Key rotation?
- General-Purpose Keyring Key-Management
Functions
- Keyring_file
- Is useful? How to make it profitable?
- Keyring Vault
- How does it work?
- How to make a transition from keyring_file

8.
Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Business Considerations for Disaster Recovery

9.
What is Disaster Recovery?
● Failures
○ Operational (power, network, IT systems)
○ Natural (hurricane, flood, fire, earthquake)
○ Human caused (operator error, malicious
activity, terrorism)
● Drivers
○ How fast can we get up and running
○ What data have we lost
○ How can we reduce risk
Policies, tools & procedures that ensure your data is secure
and protected in case of an outage or serious catastrophe

10.
Uptime Guarantees - Why Compromise?

11.
The Small Print

12.
“We Offer 100% Availability, But We Exclude… “
● Planned outages
○ e.g., server or network maintenance
● Failure of network, power or facilities
delivered by an upstream provider
● DOS attacks, hacker activity or other
malicious events
● Acts of God
○ e.g., weather related - hurricane, flood

13.
Low Downtime Comes at a Cost

14.
Analyzing Risk

15.
Assessing Business Impact

16.
Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Defining Disaster Recovery

17.
Stockholm to Oslo Train Breaks Down After One Hour
x

18.
Outage Timeline

19.
Recovery Time Objective (RTO)

20.
Recovery Point Objective (RPO)

21.
RTO + RPO = 0 ?
Database
Replication
Storage
Clustering
Data
Integrity
Security
Load
Balancing
Network
Bonding
File
Replication
DB
Clustering

22.
“Everything Fails,
All the Time”
Werner Vogels

23.
Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Disaster Recovery Tiers

24.
Cost of Disaster Recovery

25.
Matching Disaster Recovery Plans to the Business

26.
Backup with No Hot Site
● Physical vs Logical backup
○ High impact on RTO
● Combine Full & Incremental
○ PITR-compatible to reduce RPO
● Schrödinger’s backup
○ “The condition of any backup is
unknown until a restore is attempted”
● Encryption
● Keep a copy of latest backup in active site

27.
Backup Retention
● Local Server
○ Up to 1 week
● Local Datacenter
○ Up to 2 weeks
● Remote Datacenter
○ Up to 4 weeks
○ Plus keep monthly backups &
annual backups as required

28.
Backup with Hot Site
● We can reinstall DBs and apps
from scratch and restore data
● Recovery time predictable
● In case of AWS, pre-configured
AMIs can be used to quickly
provision the application
environment

29.
Asynchronous Replication to Hot Site
● Low RTO
○ ‘Almost current’ data
enables fast failover
● Low RPO
● Add a delayed slave to guard
against operator error
● Backup still important

30.
Synchronous Replication to Hot Site
● Highest tier of DR
○ Minimal RPO and RTO
● Data on primary site and hot
sites have same transactional
state
○ Failover instantaneous
and automatic
● Failure detection time is main
culprit that adds to RTO
● 3 sites to avoid network
partitioning

31.
Synchronous Replication to Hot Site - Database Proxy

32.
Copyright 2017 Severalnines AB
Copyright 2018 Severalnines AB
Concluding Remarks

33.
Reality Check
Disaster Recovery Planning & Testing
Source: https://www.zetta.net/resource/state-disaster-recovery-2016

34.
Source:https://uptimeinstitute.com/about-
ui/press-releases/uptime-institute-annual-
survey-results-enterprise-owned-data-
centers-still-primary-compute-venue
Geographically Distributed Datacenters on the increase
Uptime Institute Data Center Industry Survey (2017)

35.
Failover the New Normal
● Failover used to be a complex
procedure
○ Required lot of staff
○ Required availability of VPs /
technology heads
● In modern distributed infrastructure,
design for failure
● Considerations
○ How many sites?
○ How to route users to sites?
○ What goes into a failover?

36.
Source: https://severalnines.com/blog/database-tco-calculating-total-cost-ownership-mysql-management
Calculating Database TCO - Colocation

37.
Source: https://severalnines.com/blog/database-tco-calculating-total-cost-ownership-mysql-management
Calculating Database TCO - Cloud

38.
Additional Resources
Free Disaster
Recovery Whitepaper
severalnines.com/res
ources/whitepapers

39.
Additional Resources
Free Database
Backup Whitepaper
severalnines.com/res
ources/whitepapers

40.
Additional Resources
● Calculating Database TCO
○ https://severalnines.com/blog/database-tco-calculating-total-cost-
ownership-mysql-management
● Multi-DC setups for MySQL & MariaDB
○ https://severalnines.com/blog/multiple-data-center-setups-using-galera-
cluster-mysql-or-mariadb
● Contact us: info@severalnines.com

41.
43
Rate My Session