Multi-master, multi-region MySQL deployment in Amazon AWS
 

Like this? Share it with your network

Share

Multi-master, multi-region MySQL deployment in Amazon AWS

on

  • 2,145 views

MySQL data rules the cloud, but recent experience shows us that there's no substitute for maintaining copies of data, across availability zones and regions, when it comes to Amazon Web Services (AWS) ...

MySQL data rules the cloud, but recent experience shows us that there's no substitute for maintaining copies of data, across availability zones and regions, when it comes to Amazon Web Services (AWS) data resilience.

In this webinar, we discuss the multi-master capabilities of Continuent Tungsten to help you build and manage systems that spread data across multiple sites. We cover important topics such as setting up large scale topologies, handling failures, and how to handle data privacy issues like removing personally identifiable information or handling privacy law restrictions on data movement. We will conclude with a live demonstration of a distributed MySQL solution with Continuent Tungsten clusters working across multiple AWS availability zones and regions.

Statistics

Views

Total Views
2,145
Views on SlideShare
2,145
Embed Views
0

Actions

Likes
0
Downloads
14
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Multi-master, multi-region MySQL deployment in Amazon AWS Presentation Transcript

  • 1. Multi-Region, Multi-Master MySQL in the Amazon Cloud Robert Hodges, CEO Jeff Mace, Director of Services ©Continuent 2014
  • 2. Introducing Continuent • The leading provider of clustering and replication for open source DBMS • Our Product: Continuent Tungsten • Clustering - Commercial-grade HA, performance scaling and data management for MySQL • Replication - Flexible, high-performance data movement ©Continuent 2014 2
  • 3. Quick Continuent Facts • Largest Tungsten clustering installation processes up to 700M transactions daily • Over 100 customers including many household names • Multiple multi-site, multi-master deployments in production • ... And more are on the way ©Continuent 2014 3
  • 4. Tungsten Clustering in Two Minutes ©Continuent 2014 4
  • 5. Tungsten Database-as-a-Service GonzoPortal.com Tungsten clusters combine offthe-shelf open source DBMS servers into data services with: apache /php ! • 24x7 data access • Scaling of load on replicas • Simple management commands ! ...without app changes or data migration Amazon US West ©Continuent 2014 5 Connector Connector
  • 6. Cluster Detailed View Application Application Tungsten Connector Tungsten Connector Db3 Manager Manager Replicator Replicator Replicator Slave Master Data Service: sjc Slave 6 Monitoring and Control Monitoring and Control Db1 Manager ©Continuent 2014 Db2
  • 7. Tungsten Replication in Four Minutes ©Continuent 2014 7
  • 8. Tungsten Replicator Overview Master Download transactions via network DBMS Logs Slave Replicator (Transactions + Metadata) Replicator Apply using JDBC ©Continuent 2014 THL THL (Transactions + Metadata) 8
  • 9. Replication Pipelines Pipeline Stage Extract Filter Master DBMS ©Continuent 2014 Stage Apply Extract Filter Transaction History Log Stage Apply Extract Filter In-Memory Queue 9 Apply Slave DBMS
  • 10. Multiple Services Per Replicator NYC Replicator Service nyc Replicator Service nyc Frankfurt Service fra Replicator Service fra ©Continuent 2014 London 10
  • 11. Using Services for Multi-Master Replication New York Replicator Replicator lhr (slave) jfk (master) ©Continuent 2014 lhr (master) jfk (slave) 11 London
  • 12. Introducing Multi-Site Multi-Master Clustering ©Continuent 2014 12
  • 13. Why Multi-Site? Why Multi-Master? • • • Get data close to global applications • • Keep running if a site fails ©Continuent 2014 Avoid latency cost of commit between sites Keep running during intermittent network failures between sites Enable simple recovery when it comes back 13
  • 14. Multi-Site Multi-Master Clustering: the Marketecture Amazon Region Amazon Region Application Applications Application Applications No app or data coupling across regions Live data and apps in all regions Multi-AZ deployment ©Continuent 2014 Async replication Run in Amazon between sites or on-premise 14
  • 15. Detailed Topology US-West-1 US-East-1 master master east1 slave west1 east1 west slave Important Note: Continuent recommends 3 nodes for all clusters ©Continuent 2014 15
  • 16. Demo Setting up Multi-Master Clustering across Sites EU-West-1 US-East-1 ©Continuent 2014 US-West-1 16
  • 17. Failures in Multi-Master Topologies ©Continuent 2014 17
  • 18. Best Practice Deployment in Amazon Multi-AZ clustering with asynchronous multimaster across regions AZ US- West-1 AZ AZ AZ AZ EU- West-1 AZ US-East-1 Region ©Continuent 2014 AZ AZ AZ AZ Availability Zone 18
  • 19. Normal Multi-Master Configuration US-West-1 US-East-1 master master east1 slave ©Continuent 2014 west1 east1 west 19 slave
  • 20. Failed master US-West-1 X master master east1 master ©Continuent 2014 US-East-1 west1 east1 west 20 slave
  • 21. Failed Slave US-West-1 US-East-1 master master east1 X ©Continuent 2014 slave west1 east1 west 21 slave
  • 22. Failed Multi-Master Replicator US-West-1 US-East-1 east1 X east1 west master slave ©Continuent 2014 master west1 22 slave
  • 23. Full-Site Failure US-West-1 X US-East-1 master slave ©Continuent 2014 master east1 west1 east1 west 23 slave
  • 24. Enabling Data Replication Policies with Filters ©Continuent 2014 24
  • 25. What Are Replication Filters? Master Pipeline Stage Extract Filter Stage Apply Extract Filter Apply Inspect Change Drop Master DBMS ©Continuent 2014 Transaction History Log 25 In-Memory Queue
  • 26. Problem: Handling PII Data CREATE TABLE `personal_info` ( `id` int(11) NOT NULL AUTO_INCREMENT, `userid` varchar(250) DEFAULT NULL, `personal_code` varchar(250) DEFAULT NULL, `birth_date` date DEFAULT NULL, `email` varchar(250) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 Requirement 1: Do not replicate personally identifiable information ©Continuent 2014 26
  • 27. Using dropcolumns.js Built-In Filter Master Pipeline Stage Extract Filter(s) Stage Apply Extract Filter Apply colnames dropcolumns Master DBMS Transaction History Log dropcolumns.json ©Continuent 2014 27 In-Memory Queue
  • 28. Problem: Application-Specific Rules mysql> select * from personal_info; +----+--------+---------------+------------+-----------------+---------+ | id | userid | personal_code | birth_date | email | newdata | +----+--------+---------------+------------+-----------------+---------+ | 3 | fre11 | secret | 1985-01-16 | bobr@gmail.com | new | | 4 | bill | verysecret | 1995-01-16 | billk@gmail.com | newer | +----+--------+---------------+------------+-----------------+---------+ ! Requirement 2: prevent userid ‘bill’ from replicating ©Continuent 2014 28
  • 29. Adding Rules using Custom Filters • • Javascript vs. Java Filters Put Javascript file in: ${replicator-home}/samples/extensions/javascript • Add .tpl file for your filter to: ${replicator-home}/samples/conf/filters/default ©Continuent 2014 29
  • 30. Demo Adding filters to replication ©Continuent 2014 30
  • 31. Conclusion ©Continuent 2014 31
  • 32. How Do I Get Started? • Documentation for all products http://docs.continuent.com • Download registration for Tungsten clustering: http://www.continuent.com/downloads/software • Downloads for open source Tungsten Replicator http://code.google.com/p/tungsten-replicator/ • Vagrant project for setting up test topologies https://github.com/continuent/continuent-vagrant/tree/master/ examples/MSMM ©Continuent 2014 32
  • 33. 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: sales@continuent.com Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs Continuent Web Page: http://www.continuent.com ! Tungsten Replicator: http://code.google.com/p/tungsten-replicator ©Continuent 2014