Mysql User Camp : 20-June-14 : Mysql Fabric

5,610 views

Published on

This pres

1 Comment
0 Likes
Statistics
Notes
  • I create 3 groups, global-group, group_id-1 and group_id-2, then built a shard. Till now, everything is OK. But when i split the shard with a new group named group_id-3, there is an error occurred. The message is as follows:
    # mysqlfabric sharding split_shard 2 group_id-3 –split_value=100000
    Password for admin:
    Procedure :
    { uuid = a0eaff93-f6fe-4a7d-9c44-e7e53882fbfe,
    finished = True,
    success = False,
    return = BackupError: (‘Error while restoring the backup using the mysql client\n, %s’, “ERROR 1840 (HY000) at line 24 in file: ‘MySQL_132.228.239.19_3316.sql’: @@GLOBAL.GTID_PURGED can only be set when @@GLOBAL.GTID_EXECUTED is empty.\n”),
    activities =
    }
    I run the “reset master” command first, but the error is still there.what is this error?

    Regards
    Shen
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

No Downloads
Views
Total views
5,610
On SlideShare
0
From Embeds
0
Number of Embeds
3,226
Actions
Shares
0
Downloads
21
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

Mysql User Camp : 20-June-14 : Mysql Fabric

  1. 1. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.1 MySQL Fabric
  2. 2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |2 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Safe Harbor Statement
  3. 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3 Program Agenda  Handling Scaling  What is sharding?  Managing a Sharded Database  Working with a Sharded Database
  4. 4. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |4 What is this shardin g? What are the benefits of sharding ? How do I shard my database ?
  5. 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |5 It's all about scaling... ● Start with a single server The Growing Enterprise
  6. 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |6 It's all about scaling... ● Start with a single server ● More and more page requests ● ...more and more reads ● What to do? ● Scale out! The Growing Enterprise
  7. 7. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |7 It's all about scaling... ● Start with a single server ● More and more page requests ● ...more and more reads ● What to do? ● Scale out! ● Replicate to read servers The Growing Enterprise
  8. 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |8 It's all about scaling... ● Start with a single server ● More and more page requests ● ...more and more reads ● What to do? ● Scale out! ● Replicate to read servers ● Perform a read-write split ● Writes go to master ● Reads go to read servers The Growing Enterprise
  9. 9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |9 It's all about scaling... ● More and more updates ● Write load increases ● What now? The Growing Enterprise
  10. 10. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |10 It's all about scaling... ● More and more updates ● Write load increases ● What now? ● Add another master? The Growing Enterprise
  11. 11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |11 It's all about scaling... ● More and more updates ● Write load increases ● What now? ● Add another master? ● Doesn't work... ● Write load is replicated The Growing Enterprise
  12. 12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |12 It's all about scaling... ● More and more updates ● Write load increases ● What now? ● Add another master? ● Doesn't work... ● Write load is replicated ● Partition database ● Distribute writes ● Called sharding The Growing Enterprise UID 10000-20000 UID 20001-40000
  13. 13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |13 Benefits of Sharding ● Write scalability ● Can handle more writes ● Large data set ● Database too large ● Does not fit on single server ● Improved performance ● Smaller index size ● Smaller working set ● Improve performance UID 10000-20000 UID 20001-40000
  14. 14. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |14 High-level Architecture ● What components do we need? ● How are they deployed?
  15. 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |15 Components for a Sharding Solution Shard #2 Shard #1 Shard #3 Switch State Store Executor QUERY KEY KEY QUERY Contain decision logic for distributing queries Contain information about location of shards SHARD#
  16. 16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |16 Transaction and Shard Key Handling ● How are the transactions handled? ● How do get the shard key for a transaction? ● How to compute shard from key?
  17. 17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |17 BEGIN SELECT salary INTO @s FROM salaries WHERE emp_no = 20101; SET @s = 1.1 * @s; INSERT INTO salaries VALUES (20101, @s); COMMIT BEGIN INSERT INTO ...  COMMIT Sharding key? Ah, there it is! Session state? Hmm... looks like a read transaction Oops.. it was a write transaction! Transaction done! Clear session state? New transaction! Different shard? What about the session state? Transaction Handling
  18. 18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |18 Transaction Handling ● Detecting transaction boundaries ● Managing session state ● Move session state between servers – Easy to use – Expensive and error prone ● Reset state after each transaction – Transactions start with default session state What about crashes? Where do I store the session state?
  19. 19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |19 Mapping the Sharding Key ● What is a sharding key? ● Single column ● Multi column – Same table? – Different tables? ● How is the key transformed? ● Hash ● Range ● User-defined Compute Shard# KeyShard# (X) (X,Y,...) RANGE HASH Something else
  20. 20. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |20 Granularity of Sharding ● Can multiple tables be sharded with the same key? ● Can we shard different tables different ways? ● Do we allow global tables? ● Do we allow cross-database queries?
  21. 21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |21 Sharded Tables Table Rows salaries 284 404 700 titles 44 330 800 employees 30 002 400 dept_emp 33 160 300 dept_manager 2 400 departments 900 In desperate need of sharding! Foreign keys
  22. 22. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |22 Multi-table Query with Sharded Tables SELECT first_name, last_name, salary FROM salaries JOIN employees USING (emp_no) WHERE emp_no = 21012 AND CURRENT_DATE BETWEEN from_date AND to_date; ● Referential Integrity Constraint ● Example query joining salaries and employees ● Same key, same shard: co-locate rows for same user ● JOIN normally based on equality ● Using non-equality defeats purpose of foreign key
  23. 23. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |23 Global Tables Table Rows salaries 284 404 700 titles 44 330 800 employees 30 002 400 dept_emp 33 160 300 dept_manager 2 400 departments 900 Do not really need to be sharded
  24. 24. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |24 Multi-table Query with Global Tables SELECT first_name, last_name, GROUP_CONCAT(dept_name) FROM employees JOIN dept_emp USING (emp_no) JOIN departments USING (dept_no) WHERE emp_no = 21012 GROUP BY emp_no; ● JOIN with departments table ● Has no employee number, hence no sharding key ● Table need to be present on all shards ● How do we update global tables?
  25. 25. 25 Insert Picture Here Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 | An extensible and easy-to-use framework for managing a farm of MySQL servers supporting high- availability and sharding MySQL Fabric
  26. 26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |26 MySQL Fabric Architecture High Availability Groups Application XML-RPC SQL SQL Connector Connector Connector Operator MySQL Fabric Node Application Servers
  27. 27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |27 High-Level Components ● Fabric-aware Connectors ● Python, PHP, and Java ● Enhanced Connector API ● MySQL Fabric Node ● Manage information about farm ● Provide status information ● Execute procedures ● MySQL Servers ● Organized in High-Availability Groups ● Handling application data High Availability Group Application Connector Connector Connector MySQL Fabric Node
  28. 28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |28 MySQL Fabric Node Architecture MySQL MySQL Fabric Framework Executor State Store (Persister) Sh HA MySQLAMQP XML-RPC ?? Connector Connector Connector Protocols Extensions Backing Store
  29. 29. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |29 MySQL Fabric: Prerequisites ● MySQL Servers (version 5.6.10 or later) ● Backing store database server ● Application datatabase servers ● Python 2.6 or 2.7 ● No support for 3.x yet ● MySQL Utilities 1.4 ● Available at https://dev.mysql.com/downloads/tools/utilities ● “Development release” tab
  30. 30. 30 Insert Picture Here Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 | Architecture for High-Availability
  31. 31. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |31 High-Availability Group Concept ● Group of servers ● Hardware redundancy ● Data redundancy ● Generic Concept ● Implementation-independent ● Self-managed or externally managed ● Different Types ● Primary-Backup (Master-Slave) ● Shared or Replicated Storage ● MySQL Cluster Done! Examples Only Not Yet Implemented
  32. 32. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |32 High-Availability Group Concept ● Abstract Concept ● Set of servers ● Server attributes ● Connector Attributes ● Connection information ● Mode: read-only, read- write, ... ● Weight: distribute load ● Management Attributes ● Status: state/role of the server Status: Primary Mode: Read-Write Host: server-1.example.com
  33. 33. 33 Insert Picture Here Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 | Architecture for Sharding
  34. 34. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |34 Sharding Architecture Shards MySQL Fabric Node Application Connector Connector Connector Global Group Global Updates Shard Updates Replication Support global update for off-line shards
  35. 35. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |35 MySQL Fabric: Sharding Goals & Features ● Connector API Extensions ● Support Transactions ● Support full SQL ● Decision logic in connector ● Reducing network load ● Shard Multiple Tables ● Using same key ● Global Updates ● Global tables ● Schema updates ● Sharding Functions ● Range ● (Consistent) Hash ● Shard Operations ● Using built-in executor ● Shard move ● Shard split
  36. 36. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.| Percona Live | April 3, 2014 |36 Thank you!

×