2. About US
Karthik.P.R
• 4 + years of Industry experience as MySQL DBA.
• 1+ year at Yahoo! Administrating MySQL servers.
• Area of Focus MySQL HA and MySQL Sharding
• Student : M.Sc ( FOSS)
• Blogger : remotemysqldba.blogspot.in
3. PROGRAM AGENDA
• Scaling MySQL
• Available Sharding Tools
• MySQL Fabric
4. Scaling MySQL
• Scaling MySQL
• Available Sharding Tools
• MySQL Fabric
• Handling Shards
6. Scaling MySQL
• As reads and writes grows
Scale Vertically.
• Increase system resources
• Memory
• CPU
• Hard Disk
7. Scaling MySQL
Scaling reads using replication.
• Replication is the best solution.
• Replication is Async.
• Failover has to be set.
• Single writer
Write Queries Read Queries
8. Scaling MySQL
Scaling reads using Galera
• Read/ Write on any node
• Synchronous Replication
• A good HA solution.
• Easy to add nodes.
Write / Read
Queries
9. Scaling MySQL
Scaling Writes
• Replication can’t scale writes
• Partitioning is needed.
• Distributes the writes
• Horizontal scaling or Sharding.
ID 1 - 1000 ID 1001 - 2000
10. Scaling MySQL
Components .
•Shard Key
– Range
– Hash
– List
•Meta data store
•Managing data set
•High Availability of shards.
•Database and schema changes.
11. Scaling MySQL
Sharding Architecture
Proxy layer
State Store
Shard A
Shard B
Shard C
Application
12. Scaling MySQL
Advantages of Sharding
•Improved Performance
•Smaller Data set on local node.
•Handling large data set
•Scale well horizontally.
•Only small data set is affected on a node failure.
Disadvantages of Sharding
•Managing the shards
•Code change is need at some cases.
•Splitting Shards
•Maintaining the HA
13. Available Sharding Tools
• Sharding is used widely in Large Scale MySQL Deployments.
• Large organization built their own tools for sharding.
Popular MySQL Shards,
Facebook
• Twitter
• Tumblr
• Flickr
• Youtube
• Dropbox
14. Available Sharding Tools
Sharding Tools.
1) Saclebase ( Closed Source)
2) Jetpants (Tumblr )
3) Vitess ( Youtube )
4) MySQL Fabric ( Oracle MySQL )
16. MySQL FABRIC
• MySQL Fabric is the frame work to maintain shards and High Availability.
• Downloaded from MySQL Utilities.
• Good CLI commands.
• Works on MySQL > 5.6.10
• XML RPC for Python , php and Java.
• Minimized downtime of shard.
17. MySQL FABRIC ARCHITECTURE
Application XML-RPC
Connector
XML-RPC
FABRIC
NODE
Global Group
Group 1 Group 2 Group 3 Group 4
Back Store
18. MySQL FABRIC
FABRIC Components.
• Fabric node.
• Fabric aware connectors ( XML RPC )
• Server Groups.
– Global Group
– Shard Group
• Back Store
19. MySQL FABRIC
Fabric node
• Stores Fabric Config
• Shard Moving
• Shard splitting
• High Availability monitoring
Back Store
• Global tables info
• Shard key info and mapping
• Shard monitoring
Fabric Node Back store
20. MySQL FABRIC
Fabric aware connectors
• Supports Python/ php / Java Connectors
• Fetch Shard info
• Local cache
• Avoids the proxy hop.
Application XML-RPC
Connector
21. MySQL FABRIC
Server Groups
• Stores actual data
– Global Group
– Shard Group
• Primary – Read / Write
• Secondary – Reads ( failover)
• Spare ( On request )
Server Group
Primary Node Secondary Node
Spare Node
22. MySQL FABRIC
Server Groups ( Global Group )
• Stores Global Tables and schema
• Schema changes
• Replicates to Shard groups
Global Group Shard Group
Global Writes
Fabric Node
23. MySQL FABRIC ARCHITECTURE ( Again )
Application XML-RPC
Connector
XML-RPC
FABRIC
NODE
Global Group
Group 1 Group 2 Group 3 Group 4
Back Store
24. MySQL FABRIC
Advantages:
• Ease of Failover with GTID.
• Shard Split is easy
• RPC avoid network hops.
Dis Advantages
• Fabric node will be single point of failure
• No Cross Shard . ( Shard query can help here)
• Limited Connectors.
