Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scaling MySQL using Fabric

1,683 views

Published on

This is my presentation on MySQL Fabric in LSPE meet up ( Large Scale Production Engineering ) held at Yahoo! India on 13-09-2014.

Published in: Technology
  • Be the first to comment

Scaling MySQL using Fabric

  1. 1. Scaling Persistent Store Using MySQL FABRIC P.R.KARTHIK MySQL DBA
  2. 2. About US Karthik.P.R • 4 + years of Industry experience as MySQL DBA. • 1+ year at Yahoo! Administrating MySQL servers. • Area of Focus MySQL HA and MySQL Sharding • Student : M.Sc ( FOSS) • Blogger : remotemysqldba.blogspot.in
  3. 3. PROGRAM AGENDA • Scaling MySQL • Available Sharding Tools • MySQL Fabric
  4. 4. Scaling MySQL • Scaling MySQL • Available Sharding Tools • MySQL Fabric • Handling Shards
  5. 5. Scaling MySQL • Starts with a single node
  6. 6. Scaling MySQL • As reads and writes grows Scale Vertically. • Increase system resources • Memory • CPU • Hard Disk
  7. 7. Scaling MySQL Scaling reads using replication. • Replication is the best solution. • Replication is Async. • Failover has to be set. • Single writer Write Queries Read Queries
  8. 8. Scaling MySQL Scaling reads using Galera • Read/ Write on any node • Synchronous Replication • A good HA solution. • Easy to add nodes. Write / Read Queries
  9. 9. Scaling MySQL Scaling Writes • Replication can’t scale writes • Partitioning is needed. • Distributes the writes • Horizontal scaling or Sharding. ID 1 - 1000 ID 1001 - 2000
  10. 10. Scaling MySQL Components . •Shard Key – Range – Hash – List •Meta data store •Managing data set •High Availability of shards. •Database and schema changes.
  11. 11. Scaling MySQL Sharding Architecture Proxy layer State Store Shard A Shard B Shard C Application
  12. 12. Scaling MySQL Advantages of Sharding •Improved Performance •Smaller Data set on local node. •Handling large data set •Scale well horizontally. •Only small data set is affected on a node failure. Disadvantages of Sharding •Managing the shards •Code change is need at some cases. •Splitting Shards •Maintaining the HA
  13. 13. Available Sharding Tools • Sharding is used widely in Large Scale MySQL Deployments. • Large organization built their own tools for sharding. Popular MySQL Shards,  Facebook • Twitter • Tumblr • Flickr • Youtube • Dropbox
  14. 14. Available Sharding Tools Sharding Tools. 1) Saclebase ( Closed Source) 2) Jetpants (Tumblr ) 3) Vitess ( Youtube ) 4) MySQL Fabric ( Oracle MySQL )
  15. 15. MySQL FABRIC
  16. 16. MySQL FABRIC • MySQL Fabric is the frame work to maintain shards and High Availability. • Downloaded from MySQL Utilities. • Good CLI commands. • Works on MySQL > 5.6.10 • XML RPC for Python , php and Java. • Minimized downtime of shard.
  17. 17. MySQL FABRIC ARCHITECTURE Application XML-RPC Connector XML-RPC FABRIC NODE Global Group Group 1 Group 2 Group 3 Group 4 Back Store
  18. 18. MySQL FABRIC FABRIC Components. • Fabric node. • Fabric aware connectors ( XML RPC ) • Server Groups. – Global Group – Shard Group • Back Store
  19. 19. MySQL FABRIC Fabric node • Stores Fabric Config • Shard Moving • Shard splitting • High Availability monitoring Back Store • Global tables info • Shard key info and mapping • Shard monitoring Fabric Node Back store
  20. 20. MySQL FABRIC Fabric aware connectors • Supports Python/ php / Java Connectors • Fetch Shard info • Local cache • Avoids the proxy hop. Application XML-RPC Connector
  21. 21. MySQL FABRIC Server Groups • Stores actual data – Global Group – Shard Group • Primary – Read / Write • Secondary – Reads ( failover) • Spare ( On request ) Server Group Primary Node Secondary Node Spare Node
  22. 22. MySQL FABRIC Server Groups ( Global Group ) • Stores Global Tables and schema • Schema changes • Replicates to Shard groups Global Group Shard Group Global Writes Fabric Node
  23. 23. MySQL FABRIC ARCHITECTURE ( Again ) Application XML-RPC Connector XML-RPC FABRIC NODE Global Group Group 1 Group 2 Group 3 Group 4 Back Store
  24. 24. MySQL FABRIC Advantages: • Ease of Failover with GTID. • Shard Split is easy • RPC avoid network hops. Dis Advantages • Fabric node will be single point of failure • No Cross Shard . ( Shard query can help here) • Limited Connectors.
  25. 25. THANK YOU

×