Your SlideShare is downloading. ×
Database2011 MySQL Sharding
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Database2011 MySQL Sharding

4,915
views

Published on

How MySQL become an Internet standard? based on a presentation I gave at the Database2011 conference: Who uses MySQL? (everyone inc. Facebook, Twitter and Google), What are MySQL limitations? (multi …

How MySQL become an Internet standard? based on a presentation I gave at the Database2011 conference: Who uses MySQL? (everyone inc. Facebook, Twitter and Google), What are MySQL limitations? (multi threading, INSERTS, SELECTS), How the big boys solved these issues? (Sharding, Partitioning), What are the pros and cons for each Sharding strategy? (Vertical, Horizonal: directory based, static hashing, signature based), How to implement Sharding from an applicative view? How to take care of reporting? What are the emerging solutions in this field? (Gizzard, ScaleBase, Xeround and Akibian)

Published in: Technology

1 Comment
18 Likes
Statistics
Notes
  • The best alternative to sharding is CLUSTRIX ! It is seemlesss to implement and let's all developers wory about developing! not sharding. I can tell you more about it if you like, mail me at kenwaj@yahoo.com
    KW
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
4,915
On Slideshare
0
From Embeds
0
Number of Embeds
21
Actions
Shares
0
Downloads
3
Comments
1
Likes
18
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • The world is changing more and more fast You have to minimize NRE You must support unexpected demand You must provide top service (people now leaves cell operator after single incident, rather then 5, 10 years ago Firms a vulnerable – Citi worth 20Bill $ instead of 200 Bill a year ago Break and Mortar bookstores 15 years ago and Amazon…. Will it happen again to banks, insurance, real estate agencies… IS YOUR MARKET THE NEXT FOR PENETRATION – Finance? Real Estate? How to win a rival that is not existing yet? http://www.johnmwillis.com/ibm/cloud-computing-and-the-enterprise/ -
  • Sharded database – OLTP Little/No reporting OLAP – must be implemented for reporting Loads data from sharded DBs Custom mechanism Any commercial
  • Start with nothing: storage, FW, LB, Server and grow… Can buy servers for more than hour
  • Transcript

    • 1. Moshe Kaplan Database2011, Jan 13 [email_address] http://blogs.microsoft.co.il/blogs/vprnd http://top-performance.blogspot.com
    • 2. Presentation Objectives
      • Who is using MySQL?
      • MySQL Limitations
      • How to get over this?
      • Sharding Case Studies
      • Products
    • 3. Who is Using MySQL?
    • 4. MySQL Limitations
      • 50-100M Records/Table
      • 50 SELECT Statements/Second
    • 5. 700 Inserts/Sec In Memory Engine 3000 Inserts/Sec InnoDB Engine 700 Inserts/Sec Amazon AWS Standard Large Instance
    • 6. Why Do I Care?
      • From 0 to 100 (US mass adaptation)
        • Phone: 100 yrs
        • Radio: 40 yrs
        • TV: 30 yrs
        • Mobile: 20 yrs
        • Internet: 10 yrs
        • Facebook: 2 yrs
    • 7. 100K New Users/Week
    • 8. The Network Effect
    • 9. What Should I Do?
      • Oracle
      • SQL Server
      • $$$
    • 10. Sharding
    • 11. Vertical Sharding
    • 12. Horizontal Sharding
      • Static Hashing
        • Complex growth
        • Simple
      Mod 10 = 0 Mod 10 = 1 Mod 10 = 2 Mod 10 = 3 Mod 10 = 4 Mod 10 = 5 Mod 10 = 6 Mod 10 = 7 Mod 10 = 8 Mod 10 = 9
    • 13.
      • Key locations are defined in a directory
        • Simple growth
        • Directory is SPOF
      Horizontal Sharding
    • 14. Horizontal Sharding
      • Static Hashing with Directory Mapping
        • Simple Growth
        • Small Directory still SPOF
      Mod 1000 = 4
    • 15. Horizontal Sharding
      • Each key signed by DB# generated on creation
        • Simple growth
        • New key generation is SPOF
    • 16. Limitations: Reporting
    • 17. Best Practices
      • $connection = new_db_connection("customer://1234");
      • $statement = $connection->prepare( $sql_statement, $params );
      • $result = $statement->execute();
    • 18. Lessons
      • Vertical Sharding:
        • User Actions, Users, Comments, Items
      • Horizontal Sharding
      • Denormalization
      • MySQL Replication
    • 19. ‘ s Path to Sharding
      • Single Server
      • Single Master with Multiple Read Slaves
      • Partitioned
      • Sharding
    • 20. ‘ s Path
      • Master-Master replication
      • Each Shard is 50% loaded
      • 40K queries/second
    • 21. Ad Network Reference Architecture
    • 22. Sharding Management http://dev.mysql.com/tech-resources/articles/mysql_5.0_psea1.html Storage Engine MySQL Proxy
    • 23. Grizzard http://readwriteweb.com.br/search/scala-framework/
    • 24. ScaleBase
    • 25. The Bottom Line: Grow ∞
    • 26. Startup your Engines Thank you Moshe Kaplan: mokplan@gmail.com http://top-performance.blogspot.com