Your SlideShare is downloading. ×
0
•    –    –    –    –    –
View on Slidesharehttp://profyclub.ru/docs/439
••••

••
=Big Business Opportunity
-   Enterprise        -   Vendor dependencySQL        -   Scalability constraints        -   Common interface        - Ope...
••••••••••••••
SQLTransactionsNoSQL => NoACIDStandard InterfaceExperts
DBMS     Worldwide   21,359       23,252     26,701       11.8% Market   Korea       349          395        478          ...
RDBMS is still the best choice for    mission-critical data
How to manage Big data with RDBMS?
Database Sharding
•              DB         X     Y     Z•            Shard    DB       DB          DB    X         Y          Z
Name               Type              Requirements         Interface                                      DB          ETC  ...
••••    –    –•••
•    –    –    –    –    –
CUBRID 9.0
                    
•
How to install CUBRID SHARD?
http://www.cubrid.org/downloads
•    –•    –•    –
SHARD_KEY_MODULAR         = 256SHARD_KEY_LIBRARY_NAME    = ‘’SHARD_KEY_FUNCTION_NAME   = ‘’
   id       user_id=      order_no       …
int user_get_shard_key(int type, void *val){    int mod = 2;    if (val == NULL)    {           return ERROR_ON_ARGUMENT; ...
Configuring CUBRID SHARD is very easy!
•    $> cubrid createdb shard1    $> csql -S -u dba shard1 -c "create user    shard password shard123’”    $> cubrid serve...
•    $> csql -C -u shard -p shard123    shard1@localhost -c ”CREATE TABLE users    (id BIGINT PRIMARY KEY, name VARCHAR(20...
$> cubrid shard start@ cubrid shard start ++cubrid shard start: success
connectionURL   ="jdbc:cubrid:localhost:45511:shard1:shard:shard123:";
String query = "SELECT name FROM student WHERE student_no = /*+ shard_key */ ?; ";PrepareStatement query_stmt = connection...
SELECT name FROM student WHEREstudent_no = /*+ shard_key */ ?;                               •                            ...
How did we tackle the unique ID problem?
•    –    –    –    –    –
•                      BIGINT    AUTO_INCREMENT PRIMARY KEY••
Does CUBRID SHARD support HA?
Yes!
CUBRID SHARD Performance
Description       Quantity          OS (64bit) / CPU / MEMAgent to generatload and                  8       Centos5.3 / xe...
Load Generator Performance                                                                     100000                     ...
-         Similar performance until 128 Vuser                               - When SHARD is not used, 128 Vuser is        ...
TPC-C Performance Test
•       • AWS Xlarge instance    –      • 7GB RAM           • 20 EC2 units    –    –   • Ubuntu 12.04 64-bit    –        •...
46                   44.18     42.664238                         MySQL                           CUBRID3430         TPC-C ...
CUBRID SHARD is very stable     and easy to use!
                                   
•    –    –•    –    –•    –
What’s next for CUBRID?

www.cubrid.orgEsen SagynovCUBRID Project Manageresen@cubrid.org                     CUBRID Q&A                            ...
Database sharding the right way: еasy, reliable, and open source (Esen Sagynov)
Database sharding the right way: еasy, reliable, and open source (Esen Sagynov)
Database sharding the right way: еasy, reliable, and open source (Esen Sagynov)
Database sharding the right way: еasy, reliable, and open source (Esen Sagynov)
Database sharding the right way: еasy, reliable, and open source (Esen Sagynov)
Database sharding the right way: еasy, reliable, and open source (Esen Sagynov)
Upcoming SlideShare
Loading in...5
×

Database sharding the right way: еasy, reliable, and open source (Esen Sagynov)

1,129

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,129
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Self introduction.
  • CUBRID is a fully-feature Relational Database Management System.CUBRID is not a usual open source project backed by a community, but it’s actually backed by the largest IT corporation in South Korea.
  • Today I want to talk about the importance of relational database systems.
  • Nice NoSQL vs. RDBMS discussion on one of the Russian forums http://it-talk.org/post80487.html#p80487
  • In South Korea, Enterprise Business is even more dependent on Oracle database.
  • If you ask companies who operate mission-critical services, they will tell:1) that a relational database system is still the best choice for mission-critical data;2) that service availability is more important than performance;3) that high performance is good, but predictable performance is the king.The fellows at Box.com cloud storage platform also say they choose RDBMS for mission-critical data.
  • We’ve developed Database Sharding in CUBRID!The difference between partitioning and sharding is that with partitioning you can divide the data between multiple tables within one database which have identical schema.But with sharding you divide data between tables located in different databases. Sometimes the database gets so big that mere tables partitioning is not enough, in fact, it will hinder the performance of the entire system. So we’d better add new databases otherwise called Shards.If HA is for READ distribution, Sharding is for WRITE distribution as you can write to different databases simultaneously.This feature is something mostdevelopers dream to have it on Database side rather than on the application layer. Database Sharding doesn’t just simplify the developers’ life, but also improves both the application and database performance.The Application gets rid of the sharding logic.The Database reduces the index size.Win-win!
  • We’ve developed Database Sharding in CUBRID!The difference between partitioning and sharding is that with partitioning you can divide the data between multiple tables within one database which have identical schema.But with sharding you divide data between tables located in different databases. Sometimes the database gets so big that mere tables partitioning is not enough, in fact, it will hinder the performance of the entire system. So we’d better add new databases otherwise called Shards.If HA is for READ distribution, Sharding is for WRITE distribution as you can write to different databases simultaneously.This feature is something mostdevelopers dream to have it on Database side rather than on the application layer. Database Sharding doesn’t just simplify the developers’ life, but also improves both the application and database performance.The Application gets rid of the sharding logic.The Database reduces the index size.Win-win!
  • - Talking about the open source RDBMS solutions, MySQL doesn’t provide database sharding out of the box.- Google had to significantly change MySQL replication to make it work similarly. But at the time Sun, the former owner of MySQL didn’t accept Google’s changes, resulting in a fork form mainstream without mainstream support.- Twitter has recently opened their MySQL fork.http://www.oracle.com/technetwork/database/features/availability/300461-132370.pdf
  • SHARD_KEY_MODULAR = 256SHARD_KEY_LIBRARY_NAME = stringSHARD_KEY_FUNCTION_NAME = string
  • No additional SQL parsing because of HINT.
  • Transcript of "Database sharding the right way: еasy, reliable, and open source (Esen Sagynov)"

    1. 1. • – – – – –
    2. 2. View on Slidesharehttp://profyclub.ru/docs/439
    3. 3. ••••
    4. 4. 
    5. 5. ••
    6. 6. =Big Business Opportunity
    7. 7. - Enterprise - Vendor dependencySQL - Scalability constraints - Common interface - Open SourceNoSQL - Scalable - Non-standard API
    8. 8. ••••••••••••••
    9. 9. SQLTransactionsNoSQL => NoACIDStandard InterfaceExperts
    10. 10. DBMS Worldwide 21,359 23,252 26,701 11.8% Market Korea 349 395 478 17% $MM Ratio 1.6% 1.7% 1.8%70%65%60%55% Korea50% Worldwide45%40% 2009 2010 2011 Source: Gartner, 2012
    11. 11. RDBMS is still the best choice for mission-critical data
    12. 12. How to manage Big data with RDBMS?
    13. 13. Database Sharding
    14. 14. • DB X Y Z• Shard DB DB DB X Y Z
    15. 15. Name Type Requirements Interface DB ETC DBMS w/ - HibernateHibernate shards AS framework Hibernate Java - JVM supportdbShards AS & Middleware MySQL Java, C MiddlewareGizzard (Twitter) Any storage - JVM Java Middleware &Spider for MySQL MySQL Any Storage Engine - CUBRIDCUBRID SHARD Middleware - MySQL Any - Oracle
    16. 16. •••• – –•••
    17. 17. • – – – – –
    18. 18. CUBRID 9.0
    19. 19.      
    20. 20.
    21. 21. How to install CUBRID SHARD?
    22. 22. http://www.cubrid.org/downloads
    23. 23. • –• –• –
    24. 24. SHARD_KEY_MODULAR = 256SHARD_KEY_LIBRARY_NAME = ‘’SHARD_KEY_FUNCTION_NAME = ‘’
    25. 25.  id  user_id=  order_no  …
    26. 26. int user_get_shard_key(int type, void *val){ int mod = 2; if (val == NULL) { return ERROR_ON_ARGUMENT; } switch(type) { case SHARD_U_TYPE_INT: { int ival; ival = (int) (*(int *)val); return ival % 2; } break; case SHARD_U_TYPE_STRING: return ERROR_ON_MAKE_SHARD_KEY; default: return ERROR_ON_ARGUMENT; } return ERROR_ON_MAKE_SHARD_KEY;}
    27. 27. Configuring CUBRID SHARD is very easy!
    28. 28. • $> cubrid createdb shard1 $> csql -S -u dba shard1 -c "create user shard password shard123’” $> cubrid server start shard1
    29. 29. • $> csql -C -u shard -p shard123 shard1@localhost -c ”CREATE TABLE users (id BIGINT PRIMARY KEY, name VARCHAR(20), age SMALLINT)”
    30. 30. $> cubrid shard start@ cubrid shard start ++cubrid shard start: success
    31. 31. connectionURL ="jdbc:cubrid:localhost:45511:shard1:shard:shard123:";
    32. 32. String query = "SELECT name FROM student WHERE student_no = /*+ shard_key */ ?; ";PrepareStatement query_stmt = connection.prepareStatement(query);query_stmt.setInt(1,100);ResultSet rs = query_stmt.executeQuery();// fetch resultset range key_column (hash result) shard_id min max student_no 0 63 0 student_no 64 127 1 student_no 128 191 2 student_no 192 255 3
    33. 33. SELECT name FROM student WHEREstudent_no = /*+ shard_key */ ?; • •
    34. 34. How did we tackle the unique ID problem?
    35. 35. • – – – – –
    36. 36. • BIGINT AUTO_INCREMENT PRIMARY KEY••
    37. 37. Does CUBRID SHARD support HA?
    38. 38. Yes!
    39. 39. CUBRID SHARD Performance
    40. 40. Description Quantity OS (64bit) / CPU / MEMAgent to generatload and 8 Centos5.3 / xeon 2G-8core / 8GNDrive App SimulatorCUBRID Shard 1 Centos5.3 / xeon 2.27G-16core / 24GCUBRID Broker 1 Centos5.3 / xeon 2.27G-16core / 24GMeta DB 4 Centos5.x / xeon 2.33G-4core / 8GUser DB 1 Centos5.3 / xeon 2.5G-8core / 8G
    41. 41. Load Generator Performance 100000 80000 60000 RPS 40000 20000 0 32 64 96 128 160 192 256 320 384 448 512 # of concurrent users Performance trend when load is increased60000 7050000 60 5040000 4030000 3020000 2010000 10 0 0 64 128 192 256 320 proxy cpu RPS metadb TPS Mean Time(ms)
    42. 42. - Similar performance until 128 Vuser - When SHARD is not used, 128 Vuser is maximum - In SHARD usage case, when # of Vuser is increase - maximum performance can be achieved as well as shorter response time and lower CPU utilization.64 128 192 256 320 Vuser
    43. 43. TPC-C Performance Test
    44. 44. • • AWS Xlarge instance – • 7GB RAM • 20 EC2 units – – • Ubuntu 12.04 64-bit – • CUBRID 9.0 (beta) – • MySQL 5.5.28 – – • Buffer • 2.8GB – data_buffer_size – • 2.8GB• innodb_pool_size • Default configurations
    45. 45. 46 44.18 42.664238 MySQL CUBRID3430 TPC-C Index
    46. 46. CUBRID SHARD is very stable and easy to use!
    47. 47.         
    48. 48. • – –• – –• –
    49. 49. What’s next for CUBRID?
    50. 50. 
    51. 51. www.cubrid.orgEsen SagynovCUBRID Project Manageresen@cubrid.org CUBRID Q&A www.cubrid.org/questions
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×