Cybozu Labs, Inc.
  Kazuho Oku
 Oracle (RAC)                      

    force.com (Salesforce)
    GAE DataStore
    Kumofs, hBase, etc.

              Q. Oracle       ?


Nov 30 2009                                2
PHP RDBMS                  

        RDBMS
    OSS RDBMS                                           ?
         MySQL Cluster?
         Spider Storage Engine?
         Oracle OSS          ?
                                  KVS       RDBMS

         Salesforce   GAE DataStore         SQL-like

Nov 30 2009                                                  3
MySQL (InnoDB)                  

    SELECT
                   max. 40k QPS
               memcached   1/5

    
                     OS
                   KVS

    SQL                      ACID
    
    
Nov 30 2009                              4
 ACID
    
         
    
         
               
         
               

                  
Nov 30 2009            5
(SNS           )

    
    
         
    
          :
              (JOIN          )
    
                                 
Nov 30 2009                                6
Incline & Pacific                

    
         
         
         (eventual) consistency
    
         SPOF          Proxy-less   Daemon-less


              OSS RDBMS
Nov 30 2009                                        7
Incline –              

                 RDB Sharding
                  shard
                                    




                  
             
           


                        
               
           



                                                
              Incline               
Nov 30 2009                                             8
Incline –     

                      DDL

         
         DDL
                            shard

    ACID
    


Nov 30 2009                          9
Incline –                    (1)

                           JOIN

               uid:1-2000
              uid:2001-4000
              uid:4001-6000

                  tweet
                      tweet
                        tweet

               following
                  following
                     following

              followed_by
                followed_by
                followed_by
       ...
                timeline
                   timeline
                     timeline

                        queue
                      queue
                      queue



    when uid:123 tweets, write only to his tweet table. Incline updates
    other tables automatically

Nov 30 2009                                                                                     10
Incline –                     (2)

                                                                    

               uid:1-2000
               uid:2001-4000
               uid:4001-6000

                  tweet
                       tweet
                      tweet

                following
                  following
                   following

              followed_by
                 followed_by
                followed_by
      ...
                timeline
                    timeline
                   timeline

                        queue
                       queue
                     queue




    when uid:2431 starts following uid:940 only write to his following table

Nov 30 2009                                                                                     11
Pacific –              

    
                         RDBMS

    
         
         
                  < 10

    SPOF
         
Nov 30 2009                       12
Incline & Pacific       

    Incline
         
         MySQL PostgreSQL
         Windows
    Pacific
         
    see developer.cybozu.co.jp/kazuho


Nov 30 2009                              13
 
         Incline Pacific
         DBIx::ShardManager
                              




Nov 30 2009                            14

高度に進化した分散データストアについて

  • 1.
  • 2.
     Oracle (RAC)  force.com (Salesforce)  GAE DataStore  Kumofs, hBase, etc. Q. Oracle ? Nov 30 2009 2
  • 3.
    PHP RDBMS   RDBMS  OSS RDBMS ?  MySQL Cluster?  Spider Storage Engine?  Oracle OSS ?   KVS RDBMS  Salesforce GAE DataStore SQL-like Nov 30 2009 3
  • 4.
    MySQL (InnoDB)  SELECT   max. 40k QPS  memcached 1/5     OS   KVS  SQL ACID     Nov 30 2009 4
  • 5.
     ACID                 Nov 30 2009 5
  • 6.
    (SNS )           :   (JOIN )     Nov 30 2009 6
  • 7.
    Incline & Pacific        (eventual) consistency    SPOF Proxy-less Daemon-less OSS RDBMS Nov 30 2009 7
  • 8.
    Incline –   RDB Sharding   shard Incline Nov 30 2009 8
  • 9.
    Incline –   DDL    DDL   shard  ACID   Nov 30 2009 9
  • 10.
    Incline – (1)   JOIN uid:1-2000 uid:2001-4000 uid:4001-6000 tweet tweet tweet following following following followed_by followed_by followed_by ... timeline timeline timeline queue queue queue when uid:123 tweets, write only to his tweet table. Incline updates other tables automatically Nov 30 2009 10
  • 11.
    Incline – (2)   uid:1-2000 uid:2001-4000 uid:4001-6000 tweet tweet tweet following following following followed_by followed_by followed_by ... timeline timeline timeline queue queue queue when uid:2431 starts following uid:940 only write to his following table Nov 30 2009 11
  • 12.
    Pacific –     RDBMS         < 10  SPOF   Nov 30 2009 12
  • 13.
    Incline & Pacific  Incline    MySQL PostgreSQL  Windows  Pacific    see developer.cybozu.co.jp/kazuho Nov 30 2009 13
  • 14.
       Incline Pacific  DBIx::ShardManager   Nov 30 2009 14