0
CUBRID Reference Architecture for Social Networking Service<br />Kieun Park<br />NHN Business Platform Corp.<br />2011.8<b...
46  CUBRID Reference Architecture for Social Networking Service<br />2 /<br />
Abstract<br />46  CUBRID Reference Architecture for Social Networking Service<br />3 /<br />The top ranked facebook celebr...
I Am<br />46  CUBRID Reference Architecture for Social Networking Service<br />4 /<br />박기은Kieun Park<br /><ul><li>Softwar...
Service Platform Development Center
NHN Business Platform Corp.
iamyaw@nhn.com
CUBRID Open Source DBMS
nStore Distributed Database System</li></li></ul><li>Contents<br />46  CUBRID Reference Architecture for Social Networking...
Contents<br />46  CUBRID Reference Architecture for Social Networking Service<br />6 /<br />Characteristics of online soci...
Contents<br />46  CUBRID Reference Architecture for Social Networking Service<br />7 /<br />Characteristics of online soci...
Contents<br />46  CUBRID Reference Architecture for Social Networking Service<br />8 /<br />Characteristics of online soci...
46  CUBRID Reference Architecture for Social Networking Service<br />9 /<br />Characteristics of online social networking ...
Some Infographics about Online Social Networking Service<br />46  CUBRID Reference Architecture for Social Networking Serv...
Some Infographics about Online Social Networking Service<br />46  CUBRID Reference Architecture for Social Networking Serv...
Some Infographics about Online Social Networking Service<br />46  CUBRID Reference Architecture for Social Networking Serv...
Some Infographics about Online Social Networking Service<br />46  CUBRID Reference Architecture for Social Networking Serv...
Some Infographics about Online Social Networking Service<br />46  CUBRID Reference Architecture for Social Networking Serv...
Statistics of Facebook and Twitter<br />46  CUBRID Reference Architecture for Social Networking Service<br />15 /<br />140...
Statistics of Me2Day<br />46  CUBRID Reference Architecture for Social Networking Service<br />16 /<br />Postings per day:...
Online social networking service<br />46  CUBRID Reference Architecture for Social Networking Service<br />17 /<br />Socia...
Feed Following Works<br />46  CUBRID Reference Architecture for Social Networking Service<br />18 /<br />Feeds Following<b...
Characteristics of Online Social Networking Service<br />46  CUBRID Reference Architecture for Social Networking Service<b...
Followers gets personalized feeds that aggregate streams produced those followed.
Highly variable and somewhat bit fan-out of the follows graph makes data feeding difficult to implement and requires high ...
46  CUBRID Reference Architecture for Social Networking Service<br />20 /<br />Challenges and demands on database architec...
Challenge and Demands on Database Architecture<br />46  CUBRID Reference Architecture for Social Networking Service<br />2...
Today social media generates more information in a short period of time than was previously available in the entire world ...
Not only the exponential growth of Facebook, Google+, Twitter, but also the use of more and more rich media such as user-g...
Social media now produces massive amounts of data. Facebook’s network, for instance, consists of 100 million entities gene...
46  CUBRID Reference Architecture for Social Networking Service<br />23 /<br />CUBRID unique features<br />
CUBRID<br />46  CUBRID Reference Architecture for Social Networking Service<br />24 /<br />Free<br />open source<br />is t...
<ul><li>HA feature
Reclaim deleted space
Fast serial data (cached)
LFS (large file support ) for database volume</li></ul>CUBRID<br />46  CUBRID Reference Architecture for Social Networking...
Database volume size reduced.
Multi-range scan and key limit function
Covered index</li></ul>October, 2009<br />CUBRID Cluster Project has been started.<br />September, 2009<br />CUBRID 2008 R...
HA monitoring
Full SQL function support</li></ul>CUBRID became an open source project.<br />CUBRID 2008 R1.1 stable was released.<br />N...
CUBRID Index Scan with Top-k Sorting Technique<br />46  CUBRID Reference Architecture for Social Networking Service<br />2...
CUBRID Index Scan with Top-k Sorting Technique<br />46  CUBRID Reference Architecture for Social Networking Service<br />2...
CUBRID Test Results<br />46  CUBRID Reference Architecture for Social Networking Service<br />28 /<br />Refer http://www.c...
CUBRID High Availability Feature<br />46  CUBRID Reference Architecture for Social Networking Service<br />29 /<br />CUBRI...
Upcoming SlideShare
Loading in...5
×

CUBRID Features Optimized for Social Networking Services

1,527

Published on

CUBRID has many optimizations for SNS. In this presentation CUBRID architect explains the characteristics of Social Networking Services and how CUBRID architecture is designed to meet these demands.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,527
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
39
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "CUBRID Features Optimized for Social Networking Services"

  1. 1. CUBRID Reference Architecture for Social Networking Service<br />Kieun Park<br />NHN Business Platform Corp.<br />2011.8<br />
  2. 2. 46 CUBRID Reference Architecture for Social Networking Service<br />2 /<br />
  3. 3. Abstract<br />46 CUBRID Reference Architecture for Social Networking Service<br />3 /<br />The top ranked facebook celebrity has 44 million fans. The top ranked twitter user has 11 million followers. There are over 900 million objects in the facebook site and 140 million tweets people send per day. Needless to say, these facts heavily impact on database they have. Thus, best practice in database architecture is important.<br />Online social networking (OSN) services have rapidly proliferated and changed the way data is stored and served. Social data is an enormous graph of small objects that are tightly interconnected. The service page of OSN is a view of those small objects customized to a specific viewers at a specific time. Typically, the view is aggregation of events connected by social graph which is changing constantly with users' realtime interaction. Even though the Dunbar's number shows that the number of people with whom one gets stable social relationship is relatively small as 150, in OSN site celebs have a large number of followers so that the social graph is very huge. These properties of the data lead to new challenges, and demands new database architecture to handle them.<br />The main considerations of database architecture for OSN are about scale-out and performance in addition to high availability as mandatory. the main characteristics of OSN service in terms of data are power-law scaling, data feeding frenzy and Zipfian distribution access. Data being delivered are exponentially growing according to the popularity of the service. Cost-effective database scale-out architecture is important to business requirement as well as to technical issues.<br />In this presentation, CUBRID Reference Architecture for social networking service will be shown. The presented architectures are based on best practices developed from real business cases of NHN, biggest portal service provider in Korea. Described are the helpful features to support the database architecture demands for OSN service. For example, index scan with top-k sorting technique is developed for fast feed aggregation. Also, HA, automatic sharding and clustering features of the CUBRID will be explained. Finally, the nStore, a distributed database system based on the CUBRID, will be introduced. Concept of the nStore is similar to Amazon Dynamo but different in that it support SQL.<br />
  4. 4. I Am<br />46 CUBRID Reference Architecture for Social Networking Service<br />4 /<br />박기은Kieun Park<br /><ul><li>Software/Database Architect
  5. 5. Service Platform Development Center
  6. 6. NHN Business Platform Corp.
  7. 7. iamyaw@nhn.com
  8. 8. CUBRID Open Source DBMS
  9. 9. nStore Distributed Database System</li></li></ul><li>Contents<br />46 CUBRID Reference Architecture for Social Networking Service<br />5 /<br />Characteristics of online social networking service<br />Challenges and demands on database architecture<br />CUBRID features<br />CUBRID reference architecture for social networking service<br />How fast is the data growing in online social networking service?<br />Characteristics of OSN service: Power-law scaling growth, data feeding frenzy, and Zipfian distribution access<br />How does it access database? Feed aggregation<br />
  10. 10. Contents<br />46 CUBRID Reference Architecture for Social Networking Service<br />6 /<br />Characteristics of online social networking service<br />Challenges and demands on database architecture<br />CUBRID features<br />CUBRID reference architecture for social networking service<br />Business demands and system requirements<br />Main considerations of database architecture for OSN service<br />Scale-out, performance, and high availability<br />
  11. 11. Contents<br />46 CUBRID Reference Architecture for Social Networking Service<br />7 /<br />Characteristics of online social networking service<br />Challenges and demands on database architecture<br />CUBRID unique features<br />CUBRID reference architecture for social networking service<br />Index scan with top-k sorting technique<br />High availability feature<br />Automatic sharding component<br />CUBRID Cluster System<br />nStore, a distributed database system based on the CUBRID<br />
  12. 12. Contents<br />46 CUBRID Reference Architecture for Social Networking Service<br />8 /<br />Characteristics of online social networking service<br />Challenges and demands on database architecture<br />CUBRID features<br />CUBRID reference architecture for social networking service<br />CUBRID Web Reference Architecture<br />CUBRID SNS Reference Architecture<br />
  13. 13. 46 CUBRID Reference Architecture for Social Networking Service<br />9 /<br />Characteristics of online social networking service<br />
  14. 14. Some Infographics about Online Social Networking Service<br />46 CUBRID Reference Architecture for Social Networking Service<br />10 /<br />The history and evolution of OSN are made in last 10 years.<br />Source http://blog.skloog.com/history-social-media-history-social-media-bookmarking/<br />
  15. 15. Some Infographics about Online Social Networking Service<br />46 CUBRID Reference Architecture for Social Networking Service<br />11 /<br />500 million Facebook users, 106 million Twitter users<br />Social networks with user bases larger than the population of most countries<br />Source http://www.digitalsurgeons.com/facebook-vs-twitter-infographic/<br />
  16. 16. Some Infographics about Online Social Networking Service<br />46 CUBRID Reference Architecture for Social Networking Service<br />12 /<br />The top ranked twitter user, Lady Gaga, has 11 million followers. About 55 million Tweets per day.<br />Twitter gets about 600 million queries every day.<br />(http://twitaholic.com)<br />Source http://www.digitalbuzzblog.com/infographic-twitter-statistics-facts-figures/<br />
  17. 17. Some Infographics about Online Social Networking Service<br />46 CUBRID Reference Architecture for Social Networking Service<br />13 /<br />The most followed person, Eminem, has more than 44 million fans.<br />More than 5 billion pieces of content shared each week.<br />2,716,000 messages, 1,587,000 wall posts, 10,208,000 comments in 20 minutes on Facebook.<br />(http://www.independent.co.uk)<br />Source http://www.digitalbuzzblog.com/facebook-statistics-facts-figures-for-2010/<br />Source http://www.digitalbuzzblog.com/facebook-statistics-stats-facts-2011/<br />
  18. 18. Some Infographics about Online Social Networking Service<br />46 CUBRID Reference Architecture for Social Networking Service<br />14 /<br />Have we reached a world of infinite information?<br />In a similar manner to our universe, the Internet is expanding at an incredibly rapid pace, reaching new levels of information storage and content creation every second.<br />By 2020,<br />roughly 25x1018 (quintillion)<br />information containers<br />Every minute,<br />24 hours of video<br />The growth gap<br />between<br />the digital contents created<br />and the available storage<br />Sourcehttp://www.flowtown.com/blog/have-we-reached-a-world-of-infinite-information<br />
  19. 19. Statistics of Facebook and Twitter<br />46 CUBRID Reference Architecture for Social Networking Service<br />15 /<br />140 million; the average number of Tweets people sent per day.<br />6,939;current TPSrecord.<br />More than 750 million active users.<br />There are over 900 million objects that people interact with (pages, groups, events and community pages)<br />Source http://www.facebook.com/press/info.php?statistics<br />Source http://blog.twitter.com/2011/03/numbers.html<br />
  20. 20. Statistics of Me2Day<br />46 CUBRID Reference Architecture for Social Networking Service<br />16 /<br />Postings per day: 278,461<br />Total postings: 123,456,727<br />Total photos: 10,638,089<br />
  21. 21. Online social networking service<br />46 CUBRID Reference Architecture for Social Networking Service<br />17 /<br />Social data is an enormous graph of small objects that are tightly interconnected.<br />The service page of OSN is a aggregation of events connected by social graph which is changing constantly with users' realtimeinteraction.<br />
  22. 22. Feed Following Works<br />46 CUBRID Reference Architecture for Social Networking Service<br />18 /<br />Feeds Following<br />Contents<br />(comment, photo, tag, …)<br />Follower<br />News Feeds<br />(personalized feeds)<br />Application Layer<br />Outbox<br />Inbox<br />Delivery & Aggregation<br />Engine<br />Content Management Layer<br />Cache<br />Database<br />Database<br />Data Storage Layer<br />
  23. 23. Characteristics of Online Social Networking Service<br />46 CUBRID Reference Architecture for Social Networking Service<br />19 /<br />Power-law scalinggrowth<br /><ul><li>Users follow activity and news of other users and entities.
  24. 24. Followers gets personalized feeds that aggregate streams produced those followed.
  25. 25. Highly variable and somewhat bit fan-out of the follows graph makes data feeding difficult to implement and requires high cost to operate.</li></ul>Online social networks have properties of significant clustering, small diameter, and power-law degrees.<br />Zipfiandistribution access<br />Data feeding frenzy<br />Twitter Activity<br />5% of users account for 75% of all activity, 10% account for 86% of activity, and the top 30% account for 97.4%.<br />
  26. 26. 46 CUBRID Reference Architecture for Social Networking Service<br />20 /<br />Challenges and demands on database architecture<br />
  27. 27. Challenge and Demands on Database Architecture<br />46 CUBRID Reference Architecture for Social Networking Service<br />21 /<br />From business demands to technology implementation.<br /><ul><li>Online social networking service have rapidly proliferated and changed the way data is stored and served.
  28. 28. Today social media generates more information in a short period of time than was previously available in the entire world a few generations ago.
  29. 29. Not only the exponential growth of Facebook, Google+, Twitter, but also the use of more and more rich media such as user-generated video from smart phone, is surely driving big data.</li></ul>Source http://www.itu.int/net/itunews/issues/2010/06/35.aspx<br />
  30. 30. Social media now produces massive amounts of data. Facebook’s network, for instance, consists of 100 million entities generating tens of millions of events per second. Twitter, meanwhile, funnels 140 million public tweets a day. [GigaOM research notes]<br />With enterprise data volumes moving past terabytes to tens of petabytes and more, business and IT leaders face significant opportunities and challenges from big data. For a large enterprise, big data may be in the petabytes or more; for a small or mid-size enterprise, data volumes that grow into tens of terabytes may become challenging to analyze and manage. <br />When an application is being designed, software architects need to plan for much greater application load to avoid major redesigns in the future. While scaling out web servers can be done quite easily, properly scaling out database servers is far more challenging and happens.<br />Challenge and Demands on Database Architecture<br />46 CUBRID Reference Architecture for Social Networking Service<br />22 /<br />Managing user generated socialinteraction data!<br />Coping with explosion in data volume!<br />Cost-effective scale-out to meet rapidly growing demands!<br />
  31. 31. 46 CUBRID Reference Architecture for Social Networking Service<br />23 /<br />CUBRID unique features<br />
  32. 32. CUBRID<br />46 CUBRID Reference Architecture for Social Networking Service<br />24 /<br />Free<br />open source<br />is the choice<br />of the modern<br />world<br />Powerful<br />clean architecture<br />with rich functionality<br />for competitive<br />performance<br />Enterprise<br />unique features<br />for stability<br />and reliability<br />
  33. 33. <ul><li>HA feature
  34. 34. Reclaim deleted space
  35. 35. Fast serial data (cached)
  36. 36. LFS (large file support ) for database volume</li></ul>CUBRID<br />46 CUBRID Reference Architecture for Social Networking Service<br />25 /<br />CUBRID 4.0 stable released.<br />July, 2011<br />CUBRID 3.0 stable released.<br />October, 2010<br />Official open source community, www.cubrid.org, opened.<br /><ul><li>INSERT performance enhancement
  37. 37. Database volume size reduced.
  38. 38. Multi-range scan and key limit function
  39. 39. Covered index</li></ul>October, 2009<br />CUBRID Cluster Project has been started.<br />September, 2009<br />CUBRID 2008 R2.0 stable released.<br />August, 2009<br /><ul><li>FBO (file-based object)
  40. 40. HA monitoring
  41. 41. Full SQL function support</li></ul>CUBRID became an open source project.<br />CUBRID 2008 R1.1 stable was released.<br />November, 2008<br />First internal release CUBRID 2008 R1.0<br />October, 2008<br />The development of CUBRID DBMS started.<br />2011<br />2006 <br />2007 <br />2008 <br />2009 <br />2010 <br />2012<br />
  42. 42. CUBRID Index Scan with Top-k Sorting Technique<br />46 CUBRID Reference Architecture for Social Networking Service<br />26 /<br />CUBRID does multi-range index scan.<br />My friends’ newest twenty comments<br />SELECT post_no FROM postsWHERE id IN (4, 15, 36, …) AND registered_date < 20000<br />ORDER BY registered_date DESC LIMIT 20<br />Multi-range scan<br />Single range scan with key filter<br />Disk I/O ?!<br /># of leaf pages accessed<br />> # of keys of scan result<br /># of leaf pages accessed <br />= # of keys of scan result<br />Filter out<br />On the fly sorting<br />during scan<br />Sort after scan<br />(4,10001) (4,9999) (4,875) …<br />(4,10001) (4,9999) (4,875) …<br />(36,947) (36,120) (36,3) …<br />(36,947) (36,120) (36,3) …<br />(15, 10000) (15,9999) (15, 7467) …<br />(15, 10000) (15,9999) (15, 7467) …<br />
  43. 43. CUBRID Index Scan with Top-k Sorting Technique<br />46 CUBRID Reference Architecture for Social Networking Service<br />27 /<br />SELECT * FROM tbl WHERE a IN (2, 4, 5) AND b < ‘K’<br />ORDER BY b LIMIT 3;<br />SELECT * FROM tbl WHERE a = 2 AND b < ‘K’<br />ORDER BY b LIMIT 3;<br />
  44. 44. CUBRID Test Results<br />46 CUBRID Reference Architecture for Social Networking Service<br />28 /<br />Refer http://www.cubrid.org/cubrid_mysql_sns_benchmark_test<br />Test case 1: user group 1 only<br />Test case 2: user group 2 only<br />Test case 3: 40% of user group 1, 50% of user group 2, 10% of user group 3<br />Test case 4: 10% of user group 1, 50% of user group 2, 40% of user group 3<br />User group 1: users with 50 or less friends<br />User group 2: users with 51~2000 friends<br />User group 3: users with friends up to tens of thousands<br />
  45. 45. CUBRID High Availability Feature<br />46 CUBRID Reference Architecture for Social Networking Service<br />29 /<br />CUBRID HA, highly fault-resistant DBMS enables<br /><ul><li>Non-stop 24x7 service
  46. 46. System maintenance without shutdown
  47. 47. Automatically fail-over (less than 20 sec)
  48. 48. Various acess modes (read-write, read-only)</li></ul>Application<br />CUBRID Driver<br />CUBRID Driver<br />UPDATE<br />SELECT<br />UPDATE<br />Broker<br />Active<br />Broker<br />Backup<br />Broker<br />automatic<br />switch-over<br />Read-Only<br />Mode<br />Read-Write<br />Mode<br />Standby-2<br />Server<br />@Remote IDC<br />Standby-1<br />Server<br />automatic<br />fail-over/fail-back<br />Active<br />Server<br />Database Server<br />Slave DB<br />Master DB<br />Slave DB<br />
  49. 49. CUBRID High Availability Feature<br />46 CUBRID Reference Architecture for Social Networking Service<br />30 /<br />UPDATE<br />SELECT<br />Heartbeat<br />Heartbeat<br />Log Applying<br />Log Applying<br />Log Shipping<br />(synchronous)<br />Log<br />Writer<br />Log<br />Applier<br />CUBRID<br />Server<br />Log<br />Writer<br />Log<br />Applier<br />CUBRID<br />Server<br />Slave<br />DB<br />Replication<br />Log<br />Replication<br />Log<br />Transaction<br />Log<br />Transaction<br />Log<br />Master<br />DB<br />S1-Node<br />Standby Server Node<br />A-Node<br />Active Server Node<br />Log Shipping<br />(asynchronous)<br />Heartbeat<br />SELECT<br />Log Applying<br />HA feature is based on database replication with transaction log multiplication technique.<br />Slave<br />DB<br />Replication<br />Log<br />Transaction<br />Log<br />Statement-based replication could cause data inconsistency.<br />S2-Node<br />
  50. 50. CUBRID Automatic Sharding Component<br />46 CUBRID Reference Architecture for Social Networking Service<br />31 /<br />Automatic shardingfeature enables<br /><ul><li>No more application logic
  51. 51. Scale-out DB architecture</li></ul>Features<br /><ul><li>Multiple sharding strategies</li></ul>Shard by modulus, date/time range, extendible hash<br /><ul><li>User hint-aware</li></ul>SELECT * FROM tbl WHERE nonkey=‘abc’ /* shard=1 */<br />Application<br />SELECT … WHERE key=k0008<br />UPDATE … WHERE key=k0002<br />Broker<br />Sharding<br />Metadata<br />automatic sharding<br />Expand Shard<br />k0001<br />k0005<br />K000…<br />k0002<br />k0006<br />K000…<br />k0003<br />k0007<br />K000…<br />k0004<br />k0008<br />K000…<br />Shard #1<br />Shard #2<br />Shard #3<br />Shard #4<br />Database Server<br />New Shard<br />
  52. 52. CUBRID Cluster System<br />46 CUBRID Reference Architecture for Social Networking Service<br />32 /<br />Main features of CUBRID Cluster are<br /><ul><li>Global schema
  53. 53. Distributed partition
  54. 54. Load balancing</li></ul>Users can get<br /><ul><li>Single big database view
  55. 55. Location transparency
  56. 56. Additionally, linear scalability</li></ul>Application<br />SELECT * FROM gtable<br />WHERE part_key=2 AND …<br />INSERT INTO gtable …<br />Broker<br />load balancing<br />global schema / distributed partition<br />gtable<br />part_01<br />part_05<br />gtable<br />part_02<br />part_06<br />gtable<br />part_03<br />part_07<br />gtable<br />part_04<br />part_08<br />Node #1<br />Node #2<br />Node #3<br />Node #4<br />Cluster Server<br />
  57. 57. CUBRID Cluster System<br />46 CUBRID Reference Architecture for Social Networking Service<br />33 /<br />The global schemais a single representation or a global view of all nodes where each node has its own database and schema.<br />SELECT * FROM contents<br />WHERE auth = (SELECT name FROM author WHERE …) <br />Local<br />Schema<br />User<br />Global<br />Schema<br />User<br />UPDATE local …<br />SELECT * FROM contents<br /> WHERE …<br />SELECT * FROM info, code<br /> WHERE info.id = code.id<br />INSERT INTO contents…<br />info<br />contents<br />author<br />Global Schema<br />author<br />code<br />level<br />local<br />contents<br />contents<br />contents<br />info<br />Local Schema #4<br />Local Schema #3<br />Local Schema #2<br />Local Schema #1<br />The users can access any databases through a single schema regardless of and without knowing the location of the distributed data.<br />Database #1<br />Database #2<br />Database #3<br />Database #4<br />
  58. 58. CUBRID Cluster<br />46 CUBRID Reference Architecture for Social Networking Service<br />34 /<br />Global Schema<br />Data<br />System<br />Catalog<br />Logical View<br />Logical View<br />Index<br />Physical View<br />Physical View<br />Schema<br />Schema<br />Data<br />System<br />Catalog<br />System<br />Catalog<br />Data<br />Index<br />Index<br />
  59. 59. CUBRID Cluster<br />46 CUBRID Reference Architecture for Social Networking Service<br />35 /<br />The distributed partition maps global schema onto table partitioning.<br />Partitions are resident in different nodes but accessed through global schema.<br />SELECT * FROM gtable, info <br />WHERE gtable.part_key=02<br /> AND info.id = gtable.id<br />gtable – PARTITION BY HASH (part_key)<br />info<br />part_01<br />part_02<br />part_03<br />part_04<br />Global Schema<br />part_05<br />part_06<br />part_07<br />part_08<br />Partition Data<br />Partition Data<br />Partition Data<br />Partition Data<br />part_02<br />part_03<br />part_03<br />part_01<br />info<br />part_06<br />part_07<br />part_08<br />part_05<br />Database #1<br />Database #2<br />Database #3<br />Database #4<br />
  60. 60. nStore, a distributed database system based on the CUBRID<br />46 CUBRID Reference Architecture for Social Networking Service<br />36 /<br />RDB-like tabular model<br /><ul><li>Schema, column, record
  61. 61. Index on columns (ordered search)</li></ul>Restricted data type<br /><ul><li>Integer(bigint), string, timestamp(msec), id(128bit), bool</li></ul>Data partitioned by key<br /><ul><li>E.g., user-id could be a key</li></ul>SQL-like query language<br /><ul><li>SELECT a, b, c FROM postWHERE fid IN (?, ?, ?) AND b=?ORDER BY ts LIMIT 20,CK=“iamyaw”
  62. 62. INSERT INTO post(no, id, date) VALUES (?, ?, ?),CK=“iamyaw”</li></ul>Joinsupported<br /><ul><li>Between tables in one container</li></li></ul><li>nStore, a distributed database system based on the CUBRID<br />46 CUBRID Reference Architecture for Social Networking Service<br />37 /<br />Application<br />Application<br />Application<br />REST API<br />http://server/keyspace/query?ckey=iamyaw&nsql=‘select a from tbl where k=100’&format=json<br />Data Distribution<br />Replication (3- Copy)<br />Rebalancing<br />nStore<br />nStore<br />CUBRID<br />CUBRID<br />nStore<br />nStore<br />nStore<br />Query Processing<br />Storage System<br />CUBRID<br />CUBRID<br />CUBRID<br />
  63. 63. nStore, a distributed database system based on the CUBRID<br />46 CUBRID Reference Architecture for Social Networking Service<br />38 /<br />Application<br />Container Server<br />Container (ckey=iamyaw)<br />nStore<br />Equi-join<br />REST API<br />Table A<br />Table B<br />Container Server<br />Table C<br />Indexed<br />Column<br />Equi-join<br />Container Server<br />Container Server<br />Global <br />Table G<br />Management Node<br />Indexed Column<br />Container (ckey=kieun_park)<br />Equi-join<br />Container Server<br />Table A<br />Table B<br />Tables<br />Table C<br />Indexed<br />Column<br />Distribution layer<br />RDBMS<br />Indexed Column<br />
  64. 64. nStore Test Results<br />46 CUBRID Reference Architecture for Social Networking Service<br />39 /<br />Tested using YCSB (http://research.yahoo.com/Web_Information_Management/YCSB)<br />INSERT: 50,000,000 records (1K size)<br />READ: Zifian distribution<br />READ w/ compaction: after SSTable compaction (Cassandra, Hbase)<br />READ/UPDATE: 50:50 (50,000,000 records DB)<br />READ/INSERT: 50:50 (50,000,000 records DB)<br />
  65. 65. 46 CUBRID Reference Architecture for Social Networking Service<br />40 /<br />CUBRID referencearchitecture for social networking service<br />
  66. 66. CUBRID Web Reference Architecture<br />46 CUBRID Reference Architecture for Social Networking Service<br />41 /<br />Mid-size<br />web service<br />Web Server<br />(User Interface)<br />Small-size<br />web service<br />Web Application Server (Business Logic)<br />Cache Server<br />Web Server<br />RW<br />RO<br />DB Sharding<br />master<br />master<br />master<br />master<br />CUNITOR<br />master<br />slave<br />slave<br />slave<br />slave<br />slave<br />CUBRID HA<br />CUBRID HA<br />
  67. 67. Social Networking Service Architecture<br />46 CUBRID Reference Architecture for Social Networking Service<br />42 /<br />Web Servers (User Interface)<br />Cache Layer<br />Web Application Servers (Business Logic)<br />Social Query Engine<br />Aggregation Engine<br />Delivery Engine<br />Search Engine<br />Recommendation<br />Engine<br />User Profile DB<br />Social Relation DB<br />Analytics DB<br />Feed Outbox DB<br />Feed Inbox DB<br />Search Index<br />
  68. 68. CUBRID SNS Reference Architecture<br />46 CUBRID Reference Architecture for Social Networking Service<br />43 /<br />Analytic DB<br />partitioned for OLAP<br />Application servers<br />ETL<br />Cache server farm<br />node #2<br />node #n<br />node #1<br />CUBRID Cluster<br />User profile DB<br />sharded by user-id<br />Social relation DB<br />sharded by user-id<br />Inbox/Outbox storage<br />distributed according to user-id<br />OAM<br />RW<br />RO<br />RW<br />RO<br />broker<br />broker<br />DB Sharding<br />container<br />container<br />DB Sharding<br />container<br />container<br />management<br />slave<br />slave<br />slave<br />slave<br />monitoring<br />server<br />container<br />container<br />nStore w/ CUBRID<br />CUNITOR<br />master<br />master<br />master<br />master<br />CUBRID HA<br />CUBRID HA<br />
  69. 69. Best Practices<br />46 CUBRID Reference Architecture for Social Networking Service<br />44 /<br />High available database architecture is the basic business requirements and not technical barrier anymore.<br />Automatic shardingis an effective way to scale-out DB system storing relational model data.<br />nStore is a solution for peta-byte scale data with benefits of high available and scalable distributed store.<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×