Ruby,NoSQL and TokyoCabinet<br />    盛大创新院  庄表伟<br />http://www.zhuangbiaowei.com/blog/<br />
From RDBMS to NoSQL  (1)<br />Long long ago …<br />Network Database (GE IDS 1961)<br />Hierarchical Database (IBM IMS 1968...
From RDBMS to NoSQL  (2)<br />Now is Web2.0 era<br />Have huge storage(TB/PB data size)<br />Need high performance<br />Ne...
Projects or papers…<br />Google BigTable (2004)<br />Tokyo Cabinet (2006)<br />Amazon Dynamo (2007)<br />MongoDB,CouchDB,C...
CAP (Eric Brewer 2000)<br />Consistency<br />Availability<br />Partition tolerance<br />BASE<br />Basically Available<br /...
<ul><li>Loss of data integrity
Any data is allowed
Can not ensure data consistency
Eventual Consistency
No transaction
Stored procedure convert to map/reduce script
Lack support of management system
System refactoring or rewrite</li></ul>Choose NoSQL means…<br />
<ul><li>by MikioHirabayashi (平林幹雄)
http://1978th.net/
TokyoCabinet -- lightweight database library
TokyoTyrant -- lightweight database server
Key/Value Database
Database types
Hash / B+ Tree / Fixed-length / Table
High performance
Insert: 0.4sec/1M records(2,500,000 qps)
Upcoming SlideShare
Loading in …5
×

Ruby,no sql and tokyocabinet

4,297 views

Published on

Published in: Technology, Education
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,297
On SlideShare
0
From Embeds
0
Number of Embeds
276
Actions
Shares
0
Downloads
63
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Ruby,no sql and tokyocabinet

  1. 1. Ruby,NoSQL and TokyoCabinet<br /> 盛大创新院 庄表伟<br />http://www.zhuangbiaowei.com/blog/<br />
  2. 2. From RDBMS to NoSQL (1)<br />Long long ago …<br />Network Database (GE IDS 1961)<br />Hierarchical Database (IBM IMS 1968)<br />Relational database era<br />Edgar Frank Codd -- Since 1969<br />Relational model<br />SQL<br />Management system<br />Over $ 20 billion (2008)<br />
  3. 3. From RDBMS to NoSQL (2)<br />Now is Web2.0 era<br />Have huge storage(TB/PB data size)<br />Need high performance<br />Need high scalability<br />Need high availability<br />NoSQL now coming<br />New theory support<br />New API style<br />New buzzword<br />
  4. 4. Projects or papers…<br />Google BigTable (2004)<br />Tokyo Cabinet (2006)<br />Amazon Dynamo (2007)<br />MongoDB,CouchDB,Cassandra,Voldemort (2008)<br />Redis,Riak,HBase… (2009)<br />More and more…<br />From RDBMS to NoSQL (3)<br />
  5. 5. CAP (Eric Brewer 2000)<br />Consistency<br />Availability<br />Partition tolerance<br />BASE<br />Basically Available<br />Soft-state<br />Eventual Consistency<br />MapReduce<br />From functional programming<br />Distributed computing framework<br />Important NoSQL theory<br />
  6. 6. <ul><li>Loss of data integrity
  7. 7. Any data is allowed
  8. 8. Can not ensure data consistency
  9. 9. Eventual Consistency
  10. 10. No transaction
  11. 11. Stored procedure convert to map/reduce script
  12. 12. Lack support of management system
  13. 13. System refactoring or rewrite</li></ul>Choose NoSQL means…<br />
  14. 14. <ul><li>by MikioHirabayashi (平林幹雄)
  15. 15. http://1978th.net/
  16. 16. TokyoCabinet -- lightweight database library
  17. 17. TokyoTyrant -- lightweight database server
  18. 18. Key/Value Database
  19. 19. Database types
  20. 20. Hash / B+ Tree / Fixed-length / Table
  21. 21. High performance
  22. 22. Insert: 0.4sec/1M records(2,500,000 qps)
  23. 23. Search: 0.33sec/1M records(3,000,000 qps)
  24. 24. High concurrency
  25. 25. High Scalability</li></ul>TokyoCabinet and TokyoTyrant<br />
  26. 26. <ul><li>Everyone table need a port
  27. 27. Loss table struct and data type
  28. 28. Can not access by column
  29. 29. Reduction of performance
  30. 30. Read 400,000 record/sec(TCT)
  31. 31. Read 1,800,000 record/sec(TCH)
  32. 32. Read 1,000,000 record/sec(TCB)
  33. 33. When the data more than 100 million, will performance reduction
  34. 34. Network protocol is not perfect</li></ul>TC/TT table engine insufficiency<br />
  35. 35. <ul><li>One port,more table
  36. 36. Row access and column access
  37. 37. B+ tree data store
  38. 38. New binary api protocol
  39. 39. New data storage mode
  40. 40. Config TCH
  41. 41. Index  TCB
  42. 42. Data  TCB</li></ul>About TCDatabase<br />
  43. 43. ActiveTokyoCabinet (1)<br />Config<br />
  44. 44. Model<br />ActiveTokyoCabinet (2)<br />
  45. 45. database.yml<br />development:<br /> adapter: tcdb<br /> path: ./db<br /> database: blog<br />model<br />class Post < ActiveRecord::Base<br />validates_presence_of :body, :title<br />has_many :comments<br />end<br />About tcdb-adapter (1)<br />
  46. 46. Migration<br />class CreatePosts < ActiveRecord::Migration<br /> def self.up<br />create_table :posts do |t|<br />t.string :title<br />t.text :body<br />t.timestamps<br /> end<br /> end<br /> def self.down<br />drop_table :posts<br /> end<br />end<br />About tcdb-adapter (2)<br />
  47. 47. Demo<br />
  48. 48. SQLite MySQL Oracle<br />Money<br />MySQL MySQL MySQL<br />DBA<br />RDBMS  NoSQL<br />Risk<br />RDBMS  tcdb-adapter  NoSQL<br />SQL/NoSQL Bridge<br />Storage architecture migration<br />
  49. 49. Architecture diagram (today)<br />Classic<br />ActiveTokyoCabinet<br />tcdb-adapter<br />
  50. 50. <ul><li>TCDatabase Cloud Database</li></ul>Transparent distributed<br />Running like mysql proxy<br />Access to TC / MongoDB / MySQL / more<br />Configuration and management support<br /><ul><li>Tcdb-adapter  Cloud ActiveRecord</li></ul>Almost support all of the SQL<br />Rails3 support (based arel)<br />Intergration thinking sphinx<br />One Interface, One architecture!<br /><ul><li>You can imagine more …</li></ul>The future<br />
  51. 51. Architecture diagram (future - 1)<br />
  52. 52. Architecture diagram (future - 2)<br />
  53. 53. Architecture diagram (future - 3)<br />
  54. 54. It will be open source!<br />
  55. 55. Thanks<br />

×