Versatile Storage Options
with Tokyo Cabinet
NoSQL Live
March 11, 2010
Flinn Mueller
Tokyo Cabinet
• Mikio Hirabayashi
• Mixi.jp
• LGPL
• C99/Posix
• Successor to QDBM
• mmap
Hash DB

• Key/Value
• Strings
• Extensions (Lua)
• Map/Reduce
• Forward Matching Keys
B+ Tree Store

• Key/Value
• Duplicate Keys
• Forward Matching Keys
• (Perfect Tag Store)
Fixed Length DB

• Key/Value
• Keys As Natural Number
• Value Length Limited
• Faster than Hash or B+ Tree
• Range Query
Table Store

• Tuple/Document Store
• Query (condition, order, limit)
• Full Text Search
• Union, Intersect, Diff
• Parall...
How I Use Tokyo

• Like Memcache++
• Like a Queue
• Like a Counter
• Like a Tag Cloud
{}
Like Memcache++
require 'tokyo_tyrant'
t = TokyoTyrant::DB.new

t['example.com/lorem'] = "Lorem..."
t['example.com/lorem']...
Q
Like a Queue
require 'tokyo_tyrant'
tdb = TokyoTyrant::Table.new

tdb[tdb.genuid] = {
    'url' => "http://example.com",
 ...
Like a Queue

require 'tokyo_tyrant'
tdb = TokyoTyrant::Table.new

tasks = tdb.find{ |q|
    q.condition :queue, :streq, '...
i++
Like a Counter

require 'tokyo_tyrant'
COUNTER_DB = TokyoTyrant::DB.new
...

COUNTER_DB.add_int(id)
# => 82169
zomg
pictures

       lolcatz
   funny
Like a Tag Cloud
require 'tokyo_tyrant'
TAG_DB = TokyoTyrant::BDB.new
...

tags = ['foo', 'bar']
tags.collect!{ |tag| "tag...
Related Content
articles = TAG_DB.mget(article_ids)

unpack = proc{ |k,v| v = MessagePack.unpack(v); v['__id'] = k; v }
ar...
Versatile Storage Options


• Like Memcache++
• Like a Queue
• Like a Counter
• Like a Tag Cloud
More
          http://1978th.net/tokyocabinet
          http://1978th.net/tokyotyrant/
http://groups.google.com/group/toky...
Upcoming SlideShare
Loading in …5
×

Versatile Storage Options With Tokyo Cabinet

4,800 views

Published on

Slides from my lightning talk at NoSQL Live Boston.

Published in: Technology, Business
1 Comment
6 Likes
Statistics
Notes
  • I've added the audio from the presentation here:
    http://actsasflinn.com/post/473806917/versatile-storage-options-with-tokyo-cabinet
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,800
On SlideShare
0
From Embeds
0
Number of Embeds
1,664
Actions
Shares
0
Downloads
0
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide



















  • Versatile Storage Options With Tokyo Cabinet

    1. 1. Versatile Storage Options with Tokyo Cabinet NoSQL Live March 11, 2010 Flinn Mueller
    2. 2. Tokyo Cabinet • Mikio Hirabayashi • Mixi.jp • LGPL • C99/Posix • Successor to QDBM • mmap
    3. 3. Hash DB • Key/Value • Strings • Extensions (Lua) • Map/Reduce • Forward Matching Keys
    4. 4. B+ Tree Store • Key/Value • Duplicate Keys • Forward Matching Keys • (Perfect Tag Store)
    5. 5. Fixed Length DB • Key/Value • Keys As Natural Number • Value Length Limited • Faster than Hash or B+ Tree • Range Query
    6. 6. Table Store • Tuple/Document Store • Query (condition, order, limit) • Full Text Search • Union, Intersect, Diff • Parallel Query
    7. 7. How I Use Tokyo • Like Memcache++ • Like a Queue • Like a Counter • Like a Tag Cloud
    8. 8. {}
    9. 9. Like Memcache++ require 'tokyo_tyrant' t = TokyoTyrant::DB.new t['example.com/lorem'] = "Lorem..." t['example.com/lorem'] # => "Lorem..." t['example.com/lorem/ipsum'] = "Ipsum..." t.fwmkeys('example.com/') # => ['example.com/lorem/ipsum', 'example.com/lorem']
    10. 10. Q
    11. 11. Like a Queue require 'tokyo_tyrant' tdb = TokyoTyrant::Table.new tdb[tdb.genuid] = { 'url' => "http://example.com", 'queue' => "1m", 'created_at' => Time.now.to_i, 'origin' => Socket.hostname }
    12. 12. Like a Queue require 'tokyo_tyrant' tdb = TokyoTyrant::Table.new tasks = tdb.find{ |q| q.condition :queue, :streq, '1m' q.order_by :created_at }.each{ |task| run_process(task) }
    13. 13. i++
    14. 14. Like a Counter require 'tokyo_tyrant' COUNTER_DB = TokyoTyrant::DB.new ... COUNTER_DB.add_int(id) # => 82169
    15. 15. zomg pictures lolcatz funny
    16. 16. Like a Tag Cloud require 'tokyo_tyrant' TAG_DB = TokyoTyrant::BDB.new ... tags = ['foo', 'bar'] tags.collect!{ |tag| "tag:#{tag}:articles" } hash_tags = TAG_DB.getlist(tags) article_ids = hash_tags.values.collect.flatten article_ids = article_ids.uniq.sort.reverse[0..4] articles = TAG_DB.mget(article_ids)
    17. 17. Related Content articles = TAG_DB.mget(article_ids) unpack = proc{ |k,v| v = MessagePack.unpack(v); v['__id'] = k; v } articles = articles.collect(&unpack).compact articles = articles.sort_by{ |v| v['__id'] } articles.reverse! content = Mustache.render(@@template, :articles => articles) response = [200, {"Content-Type" => "text/html"}, [content]]
    18. 18. Versatile Storage Options • Like Memcache++ • Like a Queue • Like a Counter • Like a Tag Cloud
    19. 19. More http://1978th.net/tokyocabinet http://1978th.net/tokyotyrant/ http://groups.google.com/group/tokyocabinet-users/ http://tokyocabinetwiki.pbworks.com/ http://github.com/actsasflinn/ruby-tokyotyrant http://github.com/jmettraux/rufus-tokyo/

    ×