Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Versatile Storage Options
with Tokyo Cabinet
NoSQL Live
March 11, 2010
Flinn Mueller
Tokyo Cabinet
• Mikio Hirabayashi
• Mixi.jp
• LGPL
• C99/Posix
• Successor to QDBM
• mmap
Hash DB

• Key/Value
• Strings
• Extensions (Lua)
• Map/Reduce
• Forward Matching Keys
B+ Tree Store

• Key/Value
• Duplicate Keys
• Forward Matching Keys
• (Perfect Tag Store)
Fixed Length DB

• Key/Value
• Keys As Natural Number
• Value Length Limited
• Faster than Hash or B+ Tree
• Range Query
Table Store

• Tuple/Document Store
• Query (condition, order, limit)
• Full Text Search
• Union, Intersect, Diff
• Parall...
How I Use Tokyo

• Like Memcache++
• Like a Queue
• Like a Counter
• Like a Tag Cloud
{}
Like Memcache++
require 'tokyo_tyrant'
t = TokyoTyrant::DB.new

t['example.com/lorem'] = "Lorem..."
t['example.com/lorem']...
Q
Like a Queue
require 'tokyo_tyrant'
tdb = TokyoTyrant::Table.new

tdb[tdb.genuid] = {
    'url' => "http://example.com",
 ...
Like a Queue

require 'tokyo_tyrant'
tdb = TokyoTyrant::Table.new

tasks = tdb.find{ |q|
    q.condition :queue, :streq, '...
i++
Like a Counter

require 'tokyo_tyrant'
COUNTER_DB = TokyoTyrant::DB.new
...

COUNTER_DB.add_int(id)
# => 82169
zomg
pictures

       lolcatz
   funny
Like a Tag Cloud
require 'tokyo_tyrant'
TAG_DB = TokyoTyrant::BDB.new
...

tags = ['foo', 'bar']
tags.collect!{ |tag| "tag...
Related Content
articles = TAG_DB.mget(article_ids)

unpack = proc{ |k,v| v = MessagePack.unpack(v); v['__id'] = k; v }
ar...
Versatile Storage Options


• Like Memcache++
• Like a Queue
• Like a Counter
• Like a Tag Cloud
More
          http://1978th.net/tokyocabinet
          http://1978th.net/tokyotyrant/
http://groups.google.com/group/toky...
Upcoming SlideShare
Loading in …5
×

Versatile Storage Options With Tokyo Cabinet

4,965 views

Published on

Slides from my lightning talk at NoSQL Live Boston.

Published in: Technology, Business
  • I've added the audio from the presentation here:
    http://actsasflinn.com/post/473806917/versatile-storage-options-with-tokyo-cabinet
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Versatile Storage Options With Tokyo Cabinet

  1. 1. Versatile Storage Options with Tokyo Cabinet NoSQL Live March 11, 2010 Flinn Mueller
  2. 2. Tokyo Cabinet • Mikio Hirabayashi • Mixi.jp • LGPL • C99/Posix • Successor to QDBM • mmap
  3. 3. Hash DB • Key/Value • Strings • Extensions (Lua) • Map/Reduce • Forward Matching Keys
  4. 4. B+ Tree Store • Key/Value • Duplicate Keys • Forward Matching Keys • (Perfect Tag Store)
  5. 5. Fixed Length DB • Key/Value • Keys As Natural Number • Value Length Limited • Faster than Hash or B+ Tree • Range Query
  6. 6. Table Store • Tuple/Document Store • Query (condition, order, limit) • Full Text Search • Union, Intersect, Diff • Parallel Query
  7. 7. How I Use Tokyo • Like Memcache++ • Like a Queue • Like a Counter • Like a Tag Cloud
  8. 8. {}
  9. 9. Like Memcache++ require 'tokyo_tyrant' t = TokyoTyrant::DB.new t['example.com/lorem'] = "Lorem..." t['example.com/lorem'] # => "Lorem..." t['example.com/lorem/ipsum'] = "Ipsum..." t.fwmkeys('example.com/') # => ['example.com/lorem/ipsum', 'example.com/lorem']
  10. 10. Q
  11. 11. Like a Queue require 'tokyo_tyrant' tdb = TokyoTyrant::Table.new tdb[tdb.genuid] = { 'url' => "http://example.com", 'queue' => "1m", 'created_at' => Time.now.to_i, 'origin' => Socket.hostname }
  12. 12. Like a Queue require 'tokyo_tyrant' tdb = TokyoTyrant::Table.new tasks = tdb.find{ |q| q.condition :queue, :streq, '1m' q.order_by :created_at }.each{ |task| run_process(task) }
  13. 13. i++
  14. 14. Like a Counter require 'tokyo_tyrant' COUNTER_DB = TokyoTyrant::DB.new ... COUNTER_DB.add_int(id) # => 82169
  15. 15. zomg pictures lolcatz funny
  16. 16. Like a Tag Cloud require 'tokyo_tyrant' TAG_DB = TokyoTyrant::BDB.new ... tags = ['foo', 'bar'] tags.collect!{ |tag| "tag:#{tag}:articles" } hash_tags = TAG_DB.getlist(tags) article_ids = hash_tags.values.collect.flatten article_ids = article_ids.uniq.sort.reverse[0..4] articles = TAG_DB.mget(article_ids)
  17. 17. Related Content articles = TAG_DB.mget(article_ids) unpack = proc{ |k,v| v = MessagePack.unpack(v); v['__id'] = k; v } articles = articles.collect(&unpack).compact articles = articles.sort_by{ |v| v['__id'] } articles.reverse! content = Mustache.render(@@template, :articles => articles) response = [200, {"Content-Type" => "text/html"}, [content]]
  18. 18. Versatile Storage Options • Like Memcache++ • Like a Queue • Like a Counter • Like a Tag Cloud
  19. 19. More http://1978th.net/tokyocabinet http://1978th.net/tokyotyrant/ http://groups.google.com/group/tokyocabinet-users/ http://tokyocabinetwiki.pbworks.com/ http://github.com/actsasflinn/ruby-tokyotyrant http://github.com/jmettraux/rufus-tokyo/

×