• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Tokyo Cabinet / Tyrant @ Nosql Paris
 

Tokyo Cabinet / Tyrant @ Nosql Paris

on

  • 5,879 views

 

Statistics

Views

Total Views
5,879
Views on SlideShare
5,139
Embed Views
740

Actions

Likes
13
Downloads
100
Comments
0

9 Embeds 740

http://nosql.mypopescu.com 667
http://www.slideshare.net 54
http://irr.posterous.com 6
http://translate.googleusercontent.com 5
http://www.papiersdundev.com 3
http://www.hanrss.com 2
http://webcache.googleusercontent.com 1
http://static.slidesharecdn.com 1
https://d4y2u9.appspot.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Tokyo Cabinet / Tyrant @ Nosql Paris Tokyo Cabinet / Tyrant @ Nosql Paris Presentation Transcript

  • Feedbacks: Tokyo Cabinet & Tyrant Florent Solt — Netvibes florent@solt.biz — twitter.com/florentsolt
  • Tokyo *what* ? Tokyo Cabinet http://1978th.net/tokyocabinet/ « Tokyo Cabinet is a library of routines for managing a database » 1 database = 1 file key-value storage written in C, official packages in Perl, Ruby, Java, Lua
  • Tokyo Cabinet Memory or Filesystem Compression bzip or gzip Dynamic defragmentation Hash, BTree, Fixed-Length, Table
  • Tokyo Cabinet — Hash
  • Tokyo Cabinet — BTree
  • Tokyo Cabinet — Fixed
  • Tokyo Cabinet — Table
  • Tokyo *what* ? Tokyo Tyrant http://1978th.net/tokyotyrant/ « Tokyo Tyrant is a package of network interface to the DBM called Tokyo Cabinet » binary, http and memcached protocols epoll / kqueue / eventports Written in C, official packages in Perl and Ruby
  • Tokyo Tyrant No configuration files (command line only) Lua extention Master/Slave and Master/Master async replication Backup (via the copy fonction) 1 process = 1 database
  • Examples Run a memory hash server (same as memcached) ttserver Sample queries memcat --servers=127.0.0.1:1978 1 curl 127.0.0.1:1978/1
  • Examples Run a btree on port 11211 (to simulate a memcache server) and save all data compressed on the disk ttserver -dmn -port 11211 -log /path/to/server.log /path/ to/cache.tcb#opts=ld#bnum=1000000
  • Examples Run a table with some indexes and enable master- master replication Master 1 mkdir ulog1; ttserver -dmn -port 1978 -ulog ulog1 -sid 1 -mhost localhost -mport 1979 -rts 1.rts /path/to/ table1.tct#idx=foo:dec#idx=bar Master 2 mkdir ulog2; ttserver -dmn -port 1979 -ulog ulog2 -sid 2 -mhost localhost -mport 1978 -rts 2.rts /path/to/ table2.tct#idx=foo:dec#idx=bar
  • Benchmarks or not Benchmarks ?
  • Netvibes architecture Queue Feed Sort Item Subscriber Flag Enclosure User
  • Netvibes architecture Table + Hash + Lua Table BTree Shard Queue Feed Sort Item Hash + Lua Subscriber Flag Enclosure BTree + Lua + Table Shard User Table
  • Netvibes architecture Hundreds Millions Millions Millions of millions Queue Feed Sort Item Subscriber Flag Enclosure Hundreds Dozens of millions of millions User
  • Queue implementation — Example multi queue (via prefix) support « adddouble » function helps to avoid conflicts « putkeep » function prevent overwriting
  • Queue implementation — Example « fwmkeys » finds all items with the given prefix multiple dequeue support
  • Queue implementation — Netvibes One queue / minute Automatic slot change When a queue empty and not in the slot representing the current time No duplicates Check if an item is already in the queue
  • Strengths and Weaknesses
  • Strengths and Weaknesses Weaknesses No bug tracker, no public code repository The documentation is not good enough Under heavy load, the master-master replication failed, we fallback to Master-Slave (and still have some issues) Databases can be corrupted Process killed, System crash or reboot
  • Strengths and Weaknesses Weaknesses With big tables, queries need a lot of RAM and time Tables seems slow, we may avoid them in the future and prefer Hash or BTree Need a lots of time to understand all configuration options and their impact No live backup, the copy function lock the database
  • Strengths and Weaknesses Strengths Easy to deploy and setup Easy to use It’s not a black box Good to very good performance for most of the time
  • Strengths and Weaknesses Strengths Small memory footprint A single Tyrant process can handle thousands of open connections Many command line tools Lua extensions
  • And so what ?
  • Links Google groups of english spoken users http://groups.google.com/group/tokyocabinet-users Wiki: unofficial page to collect links http://tokyocabinetwiki.pbworks.com/ Blog of Ilya Grigorik http://www.igvita.com/2009/02/13/tokyo-cabinet- beyond-key-value-store http://www.igvita.com/2009/07/13/extending-tokyo- cabinet-db-with-lua/