Your SlideShare is downloading. ×
0
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Tokyo Cabinet / Tyrant @ Nosql Paris
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Tokyo Cabinet / Tyrant @ Nosql Paris

4,644

Published on

Published in: Technology
0 Comments
13 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,644
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
104
Comments
0
Likes
13
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide


























  • Transcript

    • 1. Feedbacks: Tokyo Cabinet & Tyrant Florent Solt — Netvibes florent@solt.biz — twitter.com/florentsolt
    • 2. Tokyo *what* ? Tokyo Cabinet http://1978th.net/tokyocabinet/ « Tokyo Cabinet is a library of routines for managing a database » 1 database = 1 file key-value storage written in C, official packages in Perl, Ruby, Java, Lua
    • 3. Tokyo Cabinet Memory or Filesystem Compression bzip or gzip Dynamic defragmentation Hash, BTree, Fixed-Length, Table
    • 4. Tokyo Cabinet — Hash
    • 5. Tokyo Cabinet — BTree
    • 6. Tokyo Cabinet — Fixed
    • 7. Tokyo Cabinet — Table
    • 8. Tokyo *what* ? Tokyo Tyrant http://1978th.net/tokyotyrant/ « Tokyo Tyrant is a package of network interface to the DBM called Tokyo Cabinet » binary, http and memcached protocols epoll / kqueue / eventports Written in C, official packages in Perl and Ruby
    • 9. Tokyo Tyrant No configuration files (command line only) Lua extention Master/Slave and Master/Master async replication Backup (via the copy fonction) 1 process = 1 database
    • 10. Examples Run a memory hash server (same as memcached) ttserver Sample queries memcat --servers=127.0.0.1:1978 1 curl 127.0.0.1:1978/1
    • 11. Examples Run a btree on port 11211 (to simulate a memcache server) and save all data compressed on the disk ttserver -dmn -port 11211 -log /path/to/server.log /path/ to/cache.tcb#opts=ld#bnum=1000000
    • 12. Examples Run a table with some indexes and enable master- master replication Master 1 mkdir ulog1; ttserver -dmn -port 1978 -ulog ulog1 -sid 1 -mhost localhost -mport 1979 -rts 1.rts /path/to/ table1.tct#idx=foo:dec#idx=bar Master 2 mkdir ulog2; ttserver -dmn -port 1979 -ulog ulog2 -sid 2 -mhost localhost -mport 1978 -rts 2.rts /path/to/ table2.tct#idx=foo:dec#idx=bar
    • 13. Benchmarks or not Benchmarks ?
    • 14. Netvibes architecture Queue Feed Sort Item Subscriber Flag Enclosure User
    • 15. Netvibes architecture Table + Hash + Lua Table BTree Shard Queue Feed Sort Item Hash + Lua Subscriber Flag Enclosure BTree + Lua + Table Shard User Table
    • 16. Netvibes architecture Hundreds Millions Millions Millions of millions Queue Feed Sort Item Subscriber Flag Enclosure Hundreds Dozens of millions of millions User
    • 17. Queue implementation — Example multi queue (via prefix) support « adddouble » function helps to avoid conflicts « putkeep » function prevent overwriting
    • 18. Queue implementation — Example « fwmkeys » finds all items with the given prefix multiple dequeue support
    • 19. Queue implementation — Netvibes One queue / minute Automatic slot change When a queue empty and not in the slot representing the current time No duplicates Check if an item is already in the queue
    • 20. Strengths and Weaknesses
    • 21. Strengths and Weaknesses Weaknesses No bug tracker, no public code repository The documentation is not good enough Under heavy load, the master-master replication failed, we fallback to Master-Slave (and still have some issues) Databases can be corrupted Process killed, System crash or reboot
    • 22. Strengths and Weaknesses Weaknesses With big tables, queries need a lot of RAM and time Tables seems slow, we may avoid them in the future and prefer Hash or BTree Need a lots of time to understand all configuration options and their impact No live backup, the copy function lock the database
    • 23. Strengths and Weaknesses Strengths Easy to deploy and setup Easy to use It’s not a black box Good to very good performance for most of the time
    • 24. Strengths and Weaknesses Strengths Small memory footprint A single Tyrant process can handle thousands of open connections Many command line tools Lua extensions
    • 25. And so what ?
    • 26. Links Google groups of english spoken users http://groups.google.com/group/tokyocabinet-users Wiki: unofficial page to collect links http://tokyocabinetwiki.pbworks.com/ Blog of Ilya Grigorik http://www.igvita.com/2009/02/13/tokyo-cabinet- beyond-key-value-store http://www.igvita.com/2009/07/13/extending-tokyo- cabinet-db-with-lua/

    ×