Key value stores are popping all around the web, describing themselves as the best fit for your webapp… but take care of the hype ! Think you’ll have 10000+ writes a sec ? There’s a lot of unsaid “special features” that you have to know. Don’t trust benchmarks, even your own. Here’s how to choose the right key value store for your app ! The video is also available on youtube: http://www.youtube.com/watch?v=YZD8-EzozKQ

1. Beware of your Hype – Value store !
Ignite Velocity 2009
Jérémie BORDIER -
@ahfeel / jeremie.bordier@exalead.com

2. Get the Hype !
• Rela&onal DBMS are so 1990…
• So simple, so fast, looks powerful !
BerkeleyDB Persevere
Cassandra
Hypertable Project Voldemort
Redis Dynomite
SimpleDB MemcacheDB
Dynamo
Scalaris
Tokyo Tyrant CouchDB MongoDB
2

3. 10 000+ writes / sec !
• RDBMS only performs ±300 w / sec… wC ?
Let’s do some simple math !
3

4. State of the art mathema8cs
• Average SATA II disk seek &me: ±6ms
• Very good SCSI disk: ±3ms
1 second / 3ms =
±300 REAL writes / sec
4

5. Eventual Persistency
• Doesn’t sync the file system
• Keep that in mind !
Your servers WILL crash, you WILL lose data
5

6. Don’t care about writes…
• If can do 30 000+ lookup / sec !
Not for long !
6

7. Eventual memory
• Relies on B+Trees
• Map all the data structures in memory
What if your data goes too big ?
7

8. Lookup…
• get(“elevator”)
– Visit 8 nodes
– Read the data from disk
A R
E S O Data:
L 42
V T
E A O
S M
8

9. Why elevator ?
• American elevators are wayyyy too fast !
• Feels like a NASA training
Makes me want to throw up :(
9

10. Hardware limit !
Up to 9 RANDOM I/Os per lookup !
1 second / (9 * 3ms) = 37 get / sec
• On very good hardware !
10

11. We did that too :)
• We had similar algorithms
• Encountered horrible perf decreases…
REDUCE I/O !
Ensure only 1 I/O per lookup.
11

12. Don’t trust benchmarks
• A few million entries isn’t enough
• You’re benchmarking your Disk / OS
cache :)
(Common, DON’T BENCHMARK IN RUBY…)
12

13. Compare what’s comparable
• Distributed column stores
– BigTable like systems
• Key value stores
– Tokyo, Dynamo like systems
13

14. Distributed column stores
Google Megastore
Persistent
Hypertable SimpleDB
Eventually Replicated
Persistent
Mature Distributed
Cassandra HBase
14

15. Key – value stores
Persevere
BerkeleyDB
Mutable
MongoDB
Persistent Replicated
Dynomite
Embedded Distributed
Tokyo Cabinet Eventually
Scalaris
Mature
Persistent
MemcacheDB Immutable Redis
CouchDB
Voldemort 15

16. How to choose ?
• Maturity is priceless
• Most suitable stores:
– Persistent: BerkeleyDB, MySQL
– Ev. Persistent: Tokyo Cabinet
– Ev. Persistent + Distributed + … : Voldemort
– Distributed column store: Cassandra
16

17. You are not Google
(Well, not all of you )
• Build something
• Make it work
• Think about scaling
• Think about being hyped
Why ?
17

18. Key value pain…
• You will end up crossing data.
• Doing joins with KV stores ?
Only for ninjas !
18

19. Query 8me joins
• Coding what RDBMS are made for ? …
• Slow !
(you shouldn’t do this…)
19

20. Build 8me schema flaTening
• Not flexible !
• Needs Map Reduce (Hadoop…) to scale
Think before doing this :)
20

21. And finally..
• HRD hard drive: 160 000 random IO/sec !!!
• Yahoo announced open sourcing Sherpa !
• Resources
– hip://developer.yahoo.net/blog/archives/2009/06/nosql_meetup.html
– hip://metabrew.com/ar&cle/an&‐rdbms‐a‐list‐of‐distributed‐key‐value‐stores/
– hip://www.ryanpark.org/2008/04/top‐10‐avoid‐the‐simpledb‐hype.html
– hip://project‐voldemort.com/blog/2009/06/building‐a‐1‐tb‐data‐cycle‐at
‐linkedin‐with‐hadoop‐and‐project‐voldemort/
Thanks !
Contact: @ahfeel :) 21