Beware of your Hype Value Stores

Beware of your Hype – Value store ! Ignite Velocity 2009 Jérémie BORDIER - @ahfeel / jeremie.bordier@exalead.com

Get the Hype !  •  Rela&onal DBMS are so 1990…  •  So simple, so fast, looks powerful !  BerkeleyDB Persevere Cassandra Hypertable Project Voldemort Redis Dynomite SimpleDB MemcacheDB Dynamo Scalaris Tokyo Tyrant CouchDB MongoDB 2

10 000+ writes / sec !  •  RDBMS only performs ±300 w / sec… wC ?  Let’s do some simple math ! 3

State of the art mathema8cs  •  Average SATA II disk seek &me: ±6ms  •  Very good SCSI disk: ±3ms  1 second / 3ms = ±300 REAL writes / sec 4

Eventual Persistency  •  Doesn’t sync the ﬁle system  •  Keep that in mind !  Your servers WILL crash, you WILL lose data  5

Don’t care about writes…  •  If can do 30 000+ lookup / sec !  Not for long ! 6

Eventual memory  •  Relies on B+Trees  •  Map all the data structures in memory  What if your data goes too big ? 7

Lookup…  •  get(“elevator”)  –  Visit 8 nodes  –  Read the data from disk  A R E S O Data: L 42  V T E A O S M 8

Why elevator ?  •  American elevators are wayyyy too fast !  •  Feels like a NASA training  Makes me want to throw up :(   9

Hardware limit !  Up to 9 RANDOM I/Os per lookup ! 1 second / (9 * 3ms) = 37 get / sec •  On very good hardware !  10

We did that too :)  •  We had similar algorithms  •  Encountered horrible perf decreases…  REDUCE I/O ! Ensure only 1 I/O per lookup. 11

Don’t trust benchmarks  •  A few million entries isn’t enough  •  You’re benchmarking your Disk / OS  cache :)  (Common, DON’T BENCHMARK IN RUBY…) 12

Compare what’s comparable  •  Distributed column stores  –  BigTable like systems  •  Key value stores  –  Tokyo, Dynamo like systems  13

Distributed column stores  Google Megastore Persistent Hypertable SimpleDB Eventually Replicated Persistent Mature Distributed Cassandra HBase 14

Key – value stores  Persevere BerkeleyDB Mutable MongoDB Persistent Replicated Dynomite Embedded Distributed Tokyo Cabinet Eventually Scalaris Mature Persistent MemcacheDB Immutable Redis CouchDB Voldemort 15

How to choose ?  •  Maturity is priceless  •  Most suitable stores:  –  Persistent: BerkeleyDB, MySQL  –  Ev. Persistent: Tokyo Cabinet  –  Ev. Persistent + Distributed + … : Voldemort  –  Distributed column store: Cassandra  16

You are not Google  (Well, not all of you ) •  Build something  •  Make it work  •  Think about scaling  •  Think about being hyped  Why ? 17

Key value pain…  •  You will end up crossing data.  •  Doing joins with KV stores ?  Only for ninjas ! 18

Query 8me joins   •  Coding what RDBMS are made for ? …  •  Slow !  (you shouldn’t do this…) 19

Build 8me schema ﬂaTening   •  Not ﬂexible !  •  Needs Map Reduce (Hadoop…) to scale  Think before doing this :) 20

And ﬁnally..  •  HRD hard drive: 160 000 random IO/sec !!!  •  Yahoo announced open sourcing Sherpa !  •  Resources  –  hip://developer.yahoo.net/blog/archives/2009/06/nosql_meetup.html  –  hip://metabrew.com/ar&cle/an&‐rdbms‐a‐list‐of‐distributed‐key‐value‐stores/  –  hip://www.ryanpark.org/2008/04/top‐10‐avoid‐the‐simpledb‐hype.html  –  hip://project‐voldemort.com/blog/2009/06/building‐a‐1‐tb‐data‐cycle‐at ‐linkedin‐with‐hadoop‐and‐project‐voldemort/  Thanks ! Contact: @ahfeel :) 21

http://www.altaidevalley.com	42
http://www.slideshare.net	18
http://www.linkedin.com	6
http://altaide.typepad.com	1

Beware of your Hype Value Stores

by Jérémie BORDIER

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 67

Statistics

Beware of your Hype Value Stores Presentation Transcript