Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
PATTERNS FORKEY-VALUE STORE   JavaZone 2012   Ole-Martin Mørk       13/09/12
OM MEGOle-Martin MørkScientistBekk Consulting AStwitter: olemartin
AGENDA         INTRODUCTION         DIFFERENT KV STORES         GENERAL PATTERNS         RIAK PATTERNS         REDIS PATTE...
MAP KEY         VALUE session_1   Functional Programming You Already Know session_2   Fra enkel J2SE til Grid computing me...
PUT
GET
DEL
KEYVALUESTORE
select valuefrom lookup 	where key=?
HVORFOR BENYTTE KEY-VALUE STORES?   1                                    0   Speed
Distribution
HVORFOR BENYTTE KEY-VALUE STORES?                   1                                                    2                ...
PERSON OG ADRESSER                     Person      Epost                      Tlf                     Adresse      Twitter...
navn: “Ole-Martin Mørk”,	     	eposter:          epost: ”me@ole - martin.net”   	     	   	    	     	   type: “jobb”, epo...
{navn: “Ole-Martin Mørk”,	  	eposter: [ { epost: ”me@ole - martin.net” }, 	  	   	     	   	{ type: “jobb”, epost: “ole-ma...
HTML5 LOCALSTORAGE            Demo     Klikk!
CASSANDRA   1            9
DETTE ER FINE DATABASER   2                          0         Why?
PATTERNS
Keys
WELL-FORMED KEYS    app:space:what:id
LOOKUPS ON OTHER DATA                                       2                                                            4...
RELATIONS   user:id:1234                  userId     1234                  username   olemartin                  email    ...
Hash
HASHING <hash> modulo <antall servere> = serverId
HASHING          43           4   3                   32 2                      2                          11   1
KONSISTENT HASHING                     A                         B             C
KONSISTENT HASHING                     A                         B             C
KONSISTENT HASHING                     A                 D            D                 C       A                 C       ...
Use cases
EKSEMPLER PÅ APPLIKASJONER SOM KAN KLE K/VRDBMS + KEY/VALUE
RIAK                  Key                   Key                     Key                     Key       Bucket         Bucke...
ConsistencyAvailabilityP artition
N W R
N>W
N>R
PATTERNS
RIAK OG MAP REDUCE
MAP REDUCE   shopping:13092012:userId
SECONDARY INDEXES               4                                6                    Secondary                     indexes
SECONDARY INDEXES                                         4                                                          7   u...
Pre     PostCommit   Commit
Hash               Sorted Sets       ListsSets            Strings
PATTERNS
SELECT * FROM fooWHERE ...ORDER BY time DESC LIMIT 10
PATTERNS FOR MILJØER MED RDBMS   RDBMS                    REDIS
STATISTIKKAntall sider lest av bruker<pages:userId, count>incr pages:userIdMest leste i dag:<most_read:300-2012, <url, cou...
SERIALIZATION         100000 ms          90000 ms          80000 ms          70000 ms          60000 ms          50000 ms ...
KRYO IN REDIS<bean id="redisTemplate"	    class="org.springframework.data.redis.core.RedisTemplate">	    <property name="c...
KRYOpublic class DiggerSerializer<T> implements RedisSerializer<T> {	     private Kryo kryo = new Kryo();	     public Digg...
REDDIT KLONE1.  story:id – holder styr på id-en til en story2.  story:id, value – holder selve story-objektet3.  story:sto...
REDIS OG RIAK                +                =
Productcatalog     Produktkatalog
RIAK                    product:2       Products   product:1
POST COMMIT                    RSS         Products         Redis
REDIS        catalog:product:id    binary data        catalog:product:today:sold_zset <productId, counter>        catalog:...
RESULT
Summary
SUMMARY1.  KV-stores kommer i alle farger og fasonger2.  Gode nøkkelnavn3.  Bruk god tid på datamodellering4.  Konsistent ...
6             9Questions?
TAKK FOR     MEG!    Ole-Martin Mørkole-martin.mork@bekk.no      @olemartin
Patterns for key-value stores
Patterns for key-value stores
Patterns for key-value stores
Patterns for key-value stores
Patterns for key-value stores
Patterns for key-value stores
Patterns for key-value stores
Patterns for key-value stores
Patterns for key-value stores
Upcoming SlideShare
Loading in …5
×

Patterns for key-value stores

5,447 views

Published on

javazone, 2012, redis, riak

Published in: Technology, Business
  • good information on Key Value stores

    http://www.orderacab,co.uk
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Patterns for key-value stores

  1. PATTERNS FORKEY-VALUE STORE JavaZone 2012 Ole-Martin Mørk 13/09/12
  2. OM MEGOle-Martin MørkScientistBekk Consulting AStwitter: olemartin
  3. AGENDA INTRODUCTION DIFFERENT KV STORES GENERAL PATTERNS RIAK PATTERNS REDIS PATTERNS RIAK+REDIS PATTERNS SUMMARY
  4. MAP KEY VALUE session_1 Functional Programming You Already Know session_2 Fra enkel J2SE til Grid computing med GigaSpaces XAP session_3 Reinventing software quality session_4 Taming Java Agents session_5 Memory Leaks in Java session_6 Linking data without common identifiers session_7 Avinstaller Java nå!
  5. PUT
  6. GET
  7. DEL
  8. KEYVALUESTORE
  9. select valuefrom lookup where key=?
  10. HVORFOR BENYTTE KEY-VALUE STORES? 1 0 Speed
  11. Distribution
  12. HVORFOR BENYTTE KEY-VALUE STORES? 1 2 When all you have is a hammer..
  13. PERSON OG ADRESSER Person Epost Tlf Adresse Twitter Facebook
  14. navn: “Ole-Martin Mørk”, eposter: epost: ”me@ole - martin.net” type: “jobb”, epost: “ole-martin.mork@bekk.no” tlf: type: “mobil”, tlf: “90252338” adresser: type: “hjem” adresse: gate: “Lundliveien 30b”, sted: “Oslo” type: “jobb” adresse: postboks: “postboks 134, Sentrum”, sted: “Oslo” postnummer: “0150”
  15. {navn: “Ole-Martin Mørk”, eposter: [ { epost: ”me@ole - martin.net” }, { type: “jobb”, epost: “ole-martin.mork@bekk.no” }], tlf: [{ type: “mobil”, tlf: “90252338” }], adresser: [{ type: “hjem”, adresse: { gate: “Lundliveien 30b”, sted: “Oslo” } }, { type: “jobb”, adresse: { postboks: “postboks 134, Sentrum”, sted: “Oslo”, postnummer: “0150” } }]}
  16. HTML5 LOCALSTORAGE Demo Klikk!
  17. CASSANDRA 1 9
  18. DETTE ER FINE DATABASER 2 0 Why?
  19. PATTERNS
  20. Keys
  21. WELL-FORMED KEYS app:space:what:id
  22. LOOKUPS ON OTHER DATA 2 4 user:id:1234 userId 1234 username olemartin email olemartin@bekk.no twitter olemartin user:twitter:olemartin user:id:1234
  23. RELATIONS user:id:1234 userId 1234 username olemartin email olemartin@bekk.no twitter olemartin user:id:4321 userId 4321 username grolisbeth brother user:id:1234
  24. Hash
  25. HASHING <hash> modulo <antall servere> = serverId
  26. HASHING 43 4 3 32 2 2 11 1
  27. KONSISTENT HASHING A B C
  28. KONSISTENT HASHING A B C
  29. KONSISTENT HASHING A D D C A C B C B
  30. Use cases
  31. EKSEMPLER PÅ APPLIKASJONER SOM KAN KLE K/VRDBMS + KEY/VALUE
  32. RIAK Key Key Key Key Bucket Bucket Bucket
  33. ConsistencyAvailabilityP artition
  34. N W R
  35. N>W
  36. N>R
  37. PATTERNS
  38. RIAK OG MAP REDUCE
  39. MAP REDUCE shopping:13092012:userId
  40. SECONDARY INDEXES 4 6 Secondary indexes
  41. SECONDARY INDEXES 4 7 user:id:1234 userId 1234 username olemartin email olemartin@bekk.no twitter olemartin twitter:id_bin olemartin email:id_bin user:twitter:olemartin olemartin@bekk.no user:id:1234 user:sex_bin Male user:age_int 35
  42. Pre PostCommit Commit
  43. Hash Sorted Sets ListsSets Strings
  44. PATTERNS
  45. SELECT * FROM fooWHERE ...ORDER BY time DESC LIMIT 10
  46. PATTERNS FOR MILJØER MED RDBMS RDBMS REDIS
  47. STATISTIKKAntall sider lest av bruker<pages:userId, count>incr pages:userIdMest leste i dag:<most_read:300-2012, <url, count>,<url, count>>zrangebyscore most_read:300-2012 -inf +infUnike besøkende:<page:url:300-2012, <userId>, <userId>, <userId>>scard page:url:300-2012
  48. SERIALIZATION 100000 ms 90000 ms 80000 ms 70000 ms 60000 ms 50000 ms 40000 ms 30000 ms 20000 ms 10000 ms 0 ms
  49. KRYO IN REDIS<bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate"> <property name="connectionFactory" ref="connectionFactory"/> <property name="defaultSerializer" ref="serializer"/> </bean> <bean id="serializer" class="no.bekk.redis.DiggerSerializer"/>
  50. KRYOpublic class DiggerSerializer<T> implements RedisSerializer<T> { private Kryo kryo = new Kryo(); public DiggerSerializer() { kryo.register(Story.class); } public byte[] serialize(T t) { return new ObjectBuffer(kryo).writeClassAndObject(t); } public T deserialize(byte[] bytes) { return (T) new ObjectBuffer(kryo).readClassAndObject(bytes); } }
  51. REDDIT KLONE1.  story:id – holder styr på id-en til en story2.  story:id, value – holder selve story-objektet3.  story:stories_zset – holder styr på antall stemmer4.  story:tag:storyId – holder styr på tags knyttet til en story5.  tag:tagId – holder styr på stories knyttet til en tag
  52. REDIS OG RIAK + =
  53. Productcatalog Produktkatalog
  54. RIAK product:2 Products product:1
  55. POST COMMIT RSS Products Redis
  56. REDIS catalog:product:id binary data catalog:product:today:sold_zset <productId, counter> catalog:product:week:sold_zset <productId, counter> catalog:product:year:sold_zset <productId, counter>
  57. RESULT
  58. Summary
  59. SUMMARY1.  KV-stores kommer i alle farger og fasonger2.  Gode nøkkelnavn3.  Bruk god tid på datamodellering4.  Konsistent hashing5.  Bruk KV-store sammen med RDBMS6.  Bruk KV-store for domener som matcher7.  Bruk databasen til det den er god til8.  Kombiner gjerne flere type KV-stores9.  La kreativiteten blomstre
  60. 6 9Questions?
  61. TAKK FOR MEG! Ole-Martin Mørkole-martin.mork@bekk.no @olemartin

×