Patterns for key-value stores

4,235 views

Published on

javazone, 2012, redis, riak

Published in: Technology, Business
1 Comment
4 Likes
Statistics
Notes
  • good information on Key Value stores

    http://www.orderacab,co.uk
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,235
On SlideShare
0
From Embeds
0
Number of Embeds
57
Actions
Shares
0
Downloads
36
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide

Patterns for key-value stores

  1. PATTERNS FORKEY-VALUE STORE JavaZone 2012 Ole-Martin Mørk 13/09/12
  2. OM MEGOle-Martin MørkScientistBekk Consulting AStwitter: olemartin
  3. AGENDA INTRODUCTION DIFFERENT KV STORES GENERAL PATTERNS RIAK PATTERNS REDIS PATTERNS RIAK+REDIS PATTERNS SUMMARY
  4. MAP KEY VALUE session_1 Functional Programming You Already Know session_2 Fra enkel J2SE til Grid computing med GigaSpaces XAP session_3 Reinventing software quality session_4 Taming Java Agents session_5 Memory Leaks in Java session_6 Linking data without common identifiers session_7 Avinstaller Java nå!
  5. PUT
  6. GET
  7. DEL
  8. KEYVALUESTORE
  9. select valuefrom lookup where key=?
  10. HVORFOR BENYTTE KEY-VALUE STORES? 1 0 Speed
  11. Distribution
  12. HVORFOR BENYTTE KEY-VALUE STORES? 1 2 When all you have is a hammer..
  13. PERSON OG ADRESSER Person Epost Tlf Adresse Twitter Facebook
  14. navn: “Ole-Martin Mørk”, eposter: epost: ”me@ole - martin.net” type: “jobb”, epost: “ole-martin.mork@bekk.no” tlf: type: “mobil”, tlf: “90252338” adresser: type: “hjem” adresse: gate: “Lundliveien 30b”, sted: “Oslo” type: “jobb” adresse: postboks: “postboks 134, Sentrum”, sted: “Oslo” postnummer: “0150”
  15. {navn: “Ole-Martin Mørk”, eposter: [ { epost: ”me@ole - martin.net” }, { type: “jobb”, epost: “ole-martin.mork@bekk.no” }], tlf: [{ type: “mobil”, tlf: “90252338” }], adresser: [{ type: “hjem”, adresse: { gate: “Lundliveien 30b”, sted: “Oslo” } }, { type: “jobb”, adresse: { postboks: “postboks 134, Sentrum”, sted: “Oslo”, postnummer: “0150” } }]}
  16. HTML5 LOCALSTORAGE Demo Klikk!
  17. CASSANDRA 1 9
  18. DETTE ER FINE DATABASER 2 0 Why?
  19. PATTERNS
  20. Keys
  21. WELL-FORMED KEYS app:space:what:id
  22. LOOKUPS ON OTHER DATA 2 4 user:id:1234 userId 1234 username olemartin email olemartin@bekk.no twitter olemartin user:twitter:olemartin user:id:1234
  23. RELATIONS user:id:1234 userId 1234 username olemartin email olemartin@bekk.no twitter olemartin user:id:4321 userId 4321 username grolisbeth brother user:id:1234
  24. Hash
  25. HASHING <hash> modulo <antall servere> = serverId
  26. HASHING 43 4 3 32 2 2 11 1
  27. KONSISTENT HASHING A B C
  28. KONSISTENT HASHING A B C
  29. KONSISTENT HASHING A D D C A C B C B
  30. Use cases
  31. EKSEMPLER PÅ APPLIKASJONER SOM KAN KLE K/VRDBMS + KEY/VALUE
  32. RIAK Key Key Key Key Bucket Bucket Bucket
  33. ConsistencyAvailabilityP artition
  34. N W R
  35. N>W
  36. N>R
  37. PATTERNS
  38. RIAK OG MAP REDUCE
  39. MAP REDUCE shopping:13092012:userId
  40. SECONDARY INDEXES 4 6 Secondary indexes
  41. SECONDARY INDEXES 4 7 user:id:1234 userId 1234 username olemartin email olemartin@bekk.no twitter olemartin twitter:id_bin olemartin email:id_bin user:twitter:olemartin olemartin@bekk.no user:id:1234 user:sex_bin Male user:age_int 35
  42. Pre PostCommit Commit
  43. Hash Sorted Sets ListsSets Strings
  44. PATTERNS
  45. SELECT * FROM fooWHERE ...ORDER BY time DESC LIMIT 10
  46. PATTERNS FOR MILJØER MED RDBMS RDBMS REDIS
  47. STATISTIKKAntall sider lest av bruker<pages:userId, count>incr pages:userIdMest leste i dag:<most_read:300-2012, <url, count>,<url, count>>zrangebyscore most_read:300-2012 -inf +infUnike besøkende:<page:url:300-2012, <userId>, <userId>, <userId>>scard page:url:300-2012
  48. SERIALIZATION 100000 ms 90000 ms 80000 ms 70000 ms 60000 ms 50000 ms 40000 ms 30000 ms 20000 ms 10000 ms 0 ms
  49. KRYO IN REDIS<bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate"> <property name="connectionFactory" ref="connectionFactory"/> <property name="defaultSerializer" ref="serializer"/> </bean> <bean id="serializer" class="no.bekk.redis.DiggerSerializer"/>
  50. KRYOpublic class DiggerSerializer<T> implements RedisSerializer<T> { private Kryo kryo = new Kryo(); public DiggerSerializer() { kryo.register(Story.class); } public byte[] serialize(T t) { return new ObjectBuffer(kryo).writeClassAndObject(t); } public T deserialize(byte[] bytes) { return (T) new ObjectBuffer(kryo).readClassAndObject(bytes); } }
  51. REDDIT KLONE1.  story:id – holder styr på id-en til en story2.  story:id, value – holder selve story-objektet3.  story:stories_zset – holder styr på antall stemmer4.  story:tag:storyId – holder styr på tags knyttet til en story5.  tag:tagId – holder styr på stories knyttet til en tag
  52. REDIS OG RIAK + =
  53. Productcatalog Produktkatalog
  54. RIAK product:2 Products product:1
  55. POST COMMIT RSS Products Redis
  56. REDIS catalog:product:id binary data catalog:product:today:sold_zset <productId, counter> catalog:product:week:sold_zset <productId, counter> catalog:product:year:sold_zset <productId, counter>
  57. RESULT
  58. Summary
  59. SUMMARY1.  KV-stores kommer i alle farger og fasonger2.  Gode nøkkelnavn3.  Bruk god tid på datamodellering4.  Konsistent hashing5.  Bruk KV-store sammen med RDBMS6.  Bruk KV-store for domener som matcher7.  Bruk databasen til det den er god til8.  Kombiner gjerne flere type KV-stores9.  La kreativiteten blomstre
  60. 6 9Questions?
  61. TAKK FOR MEG! Ole-Martin Mørkole-martin.mork@bekk.no @olemartin

×