Redis — The AK-47 of Post-relational Databases


Published on

Published in: Technology
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Redis — The AK-47 of Post-relational Databases

  1. Redis — The AK-47 Of Post-relational DatabasesKarel Minařík
  2. Karel Minařík→ Independent web designer and developer→ Ruby, Rails, Git, CouchDB propagandista in .cz→ Previously: Flash Developer; Art Director; Information Architect;… (see LinkedIn)→ @karmiq at Twitter→ Redis
  3. WEAPON DESIGNAK-47Designed at the end of WWII by Mikhail KalashnikovAssault Rifle, not a “submachine gun”Simple designDesigned for mass production & low-quality manufacturingExtreme reliability, at the cost of accuracy Redis
  4. Redis
  5. Reliability Redis
  6. SIMPLICITYThe Redis ManifestoWere against complexity.We believe designing systems is a fight against complexity.Most of the time the best way to fight complexity is by notcreating it at all. Redis
  7. SIMPLICITYRedis and NoSQLWhats wrong with RDBMS when used for (many) tasksthat dont need all this complexity? The data model: nonscalable, time complexity hard to predict, and cant modelmany common problems well enough. Redis
  8. Assault rifles as the crucial invention for new rules of warfare: fighting in shorter distances and shooting while on the run. Redis
  9. Memory is the new disk.Disk is the new tape.— Jim Gray Redis
  10. Redis: Main Features Redis
  11. Simplicity SpeedPredictability VersatilityReliability Low Footprint
  12. SIMPLICITYInstallationgit  clone  redismake./src/redis-­‐server./src/redis-­‐cli Redis
  13. SIMPLICITYThe Zen of Redis(...) what Redis provides are data structures (...) Redis
  14. REDISData StructuresStringsListsSetsSorted SetsHashes Redis
  15. REDIS DATA STRUCTURESStringsSET  key  "value"  ~  4  billion  of  keysGET  key=>  "value"DEL  key Redis
  16. REDIS DATA STRUCTURESFetch multiple keys at onceSET  key1  "value1"SET  key2  "value2"MGET  key1  key2=>  "value1"=>  "value2" Redis
  17. REDIS DATA STRUCTURESExpirationEXPIRE  key  5GET  key=>  "value"TTL  key=>  1GET  key=>  (nil) Redis
  18. UsageCache Redis
  19. REDIS DATA STRUCTURESAtomic IncrementsGET  key=>  nilINCR  key=>  1INCR  key=>  2GET  key=>  2 Redis
  20. UsageCounters (downloads, hits, votes, …)$  curl  downloads:totalINCR  downloads:total:todayINCR  downloads:total:2011-­‐05-­‐10INCR  downloads:/downloads/file1.mpg:totalINCR  downloads:/downloads/file1.mpg:todayINCR  downloads:/downloads/file1.mpg:2011-­‐05-­‐10 Redis
  21. UsageCounters (downloads, hits, votes, …)#  Total  downloads  for  server,  all  timeGET  downloads:total#  Total  downloads  for  server,  todayGET  downloads:total:today#  Total  downloads  for  fileGET  downloads:/downloads/file1.mpg:total#  Total  downloads  for  file  todayINCR  downloads:/downloads/file1.mpg:today... Redis
  22. UsageCounters (downloads, hits, votes, …)#  Nightly  maintenance  at  23:59RENAME  downloads:total:today                downloads:total:yesterdayAll this runs at super-sonic speed, with minimal overhead and resource consumption.See implementation for, youll hit denormalization bloat once you start adding metrics (eg. downloads percountry, per category, …) Redis
  23. UsageVariations: Rate limiting$  curl  api:<TOKEN>:hits=>  1 if INCR(api:abc123:hits) > LIMIT return 420 Enhance Your Calm end#  Every  hour...DEL  api:<TOKEN>:hits Redis
  24. UsageGenerating unique IDsINCR  global:users_ids=>  1SET  users:1:username  "john"INCR  global:users_ids=>  2SET  users:2:username  "mary" Redis
  25. REDIS DATA STRUCTURESListsLPUSH  key  1 RPOP  key=>  1 =>  "1"LPUSH  key  2 LRANGE  key  0  -­‐1=>  2 =>  "3"LPUSH  key  3 =>  "2"=>  3 LLEN  keyLRANGE  key  0  -­‐1 =>  2=>  "3" LTRIM  key  0  1=>  "2" =>  OK=>  "1" Redis
  26. UsageIndexes (list of comments, ...)LPUSH  article:comments  <ID>Timelines (of all sorts: messages, logs, …)LPUSH  user:<ID>:inbox  "message  from  Alice"LPUSH  user:<ID>:inbox  "message  from  Bob"#  Limit  the  messages  to  100LTRIM  user:<ID>:inbox  0  99#  Get  last  10  messagesLRANGE  user:<ID>:inbox  0  9#  Get  next  10  messagesLRANGE  user:<ID>:inbox  10  19 Redis
  27. UsageQueues#  PublisherRPUSH  queue  "task-­‐1"RPUSH  queue  "task-­‐2"#  Worker  (blocks  and  waits  for  tasks)BLPOP  queue  0 Redis
  28. UsageQueues# publisher.shfor i in {1..10}; do redis-cli RPUSH "queue" "task-$i"done# worker.shwhile true; do redis-cli BLPOP "queue" 0done Demo Redis
  29. REDIS DATA STRUCTURESResque: Background Processing from Github RPUSH } LPOP O(1) … tens or millions of items … Redis
  31. TALK ON ASYNCHRONOUS PROCESSINGThe Code of the Forking PathsSlides (in Czech) Redis
  32. REDIS DATA STRUCTURESSetsSADD  key  1 SISMEMBER  key  1=>  1 =>  "1"SADD  key  2 SISMEMBER  key  5=>  2 =>  "0"SADD  key  3 SRANDMEMBER  key=>  3 =>  "<RAND>"SMEMBERS  key SREM  key  3=>  "3" =>  1=>  "1"=>  "2" Redis
  33. REDIS DATA STRUCTURESSet OperationsSADD  A  1 SADD  B  1SADD  A  2 SADD  B  3SMEMBERS  A SMEMBERS  B=>  "1" =>  "1"=>  "2" =>  "3" Redis
  34. REDIS DATA STRUCTURESSet Operations[1,2] [1,3] Union SUNION  A  B =>  1 =>  2[1,2] [1,3] =>  3 Intersection SINTER  A  B =>  1[1,2] [1,3] Difference SDIFF  A  B =>  2 Redis
  35. UsageAd servingSADD  ads:cars      "Check  out  Toyota!"SADD  ads:cars      "Check  out  Ford!"...SADD  ads:movies  "Check  out  Winters  Bone!"...SRANDMEMBER  ads:carsSRANDMEMBER  ads:moviesNote: Time complexity is O(1). “Check out ODER BY RAND()!” Redis
  36. UsageRelations (Friends/followers)SADD  users:A:follows  BSADD  users:B:follows  CSADD  users:B:follows  DSADD  users:C:follows  ASADD  users:C:follows  D Redis
  37. UsageRelations (Friends/followers)# Joint network of A and BSUNION  users:A:follows  users:B:follows1)  "C"2)  "D"3)  "B" Redis
  38. UsageRelations (Friends/followers)#  Common  for  A  and  BSINTER  users:A:follows  users:B:follows[]#  Common  for  B  and  CSINTER  users:B:follows  users:C:follows1)  "D"#  Unique  to  B  compared  to  CSDIFF  users:B:follows  users:C:follows1)  "C" Redis
  39. UsageRelations (Friends/followers)#  Whom  I  follow...SADD  friends  A SMEMBERS  friendsSADD  friends  B 1)  "A" 2)  "B"#  Who  follows  me...SADD  followers  B SMEMBERS  followersSADD  followers  C 1)  "C" 2)  "B" Redis
  40. UsageRelations (Friends/followers)#  Mutual  relationshipsSINTER  friends  followers1)  "B"#  Who  does  not  follow  me  back?SDIFF  friends  followers1)  "A"#  Who  am  I  not  following  back?SDIFF  followers  friends1)  "C" Redis
  41. Mining the Social Web Analyzing Data from Facebook, Twitter, LinkedIn, and Other Social Media Sites Matthew A. Russell Redis
  42. UsageRelations (Article tags/categories)SADD  tags:ruby  article-­‐1SADD  tags:java  article-­‐2SADD  tags:web    article-­‐1SADD  tags:web    article-­‐2 Redis
  43. UsageRelations (Article tags/categories)#  ruby  OR  javaSUNION  tags:ruby  tags:java1)  "article-­‐2"2)  "article-­‐1"#  ruby  AND  javaSINTER  tags:ruby  tags:java[]#  web  AND  NOT  rubySDIFF  tags:web  tags:ruby1)  "article-­‐2" Redis
  44. UsageFriends Online#  My  friendsSADD  friends  ASADD  friends  BSADD  friends  C#  Friend  A  connectsSADD  online:fresh  ASUNIONSTORE  online  online:fresh  online:stale#  Whos  online  now?SINTER  friends  online[A] Redis
  45. UsageFriends Online#  Every  minute,  rename  the  "fresh"  to  "stale"  ...RENAME  online:fresh  online:stale#  ...  and  update  the  "online"  setSUNIONSTORE  online  online:fresh  online:stale#  Friend  B  connectsSADD  online:fresh  BSUNIONSTORE  online  online:fresh  online:stale#  Whos  online  now?SINTER  friends  online[A,B] Redis
  46. UsageFriends Online#  Time  passes  ...#  Rename  the  "fresh"  to  "stale",  every  minute  ...RENAME  online:fresh  online:stale#  ...  and  update  the  "online"  setSUNIONSTORE  online  online:fresh  online:stale#  Whos  online  now?SINTER  friends  online[B] Redis
  47. REDIS DATA STRUCTURESSorted SetsZADD  key  100  A ZREVRANGE  key  0  -­‐1ZADD  key  10    C 1)  "A"ZADD  key  80    B 2)  "B" 3)  "C"ZRANGE  key  0  -­‐1 ZINCRBY  key  10  C1)  "C" "20"2)  "B"3)  "A" Redis
  48. REDIS DATA STRUCTURESSorted SetsZREVRANGE  key  0  -­‐1   ZREVRANGEBYSCORE  WITHSCORES key  100  501)  "A" 1)  "A"2)  "100" 2)  "B"3)  "B"4)  "80"5)  "C"6)  "20" Redis
  49. UsageLeaderboards#  User  A  got  10  pointsZINCRBY  scores  10  A#  User  B  got  15  pointsZINCRBY  scores  15  B#  User  A  got  another  10  pointsZINCRBY  scores  10  A#  Display  scoresZREVRANGE  scores  0  -­‐1  WITHSCORES1)  "A"2)  "20"3)  "B"4)  "15" Redis
  50. UsageInverted Index#  Index  document  AZINCRBY  index:foo  1  document-­‐AZINCRBY  index:foo  1  document-­‐AZINCRBY  index:bar  1  document-­‐A#  Index  document  BZINCRBY  index:foo  1  document-­‐BZINCRBY  index:baz  1  document-­‐B#  Search  for  token  foo,  sort  by  occurencesZREVRANGE  index:foo  0  -­‐1  WITHSCORES1)  "document-­‐A"2)  "2"3)  "document-­‐B"4)  "1" Redis
  51. REDIS SORTED SETS AS AN INDEXInverted Index Redis
  52. REDIS DATA STRUCTURESHashesHMSET  users:1  username  j  name  JohnHMSET  users:2  username  m  name  MaryHGETALL  users:11)  "username"2)  "j"...HKEYS  users:11)  "username"2)  "name"HSET  users:1  score  100HGET  users:1  score1)  "100" Redis
  53. UsageStructured data (Articles, users, …)HMSET  articles:1  title          "Redis  is  cool!"                                    content      "I  recently  ..."                                    published  "2011-­‐05-­‐10"HGETALL  articles:11)  "title"2)  "Redis  is  cool!"3)  "content"4)  "I  recently  ..."5)  "published"6)  "2011-­‐05-­‐10"HSET  articles:1  title  "Redis  is  very  cool!"HGET  articles:1  title"Redis  is  very  cool!" Redis
  54. UsageUser Preferences (No login)#  Save  preferences  from  <FORM>HMSET  prefs:<COOKIE  HASH>  background  #ccc  color  #333#  Keep  it  for  one  yearEXPIRE  prefs:<COOKIE  HASH>  31556926#  Retrieve  preferencesHGETALL  prefs:<COOKIE  HASH> Redis
  55. REDIS FEATURESPublish/SubscribeSUBSCRIBE    log.errorPUBLISH        log.error  "ERROR"PSUBSCRIBE  log.* “AMQP”PUBLISH        log.error  "ERROR"=>  "ERROR"=>  "ERROR"PUBLISH  "INFO"=>  "INFO" Demo Redis
  56. REDIS FEATURESDurabilityBGSAVE/usr/local/var/db/redis/dump.rdbBGREWRITEAOF#  redis.confappendonly  yes Redis
  57. REDIS FEATURESVirtual Memory#  redis.confvm-­‐enabled  yesAllows to work with data sets bigger then available RAM.Swaps less often used values to disk.Keys must still fit into RAM.#  Make  Redis  disk-­‐bound  databasevm-­‐max-­‐memory  0The future? Redis Diskstore (disk-bound by default, cache in between server and disk) Redis
  58. REDIS FEATURESReplication#  slave.confslaveof  6379$  redis-­‐server  slave.confThe future? Redis Cluster (Dynamo-like, Distributed hash table) Redis
  59. REDIS FEATURESScripting (experimental)$  git  clone  -­‐b  scripting  redis-­‐scripting$  cd  redis-­‐scripting$  make$  echo  port  7397  |  ./src/redis-­‐server  -­‐$  ./src/redis-­‐cliSET  myscript  "return  HELLO"EVAL  "return  loadstring(redis(get,  KEYS[1]))()"              1  myscript=>  "HELLO" Redis
  60. REDIS RESOURCESThe Interactive Documentation PROMPT Redis
  61. REDIS RESOURCESSimon Willison Tutorial Redis
  62. REDIS RESOURCESRedis Implementation Details Redis
  63. REDIS RESOURCESRedis Use Cases Redis
  64. REDIS RESOURCESThe Original Metaphore… Redis
  65. Thanks! d