Your SlideShare is downloading. ×
0
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Redis — The AK-47 of Post-relational Databases
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Redis — The AK-47 of Post-relational Databases

17,573

Published on

Published in: Technology
3 Comments
47 Likes
Statistics
Notes
No Downloads
Views
Total Views
17,573
On Slideshare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
368
Comments
3
Likes
47
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Redis — The AK-47 Of Post-relational DatabasesKarel Minařík
  • 2. Karel Minařík→ Independent web designer and developer→ Ruby, Rails, Git, CouchDB propagandista in .cz→ Previously: Flash Developer; Art Director; Information Architect;… (see LinkedIn)→ @karmiq at Twitter→ karmi.cz Redis
  • 3. WEAPON DESIGNAK-47Designed at the end of WWII by Mikhail KalashnikovAssault Rifle, not a “submachine gun”Simple designDesigned for mass production & low-quality manufacturingExtreme reliability, at the cost of accuracy Redis
  • 4. http://www.youtube.com/watch?v=J6c3DLlM9KA#t=8m32s Redis
  • 5. Reliability Redis
  • 6. SIMPLICITYThe Redis ManifestoWere against complexity.We believe designing systems is a fight against complexity.Most of the time the best way to fight complexity is by notcreating it at all.http://antirez.com/post/redis-manifesto.html Redis
  • 7. SIMPLICITYRedis and NoSQLWhats wrong with RDBMS when used for (many) tasksthat dont need all this complexity? The data model: nonscalable, time complexity hard to predict, and cant modelmany common problems well enough.http://antirez.com/post/MongoDB-and-Redis.html Redis
  • 8. Assault rifles as the crucial invention for new rules of warfare: fighting in shorter distances and shooting while on the run.http://www.youtube.com/watch?v=a1KBsqvKpXk#t=7m50s Redis
  • 9. Memory is the new disk.Disk is the new tape.— Jim Grayhttp://www.infoq.com/news/2008/06/ram-is-diskhttp://research.microsoft.com/en-us/um/people/gray/talks/Flash_is_Good.ppt Redis
  • 10. Redis: Main Features Redis
  • 11. Simplicity SpeedPredictability VersatilityReliability Low Footprint
  • 12. SIMPLICITYInstallationgit  clone  http://github.com/antirez/rediscd  redismake./src/redis-­‐server./src/redis-­‐cli Redis
  • 13. SIMPLICITYThe Zen of Redis(...) what Redis provides are data structures (...)http://antirez.com/post/MongoDB-and-Redis.html#c1537 Redis
  • 14. REDISData StructuresStringsListsSetsSorted SetsHashes Redis
  • 15. REDIS DATA STRUCTURESStringsSET  key  "value"  ~  4  billion  of  keysGET  key=>  "value"DEL  key Redis
  • 16. REDIS DATA STRUCTURESFetch multiple keys at onceSET  key1  "value1"SET  key2  "value2"MGET  key1  key2=>  "value1"=>  "value2" Redis
  • 17. REDIS DATA STRUCTURESExpirationEXPIRE  key  5GET  key=>  "value"TTL  key=>  1GET  key=>  (nil) Redis
  • 18. UsageCachehttp://antirez.com/post/redis-as-LRU-cache.htmlSessionshttps://github.com/mattmatt/redis-session-store Redis
  • 19. REDIS DATA STRUCTURESAtomic IncrementsGET  key=>  nilINCR  key=>  1INCR  key=>  2GET  key=>  2 Redis
  • 20. UsageCounters (downloads, hits, votes, …)$  curl  http://example.com/downloads/file1.mpgINCR  downloads:totalINCR  downloads:total:todayINCR  downloads:total:2011-­‐05-­‐10INCR  downloads:/downloads/file1.mpg:totalINCR  downloads:/downloads/file1.mpg:todayINCR  downloads:/downloads/file1.mpg:2011-­‐05-­‐10 Redis
  • 21. UsageCounters (downloads, hits, votes, …)#  Total  downloads  for  server,  all  timeGET  downloads:total#  Total  downloads  for  server,  todayGET  downloads:total:today#  Total  downloads  for  fileGET  downloads:/downloads/file1.mpg:total#  Total  downloads  for  file  todayINCR  downloads:/downloads/file1.mpg:today... Redis
  • 22. UsageCounters (downloads, hits, votes, …)#  Nightly  maintenance  at  23:59RENAME  downloads:total:today                downloads:total:yesterdayAll this runs at super-sonic speed, with minimal overhead and resource consumption.See implementation for Rubygems.org: https://gist.github.com/296921However, youll hit denormalization bloat once you start adding metrics (eg. downloads percountry, per category, …) Redis
  • 23. UsageVariations: Rate limiting$  curl  http://api.example.com/list.jsonINCR  api:<TOKEN>:hits=>  1 if INCR(api:abc123:hits) > LIMIT return 420 Enhance Your Calm end#  Every  hour...DEL  api:<TOKEN>:hits Redis
  • 24. UsageGenerating unique IDsINCR  global:users_ids=>  1SET  users:1:username  "john"INCR  global:users_ids=>  2SET  users:2:username  "mary" Redis
  • 25. REDIS DATA STRUCTURESListsLPUSH  key  1 RPOP  key=>  1 =>  "1"LPUSH  key  2 LRANGE  key  0  -­‐1=>  2 =>  "3"LPUSH  key  3 =>  "2"=>  3 LLEN  keyLRANGE  key  0  -­‐1 =>  2=>  "3" LTRIM  key  0  1=>  "2" =>  OK=>  "1" Redis
  • 26. UsageIndexes (list of comments, ...)LPUSH  article:comments  <ID>Timelines (of all sorts: messages, logs, …)LPUSH  user:<ID>:inbox  "message  from  Alice"LPUSH  user:<ID>:inbox  "message  from  Bob"#  Limit  the  messages  to  100LTRIM  user:<ID>:inbox  0  99#  Get  last  10  messagesLRANGE  user:<ID>:inbox  0  9#  Get  next  10  messagesLRANGE  user:<ID>:inbox  10  19 Redis
  • 27. UsageQueues#  PublisherRPUSH  queue  "task-­‐1"RPUSH  queue  "task-­‐2"#  Worker  (blocks  and  waits  for  tasks)BLPOP  queue  0 Redis
  • 28. UsageQueues# publisher.shfor i in {1..10}; do redis-cli RPUSH "queue" "task-$i"done# worker.shwhile true; do redis-cli BLPOP "queue" 0done Demo Redis
  • 29. REDIS DATA STRUCTURESResque: Background Processing from Github RPUSH } LPOP O(1) … tens or millions of items … https://github.com/defunkt/resque/blob/v1.13.0/lib/resque.rb#L133-138 Redis
  • 30. MESSAGE QUEUESRestMQ Redis
  • 31. TALK ON ASYNCHRONOUS PROCESSINGThe Code of the Forking PathsSlideshttp://www.slideshare.net/karmi/the-code-of-the-forking-paths-asynchronous-processing-with-resque-and-amqpVideo (in Czech)http://multimedia.vse.cz/media/Viewer/?peid=51c06c512f4645289c4e9c749dc85acc1d Redis
  • 32. REDIS DATA STRUCTURESSetsSADD  key  1 SISMEMBER  key  1=>  1 =>  "1"SADD  key  2 SISMEMBER  key  5=>  2 =>  "0"SADD  key  3 SRANDMEMBER  key=>  3 =>  "<RAND>"SMEMBERS  key SREM  key  3=>  "3" =>  1=>  "1"=>  "2" Redis
  • 33. REDIS DATA STRUCTURESSet OperationsSADD  A  1 SADD  B  1SADD  A  2 SADD  B  3SMEMBERS  A SMEMBERS  B=>  "1" =>  "1"=>  "2" =>  "3" Redis
  • 34. REDIS DATA STRUCTURESSet Operations[1,2] [1,3] Union SUNION  A  B =>  1 =>  2[1,2] [1,3] =>  3 Intersection SINTER  A  B =>  1[1,2] [1,3] Difference SDIFF  A  B =>  2http://en.wikipedia.org/wiki/Set_(mathematics)#Basic_operations Redis
  • 35. UsageAd servingSADD  ads:cars      "Check  out  Toyota!"SADD  ads:cars      "Check  out  Ford!"...SADD  ads:movies  "Check  out  Winters  Bone!"...SRANDMEMBER  ads:carsSRANDMEMBER  ads:moviesNote: Time complexity is O(1). “Check out ODER BY RAND()!” Redis
  • 36. UsageRelations (Friends/followers)SADD  users:A:follows  BSADD  users:B:follows  CSADD  users:B:follows  DSADD  users:C:follows  ASADD  users:C:follows  D Redis
  • 37. UsageRelations (Friends/followers)# Joint network of A and BSUNION  users:A:follows  users:B:follows1)  "C"2)  "D"3)  "B" Redis
  • 38. UsageRelations (Friends/followers)#  Common  for  A  and  BSINTER  users:A:follows  users:B:follows[]#  Common  for  B  and  CSINTER  users:B:follows  users:C:follows1)  "D"#  Unique  to  B  compared  to  CSDIFF  users:B:follows  users:C:follows1)  "C" Redis
  • 39. UsageRelations (Friends/followers)#  Whom  I  follow...SADD  friends  A SMEMBERS  friendsSADD  friends  B 1)  "A" 2)  "B"#  Who  follows  me...SADD  followers  B SMEMBERS  followersSADD  followers  C 1)  "C" 2)  "B" Redis
  • 40. UsageRelations (Friends/followers)#  Mutual  relationshipsSINTER  friends  followers1)  "B"#  Who  does  not  follow  me  back?SDIFF  friends  followers1)  "A"#  Who  am  I  not  following  back?SDIFF  followers  friends1)  "C" Redis
  • 41. Mining the Social Web Analyzing Data from Facebook, Twitter, LinkedIn, and Other Social Media Sites Matthew A. Russell http://books.google.com/books?id=SYM1lrQdrdsC&lpg=PP1&pg=PA94http://oreilly.com/catalog/0636920010203 Redis
  • 42. UsageRelations (Article tags/categories)SADD  tags:ruby  article-­‐1SADD  tags:java  article-­‐2SADD  tags:web    article-­‐1SADD  tags:web    article-­‐2 Redis
  • 43. UsageRelations (Article tags/categories)#  ruby  OR  javaSUNION  tags:ruby  tags:java1)  "article-­‐2"2)  "article-­‐1"#  ruby  AND  javaSINTER  tags:ruby  tags:java[]#  web  AND  NOT  rubySDIFF  tags:web  tags:ruby1)  "article-­‐2" Redis
  • 44. UsageFriends Online#  My  friendsSADD  friends  ASADD  friends  BSADD  friends  C#  Friend  A  connectsSADD  online:fresh  ASUNIONSTORE  online  online:fresh  online:stale#  Whos  online  now?SINTER  friends  online[A] Redis
  • 45. UsageFriends Online#  Every  minute,  rename  the  "fresh"  to  "stale"  ...RENAME  online:fresh  online:stale#  ...  and  update  the  "online"  setSUNIONSTORE  online  online:fresh  online:stale#  Friend  B  connectsSADD  online:fresh  BSUNIONSTORE  online  online:fresh  online:stale#  Whos  online  now?SINTER  friends  online[A,B] Redis
  • 46. UsageFriends Online#  Time  passes  ...#  Rename  the  "fresh"  to  "stale",  every  minute  ...RENAME  online:fresh  online:stale#  ...  and  update  the  "online"  setSUNIONSTORE  online  online:fresh  online:stale#  Whos  online  now?SINTER  friends  online[B] Redis
  • 47. REDIS DATA STRUCTURESSorted SetsZADD  key  100  A ZREVRANGE  key  0  -­‐1ZADD  key  10    C 1)  "A"ZADD  key  80    B 2)  "B" 3)  "C"ZRANGE  key  0  -­‐1 ZINCRBY  key  10  C1)  "C" "20"2)  "B"3)  "A" Redis
  • 48. REDIS DATA STRUCTURESSorted SetsZREVRANGE  key  0  -­‐1   ZREVRANGEBYSCORE  WITHSCORES key  100  501)  "A" 1)  "A"2)  "100" 2)  "B"3)  "B"4)  "80"5)  "C"6)  "20" Redis
  • 49. UsageLeaderboards#  User  A  got  10  pointsZINCRBY  scores  10  A#  User  B  got  15  pointsZINCRBY  scores  15  B#  User  A  got  another  10  pointsZINCRBY  scores  10  A#  Display  scoresZREVRANGE  scores  0  -­‐1  WITHSCORES1)  "A"2)  "20"3)  "B"4)  "15" Redis
  • 50. UsageInverted Index#  Index  document  AZINCRBY  index:foo  1  document-­‐AZINCRBY  index:foo  1  document-­‐AZINCRBY  index:bar  1  document-­‐A#  Index  document  BZINCRBY  index:foo  1  document-­‐BZINCRBY  index:baz  1  document-­‐B#  Search  for  token  foo,  sort  by  occurencesZREVRANGE  index:foo  0  -­‐1  WITHSCORES1)  "document-­‐A"2)  "2"3)  "document-­‐B"4)  "1" Redis
  • 51. REDIS SORTED SETS AS AN INDEXInverted Indexhttps://gist.github.com/928605 Redis
  • 52. REDIS DATA STRUCTURESHashesHMSET  users:1  username  j  name  JohnHMSET  users:2  username  m  name  MaryHGETALL  users:11)  "username"2)  "j"...HKEYS  users:11)  "username"2)  "name"HSET  users:1  score  100HGET  users:1  score1)  "100" Redis
  • 53. UsageStructured data (Articles, users, …)HMSET  articles:1  title          "Redis  is  cool!"                                    content      "I  recently  ..."                                    published  "2011-­‐05-­‐10"HGETALL  articles:11)  "title"2)  "Redis  is  cool!"3)  "content"4)  "I  recently  ..."5)  "published"6)  "2011-­‐05-­‐10"HSET  articles:1  title  "Redis  is  very  cool!"HGET  articles:1  title"Redis  is  very  cool!" Redis
  • 54. UsageUser Preferences (No login)#  Save  preferences  from  <FORM>HMSET  prefs:<COOKIE  HASH>  background  #ccc  color  #333#  Keep  it  for  one  yearEXPIRE  prefs:<COOKIE  HASH>  31556926#  Retrieve  preferencesHGETALL  prefs:<COOKIE  HASH> Redis
  • 55. REDIS FEATURESPublish/SubscribeSUBSCRIBE    log.errorPUBLISH        log.error  "ERROR"PSUBSCRIBE  log.* “AMQP”PUBLISH        log.error  "ERROR"=>  "ERROR"=>  "ERROR"PUBLISH        log.info  "INFO"=>  "INFO" Demo Redis
  • 56. REDIS FEATURESDurabilityBGSAVE/usr/local/var/db/redis/dump.rdbBGREWRITEAOF#  redis.confappendonly  yeshttp://redis.io/topics/persistence Redis
  • 57. REDIS FEATURESVirtual Memory#  redis.confvm-­‐enabled  yesAllows to work with data sets bigger then available RAM.Swaps less often used values to disk.Keys must still fit into RAM.#  Make  Redis  disk-­‐bound  databasevm-­‐max-­‐memory  0The future? Redis Diskstore (disk-bound by default, cache in between server and disk)http://redis.io/topics/virtual-memory Redis
  • 58. REDIS FEATURESReplication#  slave.confslaveof  127.0.0.1  6379$  redis-­‐server  slave.confThe future? Redis Cluster (Dynamo-like, Distributed hash table)http://redis.io/topics/replication Redis
  • 59. REDIS FEATURESScripting (experimental)$  git  clone  https://github.com/antirez/redis  -­‐b  scripting  redis-­‐scripting$  cd  redis-­‐scripting$  make$  echo  port  7397  |  ./src/redis-­‐server  -­‐$  ./src/redis-­‐cliSET  myscript  "return  HELLO"EVAL  "return  loadstring(redis(get,  KEYS[1]))()"              1  myscript=>  "HELLO"http://antirez.com/post/scripting-branch-released.html Redis
  • 60. REDIS RESOURCESThe Interactive Documentation PROMPThttp://redis.io/commands Redis
  • 61. REDIS RESOURCESSimon Willison Tutorialhttp://simonwillison.net/static/2010/redis-tutorial/ Redis
  • 62. REDIS RESOURCESRedis Implementation Detailshttp://pauladamsmith.com/articles/redis-under-the-hood.html Redis
  • 63. REDIS RESOURCESRedis Use Caseshttp://highscalability.com/blog/2010/12/6/what-the-heck-are-you-actually-using-nosql-for.html Redis
  • 64. REDIS RESOURCESThe Original Metaphore…http://flazz.me/redis-the-ak-47-of-databases Redis
  • 65. Thanks! d

×