Jordi Boggiano@seldaekhttp://nelm.io/Redis - Your AdvancedIn-Memory Key-Value Store
About Me              Belgian living in Zürich, Switzerland              Weby stuff for 10 years               http://seld...
Agenda                   Intro                   Features                   Use Cases                   Using it with PHPJ...
Redis                   Wut?Jordi Boggiano             Company nelm.ioTwitter @seldaek               Blog seld.be
Store                   You put data in itJordi Boggiano                          Company nelm.ioTwitter @seldaek         ...
Key-Value Store                    Like NoSQL?Jordi Boggiano                       Company nelm.ioTwitter @seldaek        ...
In-Memory Key-Value Store                       Like Memcached?Jordi Boggiano                                 Company nelm...
In-Memory Key-Value Store                   Memory is fast, but ephemeralJordi Boggiano                                   ...
Persist to Disk                   Fast and lastingJordi Boggiano                        Company nelm.ioTwitter @seldaek   ...
Created by @antirez                   Sponsored by VMWareJordi Boggiano                            Company nelm.ioTwitter ...
So how does it work?Jordi Boggiano                            Company nelm.ioTwitter @seldaek                             ...
Simple Text Protocol                    Human Readable!Jordi Boggiano                            Company nelm.ioTwitter @s...
Example - Client Library               1 SET key valueJordi Boggiano                                              Company ...
Example - Client Library               1 SET key value               2 > OKJordi Boggiano                                 ...
Example - Client Library               1 SET key value               2 > OK               3 GET keyJordi Boggiano         ...
Example - Client Library               1   SET key value               2   > OK               3   GET key               4 ...
Example - Low Level               1   *3               2   $3               3   SET               4   $3               5  ...
Example - Low Level               1   *3rn               2   $3rn               3   SETrn               4   $3rn          ...
Data TypesJordi Boggiano                  Company nelm.ioTwitter @seldaek                    Blog seld.be
Strings                   1   SET name Bob                   2   SET age 20                   3   MGET name age           ...
Strings                   1   SETEX age 3 20                   2   GET age                   3   > 20                   4 ...
Integers                   1   INCR count                   2   > 1                   3   INCR count                   4  ...
Hashes              01 HMSET user name Alice email                 alice@example.org              02 HGET user email      ...
Lists              01   RPUSH admins Alice              02   > 1              03   RPUSH admins Bob              04   > 2 ...
Sets              01   SADD page:3:visitors 134              02   > 1              03   SADD page:3:visitors 253          ...
Sets                   1   SADD page:6:visitors 253                   2   SADD page:6:visitors 923                   3   S...
Sorted Sets                   1   ZADD highscores 2930 Alice                   2   ZADD highscores 1546 Bob               ...
Sort of Lists? Listed Sets?              Lists                   1 array(foo, bar)              Sets                   1 s...
The basic datatypes of every                     language exist in RedisJordi Boggiano                                    ...
Process data in Redis                      instead of PHPJordi Boggiano                             Company nelm.ioTwitter...
A Few FeaturesJordi Boggiano                      Company nelm.ioTwitter @seldaek                        Blog seld.be
Atomic Operations                   1   SETNX name Alice                   2   GET name                   3   > Alice     ...
EXPIRE / EXPIREAT / PERSISTJordi Boggiano                               Company nelm.ioTwitter @seldaek                   ...
PipeliningJordi Boggiano                  Company nelm.ioTwitter @seldaek                    Blog seld.be
SORT - SQL in your NoSQL                   1   SORT key                   2   SORT key LIMIT 0 10 DESC                   3...
Append-Only File & SnapshotsJordi Boggiano                                    Company nelm.ioTwitter @seldaek             ...
Pub/Sub                   1 SUBSCRIBE foo1                   2 PSUBSCRIBE foo*                   3 PUBLISH foo0 messageJor...
Master/Slave ReplicationJordi Boggiano                                Company nelm.ioTwitter @seldaek                     ...
Transactions: MULTI / EXEC /                         DISCARD              Optimistic Locking with WATCHJordi Boggiano     ...
Upcoming FeaturesJordi Boggiano                         Company nelm.ioTwitter @seldaek                           Blog sel...
Lua Scripting: EVAL / EVALSHA                        This is great.Jordi Boggiano                             Company nelm...
EVAL               1 EVAL <body> <num_keys_in_args> [<arg1> <arg2> ... <arg_N>]               1 GET A               2 EVAL...
EVAL              Example: Atomic Conditional Decrement, Client-Side               1   WATCH foo               2   $val = ...
EVAL              Example: Atomic Conditional Decrement, Server-Side              01 EVAL "local value = tonumber(redis.ca...
Redis Cluster                   Almost as delayed as                   Duke Nukem ForeverJordi Boggiano                   ...
Speed                        Redis is FAST.                   CPU is unlikely to be the                         bottleneck...
Wait for Redis Cluster?                   If you can. Cluster looks great.Jordi Boggiano                                  ...
Use sharding, most client                      libraries can do it.Jordi Boggiano                                 Company ...
Use Edis, a protocol compatible                  storage-oriented server.                   http://inaka.github.com/edis/i...
Use it without persistence,                          store to disk/DB                   and load hot data in memory.      ...
Ok, great. But..                   What is it good for?Jordi Boggiano                            Company nelm.ioTwitter @s...
WEB SCALE                       THE CLOUD                         NOSQL                        ELASTIC                   H...
WEB SCALE                       THE CLOUD                         NOSQL                        ELASTIC                   H...
BULLSHITJordi Boggiano                Company nelm.ioTwitter @seldaek                  Blog seld.be
Get real                   "If you do actually have to scale,                   then your database isnt going to          ...
Really.                   What is it good for?Jordi Boggiano                            Company nelm.ioTwitter @seldaek   ...
Storing Sessions                           Caching Data                   Other typical Memcached usesJordi Boggiano      ...
Caching: Newspaper Site                   Expensive reads                   Fast-changing pages                   Dynamic ...
Caching: Newspaper Site              Cache generic results:                   1 SETEX <pageid>:content 600 <data>         ...
Logging, games and other                        write-heavy usages                   1 LPUSH logs "Log message"           ...
Highscore tables                   1   ZADD scores 4290 <playerid>                   2   ZADD scores 390 <playerid2>      ...
Whos online now?                   1   ZADD visits <unix> <userid>                   2   ZADD visits 1327681399 52        ...
Job Queues              Workers:                   1 BRPOP queue              Application:                   1 LPUSH queue...
Behavior Tracking                   1 SETBIT click:<item>:<date> <userid> 1                   2 GETBIT click:3:2012-01-28 ...
Last but not least, it puts the                     fun back in DatafunbasesJordi Boggiano                                ...
Usage with PHPJordi Boggiano                      Company nelm.ioTwitter @seldaek                        Blog seld.be
Extension: phpredisJordi Boggiano                           Company nelm.ioTwitter @seldaek                             Bl...
Extension: phpredis              Comes with a session handler, sharding, up to date.               1   $redis = new Redis(...
Extension: phpiredisJordi Boggiano                            Company nelm.ioTwitter @seldaek                             ...
Extension: phpiredis              Basic hiredis bindings, protocol parsing, low level.               1   $redis = phpiredi...
Plain PHP: PredisJordi Boggiano                         Company nelm.ioTwitter @seldaek                           Blog sel...
Plain PHP: Predis              Very complete, sharding, master/slave auto-select,              can use phpiredis for parsi...
Its fast. Its fun. Try it.                       try.redis-db.comJordi Boggiano                                    Company...
Thank you.                   Slides: http://slides.seld.beJordi Boggiano                                     Company nelm....
Find Out More                   Redis.io                   Try Redis                   The Little Redis Book              ...
Questions?                      jordi@nelm.io                         @seldaek                        Feedback:           ...
Upcoming SlideShare
Loading in …5
×

Redis - Your Advanced In-Memory Key-Value Store

4,428 views

Published on

Published in: Technology, Business
0 Comments
16 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,428
On SlideShare
0
From Embeds
0
Number of Embeds
67
Actions
Shares
0
Downloads
64
Comments
0
Likes
16
Embeds 0
No embeds

No notes for slide

Redis - Your Advanced In-Memory Key-Value Store

  1. 1. Jordi Boggiano@seldaekhttp://nelm.io/Redis - Your AdvancedIn-Memory Key-Value Store
  2. 2. About Me Belgian living in Zürich, Switzerland Weby stuff for 10 years http://seld.be Symfony2, Composer and other OSS contributions http://github.com/Seldaek Working at Nelmio http://nelm.io Symfony2 & frontend performance consultingJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  3. 3. Agenda Intro Features Use Cases Using it with PHPJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  4. 4. Redis Wut?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  5. 5. Store You put data in itJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  6. 6. Key-Value Store Like NoSQL?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  7. 7. In-Memory Key-Value Store Like Memcached?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  8. 8. In-Memory Key-Value Store Memory is fast, but ephemeralJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  9. 9. Persist to Disk Fast and lastingJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  10. 10. Created by @antirez Sponsored by VMWareJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  11. 11. So how does it work?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  12. 12. Simple Text Protocol Human Readable!Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  13. 13. Example - Client Library 1 SET key valueJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  14. 14. Example - Client Library 1 SET key value 2 > OKJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  15. 15. Example - Client Library 1 SET key value 2 > OK 3 GET keyJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  16. 16. Example - Client Library 1 SET key value 2 > OK 3 GET key 4 > "value"Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  17. 17. Example - Low Level 1 *3 2 $3 3 SET 4 $3 5 key 6 $5 7 value 1 +OK 1 *2 2 $3 3 GET 4 $3 5 key 1 $5 2 valueJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  18. 18. Example - Low Level 1 *3rn 2 $3rn 3 SETrn 4 $3rn 5 keyrn 6 $5rn 7 valuern 1 +OKrn 1 *2rn 2 $3rn 3 GETrn 4 $3rn 5 keyrn 1 $5rn 2 valuernJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  19. 19. Data TypesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  20. 20. Strings 1 SET name Bob 2 SET age 20 3 MGET name age 4 > Bob 5 > 20 1 GETSET name Alice 2 > BobJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  21. 21. Strings 1 SETEX age 3 20 2 GET age 3 > 20 4 // .. 3 seconds later .. 5 GET age 6 > nullJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  22. 22. Integers 1 INCR count 2 > 1 3 INCR count 4 > 2 5 INCRBY count 3 6 > 5Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  23. 23. Hashes 01 HMSET user name Alice email alice@example.org 02 HGET user email 03 > alice@example.org 04 HKEYS user 05 > name 06 > email 07 HGETALL user 08 > name 09 > Alice 10 > email 11 > alice@example.orgJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  24. 24. Lists 01 RPUSH admins Alice 02 > 1 03 RPUSH admins Bob 04 > 2 05 LINDEX admins 0 06 > Alice 07 LLEN admins 08 > 2 09 RPOP admins 10 > BobJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  25. 25. Sets 01 SADD page:3:visitors 134 02 > 1 03 SADD page:3:visitors 253 04 > 1 05 SADD page:3:visitors 253 06 > 0 07 SCARD page:3:visitors 08 > 2 09 SMEMBERS page:3:visitors 10 > 134 11 > 253 12 SISMEMBER page:3:visitors 349 13 > 0Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  26. 26. Sets 1 SADD page:6:visitors 253 2 SADD page:6:visitors 923 3 SADD page:6:visitors 13 4 SINTER page:3:visitors page:6:visitors 5 > 253Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  27. 27. Sorted Sets 1 ZADD highscores 2930 Alice 2 ZADD highscores 1546 Bob 3 ZREVRANGE highscores 0 10 WITHSCORES 4 > Alice 5 > 2930 6 > Bob 7 > 1546Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  28. 28. Sort of Lists? Listed Sets? Lists 1 array(foo, bar) Sets 1 shuffle(array(foo, bar)) Sorted Sets 1 ksort(array(3 => foo, 1 => bar))Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  29. 29. The basic datatypes of every language exist in RedisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  30. 30. Process data in Redis instead of PHPJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  31. 31. A Few FeaturesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  32. 32. Atomic Operations 1 SETNX name Alice 2 GET name 3 > Alice 4 SETNX name Bob 5 GET name 6 > Alice 1 INCR foo 2 GET foo 3 > 1 4 INCRBY foo 3 5 GET foo 6 > 4Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  33. 33. EXPIRE / EXPIREAT / PERSISTJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  34. 34. PipeliningJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  35. 35. SORT - SQL in your NoSQL 1 SORT key 2 SORT key LIMIT 0 10 DESC 3 SORT key ALPHA 4 SORT page:6:visitors BY user_*->rank GET user_* DESCJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  36. 36. Append-Only File & SnapshotsJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  37. 37. Pub/Sub 1 SUBSCRIBE foo1 2 PSUBSCRIBE foo* 3 PUBLISH foo0 messageJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  38. 38. Master/Slave ReplicationJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  39. 39. Transactions: MULTI / EXEC / DISCARD Optimistic Locking with WATCHJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  40. 40. Upcoming FeaturesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  41. 41. Lua Scripting: EVAL / EVALSHA This is great.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  42. 42. EVAL 1 EVAL <body> <num_keys_in_args> [<arg1> <arg2> ... <arg_N>] 1 GET A 2 EVAL "return redis.call(get, KEYS[1])" 1 AJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  43. 43. EVAL Example: Atomic Conditional Decrement, Client-Side 1 WATCH foo 2 $val = GET foo 3 $newVal = max(0, $val - 1); // decrement if foo > 0 client-side 4 MULTI 5 SET foo $newVal 6 EXEC This may return -ERR, or will block.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  44. 44. EVAL Example: Atomic Conditional Decrement, Server-Side 01 EVAL "local value = tonumber(redis.call(get, KEYS[1])) 02 if value == nil 03 then 04 return {err="Value at key is not integer"} 05 end 06 if value > tonumber(ARGV[1]) 07 then 08 value = value - 1 09 redis.call(set, KEYS[1], value) 10 end 11 return value" 1 foo 0 This is instant and can not fail.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  45. 45. Redis Cluster Almost as delayed as Duke Nukem ForeverJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  46. 46. Speed Redis is FAST. CPU is unlikely to be the bottleneck. What do you do when you run out of RAM?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  47. 47. Wait for Redis Cluster? If you can. Cluster looks great.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  48. 48. Use sharding, most client libraries can do it.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  49. 49. Use Edis, a protocol compatible storage-oriented server. http://inaka.github.com/edis/index.htmlJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  50. 50. Use it without persistence, store to disk/DB and load hot data in memory. A good example in these slides from WoogaJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  51. 51. Ok, great. But.. What is it good for?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  52. 52. WEB SCALE THE CLOUD NOSQL ELASTIC HORIZONTAL SCALING VERTICAL TOOJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  53. 53. WEB SCALE THE CLOUD NOSQL ELASTIC HORIZONTAL SCALING VERTICAL TOOJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  54. 54. BULLSHITJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  55. 55. Get real "If you do actually have to scale, then your database isnt going to magically do it for you."Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  56. 56. Really. What is it good for?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  57. 57. Storing Sessions Caching Data Other typical Memcached usesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  58. 58. Caching: Newspaper Site Expensive reads Fast-changing pages Dynamic tracking of whats read for usersJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  59. 59. Caching: Newspaper Site Cache generic results: 1 SETEX <pageid>:content 600 <data> Store user read-state in sets 1 SADD <pageid>:views <userid> Combine and render for each user 1 GET <pageid>:content 2 SISMEMBER <pageid>:views <userid>Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  60. 60. Logging, games and other write-heavy usages 1 LPUSH logs "Log message" 2 // keep the last 1000 entries 3 LTRIM logs 0 999Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  61. 61. Highscore tables 1 ZADD scores 4290 <playerid> 2 ZADD scores 390 <playerid2> 3 // ... 4 ZREVRANK scores <playerid> // 0 5 ZREVRANGE scores 0 10 WITHSCORES Note: players will only be listed once since it is a set.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  62. 62. Whos online now? 1 ZADD visits <unix> <userid> 2 ZADD visits 1327681399 52 3 ZADD visits 1327683245 18 4 ZRANGEBYSCORE visits <unix>-3600 <unix> 5 ZREMRANGEBYSCORE visits 0 <unix>-3600Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  63. 63. Job Queues Workers: 1 BRPOP queue Application: 1 LPUSH queue "job description" e.g. the Resque lib from GitHubJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  64. 64. Behavior Tracking 1 SETBIT click:<item>:<date> <userid> 1 2 GETBIT click:3:2012-01-28 55239Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  65. 65. Last but not least, it puts the fun back in DatafunbasesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  66. 66. Usage with PHPJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  67. 67. Extension: phpredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  68. 68. Extension: phpredis Comes with a session handler, sharding, up to date. 1 $redis = new Redis(); 2 $redis->connect(127.0.0.1, 6379); 3 $redis->watch(x); 4 $val = $redis->get(x); 5 $newVal = max(0, $val - 1); 6 $result = $redis->multi() 7 ->set(x, $newVal) 8 ->exec(); 9 echo $result !== false ? Success : Race lost, try again; http://github.com/nicolasff/phpredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  69. 69. Extension: phpiredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  70. 70. Extension: phpiredis Basic hiredis bindings, protocol parsing, low level. 1 $redis = phpiredis_connect(127.0.0.1, 6379); 2 phpiredis_command($redis, WATCH x); 3 $val = phpiredis_command($redis, GET x); 4 $newVal = max(0, $val - 1); 5 phpiredis_command($redis, MULTI); 6 phpiredis_command($redis, SET x .$newVal); 7 $result = phpiredis_command($redis, EXEC); 8 echo $result !== false ? Success : Race lost, try again; http://github.com/seppo0010/phpiredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  71. 71. Plain PHP: PredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  72. 72. Plain PHP: Predis Very complete, sharding, master/slave auto-select, can use phpiredis for parsing. 01 $redis = new PredisClient(tcp://10.0.0.1:6379); 02 03 $options = array( 04 cas => true, // enable Check-and-Set 05 watch => x, 06 retry => 10, // automatic retries 07 ); 08 09 $result = $redis->multiExec($options, function($transaction) { 10 $val = $transaction->get(x); 11 $newVal = max(0, $val - 1); 12 $transaction->multi(); 13 $transaction->set(x, $newVal); 14 }); 15 16 echo $result !== false ? Success : Race lost 10 times, giving up; http://github.com/nrk/predis http://github.com/snc/SncRedisBundle (Symfony2 integration)Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  73. 73. Its fast. Its fun. Try it. try.redis-db.comJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  74. 74. Thank you. Slides: http://slides.seld.beJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  75. 75. Find Out More Redis.io Try Redis The Little Redis Book Redis Cluster SpecificationJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
  76. 76. Questions? jordi@nelm.io @seldaek Feedback: http://joind.in/4764Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

×