Redis - Your Advanced In-Memory Key-Value Store

Jordi Boggiano@seldaekhttp://nelm.io/Redis - Your AdvancedIn-Memory Key-Value Store

About Me Belgian living in Zürich, Switzerland Weby stuff for 10 years http://seld.be Symfony2, Composer and other OSS contributions http://github.com/Seldaek Working at Nelmio http://nelm.io Symfony2 & frontend performance consultingJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Agenda Intro Features Use Cases Using it with PHPJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Redis Wut?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Store You put data in itJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Key-Value Store Like NoSQL?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

In-Memory Key-Value Store Like Memcached?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

In-Memory Key-Value Store Memory is fast, but ephemeralJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Persist to Disk Fast and lastingJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Created by @antirez Sponsored by VMWareJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

So how does it work?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Simple Text Protocol Human Readable!Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Example - Client Library 1 SET key valueJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Example - Client Library 1 SET key value 2 > OKJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Example - Client Library 1 SET key value 2 > OK 3 GET keyJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Example - Client Library 1 SET key value 2 > OK 3 GET key 4 > "value"Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Example - Low Level 1 *3 2 $3 3 SET 4 $3 5 key 6 $5 7 value 1 +OK 1 *2 2 $3 3 GET 4 $3 5 key 1 $5 2 valueJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Example - Low Level 1 *3rn 2 $3rn 3 SETrn 4 $3rn 5 keyrn 6 $5rn 7 valuern 1 +OKrn 1 *2rn 2 $3rn 3 GETrn 4 $3rn 5 keyrn 1 $5rn 2 valuernJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Data TypesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Strings 1 SET name Bob 2 SET age 20 3 MGET name age 4 > Bob 5 > 20 1 GETSET name Alice 2 > BobJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Strings 1 SETEX age 3 20 2 GET age 3 > 20 4 // .. 3 seconds later .. 5 GET age 6 > nullJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Integers 1 INCR count 2 > 1 3 INCR count 4 > 2 5 INCRBY count 3 6 > 5Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Hashes 01 HMSET user name Alice email alice@example.org 02 HGET user email 03 > alice@example.org 04 HKEYS user 05 > name 06 > email 07 HGETALL user 08 > name 09 > Alice 10 > email 11 > alice@example.orgJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Lists 01 RPUSH admins Alice 02 > 1 03 RPUSH admins Bob 04 > 2 05 LINDEX admins 0 06 > Alice 07 LLEN admins 08 > 2 09 RPOP admins 10 > BobJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Sets 01 SADD page:3:visitors 134 02 > 1 03 SADD page:3:visitors 253 04 > 1 05 SADD page:3:visitors 253 06 > 0 07 SCARD page:3:visitors 08 > 2 09 SMEMBERS page:3:visitors 10 > 134 11 > 253 12 SISMEMBER page:3:visitors 349 13 > 0Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Sets 1 SADD page:6:visitors 253 2 SADD page:6:visitors 923 3 SADD page:6:visitors 13 4 SINTER page:3:visitors page:6:visitors 5 > 253Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Sorted Sets 1 ZADD highscores 2930 Alice 2 ZADD highscores 1546 Bob 3 ZREVRANGE highscores 0 10 WITHSCORES 4 > Alice 5 > 2930 6 > Bob 7 > 1546Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Sort of Lists? Listed Sets? Lists 1 array(foo, bar) Sets 1 shuffle(array(foo, bar)) Sorted Sets 1 ksort(array(3 => foo, 1 => bar))Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

The basic datatypes of every language exist in RedisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Process data in Redis instead of PHPJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

A Few FeaturesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Atomic Operations 1 SETNX name Alice 2 GET name 3 > Alice 4 SETNX name Bob 5 GET name 6 > Alice 1 INCR foo 2 GET foo 3 > 1 4 INCRBY foo 3 5 GET foo 6 > 4Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

EXPIRE / EXPIREAT / PERSISTJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

PipeliningJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

SORT - SQL in your NoSQL 1 SORT key 2 SORT key LIMIT 0 10 DESC 3 SORT key ALPHA 4 SORT page:6:visitors BY user_*->rank GET user_* DESCJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Append-Only File & SnapshotsJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Pub/Sub 1 SUBSCRIBE foo1 2 PSUBSCRIBE foo* 3 PUBLISH foo0 messageJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Master/Slave ReplicationJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Transactions: MULTI / EXEC / DISCARD Optimistic Locking with WATCHJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Upcoming FeaturesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Lua Scripting: EVAL / EVALSHA This is great.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

EVAL 1 EVAL <body> <num_keys_in_args> [<arg1> <arg2> ... <arg_N>] 1 GET A 2 EVAL "return redis.call(get, KEYS[1])" 1 AJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

EVAL Example: Atomic Conditional Decrement, Client-Side 1 WATCH foo 2 $val = GET foo 3 $newVal = max(0, $val - 1); // decrement if foo > 0 client-side 4 MULTI 5 SET foo $newVal 6 EXEC This may return -ERR, or will block.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

EVAL Example: Atomic Conditional Decrement, Server-Side 01 EVAL "local value = tonumber(redis.call(get, KEYS[1])) 02 if value == nil 03 then 04 return {err="Value at key is not integer"} 05 end 06 if value > tonumber(ARGV[1]) 07 then 08 value = value - 1 09 redis.call(set, KEYS[1], value) 10 end 11 return value" 1 foo 0 This is instant and can not fail.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Redis Cluster Almost as delayed as Duke Nukem ForeverJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Speed Redis is FAST. CPU is unlikely to be the bottleneck. What do you do when you run out of RAM?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Wait for Redis Cluster? If you can. Cluster looks great.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Use sharding, most client libraries can do it.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Use Edis, a protocol compatible storage-oriented server. http://inaka.github.com/edis/index.htmlJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Use it without persistence, store to disk/DB and load hot data in memory. A good example in these slides from WoogaJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Ok, great. But.. What is it good for?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

WEB SCALE THE CLOUD NOSQL ELASTIC HORIZONTAL SCALING VERTICAL TOOJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

BULLSHITJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Get real "If you do actually have to scale, then your database isnt going to magically do it for you."Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Really. What is it good for?Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Storing Sessions Caching Data Other typical Memcached usesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Caching: Newspaper Site Expensive reads Fast-changing pages Dynamic tracking of whats read for usersJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Caching: Newspaper Site Cache generic results: 1 SETEX <pageid>:content 600 <data> Store user read-state in sets 1 SADD <pageid>:views <userid> Combine and render for each user 1 GET <pageid>:content 2 SISMEMBER <pageid>:views <userid>Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Logging, games and other write-heavy usages 1 LPUSH logs "Log message" 2 // keep the last 1000 entries 3 LTRIM logs 0 999Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Highscore tables 1 ZADD scores 4290 <playerid> 2 ZADD scores 390 <playerid2> 3 // ... 4 ZREVRANK scores <playerid> // 0 5 ZREVRANGE scores 0 10 WITHSCORES Note: players will only be listed once since it is a set.Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Whos online now? 1 ZADD visits <unix> <userid> 2 ZADD visits 1327681399 52 3 ZADD visits 1327683245 18 4 ZRANGEBYSCORE visits <unix>-3600 <unix> 5 ZREMRANGEBYSCORE visits 0 <unix>-3600Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Job Queues Workers: 1 BRPOP queue Application: 1 LPUSH queue "job description" e.g. the Resque lib from GitHubJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Behavior Tracking 1 SETBIT click:<item>:<date> <userid> 1 2 GETBIT click:3:2012-01-28 55239Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Last but not least, it puts the fun back in DatafunbasesJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Usage with PHPJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Extension: phpredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Extension: phpredis Comes with a session handler, sharding, up to date. 1 $redis = new Redis(); 2 $redis->connect(127.0.0.1, 6379); 3 $redis->watch(x); 4 $val = $redis->get(x); 5 $newVal = max(0, $val - 1); 6 $result = $redis->multi() 7 ->set(x, $newVal) 8 ->exec(); 9 echo $result !== false ? Success : Race lost, try again; http://github.com/nicolasff/phpredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Extension: phpiredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Extension: phpiredis Basic hiredis bindings, protocol parsing, low level. 1 $redis = phpiredis_connect(127.0.0.1, 6379); 2 phpiredis_command($redis, WATCH x); 3 $val = phpiredis_command($redis, GET x); 4 $newVal = max(0, $val - 1); 5 phpiredis_command($redis, MULTI); 6 phpiredis_command($redis, SET x .$newVal); 7 $result = phpiredis_command($redis, EXEC); 8 echo $result !== false ? Success : Race lost, try again; http://github.com/seppo0010/phpiredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Plain PHP: PredisJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Plain PHP: Predis Very complete, sharding, master/slave auto-select, can use phpiredis for parsing. 01 $redis = new PredisClient(tcp://10.0.0.1:6379); 02 03 $options = array( 04 cas => true, // enable Check-and-Set 05 watch => x, 06 retry => 10, // automatic retries 07 ); 08 09 $result = $redis->multiExec($options, function($transaction) { 10 $val = $transaction->get(x); 11 $newVal = max(0, $val - 1); 12 $transaction->multi(); 13 $transaction->set(x, $newVal); 14 }); 15 16 echo $result !== false ? Success : Race lost 10 times, giving up; http://github.com/nrk/predis http://github.com/snc/SncRedisBundle (Symfony2 integration)Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Its fast. Its fun. Try it. try.redis-db.comJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Thank you. Slides: http://slides.seld.beJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Find Out More Redis.io Try Redis The Little Redis Book Redis Cluster SpecificationJordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Questions? jordi@nelm.io @seldaek Feedback: http://joind.in/4764Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be

Redis - Your Advanced In-Memory Key-Value Store

by Jordi Boggiano on Oct 10, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 12

Statistics

Redis - Your Advanced In-Memory Key-Value Store — Presentation Transcript

http://a0.twimg.com	6
http://paper.li	5
http://us-w1.rockmelt.com	1