Redis Use Patterns
An Introduction to the SQL Practitioner
@ItamarHaber #DevConTLV 2014
About
A Redis Geek and Chief Developers Advocate at .com
We provide Redis Cloud – an enterprise-class Redis service for
de...
What’s Redis? (REmote DIctionary Server)
• Open-source (BSD), in-memory, persist-able key-value advanced datastore
• Key-v...
Why Redis? Because It Is Fun!
• Simplicity  rich functionality, great flexibility
• Performance  easily serves 100K’s of...
Redis
Makes
You
.
.
.
.
THINK!
• about how data is stored
• about how data is accessed
• about efficiency
• about performa...
Pattern: Caching Calls to the DB
Motivation: quick responses, reduce load on DBMS
How: keep the statement's results using ...
STRINGs
• Are the most basic data type
• Are binary-safe
• Is used for storing:
• Strings (duh) – APPEND, GETRANGE,
SETRAN...
Pattern: Avoiding Calls to the DB
Motivation: server-side storage and sharing of data that doesn't need a
full-fledged RDB...
The HASH Data Type
• Acts as a Redis-within-Redis  contains key-value pairs
• Have their own commands: HINCRBY, HINCRBYFL...
Denormalization
• Non relational  no foreign keys, no
referential integrity constraints
• Thus, data normalization isn't ...
Pattern: Lists of Items
Motivation: keeping track of a sequence, e.g. last viewed profiles
How: use Redis' LIST data type
...
Key Points About Key Names
• Key names are "limited" to 512MB (also the values btw)
• To conserve RAM & CPU, try avoid usi...
Pattern: Queues (apropos the list data type)
Motivation: a producer-consumer use case, asynchronous job
management, e.g. p...
Is Redis ACID? (mostly) Yes!
• Redis is (mostly) single threaded, hence every
operation is
• Atomic
• Consistent
• Isolate...
Pattern: Searching
Motivation: finding keys in the database, for example all the users
How #1: use a LIST to store key nam...
Pattern: Indexing
Motivation: Redis doesn't have indices, you need to maintain them
How: the SET data type (a collection o...
Pattern: Relationships
Motivation: Redis doesn't have foreign keys, you need to maintain them
> SADD user:1:friends 3 4 5 ...
ZSETs (Sorted Sets)
I HAVE
CDO
IT'S LIKE
OCD
BUT ALL THE LETTERS ARE
IN ALPHABETICAL ORDER
AS THEY SHOULD BE
• Are just li...
Pattern: Sorting
Motivation: anything that needs to be sorted
How: ZSETs
> ZADD friends_count 3 1 1 2 999 3
> ZREVRANGE fr...
The SORT Command
• A command that sorts LISTs, SETs and SORTED SETs
• SORT's syntax is the most complex (comparatively) bu...
Pattern: Counting Things
Motivation: statistics, real-time analytics, dashboards, throttling
How #1: use the *INCR command...
Pattern: Counting Unique Items
How #1: SADD items and SCARD for the count
Problem: more unique items  more RAM 
How #2: ...
Wait, There's More!
• There are 107 additional commands that we didn't cover 
• Expiration and eviction policies
• Publis...
Where Next?
• Try the interactive demo and get a free 25MB Redis database in the
cloud at http://redislabs.com
• Need help...
Redis Use Patterns (DevconTLV June 2014)
Upcoming SlideShare
Loading in...5
×

Redis Use Patterns (DevconTLV June 2014)

6,452
-1

Published on

An introduction to Redis for the SQL practitioner, covering data types and common use cases.

The video of this session can be found at: https://www.youtube.com/watch?v=8Unaug_vmFI

Published in: Software, Technology
0 Comments
19 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,452
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
135
Comments
0
Likes
19
Embeds 0
No embeds

No notes for slide

Redis Use Patterns (DevconTLV June 2014)

  1. 1. Redis Use Patterns An Introduction to the SQL Practitioner @ItamarHaber #DevConTLV 2014
  2. 2. About A Redis Geek and Chief Developers Advocate at .com We provide Redis Cloud – an enterprise-class Redis service for developers (infinitely scalable, highly-available with auto failover, top performing hosted Redis off AWS, Google, Azure & IBM SoftLayer) Get your free t-shirt and sticker outside!
  3. 3. What’s Redis? (REmote DIctionary Server) • Open-source (BSD), in-memory, persist-able key-value advanced datastore • Key-value means something like CREATE TABLE redis ( k VARCHAR(512MB) NOT NULL, v VARCHAR(512MB), PRIMARY KEY (k) ); • 6 data types, 160 commands, blazing fast • Created in 2009 by @antirez (a.k.a Salvatore Sanfilippo) • Source: https://github.com/antirez/redis • Website: http://redis.io
  4. 4. Why Redis? Because It Is Fun! • Simplicity  rich functionality, great flexibility • Performance  easily serves 100K’s of ops/sec • Lightweight  ~ 2MB footprint • Production proven (name dropping)  Twitter Pintrest StackOverflow teowaki many more …
  5. 5. Redis Makes You . . . . THINK! • about how data is stored • about how data is accessed • about efficiency • about performance • about the network • … • Redis is a database construction kit • Beware of Maslow's "Golden" Gavel/Law of Instrument: "If all you have is a hammer, everything looks like a nail"
  6. 6. Pattern: Caching Calls to the DB Motivation: quick responses, reduce load on DBMS How: keep the statement's results using the Redis STRING data type def get_results(sql): hash = md5.new(sql).digest() result = redis.get(hash) if result is None: result = db.execute(sql) redis.set(hash, result) # or use redis.setex to set a TTL for the key return result
  7. 7. STRINGs • Are the most basic data type • Are binary-safe • Is used for storing: • Strings (duh) – APPEND, GETRANGE, SETRANGE, STRLEN • Integers – INCR, INCRBY, DECR, DECRBY • Floats – INCRBYFLOAT • Bits – SETBIT, GETBIT, BITPOS, BITCOUNT, BITOP http://xkcd.com/171/
  8. 8. Pattern: Avoiding Calls to the DB Motivation: server-side storage and sharing of data that doesn't need a full-fledged RDBMS, e.g. sessions and shopping carts How: depending on the case, use STRING or HASH to store data in Redis def add_to_cart(session, product, quantity): if quantity > 0: redis.hset('cart:' + session, product, quantity) else: redis.hrem('cart:' + session, product) def get_cart_contents(session): return redis.hgetall('cart:' + session)
  9. 9. The HASH Data Type • Acts as a Redis-within-Redis  contains key-value pairs • Have their own commands: HINCRBY, HINCRBYFLOAT, HLEN, HKEYS, HVALS… • Usually used for aggregation, i.e. keeping related data together for easy fetching/updating (remember that Redis is not a relational database). Example: Using separate keys Using hash aggregation user:1:id  1 user:1 id  1 user:1:fname  Foo fname  Foo user:1:lname  Bar lname  Bar user:1:email  foo@acme.com email  foo@acme.com
  10. 10. Denormalization • Non relational  no foreign keys, no referential integrity constraints • Thus, data normalization isn't practical • Be prepared to have duplicated data, e.g.: > HSET user:1 country Mordor > HSET user:2 country Mordor … • Tradeoff: Processing Complexity ↔ Data Volume
  11. 11. Pattern: Lists of Items Motivation: keeping track of a sequence, e.g. last viewed profiles How: use Redis' LIST data type def view_product(uid, product): redis.lpush('user:' + uid + ':viewed', product) redis.ltrim('user:' + uid + ':viewed', 0, 9) … def get_last_viewed_products(uid): return redis.lrange('user:' + uid + ':viewed', 0, -1)
  12. 12. Key Points About Key Names • Key names are "limited" to 512MB (also the values btw) • To conserve RAM & CPU, try avoid using unnecessarily_longish_names_for_your_redis_keys because they are more expensive to store and compare (unlike an RDBMS's column names, key names are saved for each key-value pair) • On the other hand, don't be too stringent (e.g 'u:<uid>:r') • Although not mandatory, the convention is to use colons (':') to separate the parts of the key's name • Your schema is your keys' names so keep them in order
  13. 13. Pattern: Queues (apropos the list data type) Motivation: a producer-consumer use case, asynchronous job management, e.g. processing photo uploads def enqueue(queue, item): redis.lpush(queue, item) def dequeue(queue): return redis.rpop(queue) # or use brpop for blocking pop
  14. 14. Is Redis ACID? (mostly) Yes! • Redis is (mostly) single threaded, hence every operation is • Atomic • Consistent • Isolated • WATCH/MULTI/EXEC allow something like transactions (no rollbacks) • Server-side Lua scripts ("stored procedures") also behave like transactions • Durability is configurable and is a tradeoff between efficiency and safety
  15. 15. Pattern: Searching Motivation: finding keys in the database, for example all the users How #1: use a LIST to store key names How #2: the *SCAN commands def do_something_with_all_users(): first = True cursor = 0 while cursor != 0 or first: first = False cursor, data = redis.scan(cursor, 'user:*') do_something(data)
  16. 16. Pattern: Indexing Motivation: Redis doesn't have indices, you need to maintain them How: the SET data type (a collection of unordered unique members) def update_country_idx(country, uid): redis.sadd('country:' + country, uid) def get_users_in_country(country): return redis.smembers('country:' + country)
  17. 17. Pattern: Relationships Motivation: Redis doesn't have foreign keys, you need to maintain them > SADD user:1:friends 3 4 5 // Foo is social and makes friends > SCARD user:1:friends // How many friends does Foo have? > SINTER user:1:friends user:2:friends // Common friends > SDIFF user:1:friends user:2:friends // Exclusive friends > SUNION user:1:friends user:2:friends // All the friends
  18. 18. ZSETs (Sorted Sets) I HAVE CDO IT'S LIKE OCD BUT ALL THE LETTERS ARE IN ALPHABETICAL ORDER AS THEY SHOULD BE • Are just like SETs: • Members are unique • ZADD, ZCARD, ZINCRBY, … • ZSET members have a score that's used for sorting • ZCOUNT, ZRANGE, ZRANGEBYSCORE • When the scores are identical, members are sorted alphabetically • Lexicographical ranges are also supported: • ZLEXCOUNT, ZRANGEBYLEX
  19. 19. Pattern: Sorting Motivation: anything that needs to be sorted How: ZSETs > ZADD friends_count 3 1 1 2 999 3 > ZREVRANGE friends_count 0 -1 3 1 2 Set members (uids) Scores (friends count)
  20. 20. The SORT Command • A command that sorts LISTs, SETs and SORTED SETs • SORT's syntax is the most complex (comparatively) but SQLers should feel right at home with it: SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination] • SORT is also expensive in terms of complexity  O(N+M*log(M)) • BTW, SORT is perhaps the only ad-hoc-like command in Redis
  21. 21. Pattern: Counting Things Motivation: statistics, real-time analytics, dashboards, throttling How #1: use the *INCR commands How #2: use a little bit of BIT* def user_log_login(uid): joined = redis.hget('user:' + uid, 'joined') d0 = datetime.strptime(joined, '%Y-%m-$d') d1 = datetime.date.today() delta = d1 – d0 redis.setbit('user:' + uid + ':logins', delta, 1) def user_logins_count(uid): return redis.bitcount( 'user:' + uid + ':logins', 0, -1) I love to COUNT(*) data! One datum, two data, three data! HA HA HA HA!
  22. 22. Pattern: Counting Unique Items How #1: SADD items and SCARD for the count Problem: more unique items  more RAM  How #2: the HyperLogLog data structure > PFADD counter item1 item2 item3 … • HLL is a probabilistic data structure that counts (PFCOUNT) unique items • Sacrifices accuracy: standard error of 0.81% • Gains: constant complexity and memory – 12KB per counter • Bonus: HLLs are merge-able with PFMERGE 
  23. 23. Wait, There's More! • There are 107 additional commands that we didn't cover  • Expiration and eviction policies • Publish/Subscribe • Data persistency and durability • Server-side scripting with Lua • Master-Slave(s) replication • High availability with Sentinel • Redis v3 == Cluster (currently in beta) • …
  24. 24. Where Next? • Try the interactive demo and get a free 25MB Redis database in the cloud at http://redislabs.com • Need help? • RTFM: http://redis.io/documentation • Ask the redis-db mailing list • Visit #redis on Freenode IRC • Email me: itamar@ .com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×