Redis Use Patterns (DevconTLV June 2014)
Upcoming SlideShare
Loading in...5
×
 

Redis Use Patterns (DevconTLV June 2014)

on

  • 3,715 views

An introduction to Redis for the SQL practitioner, covering data types and common use cases.

An introduction to Redis for the SQL practitioner, covering data types and common use cases.

The video of this session can be found at: https://www.youtube.com/watch?v=8Unaug_vmFI

Statistics

Views

Total Views
3,715
Views on SlideShare
3,614
Embed Views
101

Actions

Likes
1
Downloads
18
Comments
0

5 Embeds 101

https://twitter.com 77
http://redislabs.com 14
https://redislabs.com 7
http://www.slideee.com 2
https://qa.redislabs.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Redis Use Patterns (DevconTLV June 2014) Redis Use Patterns (DevconTLV June 2014) Presentation Transcript

  • Redis Use Patterns An Introduction to the SQL Practitioner @ItamarHaber #DevConTLV 2014
  • About A Redis Geek and Chief Developers Advocate at .com We provide Redis Cloud – an enterprise-class Redis service for developers (infinitely scalable, highly-available with auto failover, top performing hosted Redis off AWS, Google, Azure & IBM SoftLayer) Get your free t-shirt and sticker outside!
  • What’s Redis? (REmote DIctionary Server) • Open-source (BSD), in-memory, persist-able key-value advanced datastore • Key-value means something like CREATE TABLE redis ( k VARCHAR(512MB) NOT NULL, v VARCHAR(512MB), PRIMARY KEY (k) ); • 6 data types, 160 commands, blazing fast • Created in 2009 by @antirez (a.k.a Salvatore Sanfilippo) • Source: https://github.com/antirez/redis • Website: http://redis.io
  • Why Redis? Because It Is Fun! • Simplicity  rich functionality, great flexibility • Performance  easily serves 100K’s of ops/sec • Lightweight  ~ 2MB footprint • Production proven (name dropping)  Twitter Pintrest StackOverflow teowaki many more …
  • Redis Makes You . . . . THINK! • about how data is stored • about how data is accessed • about efficiency • about performance • about the network • … • Redis is a database construction kit • Beware of Maslow's "Golden" Gavel/Law of Instrument: "If all you have is a hammer, everything looks like a nail"
  • Pattern: Caching Calls to the DB Motivation: quick responses, reduce load on DBMS How: keep the statement's results using the Redis STRING data type def get_results(sql): hash = md5.new(sql).digest() result = redis.get(hash) if result is None: result = db.execute(sql) redis.set(hash, result) # or use redis.setex to set a TTL for the key return result
  • STRINGs • Are the most basic data type • Are binary-safe • Is used for storing: • Strings (duh) – APPEND, GETRANGE, SETRANGE, STRLEN • Integers – INCR, INCRBY, DECR, DECRBY • Floats – INCRBYFLOAT • Bits – SETBIT, GETBIT, BITPOS, BITCOUNT, BITOP http://xkcd.com/171/
  • Pattern: Avoiding Calls to the DB Motivation: server-side storage and sharing of data that doesn't need a full-fledged RDBMS, e.g. sessions and shopping carts How: depending on the case, use STRING or HASH to store data in Redis def add_to_cart(session, product, quantity): if quantity > 0: redis.hset('cart:' + session, product, quantity) else: redis.hrem('cart:' + session, product) def get_cart_contents(session): return redis.hgetall('cart:' + session)
  • The HASH Data Type • Acts as a Redis-within-Redis  contains key-value pairs • Have their own commands: HINCRBY, HINCRBYFLOAT, HLEN, HKEYS, HVALS… • Usually used for aggregation, i.e. keeping related data together for easy fetching/updating (remember that Redis is not a relational database). Example: Using separate keys Using hash aggregation user:1:id  1 user:1 id  1 user:1:fname  Foo fname  Foo user:1:lname  Bar lname  Bar user:1:email  foo@acme.com email  foo@acme.com
  • Denormalization • Non relational  no foreign keys, no referential integrity constraints • Thus, data normalization isn't practical • Be prepared to have duplicated data, e.g.: > HSET user:1 country Mordor > HSET user:2 country Mordor … • Tradeoff: Processing Complexity ↔ Data Volume
  • Pattern: Lists of Items Motivation: keeping track of a sequence, e.g. last viewed profiles How: use Redis' LIST data type def view_product(uid, product): redis.lpush('user:' + uid + ':viewed', product) redis.ltrim('user:' + uid + ':viewed', 0, 9) … def get_last_viewed_products(uid): return redis.lrange('user:' + uid + ':viewed', 0, -1)
  • Key Points About Key Names • Key names are "limited" to 512MB (also the values btw) • To conserve RAM & CPU, try avoid using unnecessarily_longish_names_for_your_redis_keys because they are more expensive to store and compare (unlike an RDBMS's column names, key names are saved for each key-value pair) • On the other hand, don't be too stringent (e.g 'u:<uid>:r') • Although not mandatory, the convention is to use colons (':') to separate the parts of the key's name • Your schema is your keys' names so keep them in order
  • Pattern: Queues (apropos the list data type) Motivation: a producer-consumer use case, asynchronous job management, e.g. processing photo uploads def enqueue(queue, item): redis.lpush(queue, item) def dequeue(queue): return redis.rpop(queue) # or use brpop for blocking pop
  • Is Redis ACID? (mostly) Yes! • Redis is (mostly) single threaded, hence every operation is • Atomic • Consistent • Isolated • WATCH/MULTI/EXEC allow something like transactions (no rollbacks) • Server-side Lua scripts ("stored procedures") also behave like transactions • Durability is configurable and is a tradeoff between efficiency and safety
  • Pattern: Searching Motivation: finding keys in the database, for example all the users How #1: use a LIST to store key names How #2: the *SCAN commands def do_something_with_all_users(): first = True cursor = 0 while cursor != 0 or first: first = False cursor, data = redis.scan(cursor, 'user:*') do_something(data)
  • Pattern: Indexing Motivation: Redis doesn't have indices, you need to maintain them How: the SET data type (a collection of unordered unique members) def update_country_idx(country, uid): redis.sadd('country:' + country, uid) def get_users_in_country(country): return redis.smembers('country:' + country)
  • Pattern: Relationships Motivation: Redis doesn't have foreign keys, you need to maintain them > SADD user:1:friends 3 4 5 // Foo is social and makes friends > SCARD user:1:friends // How many friends does Foo have? > SINTER user:1:friends user:2:friends // Common friends > SDIFF user:1:friends user:2:friends // Exclusive friends > SUNION user:1:friends user:2:friends // All the friends
  • ZSETs (Sorted Sets) I HAVE CDO IT'S LIKE OCD BUT ALL THE LETTERS ARE IN ALPHABETICAL ORDER AS THEY SHOULD BE • Are just like SETs: • Members are unique • ZADD, ZCARD, ZINCRBY, … • ZSET members have a score that's used for sorting • ZCOUNT, ZRANGE, ZRANGEBYSCORE • When the scores are identical, members are sorted alphabetically • Lexicographical ranges are also supported: • ZLEXCOUNT, ZRANGEBYLEX
  • Pattern: Sorting Motivation: anything that needs to be sorted How: ZSETs > ZADD friends_count 3 1 1 2 999 3 > ZREVRANGE friends_count 0 -1 3 1 2 Set members (uids) Scores (friends count)
  • The SORT Command • A command that sorts LISTs, SETs and SORTED SETs • SORT's syntax is the most complex (comparatively) but SQLers should feel right at home with it: SORT key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE destination] • SORT is also expensive in terms of complexity  O(N+M*log(M)) • BTW, SORT is perhaps the only ad-hoc-like command in Redis
  • Pattern: Counting Things Motivation: statistics, real-time analytics, dashboards, throttling How #1: use the *INCR commands How #2: use a little bit of BIT* def user_log_login(uid): joined = redis.hget('user:' + uid, 'joined') d0 = datetime.strptime(joined, '%Y-%m-$d') d1 = datetime.date.today() delta = d1 – d0 redis.setbit('user:' + uid + ':logins', delta, 1) def user_logins_count(uid): return redis.bitcount( 'user:' + uid + ':logins', 0, -1) I love to COUNT(*) data! One datum, two data, three data! HA HA HA HA!
  • Pattern: Counting Unique Items How #1: SADD items and SCARD for the count Problem: more unique items  more RAM  How #2: the HyperLogLog data structure > PFADD counter item1 item2 item3 … • HLL is a probabilistic data structure that counts (PFCOUNT) unique items • Sacrifices accuracy: standard error of 0.81% • Gains: constant complexity and memory – 12KB per counter • Bonus: HLLs are merge-able with PFMERGE 
  • Wait, There's More! • There are 107 additional commands that we didn't cover  • Expiration and eviction policies • Publish/Subscribe • Data persistency and durability • Server-side scripting with Lua • Master-Slave(s) replication • High availability with Sentinel • Redis v3 == Cluster (currently in beta) • …
  • Where Next? • Try the interactive demo and get a free 25MB Redis database in the cloud at http://redislabs.com • Need help? • RTFM: http://redis.io/documentation • Ask the redis-db mailing list • Visit #redis on Freenode IRC • Email me: itamar@ .com