Redis Overview


Published on

An overview of Redis for Philly.rb on 2010-02-17

Published in: Technology

Redis Overview

  1. 1. Redis REmote DIctionary Server Ryan Findley, Philly.rb, 2010-02-16
  2. 2. Disclaimer • I’m not using this in production (yet).
  3. 3. Kind of like Memcache • Key-value store • All data lives in memory • Keys can expire (or not) • Fast, Light-weight
  4. 4. And More • Persistence • Multiple databases • Queryable keyspace • Support for integer counters • Higher level data structures • Atomic operations • Ability to paginate lists without mutating them • Master-slave replication • Optional VM feature (new)
  5. 5. More: Command Reference: http:// Other Goodness CommandReference • Written in C (C99 standard) • No dependencies • Detailed command reference that lists time complexity in Big-O notation. • Very easy to build / install (it’ll probably “just work”)
  6. 6. Persistence(1/2) - Snapshotting • This is the default behavior • Periodic, asynchronous dump to disk • Can be every N changes or every N seconds • For example, every 15 min if at least 1 change and every 5 min if at least 10 changes • Since data is written asynchronously, data can be lost during a crash.
  7. 7. Persistence(2/2) - Append Only File See more here: wiki/AppendOnlyFileHowto • Available as of version 1.1 • Works like a journal • Every write command is logged ASAP • Commands replayed when the server is restarted • Configurable speed/safety (safest by default)
  8. 8. More here: p/redis/wiki/ SelectCommand p/redis/wiki/ MoveCommand Multiple Databases • Use the SELECT command; 0-15 are valid by default, but you can add more in redis.conf • Useful for segmenting key namespaces, especially when key queries are needed • MOVE command can be used as a locking primitive
  9. 9. Command Reference
  10. 10. More: redis/wiki/ IntroductionToRedisDataTypes Data Structures • Strings • Strings-as-integers (used by INCR/DECR) • List of Strings • Set of Strings (unique list) • Sorted Set of Strings (and weights) (ZSET)
  11. 11. Command overview • Strings: get, set, increment, decrement • Lists: push, pop, length, range, trim • Sets: add, remove, move, length, intersect union, diff, random • Other: save, lastsave can be used to force & verify disk persistence
  12. 12. More: p/redis/wiki/ RpoplpushCommand Atomic Operations • LPUSH / RPUSH: append to head/tail of list • LPOP / RPOP: return & remove first/last element of list • RPOPLPUSH: return & remove the last element of source list and push to the head of the destination list • GETSET: set a key to a new value and return the old value • MGET/MSET: get or set multiple keys to multiple values • SMOVE: move a value from one set to another • No way to group commands (transactions)
  13. 13. More here: p/redis/wiki/ ReplicationHowto Replication • Slaves can be used for scalability or redundancy • A master can have multiple slaves. • Slaves are able to accept other slave connections • Redis replication is non-blocking on the master, but blocking on the slave (can’t respond to queries during initial sync) • Slaves are able to automatically reconnect after an outage
  14. 14. Blog post about Redis VM: post/redis-virtual- memory-story.html Optional VM feature • Doesn’t use OS virtual memory • Still new, may not scale well in certain situations. • Don’t know how this will affect replication (initial sync / re-sync may be slow, but shouldn’t block the master) • All keys stay in memory. Minimum req’s: • 1 million keys ~ 160M • 10 million keys ~ 1.6G
  15. 15. Syntax highlighted sample here: b16679cbb5b9573e078c Setup • git clone git:// && cd redis • make • ./redis-server • make test • (review sample config file)
  16. 16. Redis in Ruby • Redis library: “gem install redis” • All Redis commands can be called like methods on the connection object as described in the Command Reference
  17. 17. Example Uses • A Memcached replacement (fast) • A work queue (sets & lists) • Fast auto-completion (persistent, queryable) • ORDER BY RAND() replacement (sets)
  18. 18. Memcache Replacement • Why? Same as Memcached, but you get more “for free”: • Many databases from a single instance of Redis (instead of using namespaces) • Ability to easily backup/transfer state (dump.rdb) • Watch live commands on a running instance with the MONITOR command (instead of restarting with -v) • Opportunity to use Redis for other things once you’ve switched
  19. 19. Memcache Replacement Note: namespace was replaced with a DB number • It’s pretty easy:
  20. 20. More: Redis allows negative http:/ Work Queue / 2010/01/usage-stats-and-redis.html array indices to count backwards from the end. A VERY simple example • LPUSH and RPOP were made for this: 1 2 3 4 5 6
  21. 21. “Not a ‘better DelayedJob’”. More: Compare them and see what is best for your project defunkt/resque blog/542-introducing- Work Queue - Resque • Created by Chris Wanstrath. Heavily inspired by DelayedJob. Currently used by GitHub. Provides: • A Ruby library for creating, querying, and processing jobs • A Rake task for starting a worker which processes jobs • A Sinatra app for monitoring queues, jobs, and workers.
  22. 22. Fast Auto-Completion • Lives in memory, doesn’t expire, persists
  23. 23. Replace MySQL ORDER More: BY RAND() usecase-replacing-mysql-order-by-rand • ORDER BY RAND() is very slow (even with a LIMIT clause) • Duplicating the list of IDs as a Redis set is relatively cheap • SRANDMEMBER is fast
  24. 24. Other Interesting Uses of Redis • Nginx HTTP Redis module for caching with Redis (ngx_http_redis) • Realtime site usage statistics, use alongside Google analytics • EZMobius’ Fair Work Scheduler example • Sikwamic: Simple Redis-backed Comet server • RestMQ - A REST/JSON/HTTP based message queue built on Redis • redis-textsearch - A simple full-text search for multiple data stores
  25. 25. Further reading • • • • a_collection_of_redis_use_cases.html • comet.html • • mysql-order-by-rand •