Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Get more than a cache back! - ConFoo Montreal


Published on

In this session, we’ll see that Redis is more than just an in-memory cache system we can use in our applications. Let’s explore what Redis is, what the different data types are and why we should care. And once we grasp how Redis stores its stuff, we’ll delve into how we can use it to its fullest extent: searching the key-value store, transactions, pub/sub support and scripting.

Published in: Technology
  • ⇒ ⇐ is a good website if you’re looking to get your essay written for you. You can also request things like research papers or dissertations. It’s really convenient and helpful.
    Are you sure you want to  Yes  No
    Your message goes here

Get more than a cache back! - ConFoo Montreal

  1. 1. RedisGet more than a cache back! Maarten Balliauw @maartenballiauw
  2. 2. Who am I? Maarten Balliauw Antwerp, Belgium Developer Advocate, JetBrains Founder, MyGet Focus on web & cloud @maartenballiauw
  3. 3. Agenda Redis Data types Transactions Pub/sub Scripting Sharding/partitioning Patterns
  4. 4. Redis
  5. 5. Redis “Redis is an open source, BSD licensed, networked, single-threaded, in-memory key- value cache and store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets, sorted sets, bitmaps and hyperloglogs.” REmote DIctionary Server
  6. 6. Redis Key-value cache and store (value can be a couple of things) In-memory (no persistence, but you can) Single-threaded (atomic operations & transactions) Networked (it’s a server and it does master/slave) Some other stuff (scripting, pub/sub, Sentinel, snapshot, …) Things to remember
  7. 7. Persistence? Dump memory to disk (faster restore after restart) AOF (append-only files) RDB (Redis DB snapshots) With or without: all your data must fit in memory
  8. 8. Redis 101 demo
  9. 9. The Little Redis Book By Karl Seguin
  10. 10. Data types Type Example Key Example value String cache:/home <html><head><title>Home page</title> Hash categories:1 Field name description numproducts Member Ships Pirate ships for sale 50000 Set categories:1:products 20 11 56 89 32 4 List products:20:comments [0] -> “That skull flag is awesome!” [1] -> “Parrot bit my finger, not happy!” [2] -> “Parrot keeps swearing at me.” Sorted set products:bestsellers Field Eye Patch Parrot Score 84632 82120 + Bitmap, Hyperloglog
  11. 11. Data types demo
  12. 12. Data types Type Example Key Example value String cache:/home user:nextid <html><head><title>Home page</title> 2 Hash user:1 Field name handle Member Maarten @maartenballiauw Set user:1:followers 10 42 99 85 List user:1:timeline [0] -> { ... } [1] -> { ... } [2] -> { ... } Sorted set trending:no:tags Field #ndcoslo #redis Score 84632 82120 + Bitmap, Hyperloglog
  13. 13. Keys Good practice: use a pattern for keys E.g. users:1:maarten Get by id: KEYS users:1:* GET ...... Get by name: KEYS users:*:maarten GET ...... O(N) operation – use with caution!
  14. 14. But... client-side? Lots of options!
  15. 15. Connecting to Redis (C# / Node – pick one!) demo
  16. 16. Transactions MULTI, EXEC, DISCARD, WATCH No rollbacks (just discard queue) Failures are because of you, not Redis Optimistic locking with WATCH Only execute transaction queue if watched keys did not change
  17. 17. Transactions demo
  18. 18. Pub/Sub (P)SUBSCRIBE, UNSUBSCRIBE, PUBLISH Subscribe to a queue SUBSCRIBE news (or PSUBSCRIBE news:*) Send data to a queue PUBLISH news “This just in!” (optional) Keyspace notifications CONFIG SET notify-keyspace-events KEA PSUBSCRIBE '__key*__:*' (or __keyspace@0__:foo)
  19. 19. Pub/Sub demo
  20. 20. Scripting Run Lua scripts on Redis EVAL ‘return “Hello, World!”’ 0 Scripts are cached (SCRIPT FLUSH) Scripts can be used as functions Although we must pass the body all the time (or use SCRIPT LOAD + EVALSHA) Helper functions available! Base, table, string, math, debug, struct, cjson, cmsgpack, redis.sha1hex
  21. 21. Scripting demo
  22. 22. What if I need more memory? Sharding On client (consistent hashing) Using a proxy (Twemproxy - Cluster On server (Redis Cluster) tip: always connect to >= 2 nodes var Redis = require('ioredis'); var cluster = new Redis.Cluster([ { port: 6380, host: '' }, { port: 6381, host: '' } ]); cluster.set('foo', 'bar'); cluster.get('foo', function (err, res) { // ... });
  23. 23. Patterns
  24. 24. When can I use Redis? Output cache Session state General-purpose cache (e.g. for database) “Cache with benefits” Pub/sub Generally: when use case maps and data fits in RAM
  25. 25. When should I avoid Redis? More data than can fit in RAM Can use multiple, but even then: SUM(RAM) == max. data size Data and query model fits well in a relational model Materialize certain queries in Redis if needed Avoid Redis for large objects... ...with lots of concurrent read/write operations
  26. 26. Counting stuff How would you count “likes” if you were Facebook? SQL UPDATE posts SET likes = likes + 1 WHERE postid = 1234 Locking! Will slow down the application... Redis INCR post:12345 (Every once in a while, check keys to persist)
  27. 27. Counting stuff (+ popularity) And how would you get the 5 most popular posts? Use a sorted set Elements are scored (= # of likes) We can use ZREVRANGE to get the top X posts ZREVRANGE posts:likes 0 5 withscores
  28. 28. Get the latest 5 product reviews No need to go to the database! Use a Redis List, truncated at 5 items Need more? Query the database
  29. 29. Rate limiting API should only allows 5 requests per 60 seconds Redis List Record 5 latest requests If we’re at 5, check the leftmost item versus current time 1 2 3 4 5 12:20:25 12:20:22 12:20:21 12:20:18 12:19:50
  30. 30. Autocompletion How to provide autocompletion over our million- records product catalog? SQL “LIKE” Lucene / ElasticSearch Redis Sorted Set
  31. 31. Intersecting collections “Which friends do we have in common?” Algorithm could be: Find friends that we have in common Find friends they have in common Score the # of connections
  32. 32. Sort We have a series of bugs Priority, Status, Title, ... A user follows a few of these bugs How to store? How to query? (give me my top 5 bugs by priority) hset bugs:1 priority 5 hset bugs:2 priority 2 hset bugs:3 priority 1 SORT bugsifollow BY bugs:*->priority
  33. 33. Conclusion
  34. 34. Conclusion Redis is not just a cache Data types Transactions Pub/sub Scripting Sharding/partitioning Patterns 1 thing to remember:
  35. 35. Thank you! @maartenballiauw