Successfully reported this slideshow.

Redis and Groovy and Grails - gr8conf 2011

69

Share

Upcoming SlideShare
Redis介绍
Redis介绍
Loading in …3
×
1 of 83
1 of 83

Redis and Groovy and Grails - gr8conf 2011

69

Share

Download to read offline

Presentation at the gr8conf US 2011 on using Redis with some additional info about usage within Groovy and Grails applications.

Presentation at the gr8conf US 2011 on using Redis with some additional info about usage within Groovy and Grails applications.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Redis and Groovy and Grails - gr8conf 2011

  1. 1. Redis & Groovy & Grails by Ted Naleid http://naleid.com Monday, June 20, 2011
  2. 2. “Redis is a collection of data structures exposed over the network” from: http://nosql.mypopescu.com/post/5403851771/what-is-redis Monday, June 20, 2011
  3. 3. key/value store like memcached on steroids Monday, June 20, 2011
  4. 4. Strings, Integers, Lists, Hashes, Sets & Sorted Sets (& commonly expected operations with each data type) Monday, June 20, 2011
  5. 5. in-memory storage Monday, June 20, 2011
  6. 6. single-threaded Monday, June 20, 2011
  7. 7. fast Monday, June 20, 2011
  8. 8. “Memory is the new Disk, Disk is the new Tape” - Jim Gray Monday, June 20, 2011
  9. 9. Relative Latency CPU Register - 1x L2 Cache - 10x Memory - 100x Disk - 10,000,000x analogy from “Redis - Memory as the New Disk” - Tim Lossen & http://en.wikipedia.org/wiki/Orders_of_magnitude_(speed) Monday, June 20, 2011
  10. 10. CPU Register 1 yard photo: http://www.flickr.com/photos/limonada/904754668/ Monday, June 20, 2011
  11. 11. L2 Cache 10 yards photo: http://www.flickr.com/photos/plentyofants/2749262107 Monday, June 20, 2011
  12. 12. Memory 100 yards photo: http://www.flickr.com/photos/billmcintyre/264905933 Monday, June 20, 2011
  13. 13. Disk Minneapolis to New York to Miami to Seattle ~5600 miles Monday, June 20, 2011
  14. 14. Simple wire protocol that matches API Monday, June 20, 2011
  15. 15. % telnet localhost 6379 Escape character is '^]'. set foo bar +OK get foo $3 bar rpush mylist first :1 rpush mylist second :2 lrange mylist 0 -1 *2 $5 first $6 second Monday, June 20, 2011
  16. 16. clients for every* language *well not every language, but all the popular/semi-popular ones, you can easily write one if your language doesn’t have one Monday, June 20, 2011
  17. 17. No dependencies (451KB download) Monday, June 20, 2011
  18. 18. Used in production by high traffic sites that you’ve used Monday, June 20, 2011
  19. 19. stackoverflow, craigslist, github, disqus, digg & blizzard entertainment Monday, June 20, 2011
  20. 20. financially supported by VMware/SpringSource Monday, June 20, 2011
  21. 21. simple data structures make redis flexible Monday, June 20, 2011
  22. 22. write-through caching/ memoization Monday, June 20, 2011
  23. 23. producer/consumer message queues Monday, June 20, 2011
  24. 24. publish/subscribe Monday, June 20, 2011
  25. 25. atomic sequences/counters Monday, June 20, 2011
  26. 26. other uses... distributed locks, tag clouds, session tokens, auto-complete prefixes, API rate limiting, leaderboards, capped logs, random set items, A/B testing data storage, unique per user product pricing/sorting Monday, June 20, 2011
  27. 27. Redis Commands Monday, June 20, 2011
  28. 28. Great Online Reference http://redis.io/commands Monday, June 20, 2011
  29. 29. Strings set Redis REPL Groovy > set foo bar redis.set("foo", "bar") foo bar OK OK Monday, June 20, 2011
  30. 30. Strings get Redis REPL Groovy foo bar > get foo redis.get("foo") "bar" "bar" bar Monday, June 20, 2011
  31. 31. Exists exists (check key existence) Redis REPL Groovy foo (nil) > exists foo redis.exists("foo") (integer) 0 <= false 0 Monday, June 20, 2011
  32. 32. Integers incr (increment) Redis REPL Groovy foo 1 > incr foo redis.incr("foo") (integer) 1 <= 1 1 Monday, June 20, 2011
  33. 33. Integers decr (decrement) Redis REPL Groovy foo 0 > decr foo redis.decr("foo") (integer) 0 <= 0 0 Monday, June 20, 2011
  34. 34. Lists rpush (right push) Redis REPL Groovy > rpush foo baz redis.rpush("foo", "baz") foo baz (integer) 1 <= 1 Monday, June 20, 2011
  35. 35. Lists rpush (right push) Redis REPL Groovy > rpush foo qux redis.rpush("foo", "qux") foo baz qux (integer) 2 <= 2 Monday, June 20, 2011
  36. 36. Lists lpush (left push) Redis REPL Groovy > lpush foo bar redis.lpush("foo", "bar") foo bar baz qux (integer) 3 <= 3 Monday, June 20, 2011
  37. 37. Lists lrange (slice of list) Redis REPL Groovy foo bar baz qux > lrange foo 0 -1 redis.lrange("foo", 0, -1) 1) "bar" <= [bar, baz, qux] 2) "baz" 3) "qux" bar baz qux Monday, June 20, 2011
  38. 38. Lists lrange (slice of list) Redis REPL Groovy foo bar baz qux > lrange foo 0 1 redis.lrange("foo", 0, 1) 1) "bar" <= [bar, baz] 2) "baz" bar baz Monday, June 20, 2011
  39. 39. Lists lpop (left pop) Redis REPL Groovy foo baz qux > lpop foo redis.lpop("foo") "bar" <= "bar" bar Monday, June 20, 2011
  40. 40. Lists rpop (right pop) Redis REPL Groovy foo baz > rpop foo redis.rpop("foo") "qux" <= "qux" qux Monday, June 20, 2011
  41. 41. Hashes hset (set key → value) Redis REPL Groovy > hset foo bar baz redis.hset("foo", "bar", "baz") foo bar baz (integer) 1 <= 1 Monday, June 20, 2011
  42. 42. Hashes hset (set key → value) Redis REPL Groovy bar baz > hset foo qux quxx redis.hset("foo", "qux", "quxx") foo (integer) 1 <= 1 qux quxx Monday, June 20, 2011
  43. 43. Hashes hget (get value for key) Redis REPL Groovy bar baz foo qux quxx > hget foo bar redis.hget("foo", "bar") "baz" <= "baz" baz Monday, June 20, 2011
  44. 44. Hashes hgetall (get all keys/values) Redis REPL Groovy bar baz foo > hgetall foo qux quxx 1) "bar" redis.hgetAll("foo") 2) "baz" <= [bar:baz, qux:quxx] 3) "qux" 4) "quxx" bar baz qux quxx Monday, June 20, 2011
  45. 45. Hashes hvals (hash values) Redis REPL Groovy bar baz foo qux quxx > hvals foo redis.hvals("foo") 1) "baz" <= [baz, quxx] 2) "quxx" baz quxx Monday, June 20, 2011
  46. 46. Hashes hkeys (hash keys) Redis REPL Groovy bar baz foo qux quxx > hkeys foo redis.hkeys("foo") 1) "bar" <= [bar, qux] 2) "qux" bar qux Monday, June 20, 2011
  47. 47. Sets sadd (set add) Redis REPL Groovy > sadd m1 jan redis.sadd("m1", "jan") m1 jan (integer) 1 <= 1 Monday, June 20, 2011
  48. 48. Sets sadd (set add) Redis REPL Groovy feb > sadd m1 feb redis.sadd("m1", "feb") m1 (integer) 1 <= 1 jan Monday, June 20, 2011
  49. 49. Sets sismember (membership test) Redis REPL Groovy feb m1 jan > sismember m1 jan redis.sismember("m1", "jan") (integer) 1 <= true 1 Monday, June 20, 2011
  50. 50. Sets sismember (membership test) Redis REPL Groovy feb m1 jan > sismember m1 mar redis.sismember("m1", "mar") (integer) 0 <= false 0 Monday, June 20, 2011
  51. 51. Sets smembers (get full set) Redis REPL Groovy feb m1 jan > smembers m1 redis.smembers("m1") 1) "feb" <= [feb, jan] 2) "jan" feb jan Monday, June 20, 2011
  52. 52. Sets sinter (set intersection) Redis REPL Groovy feb feb m1 m2 jan mar > sinter m1 m2 redis.sinter("m1", "m2") 1) "feb" <= ["feb"] feb Monday, June 20, 2011
  53. 53. Sets sdiff (set difference) Redis REPL Groovy feb feb m1 m2 jan mar > sdiff m1 m2 redis.sdiff("m1", "m2") 1) "jan" <= ["jan"] jan Monday, June 20, 2011
  54. 54. Sets sunion (set union) Redis REPL Groovy feb feb m1 m2 > sunion m1 m2 jan mar 1) "mar" redis.sunion("m1", "m2") 2) "jan" mar <= ["mar", "jan", "feb"] 3) "feb" jan feb Monday, June 20, 2011
  55. 55. Sorted Sets zadd (add with score) Redis REPL Groovy > zadd z1 1 jan redis.zadd("z1", 1, "jan") z1 1 jan (integer) 1 <= 1 Monday, June 20, 2011
  56. 56. Sorted Sets zscore (score for member) Redis REPL Groovy 1 jan z1 2 feb > zscore z1 feb 3 mar redis.zscore("z1", "feb") "2" <= 2.0 2 Monday, June 20, 2011
  57. 57. Sorted Sets zrange (sorted subset) Redis REPL Groovy 1 jan z1 2 feb > zrange z1 0 1 withscores 1) "jan" 3 mar redis.zrangeWithScores("z1", 0, 1) 2) "1" <= [["jan", 1], ["feb", 2]] 3) "feb" 4) "2" 1 jan 2 feb Monday, June 20, 2011
  58. 58. Sorted Sets zrangebyscore (subset having score range) Redis REPL Groovy 1 jan z1 2 feb > zrangebyscore z1 2 3 withscores 1) "feb" 3 mar redis.zrangeByScoreWithScores("z1",2,3) 2) "2" <= [["feb", 2], ["mar", 3]] 3) "mar" 4) "3" 2 feb 3 mar Monday, June 20, 2011
  59. 59. Groovy Usage Monday, June 20, 2011
  60. 60. Grape @Grab Annotation #! /usr/bin/env groovy @Grab('redis.clients:jedis:2.0.0') def redis = new redis.clients.jedis.Jedis("localhost") assert "PONG" == redis.ping() Monday, June 20, 2011
  61. 61. Producer/Consumer Example Monday, June 20, 2011
  62. 62. Producer pushes work on a list with lpush @Grab('redis.clients:jedis:2.0.0') redis = new redis.clients.jedis.Jedis("localhost") args.each { redis.lpush("welcome-wagon", it) } Monday, June 20, 2011
  63. 63. Consumer uses blpop (blocking left pop from list) @Grab('redis.clients:jedis:2.0.0') redis = new redis.clients.jedis.Jedis("localhost") println "Joining the welcome-wagon!" while (true) { def name = redis.blpop(0, "welcome-wagon")[1] println "Welcome ${name}!" } Monday, June 20, 2011
  64. 64. Mass Producer srandmember to randomly pick female name from set @Grab('redis.clients:jedis:2.0.0') redis = new redis.clients.jedis.Jedis("localhost") if (!redis.exists("female-names")) { new File("./female-names.txt").eachLine {redis.sadd("female-names",it)} } for (i in 1..100000) { redis.lpush("welcome-wagon", redis.srandmember("female-names")) if (i % 1000 == 0) println "Adding $i" } female-names.txt from: http://antirez.com/post/autocomplete-with-redis.html Monday, June 20, 2011
  65. 65. Groovy Demo Monday, June 20, 2011
  66. 66. Grails Redis Plugin https://github.com/grails-plugins/grails-redis Monday, June 20, 2011
  67. 67. Plugin Config in Config.xml grails { redis { poolConfig { // jedis pool specific tweaks here, see jedis docs & src // ex: numTestsPerEvictionRun = 4 } port = 6379 host = "localhost" } } Monday, June 20, 2011
  68. 68. Provides Caching Through Memoization Monday, June 20, 2011
  69. 69. RedisTagLib <redis:memoize key="mykey" expire="3600"> <!-- insert expensive to generate GSP content here content will be executed once, subsequent calls will pull from redis (redis.get(“mykey”)) till the key expires --> </redis:memoize> Monday, June 20, 2011
  70. 70. RedisService Spring bean wraps pool connection // overrides propertyMissing and methodMissing to delegate to redis def redisService redisService.foo = "bar" assert "bar" == redisService.foo redisService.sadd("months", "february") assert true == redisService.sismember("months", "february") Monday, June 20, 2011
  71. 71. RedisService template methods manage pooled Redis connection redisService.withRedis { Jedis redis -> redis.set("foo", "bar") } Monday, June 20, 2011
  72. 72. RedisService template methods manage pooled Redis connection redisService.withRedis { Jedis redis -> redisService.withTransaction { Transaction transaction -> transaction.set("qux", "baz") assertNull redis.get("qux") } assertEquals "baz", redis.get("qux") } Monday, June 20, 2011
  73. 73. RedisService String memoization redisService.memoize("my-key") { Jedis redis -> // expensive operation we only want to execute once } def ONE_HOUR = 3600 // with optional timeout in seconds redisService.memoize("my-key-with-timeout", ONE_HOUR) { Jedis redis -> // expensive operation we want to execute every hour } Monday, June 20, 2011
  74. 74. RedisService Domain Class memoization (stores IDs hydrates from DB) def key = "user:$id:friends-books" redisService.memoizeDomainList(Book, key, ONE_HOUR) { redis -> // expensive process to calculate all friend’s books // stores list of Book ids, hydrates them from DB } Monday, June 20, 2011
  75. 75. Example Showing Products with Sort/Filter/Pagination Criteria Monday, June 20, 2011
  76. 76. Other Memoization Methods memoizeHash, memoizeHashField, memoizeScore (sorted set score) Monday, June 20, 2011
  77. 77. Grails Redis-GORM Plugin http://grails.github.com/inconsequential/redis/ Monday, June 20, 2011
  78. 78. Uses SpringData to abstract data layer Monday, June 20, 2011
  79. 79. Can be used in conjunction with Hibernate Monday, June 20, 2011
  80. 80. Partial support for GORM including Dynamic Finders, Criteria, Named Queries and “Transactions” Monday, June 20, 2011
  81. 81. Limitations It requires explicit index mapping on fields you want to query package com.example class Author { String name static mapWith = "redis" static hasMany = [books: Book] static mapping = { name index:true } } Monday, June 20, 2011
  82. 82. Under The Covers MONITOR output for new Author(name: "Stephen King").save() 1308027697.922839 "INCR" "com.example.Author.next_id" 1308027697.940021 "HMSET" "com.example.Author:1" "name" "Stephen King" "version" "0" 1308027697.940412 "SADD" "com.example.Author.all" "1" 1308027697.943318 "SADD" "com.example.Author:id:1" "1" 1308027697.943763 "ZADD" "com.example.Author:id:sorted" "1.0" "1" 1308027697.944911 "SADD" "com.example.Author:name:Stephen+King" "1" Monday, June 20, 2011
  83. 83. Questions? Monday, June 20, 2011

×