Redis and Groovy and Grails - gr8conf 2011Presentation Transcript
Redis & Groovy & Grails by Ted Naleid http://naleid.comMonday, June 20, 2011
“Redis is a collection of data structures exposed over the network” from: http://nosql.mypopescu.com/post/5403851771/what-is-redisMonday, June 20, 2011
key/value store like memcached on steroidsMonday, June 20, 2011
Strings, Integers, Lists, Hashes, Sets & Sorted Sets (& commonly expected operations with each data type)Monday, June 20, 2011
in-memory storageMonday, June 20, 2011
single-threadedMonday, June 20, 2011
fastMonday, June 20, 2011
“Memory is the new Disk, Disk is the new Tape” - Jim GrayMonday, June 20, 2011
Relative Latency CPU Register - 1x L2 Cache - 10x Memory - 100x Disk - 10,000,000x analogy from “Redis - Memory as the New Disk” - Tim Lossen & http://en.wikipedia.org/wiki/Orders_of_magnitude_(speed)Monday, June 20, 2011
CPU Register 1 yard photo: http://www.flickr.com/photos/limonada/904754668/Monday, June 20, 2011
L2 Cache 10 yards photo: http://www.flickr.com/photos/plentyofants/2749262107Monday, June 20, 2011
Memory 100 yards photo: http://www.flickr.com/photos/billmcintyre/264905933Monday, June 20, 2011
Disk Minneapolis to New York to Miami to Seattle ~5600 milesMonday, June 20, 2011
Simple wire protocol that matches APIMonday, June 20, 2011
% telnet localhost 6379 Escape character is ^]. set foo bar +OK get foo $3 bar rpush mylist first :1 rpush mylist second :2 lrange mylist 0 -1 *2 $5 first $6 secondMonday, June 20, 2011
clients for every* language *well not every language, but all the popular/semi-popular ones, you can easily write one if your language doesn’t have oneMonday, June 20, 2011
No dependencies (451KB download)Monday, June 20, 2011
Used in production by high traffic sites that you’ve usedMonday, June 20, 2011
financially supported by VMware/SpringSourceMonday, June 20, 2011
simple data structures make redis flexibleMonday, June 20, 2011
write-through caching/ memoizationMonday, June 20, 2011
producer/consumer message queuesMonday, June 20, 2011
publish/subscribeMonday, June 20, 2011
atomic sequences/countersMonday, June 20, 2011
other uses... distributed locks, tag clouds, session tokens, auto-complete prefixes, API rate limiting, leaderboards, capped logs, random set items, A/B testing data storage, unique per user product pricing/sortingMonday, June 20, 2011
Redis CommandsMonday, June 20, 2011
Great Online Reference http://redis.io/commandsMonday, June 20, 2011
Strings set Redis REPL Groovy > set foo bar redis.set("foo", "bar") foo bar OK OKMonday, June 20, 2011
Strings get Redis REPL Groovy foo bar > get foo redis.get("foo") "bar" "bar" barMonday, June 20, 2011
Hashes hkeys (hash keys) Redis REPL Groovy bar baz foo qux quxx > hkeys foo redis.hkeys("foo") 1) "bar" <= [bar, qux] 2) "qux" bar quxMonday, June 20, 2011
Sets sadd (set add) Redis REPL Groovy > sadd m1 jan redis.sadd("m1", "jan") m1 jan (integer) 1 <= 1Monday, June 20, 2011
Sets sadd (set add) Redis REPL Groovy feb > sadd m1 feb redis.sadd("m1", "feb") m1 (integer) 1 <= 1 janMonday, June 20, 2011
Sets sismember (membership test) Redis REPL Groovy feb m1 jan > sismember m1 jan redis.sismember("m1", "jan") (integer) 1 <= true 1Monday, June 20, 2011
Sets sismember (membership test) Redis REPL Groovy feb m1 jan > sismember m1 mar redis.sismember("m1", "mar") (integer) 0 <= false 0Monday, June 20, 2011
Sets smembers (get full set) Redis REPL Groovy feb m1 jan > smembers m1 redis.smembers("m1") 1) "feb" <= [feb, jan] 2) "jan" feb janMonday, June 20, 2011
Sets sinter (set intersection) Redis REPL Groovy feb feb m1 m2 jan mar > sinter m1 m2 redis.sinter("m1", "m2") 1) "feb" <= ["feb"] febMonday, June 20, 2011
Sets sdiff (set difference) Redis REPL Groovy feb feb m1 m2 jan mar > sdiff m1 m2 redis.sdiff("m1", "m2") 1) "jan" <= ["jan"] janMonday, June 20, 2011
Sets sunion (set union) Redis REPL Groovy feb feb m1 m2 > sunion m1 m2 jan mar 1) "mar" redis.sunion("m1", "m2") 2) "jan" mar <= ["mar", "jan", "feb"] 3) "feb" jan febMonday, June 20, 2011
Sorted Sets zadd (add with score) Redis REPL Groovy > zadd z1 1 jan redis.zadd("z1", 1, "jan") z1 1 jan (integer) 1 <= 1Monday, June 20, 2011
Sorted Sets zscore (score for member) Redis REPL Groovy 1 jan z1 2 feb > zscore z1 feb 3 mar redis.zscore("z1", "feb") "2" <= 2.0 2Monday, June 20, 2011
Sorted Sets zrange (sorted subset) Redis REPL Groovy 1 jan z1 2 feb > zrange z1 0 1 withscores 1) "jan" 3 mar redis.zrangeWithScores("z1", 0, 1) 2) "1" <= [["jan", 1], ["feb", 2]] 3) "feb" 4) "2" 1 jan 2 febMonday, June 20, 2011
Sorted Sets zrangebyscore (subset having score range) Redis REPL Groovy 1 jan z1 2 feb > zrangebyscore z1 2 3 withscores 1) "feb" 3 mar redis.zrangeByScoreWithScores("z1",2,3) 2) "2" <= [["feb", 2], ["mar", 3]] 3) "mar" 4) "3" 2 feb 3 marMonday, June 20, 2011
Groovy UsageMonday, June 20, 2011
Grape @Grab Annotation #! /usr/bin/env groovy @Grab(redis.clients:jedis:2.0.0) def redis = new redis.clients.jedis.Jedis("localhost") assert "PONG" == redis.ping()Monday, June 20, 2011
Producer/Consumer ExampleMonday, June 20, 2011
Producer pushes work on a list with lpush @Grab(redis.clients:jedis:2.0.0) redis = new redis.clients.jedis.Jedis("localhost") args.each { redis.lpush("welcome-wagon", it) }Monday, June 20, 2011
Consumer uses blpop (blocking left pop from list) @Grab(redis.clients:jedis:2.0.0) redis = new redis.clients.jedis.Jedis("localhost") println "Joining the welcome-wagon!" while (true) { def name = redis.blpop(0, "welcome-wagon")[1] println "Welcome ${name}!" }Monday, June 20, 2011
Mass Producer srandmember to randomly pick female name from set @Grab(redis.clients:jedis:2.0.0) redis = new redis.clients.jedis.Jedis("localhost") if (!redis.exists("female-names")) { new File("./female-names.txt").eachLine {redis.sadd("female-names",it)} } for (i in 1..100000) { redis.lpush("welcome-wagon", redis.srandmember("female-names")) if (i % 1000 == 0) println "Adding $i" } female-names.txt from: http://antirez.com/post/autocomplete-with-redis.htmlMonday, June 20, 2011
Groovy DemoMonday, June 20, 2011
Grails Redis Plugin https://github.com/grails-plugins/grails-redisMonday, June 20, 2011
Plugin Config in Config.xml grails { redis { poolConfig { // jedis pool specific tweaks here, see jedis docs & src // ex: numTestsPerEvictionRun = 4 } port = 6379 host = "localhost" } }Monday, June 20, 2011
Provides Caching Through MemoizationMonday, June 20, 2011
RedisTagLib <redis:memoize key="mykey" expire="3600"> <!-- insert expensive to generate GSP content here content will be executed once, subsequent calls will pull from redis (redis.get(“mykey”)) till the key expires --> </redis:memoize>Monday, June 20, 2011
RedisService Spring bean wraps pool connection // overrides propertyMissing and methodMissing to delegate to redis def redisService redisService.foo = "bar" assert "bar" == redisService.foo redisService.sadd("months", "february") assert true == redisService.sismember("months", "february")Monday, June 20, 2011
RedisService template methods manage pooled Redis connection redisService.withRedis { Jedis redis -> redis.set("foo", "bar") }Monday, June 20, 2011
RedisService String memoization redisService.memoize("my-key") { Jedis redis -> // expensive operation we only want to execute once } def ONE_HOUR = 3600 // with optional timeout in seconds redisService.memoize("my-key-with-timeout", ONE_HOUR) { Jedis redis -> // expensive operation we want to execute every hour }Monday, June 20, 2011
RedisService Domain Class memoization (stores IDs hydrates from DB) def key = "user:$id:friends-books" redisService.memoizeDomainList(Book, key, ONE_HOUR) { redis -> // expensive process to calculate all friend’s books // stores list of Book ids, hydrates them from DB }Monday, June 20, 2011
Example Showing Products with Sort/Filter/Pagination CriteriaMonday, June 20, 2011
Other Memoization Methods memoizeHash, memoizeHashField, memoizeScore (sorted set score)Monday, June 20, 2011
Grails Redis-GORM Plugin http://grails.github.com/inconsequential/redis/Monday, June 20, 2011
Uses SpringData to abstract data layerMonday, June 20, 2011
Can be used in conjunction with HibernateMonday, June 20, 2011
Partial support for GORM including Dynamic Finders, Criteria, Named Queries and “Transactions”Monday, June 20, 2011
Limitations It requires explicit index mapping on fields you want to query package com.example class Author { String name static mapWith = "redis" static hasMany = [books: Book] static mapping = { name index:true } }Monday, June 20, 2011
Under The Covers MONITOR output for new Author(name: "Stephen King").save() 1308027697.922839 "INCR" "com.example.Author.next_id" 1308027697.940021 "HMSET" "com.example.Author:1" "name" "Stephen King" "version" "0" 1308027697.940412 "SADD" "com.example.Author.all" "1" 1308027697.943318 "SADD" "com.example.Author:id:1" "1" 1308027697.943763 "ZADD" "com.example.Author:id:sorted" "1.0" "1" 1308027697.944911 "SADD" "com.example.Author:name:Stephen+King" "1"Monday, June 20, 2011
1–3 of 3 previous next Post a comment