Redis & Groovy & Grails
by Ted Naleid
http://naleid.com
Monday, June 20, 2011

“Redis is a collection of data
structures exposed over the
network”
from: http://nosql.mypopescu.com/post/5403851771/what-is-redis
Monday, June 20, 2011

key/value store
like memcached on steroids
Monday, June 20, 2011

Strings, Integers,
Lists, Hashes,
Sets & Sorted Sets
(& commonly expected operations with each data type)
Monday, June 20, 2011

in-memory storage
Monday, June 20, 2011

single-threaded
Monday, June 20, 2011

fast
Monday, June 20, 2011

“Memory is the new Disk,
Disk is the new Tape”
- Jim Gray
Monday, June 20, 2011

Relative Latency
CPU Register - 1x
L2 Cache - 10x
Memory - 100x
Disk - 10,000,000x
analogy from “Redis - Memory as the New Disk” - Tim Lossen &
http://en.wikipedia.org/wiki/Orders_of_magnitude_(speed)
Monday, June 20, 2011

CPU Register
1 yard
photo: http://www.flickr.com/photos/limonada/904754668/
Monday, June 20, 2011

L2 Cache
10 yards
photo: http://www.flickr.com/photos/plentyofants/2749262107
Monday, June 20, 2011

Memory
100 yards
photo: http://www.flickr.com/photos/billmcintyre/264905933
Monday, June 20, 2011

Disk
Minneapolis to New York to Miami
to Seattle
~5600 miles
Monday, June 20, 2011

Simple wire protocol that
matches API
Monday, June 20, 2011

% telnet localhost 6379
Escape character is '^]'.
set foo bar
+OK
get foo
$3
bar
rpush mylist first
:1
rpush mylist second
:2
lrange mylist 0 -1
*2
$5
first
$6
second
Monday, June 20, 2011

clients for every* language
*well not every language, but all the popular/semi-popular ones, you can easily write one if
your language doesn’t have one
Monday, June 20, 2011

No dependencies
(451KB download)
Monday, June 20, 2011

Used in production by high
traffic sites that you’ve used
Monday, June 20, 2011

stackoverflow, craigslist,
github, disqus, digg &
blizzard entertainment
Monday, June 20, 2011

financially supported by
VMware/SpringSource
Monday, June 20, 2011

simple data structures make
redis flexible
Monday, June 20, 2011

write-through caching/
memoization
Monday, June 20, 2011

producer/consumer
message queues
Monday, June 20, 2011

publish/subscribe
Monday, June 20, 2011

atomic sequences/counters
Monday, June 20, 2011

other uses...
distributed locks, tag clouds, session tokens,
auto-complete prefixes, API rate limiting, leaderboards,
capped logs, random set items, A/B testing data storage,
unique per user product pricing/sorting
Monday, June 20, 2011

Redis Commands
Monday, June 20, 2011

Great Online Reference
http://redis.io/commands
Monday, June 20, 2011

Strings
set
Redis REPL Groovy
> set foo bar redis.set("foo", "bar")
foo bar
OK OK
Monday, June 20, 2011

Strings
get
Redis REPL Groovy
foo bar
> get foo redis.get("foo")
"bar" "bar"
bar
Monday, June 20, 2011

Exists
exists (check key existence)
Redis REPL Groovy
foo (nil)
> exists foo redis.exists("foo")
(integer) 0 <= false
0
Monday, June 20, 2011

Integers
incr (increment)
Redis REPL Groovy
foo 1
> incr foo redis.incr("foo")
(integer) 1 <= 1
1
Monday, June 20, 2011

Integers
decr (decrement)
Redis REPL Groovy
foo 0
> decr foo redis.decr("foo")
(integer) 0 <= 0
0
Monday, June 20, 2011

Lists
rpush (right push)
Redis REPL Groovy
> rpush foo baz redis.rpush("foo", "baz")
foo baz
(integer) 1 <= 1
Monday, June 20, 2011

Lists
rpush (right push)
Redis REPL Groovy
> rpush foo qux redis.rpush("foo", "qux")
foo baz qux
(integer) 2 <= 2
Monday, June 20, 2011

Lists
lpush (left push)
Redis REPL Groovy
> lpush foo bar redis.lpush("foo", "bar")
foo bar baz qux
(integer) 3 <= 3
Monday, June 20, 2011

Lists
lrange (slice of list)
Redis REPL Groovy
foo bar baz qux
> lrange foo 0 -1 redis.lrange("foo", 0, -1)
1) "bar" <= [bar, baz, qux]
2) "baz"
3) "qux"
bar baz qux
Monday, June 20, 2011

Lists
lrange (slice of list)
Redis REPL Groovy
foo bar baz qux
> lrange foo 0 1 redis.lrange("foo", 0, 1)
1) "bar" <= [bar, baz]
2) "baz"
bar baz
Monday, June 20, 2011

Lists
lpop (left pop)
Redis REPL Groovy
foo baz qux
> lpop foo redis.lpop("foo")
"bar" <= "bar"
bar
Monday, June 20, 2011

Lists
rpop (right pop)
Redis REPL Groovy
foo baz
> rpop foo redis.rpop("foo")
"qux" <= "qux"
qux
Monday, June 20, 2011

Hashes
hset (set key → value)
Redis REPL Groovy
> hset foo bar baz redis.hset("foo", "bar", "baz")
foo bar baz
(integer) 1 <= 1
Monday, June 20, 2011

Hashes
hset (set key → value)
Redis REPL Groovy
bar baz
> hset foo qux quxx redis.hset("foo", "qux", "quxx")
foo
(integer) 1 <= 1
qux quxx
Monday, June 20, 2011

Hashes
hget (get value for key)
Redis REPL Groovy
bar baz
foo
qux quxx
> hget foo bar redis.hget("foo", "bar")
"baz" <= "baz"
baz
Monday, June 20, 2011

Hashes
hgetall (get all keys/values)
Redis REPL Groovy
bar baz
foo
> hgetall foo qux quxx
1) "bar" redis.hgetAll("foo")
2) "baz" <= [bar:baz, qux:quxx]
3) "qux"
4) "quxx" bar baz
qux quxx
Monday, June 20, 2011

Hashes
hvals (hash values)
Redis REPL Groovy
bar baz
foo
qux quxx
> hvals foo redis.hvals("foo")
1) "baz" <= [baz, quxx]
2) "quxx"
baz quxx
Monday, June 20, 2011

Hashes
hkeys (hash keys)
Redis REPL Groovy
bar baz
foo
qux quxx
> hkeys foo redis.hkeys("foo")
1) "bar" <= [bar, qux]
2) "qux"
bar qux
Monday, June 20, 2011

Sets
sadd (set add)
Redis REPL Groovy
> sadd m1 jan redis.sadd("m1", "jan")
m1 jan
(integer) 1 <= 1
Monday, June 20, 2011

Sets
sadd (set add)
Redis REPL Groovy
feb
> sadd m1 feb redis.sadd("m1", "feb")
m1
(integer) 1 <= 1
jan
Monday, June 20, 2011

Sets
sismember (membership test)
Redis REPL Groovy
feb
m1
jan
> sismember m1 jan redis.sismember("m1", "jan")
(integer) 1 <= true
1
Monday, June 20, 2011

Sets
sismember (membership test)
Redis REPL Groovy
feb
m1
jan
> sismember m1 mar redis.sismember("m1", "mar")
(integer) 0 <= false
0
Monday, June 20, 2011

Sets
smembers (get full set)
Redis REPL Groovy
feb
m1
jan
> smembers m1 redis.smembers("m1")
1) "feb" <= [feb, jan]
2) "jan"
feb
jan
Monday, June 20, 2011

Sets
sinter (set intersection)
Redis REPL Groovy
feb feb
m1 m2
jan mar
> sinter m1 m2 redis.sinter("m1", "m2")
1) "feb" <= ["feb"]
feb
Monday, June 20, 2011

Sets
sdiff (set difference)
Redis REPL Groovy
feb feb
m1 m2
jan mar
> sdiff m1 m2 redis.sdiff("m1", "m2")
1) "jan" <= ["jan"]
jan
Monday, June 20, 2011

Sets
sunion (set union)
Redis REPL Groovy
feb feb
m1 m2
> sunion m1 m2 jan mar
1) "mar" redis.sunion("m1", "m2")
2) "jan" mar <= ["mar", "jan", "feb"]
3) "feb"
jan
feb
Monday, June 20, 2011

Sorted Sets
zadd (add with score)
Redis REPL Groovy
> zadd z1 1 jan redis.zadd("z1", 1, "jan")
z1 1 jan
(integer) 1 <= 1
Monday, June 20, 2011

Sorted Sets
zscore (score for member)
Redis REPL Groovy
1 jan
z1 2 feb
> zscore z1 feb 3 mar redis.zscore("z1", "feb")
"2" <= 2.0
2
Monday, June 20, 2011

Sorted Sets
zrange (sorted subset)
Redis REPL Groovy
1 jan
z1 2 feb
> zrange z1 0 1 withscores
1) "jan" 3 mar redis.zrangeWithScores("z1", 0, 1)
2) "1" <= [["jan", 1], ["feb", 2]]
3) "feb"
4) "2"
1 jan
2 feb
Monday, June 20, 2011

Sorted Sets
zrangebyscore (subset having score range)
Redis REPL Groovy
1 jan
z1 2 feb
> zrangebyscore z1 2 3 withscores
1) "feb" 3 mar redis.zrangeByScoreWithScores("z1",2,3)
2) "2" <= [["feb", 2], ["mar", 3]]
3) "mar"
4) "3"
2 feb
3 mar
Monday, June 20, 2011

Groovy Usage
Monday, June 20, 2011

Grape @Grab Annotation
#! /usr/bin/env groovy
@Grab('redis.clients:jedis:2.0.0')
def redis = new redis.clients.jedis.Jedis("localhost")
assert "PONG" == redis.ping()
Monday, June 20, 2011

Producer/Consumer
Example
Monday, June 20, 2011

Producer
pushes work on a list with lpush
@Grab('redis.clients:jedis:2.0.0')
redis = new redis.clients.jedis.Jedis("localhost")
args.each { redis.lpush("welcome-wagon", it) }
Monday, June 20, 2011

Consumer
uses blpop (blocking left pop from list)
@Grab('redis.clients:jedis:2.0.0')
redis = new redis.clients.jedis.Jedis("localhost")
println "Joining the welcome-wagon!"
while (true) {
def name = redis.blpop(0, "welcome-wagon")[1]
println "Welcome ${name}!"
}
Monday, June 20, 2011

Mass Producer
srandmember to randomly pick female name from set
@Grab('redis.clients:jedis:2.0.0')
redis = new redis.clients.jedis.Jedis("localhost")
if (!redis.exists("female-names")) {
new File("./female-names.txt").eachLine {redis.sadd("female-names",it)}
}
for (i in 1..100000) {
redis.lpush("welcome-wagon", redis.srandmember("female-names"))
if (i % 1000 == 0) println "Adding $i"
}
female-names.txt from: http://antirez.com/post/autocomplete-with-redis.html
Monday, June 20, 2011

Groovy Demo
Monday, June 20, 2011

Grails Redis Plugin
https://github.com/grails-plugins/grails-redis
Monday, June 20, 2011

Plugin Config
in Config.xml
grails {
redis {
poolConfig {
// jedis pool specific tweaks here, see jedis docs & src
// ex: numTestsPerEvictionRun = 4
}
port = 6379
host = "localhost"
}
}
Monday, June 20, 2011

Provides Caching Through
Memoization
Monday, June 20, 2011

RedisTagLib
<redis:memoize key="mykey" expire="3600">
<!--
insert expensive to generate GSP content here
content will be executed once, subsequent calls
will pull from redis (redis.get(“mykey”)) till the key expires
-->
</redis:memoize>
Monday, June 20, 2011

RedisService
Spring bean wraps pool connection
// overrides propertyMissing and methodMissing to delegate to redis
def redisService
redisService.foo = "bar"
assert "bar" == redisService.foo
redisService.sadd("months", "february")
assert true == redisService.sismember("months", "february")
Monday, June 20, 2011

RedisService
template methods manage pooled Redis connection
redisService.withRedis { Jedis redis ->
redis.set("foo", "bar")
}
Monday, June 20, 2011

RedisService
template methods manage pooled Redis connection
redisService.withRedis { Jedis redis ->
redisService.withTransaction { Transaction transaction ->
transaction.set("qux", "baz")
assertNull redis.get("qux")
}
assertEquals "baz", redis.get("qux")
}
Monday, June 20, 2011

RedisService
String memoization
redisService.memoize("my-key") { Jedis redis ->
// expensive operation we only want to execute once
}
def ONE_HOUR = 3600 // with optional timeout in seconds
redisService.memoize("my-key-with-timeout", ONE_HOUR) { Jedis redis ->
// expensive operation we want to execute every hour
}
Monday, June 20, 2011

RedisService
Domain Class memoization (stores IDs hydrates from DB)
def key = "user:$id:friends-books"
redisService.memoizeDomainList(Book, key, ONE_HOUR) { redis ->
// expensive process to calculate all friend’s books
// stores list of Book ids, hydrates them from DB
}
Monday, June 20, 2011

Example
Showing Products with Sort/Filter/Pagination Criteria
Monday, June 20, 2011

Other Memoization Methods
memoizeHash, memoizeHashField,
memoizeScore (sorted set score)
Monday, June 20, 2011

Grails Redis-GORM Plugin
http://grails.github.com/inconsequential/redis/
Monday, June 20, 2011

Uses SpringData to abstract
data layer
Monday, June 20, 2011

Can be used in conjunction
with Hibernate
Monday, June 20, 2011

Partial support for GORM
including Dynamic Finders, Criteria, Named Queries and “Transactions”
Monday, June 20, 2011

Limitations
It requires explicit index mapping on fields you want to query
package com.example
class Author {
String name
static mapWith = "redis"
static hasMany = [books: Book]
static mapping = {
name index:true
}
}
Monday, June 20, 2011

Under The Covers
MONITOR output for new Author(name: "Stephen King").save()
1308027697.922839 "INCR" "com.example.Author.next_id"
1308027697.940021 "HMSET" "com.example.Author:1" "name" "Stephen King" "version" "0"
1308027697.940412 "SADD" "com.example.Author.all" "1"
1308027697.943318 "SADD" "com.example.Author:id:1" "1"
1308027697.943763 "ZADD" "com.example.Author:id:sorted" "1.0" "1"
1308027697.944911 "SADD" "com.example.Author:name:Stephen+King" "1"
Monday, June 20, 2011

Questions?
Monday, June 20, 2011