Redis is an open source advanced key-value store. It can contain strings, hashes, lists, sets and sorted sets. Its greatest feature is that you can run atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing to a list; computing set intersection, union and difference. This talk shows how use Redis and Ruby to build an high scalable, reliable and concurrent infrastructure of services using patterns and paradigm that this tool give us. All the examples coming from a large scale application that handles hundred millions of messages every day.

1. Save time, money and be happy!
Ruby & Redis Like a Pro
Stefano Fontanelli <s.fontanelli@gmail.com>
Italian Ruby Day Conference 2013
Friday, June 14, 13

2. Milan, 14th June 2013 Italian Ruby Day Conference 2013
About Me
Keep in touch!
http://about.me/stefanofontanelli
Friday, June 14, 13

3. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Table of content
• What is Redis?
• Redis in action
• Advanced use: a real use case
• The Reliable Queue Pattern
• Locking with Redis
• The Publish/Subscribe Messaging Paradigm
• Drawbacks
• Conclusions
Friday, June 14, 13

4. Milan, 14th June 2013 Italian Ruby Day Conference 2013
What is Redis?
More Information: http://redis.io
Friday, June 14, 13

5. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Redis: an advanced key-value store
Credits: http://strata.oreilly.com/2013/03/large-scale-data-collection-and-real-time-analytics-using-redis.html
Friday, June 14, 13

6. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Outstanding performance
No. of connections
Source: http://redis.io/topics/benchmarks
Friday, June 14, 13

7. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Why I love Redis
• Up & Running in few minutes
• brew install redis
• redis-server
• Impressive performance[1]
• Data types are similar to Ruby’s ones [2]
• Known time complexity
• Atomic operations
• Advanced features are outstanding!
• Keys with a limited time-to-live
• Pub/Sub messaging paradigm
• Easily share your data structures between processes!
[1] http://redis.io/topics/benchmarks [2] http://redis.io/topics/data-types
Friday, June 14, 13

8. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Redis in action: strings
More information: http://redis.io/topics/data-types
> get key => key
> set key, value => key = value
> incr key => key = 0
key += 1
> decr key => key = 0
key += 1
> incrby key, value => key = 0
key += value
Friday, June 14, 13

9. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Redis in action: lists
More information: http://redis.io/topics/data-types
> lpush key, value => key = []
key << value
> lpop key, value => key.pop
> llen key => key.length
LPUSH RPOP
RPUSHLPOP
Friday, June 14, 13

10. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Redis in action: hashes
More information: http://redis.io/topics/data-types
> hget k, f, v => k[f]
> hset k, f, v => k = {}
k[f] = v
> hkeys k => k.keys
Friday, June 14, 13

11. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Redis in action: sets
More information: http://redis.io/topics/data-types
> sadd k, v => k = Set.new
k << v
> srem k, v => k.delete v
> sdiff k1, k2 => k1 - k2
> sunion k1, k2 => k1 + k2
> sismember k, v => key.member? v
Friday, June 14, 13

12. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Redis in action: ordered set
More information: http://redis.io/topics/data-types
Friday, June 14, 13

13. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Complex things made easy: a real use case
Friday, June 14, 13

14. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Complex things made easy: a real use case (2)
Q1
Q2
Q3
Q4
Q5
Q6
S1
S2
S3
Qi = queue i | Sj = service j
Friday, June 14, 13

15. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Exchange data between services: queues
Friday, June 14, 13

16. LPUSH RPOP
RPUSHLPOP
Milan, 14th June 2013 Italian Ruby Day Conference 2013
The Reliable Queue Pattern (1/2)
Friday, June 14, 13

17. Messages queueProducer Consumer
Processing list
/dev/null
push
push
pop
pop
Milan, 14th June 2013 Italian Ruby Day Conference 2013
The Reliable Queue Pattern (2/2)
Friday, June 14, 13

18. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Reliable Queue in Ruby
Fork & Play:
http://github.com/stefanofontanelli/backend-toolkit
• A fully working implementation in less than 30 lines
• Interface similar to Ruby’s one
• q = Queue.new ‘my_first_reliable_queue’
• q << {‘message’ => ‘I love ruby!’}
• msg = q.pop
• You don’t need polling anymore!
Friday, June 14, 13

19. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Don’t mess up your data, use locks!
Friday, June 14, 13

20. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Locking with Redis
• Use SETNX as lock primitive
• SETNX mylock <timestamp + timeout + 1>
• 1/true lock acquired,
• 0/false lock not acquired.
• Use GET and GETSET to detect deadlocks
1. GET mylock can be use to identify an expired lock;
2. GETSET mylock <timestamp + timeout + 1>
atomically sets key to value and returns the old value;
3. the lock was acquired
when old value is still an expired timestamp.
Friday, June 14, 13

21. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Locking with Redis: a Ruby implementation
Fork & Play:
http://github.com/stefanofontanelli/backend-toolkit
• A fully working implementation in ~50 lines of code
• Easy to use
• Lock.new.acquire(obj) do |lock|
# put your code here
lock.keep_alive obj
# put your code here
end
• No need to explicitly release lock.
Friday, June 14, 13

22. Milan, 14th June 2013 Italian Ruby Day Conference 2013
What about services?
Fork & Play:
http://github.com/stefanofontanelli/backend-toolkit
• You can simply define class MyService < Daemon
• It must implement the process(msg, lock) method
• That’s all. Your service is ready to start!
• MyService.new(‘input_queue’,‘output_queue’).run
Friday, June 14, 13

23. Milan, 14th June 2013 Italian Ruby Day Conference 2013
The Publish/Subscribe Messaging Paradigm[1]
[1] http://redis.io/topics/pubsub
Friday, June 14, 13

24. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Pub/Sub pattern using Ruby & Redis
require ‘redis’
redis = Redis.new
redis.subscribe(:mychannel) do |on|
on.message do |channel, message|
# puts your code here
end
end
[1] https://github.com/redis/redis-rb/blob/master/examples/pubsub.rb
Friday, June 14, 13

25. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Redis is terrific. No drawbacks?
Friday, June 14, 13

26. Milan, 14th June 2013 Italian Ruby Day Conference 2013
How can we address these issues?
Friday, June 14, 13

27. Milan, 14th June 2013 Italian Ruby Day Conference 2013
Conclusions
Friday, June 14, 13

28. Milan, 14th June 2013 Italian Ruby Day Conference 2013
The End
Any Questions?
Keep in touch!
http://about.me/stefanofontanelli
Friday, June 14, 13