Ruby & Redis like a pro

  • 1,061 views
Uploaded on

Redis is an open source advanced key-value store. It can contain strings, hashes, lists, sets and sorted sets. Its greatest feature is that you can run atomic operations on these types, like appending …

Redis is an open source advanced key-value store. It can contain strings, hashes, lists, sets and sorted sets. Its greatest feature is that you can run atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing to a list; computing set intersection, union and difference.
This talk shows how use Redis and Ruby to build an high scalable, reliable and concurrent infrastructure of services using patterns and paradigm that this tool give us. All the examples coming from a large scale application that handles hundred millions of messages every day.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,061
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
4

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Save time, money and be happy!Ruby & Redis Like a ProStefano Fontanelli <s.fontanelli@gmail.com>Italian Ruby Day Conference 2013Friday, June 14, 13
  • 2. Milan, 14th June 2013 Italian Ruby Day Conference 2013About MeKeep in touch!http://about.me/stefanofontanelliFriday, June 14, 13
  • 3. Milan, 14th June 2013 Italian Ruby Day Conference 2013Table of content• What is Redis?• Redis in action• Advanced use: a real use case• The Reliable Queue Pattern• Locking with Redis• The Publish/Subscribe Messaging Paradigm• Drawbacks• ConclusionsFriday, June 14, 13
  • 4. Milan, 14th June 2013 Italian Ruby Day Conference 2013What is Redis?More Information: http://redis.ioFriday, June 14, 13
  • 5. Milan, 14th June 2013 Italian Ruby Day Conference 2013Redis: an advanced key-value storeCredits: http://strata.oreilly.com/2013/03/large-scale-data-collection-and-real-time-analytics-using-redis.htmlFriday, June 14, 13
  • 6. Milan, 14th June 2013 Italian Ruby Day Conference 2013Outstanding performanceNo. of connectionsSource: http://redis.io/topics/benchmarksFriday, June 14, 13
  • 7. Milan, 14th June 2013 Italian Ruby Day Conference 2013Why I love Redis• Up & Running in few minutes• brew install redis• redis-server• Impressive performance[1]• Data types are similar to Ruby’s ones [2]• Known time complexity• Atomic operations• Advanced features are outstanding!• Keys with a limited time-to-live• Pub/Sub messaging paradigm• Easily share your data structures between processes![1] http://redis.io/topics/benchmarks [2] http://redis.io/topics/data-typesFriday, June 14, 13
  • 8. Milan, 14th June 2013 Italian Ruby Day Conference 2013Redis in action: stringsMore information: http://redis.io/topics/data-types> get key => key> set key, value => key = value> incr key => key = 0key += 1> decr key => key = 0key += 1> incrby key, value => key = 0key += valueFriday, June 14, 13
  • 9. Milan, 14th June 2013 Italian Ruby Day Conference 2013Redis in action: listsMore information: http://redis.io/topics/data-types> lpush key, value => key = []key << value> lpop key, value => key.pop> llen key => key.lengthLPUSH RPOPRPUSHLPOPFriday, June 14, 13
  • 10. Milan, 14th June 2013 Italian Ruby Day Conference 2013Redis in action: hashesMore information: http://redis.io/topics/data-types> hget k, f, v => k[f]> hset k, f, v => k = {}k[f] = v> hkeys k => k.keysFriday, June 14, 13
  • 11. Milan, 14th June 2013 Italian Ruby Day Conference 2013Redis in action: setsMore information: http://redis.io/topics/data-types> sadd k, v => k = Set.newk << v> srem k, v => k.delete v> sdiff k1, k2 => k1 - k2> sunion k1, k2 => k1 + k2> sismember k, v => key.member? vFriday, June 14, 13
  • 12. Milan, 14th June 2013 Italian Ruby Day Conference 2013Redis in action: ordered setMore information: http://redis.io/topics/data-typesFriday, June 14, 13
  • 13. Milan, 14th June 2013 Italian Ruby Day Conference 2013Complex things made easy: a real use caseFriday, June 14, 13
  • 14. Milan, 14th June 2013 Italian Ruby Day Conference 2013Complex things made easy: a real use case (2)Q1Q2Q3Q4Q5Q6S1S2S3Qi = queue i | Sj = service jFriday, June 14, 13
  • 15. Milan, 14th June 2013 Italian Ruby Day Conference 2013Exchange data between services: queuesFriday, June 14, 13
  • 16. LPUSH RPOPRPUSHLPOPMilan, 14th June 2013 Italian Ruby Day Conference 2013The Reliable Queue Pattern (1/2)Friday, June 14, 13
  • 17. Messages queueProducer ConsumerProcessing list/dev/nullpushpushpoppopMilan, 14th June 2013 Italian Ruby Day Conference 2013The Reliable Queue Pattern (2/2)Friday, June 14, 13
  • 18. Milan, 14th June 2013 Italian Ruby Day Conference 2013Reliable Queue in RubyFork & Play:http://github.com/stefanofontanelli/backend-toolkit• A fully working implementation in less than 30 lines• Interface similar to Ruby’s one• q = Queue.new ‘my_first_reliable_queue’• q << {‘message’ => ‘I love ruby!’}• msg = q.pop• You don’t need polling anymore!Friday, June 14, 13
  • 19. Milan, 14th June 2013 Italian Ruby Day Conference 2013Don’t mess up your data, use locks!Friday, June 14, 13
  • 20. Milan, 14th June 2013 Italian Ruby Day Conference 2013Locking with Redis• Use SETNX as lock primitive• SETNX mylock <timestamp + timeout + 1>• 1/true lock acquired,• 0/false lock not acquired.• Use GET and GETSET to detect deadlocks1. GET mylock can be use to identify an expired lock;2. GETSET mylock <timestamp + timeout + 1>atomically sets key to value and returns the old value;3. the lock was acquiredwhen old value is still an expired timestamp.Friday, June 14, 13
  • 21. Milan, 14th June 2013 Italian Ruby Day Conference 2013Locking with Redis: a Ruby implementationFork & Play:http://github.com/stefanofontanelli/backend-toolkit• A fully working implementation in ~50 lines of code• Easy to use• Lock.new.acquire(obj) do |lock|# put your code herelock.keep_alive obj# put your code hereend• No need to explicitly release lock.Friday, June 14, 13
  • 22. Milan, 14th June 2013 Italian Ruby Day Conference 2013What about services?Fork & Play:http://github.com/stefanofontanelli/backend-toolkit• You can simply define class MyService < Daemon• It must implement the process(msg, lock) method• That’s all. Your service is ready to start!• MyService.new(‘input_queue’,‘output_queue’).runFriday, June 14, 13
  • 23. Milan, 14th June 2013 Italian Ruby Day Conference 2013The Publish/Subscribe Messaging Paradigm[1][1] http://redis.io/topics/pubsubFriday, June 14, 13
  • 24. Milan, 14th June 2013 Italian Ruby Day Conference 2013Pub/Sub pattern using Ruby & Redisrequire ‘redis’redis = Redis.newredis.subscribe(:mychannel) do |on|on.message do |channel, message|# puts your code hereendend[1] https://github.com/redis/redis-rb/blob/master/examples/pubsub.rbFriday, June 14, 13
  • 25. Milan, 14th June 2013 Italian Ruby Day Conference 2013Redis is terrific. No drawbacks?Friday, June 14, 13
  • 26. Milan, 14th June 2013 Italian Ruby Day Conference 2013How can we address these issues?Friday, June 14, 13
  • 27. Milan, 14th June 2013 Italian Ruby Day Conference 2013ConclusionsFriday, June 14, 13
  • 28. Milan, 14th June 2013 Italian Ruby Day Conference 2013The EndAny Questions?Keep in touch!http://about.me/stefanofontanelliFriday, June 14, 13