Redis, Resque & Friends
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Redis, Resque & Friends

on

  • 6,814 views

RubyFuZa presentation giving an overview of redis with a couple of examples. Also quick mentions of Resque, Resque-Scheduler and Resque-Retry

RubyFuZa presentation giving an overview of redis with a couple of examples. Also quick mentions of Resque, Resque-Scheduler and Resque-Retry

Statistics

Views

Total Views
6,814
Views on SlideShare
6,175
Embed Views
639

Actions

Likes
11
Downloads
101
Comments
0

6 Embeds 639

http://christopherspring.com 529
http://www.scoop.it 101
http://chrisspring.posterous.com 5
http://translate.googleusercontent.com 2
http://import-ydsf.posterous.com 1
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Redis, Resque & Friends Presentation Transcript

  • 1. Redis, Resque and Friends Christopher Spring https://github.com/autonomous @autonomous www.christopherspring.com
  • 2. WTF is Redis?
  • 3. WTF is Redis?Remote Dictionary Server
  • 4. Key-value storage... # Insert a value for a key SET some:key “value” # Retrieve value for a key GET some:key # => “value”
  • 5. ... on steroids!# ListsRPUSH chores "Mow lawn"LPUSH chores "Get dressed"# SetsSADD sith "Darth Maul"SUNION sith evil# Sorted setsZADD background:workers 50 "w1.oss.com"ZADD background:workers 19 "w3.oss.com"# HashHMSET client username "womble" password "secret_womble"HGET client password
  • 6. Strings
  • 7. Hashes
  • 8. Lists
  • 9. Sets
  • 10. Sorted sets
  • 11. $ brew install redis$ redis-server /usr/local/etc/redis-conf
  • 12. Example: Tagging$ redis-cli...redis> SADD post:17:tag "ruby"redis> SADD post:17:tag "rails"redis> SADD post:17:tag "redis"redis> SADD post:20:tag "resque"redis> SADD post:20:tag "redis"redis> SINTER post:17:tag post:20:tag1. "redis"
  • 13. Transactions
  • 14. Example: Transactions $ redis-cli ... redis> SET "player:1:coins" 30 redis> SET "player:2:coins" 63 # ... Players decide to trade some coin redis> MULTI redis> INCRBY "player:1:coins" 15 redis> DECRBY "player:2:coins" 15 redis> EXEC # 1. (integer) 45 # 2. (integer) 48
  • 15. In-memory storage • Simple single process event driven design • No locks • 100K+ operations per second
  • 16. Asynchronous persistence • Snapshotting - save 60 1000 # dump every 60 seconds if at least 1000 keys changed • Append-only file - durable with log rewrite support
  • 17. Master-slave replication slaveof 192.168.1.1 6379 • Multiple slaves • Slaves of slaves • Scalability • Defer save to slaves
  • 18. ... I’ve got too much data!
  • 19. Redis Virtual Memory • Only values swapped to disk • Most recent values in memory • Predictable memory usage...
  • 20. 1M keys: 160 MB10M keys: 1.6 GB100M keys: 16 GB
  • 21. MySQL + memcached • Multiple copies of data • Mainly scales reads • No higher level functions
  • 22. Client libraries for all: • C • Java • C# • Lua • C++ • Node.js • Clojure • Objective-c • Common List • Perl • Erlang • PHP • Go • Python • Haskell • Scala • haXe • Smalltalk • Io • Tcl
  • 23. Client libraries for all: • C • Java • C# • Lua • C++ • Node.js • Clojure • Objective-c • Common List • Perl • Erlang • PHP • Go • Python • Haskell • Scala • haXe • Smalltalk • Io • Tcl
  • 24. $ sudo gem install redis
  • 25. Redis as a database...
  • 26. require rubygems; require redis; require jsonclass RubyFuZa def initialize args class Dude < Struct.new(:first_name, :last_name); end @db = args[:db] @duration = args[:duration] # seconds @namespace = "#{self.class}:#{args[:id]}" chris = Dude.new Chris, Spring @s_key = "#{@namespace}:speakers" marc = Dude.new Marc, Heiligers @a_key = "#{@namespace}:attendees" steve = Dude.new Steve, Hoffmeier # Is jy my pa? end def start! redis = Redis.new return "Already started!" if started? two_days = 2*24*60*60 # seconds @db[@namespace] = started conf = RubyFuZa.new( @db.expire( @namespace, @duration ) :db => redis, end :duration => two_days, :id => "2011") def started? @db[@namespace] == started end conf.add_speaker chris # true conf.add_attendee marc # true def how_much_longer? conf.attendees # [chris, marc] t = @db.ttl(@namespace) t == -1 ? Its over! : t end conf.started? # false conf.start! # true def add_speaker( speaker ) conf.started? # true @db.sadd( @s_key, speaker.to_json ) end sleep( 2000 ) def add_attendee( attendee ) conf.how_much_longer? # two_days - 2000 @db.sadd(@a_key, attendee.to_json) end def attendees @db.sinter(@s_key, @a_key).map{ |sa| JSON.parse( sa )} endend
  • 27. Publish/Subscribe
  • 28. require rubygems $ redis-clirequire "redis" ... redis> PUBLISH rubyfuza:chat "Smoke me aredis = Redis.connect kipper..." redis> PUBLISH rubyfuza:chat "Ill be backtrap(:INT) { puts; exit } for breakfast." redis> PUBLISH rubyfuza:chat "exit"redis.subscribe(rubyfuza:chat) do |on| on.subscribe do |channel, subscriptions| puts "Subscribed to ##{channel}" end on.message do |channel, message| puts "##{channel}: #{message}" redis.unsubscribe if message == "exit" end on.unsubscribe do |channel, subscriptions| puts "Unsubscribed from ##{channel}" endend
  • 29. Resque• Backed by redis• Asynchronous job server• Multiple queues• Sinatra based web-ui
  • 30. class PostalWorker @queue :post_box def self.perform( user_id ) User.where( :id => user_id ) msg = UserMail.welcome_message( user ) msg.deliver endend...Resque.enqueue( PostalWorker, user.id )
  • 31. Resque-Scheduler• Queue jobs in the future!• Recurring cron-style queueing• Delayed jobs
  • 32. database_cleaning: cron: "0 0 * * *" class: CleanupWorker args: description: "This jobs removes junk from the DB"
  • 33. Resque.enqueue_at( 5.days.from_now, SendFollowUpEmail, :user_id => current_user.id)
  • 34. Resque-retry• Retry failed jobs• Set number of retries• Set delay between retries• Exponential back-off• Delays built on resque-scheduler
  • 35. class WebHookWorker extend Resque::Plugins::Retry @queue = :web_hooks @retry_limit = 10 @retry_delay = 5*60 # Seconds def self.perform(*args) # ... trigger web-hook endend
  • 36. In summary...
  • 37. Data structure server • Strings • Hashes • Lists • Sets • Sorted sets
  • 38. Main use cases: • Database • Cache • Messaging
  • 39. Features:• Fast• Atomic operations• Transactions• Master-Slave• Persist to disk• Expiring keys• Pub - Sub• Redis Virtual Memory
  • 40. Baie dankie!
  • 41. Questions?
  • 42. Links• https://github.com/bblimke/copy-with-style-tmbundle• http://redis.io/• https://github.com/antirez/redis• https://github.com/ezmobius/redis-rb• http://antirez.com/• https://github.com/defunkt/resque• https://github.com/bvandenbos/resque-scheduler• https://github.com/lantins/resque-retry