Redis, Resque & Friends
Upcoming SlideShare
Loading in...5
×
 

Redis, Resque & Friends

on

  • 6,724 views

RubyFuZa presentation giving an overview of redis with a couple of examples. Also quick mentions of Resque, Resque-Scheduler and Resque-Retry

RubyFuZa presentation giving an overview of redis with a couple of examples. Also quick mentions of Resque, Resque-Scheduler and Resque-Retry

Statistics

Views

Total Views
6,724
Views on SlideShare
6,089
Embed Views
635

Actions

Likes
11
Downloads
101
Comments
0

6 Embeds 635

http://christopherspring.com 526
http://www.scoop.it 100
http://chrisspring.posterous.com 5
http://translate.googleusercontent.com 2
http://import-ydsf.posterous.com 1
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Redis, Resque & Friends Redis, Resque & Friends Presentation Transcript

  • Redis, Resque and Friends Christopher Spring https://github.com/autonomous @autonomous www.christopherspring.com
  • WTF is Redis?
  • WTF is Redis?Remote Dictionary Server
  • Key-value storage... # Insert a value for a key SET some:key “value” # Retrieve value for a key GET some:key # => “value”
  • ... on steroids!# ListsRPUSH chores "Mow lawn"LPUSH chores "Get dressed"# SetsSADD sith "Darth Maul"SUNION sith evil# Sorted setsZADD background:workers 50 "w1.oss.com"ZADD background:workers 19 "w3.oss.com"# HashHMSET client username "womble" password "secret_womble"HGET client password
  • Strings
  • Hashes
  • Lists
  • Sets
  • Sorted sets
  • $ brew install redis$ redis-server /usr/local/etc/redis-conf
  • Example: Tagging$ redis-cli...redis> SADD post:17:tag "ruby"redis> SADD post:17:tag "rails"redis> SADD post:17:tag "redis"redis> SADD post:20:tag "resque"redis> SADD post:20:tag "redis"redis> SINTER post:17:tag post:20:tag1. "redis"
  • Transactions
  • Example: Transactions $ redis-cli ... redis> SET "player:1:coins" 30 redis> SET "player:2:coins" 63 # ... Players decide to trade some coin redis> MULTI redis> INCRBY "player:1:coins" 15 redis> DECRBY "player:2:coins" 15 redis> EXEC # 1. (integer) 45 # 2. (integer) 48
  • In-memory storage • Simple single process event driven design • No locks • 100K+ operations per second
  • Asynchronous persistence • Snapshotting - save 60 1000 # dump every 60 seconds if at least 1000 keys changed • Append-only file - durable with log rewrite support
  • Master-slave replication slaveof 192.168.1.1 6379 • Multiple slaves • Slaves of slaves • Scalability • Defer save to slaves
  • ... I’ve got too much data!
  • Redis Virtual Memory • Only values swapped to disk • Most recent values in memory • Predictable memory usage...
  • 1M keys: 160 MB10M keys: 1.6 GB100M keys: 16 GB
  • MySQL + memcached • Multiple copies of data • Mainly scales reads • No higher level functions
  • Client libraries for all: • C • Java • C# • Lua • C++ • Node.js • Clojure • Objective-c • Common List • Perl • Erlang • PHP • Go • Python • Haskell • Scala • haXe • Smalltalk • Io • Tcl
  • Client libraries for all: • C • Java • C# • Lua • C++ • Node.js • Clojure • Objective-c • Common List • Perl • Erlang • PHP • Go • Python • Haskell • Scala • haXe • Smalltalk • Io • Tcl
  • $ sudo gem install redis
  • Redis as a database...
  • require rubygems; require redis; require jsonclass RubyFuZa def initialize args class Dude < Struct.new(:first_name, :last_name); end @db = args[:db] @duration = args[:duration] # seconds @namespace = "#{self.class}:#{args[:id]}" chris = Dude.new Chris, Spring @s_key = "#{@namespace}:speakers" marc = Dude.new Marc, Heiligers @a_key = "#{@namespace}:attendees" steve = Dude.new Steve, Hoffmeier # Is jy my pa? end def start! redis = Redis.new return "Already started!" if started? two_days = 2*24*60*60 # seconds @db[@namespace] = started conf = RubyFuZa.new( @db.expire( @namespace, @duration ) :db => redis, end :duration => two_days, :id => "2011") def started? @db[@namespace] == started end conf.add_speaker chris # true conf.add_attendee marc # true def how_much_longer? conf.attendees # [chris, marc] t = @db.ttl(@namespace) t == -1 ? Its over! : t end conf.started? # false conf.start! # true def add_speaker( speaker ) conf.started? # true @db.sadd( @s_key, speaker.to_json ) end sleep( 2000 ) def add_attendee( attendee ) conf.how_much_longer? # two_days - 2000 @db.sadd(@a_key, attendee.to_json) end def attendees @db.sinter(@s_key, @a_key).map{ |sa| JSON.parse( sa )} endend
  • Publish/Subscribe
  • require rubygems $ redis-clirequire "redis" ... redis> PUBLISH rubyfuza:chat "Smoke me aredis = Redis.connect kipper..." redis> PUBLISH rubyfuza:chat "Ill be backtrap(:INT) { puts; exit } for breakfast." redis> PUBLISH rubyfuza:chat "exit"redis.subscribe(rubyfuza:chat) do |on| on.subscribe do |channel, subscriptions| puts "Subscribed to ##{channel}" end on.message do |channel, message| puts "##{channel}: #{message}" redis.unsubscribe if message == "exit" end on.unsubscribe do |channel, subscriptions| puts "Unsubscribed from ##{channel}" endend
  • Resque• Backed by redis• Asynchronous job server• Multiple queues• Sinatra based web-ui
  • class PostalWorker @queue :post_box def self.perform( user_id ) User.where( :id => user_id ) msg = UserMail.welcome_message( user ) msg.deliver endend...Resque.enqueue( PostalWorker, user.id )
  • Resque-Scheduler• Queue jobs in the future!• Recurring cron-style queueing• Delayed jobs
  • database_cleaning: cron: "0 0 * * *" class: CleanupWorker args: description: "This jobs removes junk from the DB"
  • Resque.enqueue_at( 5.days.from_now, SendFollowUpEmail, :user_id => current_user.id)
  • Resque-retry• Retry failed jobs• Set number of retries• Set delay between retries• Exponential back-off• Delays built on resque-scheduler
  • class WebHookWorker extend Resque::Plugins::Retry @queue = :web_hooks @retry_limit = 10 @retry_delay = 5*60 # Seconds def self.perform(*args) # ... trigger web-hook endend
  • In summary...
  • Data structure server • Strings • Hashes • Lists • Sets • Sorted sets
  • Main use cases: • Database • Cache • Messaging
  • Features:• Fast• Atomic operations• Transactions• Master-Slave• Persist to disk• Expiring keys• Pub - Sub• Redis Virtual Memory
  • Baie dankie!
  • Questions?
  • Links• https://github.com/bblimke/copy-with-style-tmbundle• http://redis.io/• https://github.com/antirez/redis• https://github.com/ezmobius/redis-rb• http://antirez.com/• https://github.com/defunkt/resque• https://github.com/bvandenbos/resque-scheduler• https://github.com/lantins/resque-retry