Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
ResqueBackground processingSystem administration on a multi-        instances setup      Experience feedback            Pl...
My project : bookandgolf.com
bookandgolf.comAt launch :  • more than 60 golfs to synchronise  • 4 different APIs  • More than 350 000 slots (starts) to...
nginx          resque-web:web             redis-server:db       worker 1                        Current       worker 2:app...
Resque job classclass MyKikoololSleepJob  @queue = :medium  def self.perform(sleep_duration)    sleep sleep_duration  ende...
Resque workersrake environment resque:work QUEUE=mediumrails@www:~$ ps aux | grep resquerails  8950  sh -c cd /home/rails/...
Resque workers Unix signals• QUIT - Wait for child to finish processing then exit• TERM / INT - Immediately kill child then...
Resque-web (Sinatra webapp)
God setup - my way!Versioned god config files by instance name, ie :•   god / app.god•   god / web.god•   god / db.god•   ...
God config# ResqueGod.watch do |w|  w.env = { RAILS_ROOT => rails_root,              RAILS_ENV  => rails_env }  w.name    ...
Workers graceful restartRake task to restart workers, the "graceful"waynamespace :resque do  task :restart_workers => :env...
Workers graceful restartnamespace :resque do  task :restart_workers, :roles => [:app, :db] do    rake "resque:restart_work...
Resque-web - Final setup• Launched externally with nginx reverse proxy (easy to add  http basic auth)• External Ruby confi...
Resque-web God monitoring%w{5678}.each do |port|  God.watch do |w|    w.env = { RAILS_ROOT => rails_root,              RAI...
ActiveRecord - timeout / stale :(Fast workaround...   ...   ActiveRecord::Base.connection.reconnect!   ...
Resque plugins• defunkt / resque-lockPrevent two workers from working on the sameJob class with same arguments
Resque plugins• jayniz / resque-lonerTIP :Do not forget to :  require "resque-loner"in your Resque-web config.rb file! or....
Other plugins...Look @ github.com/defunkt/resqueWiki!And remember that with a goodsystem architecture, Resque is just...
Background processing with Resque
Background processing with Resque
Background processing with Resque
Background processing with Resque
Upcoming SlideShare
Loading in …5
×

Background processing with Resque

15,151 views

Published on

System administration on a multi-instances architecture, experience feedback, plugins I like

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Background processing with Resque

  1. 1. ResqueBackground processingSystem administration on a multi- instances setup Experience feedback Plugins Nicolas Blanco Novapost
  2. 2. My project : bookandgolf.com
  3. 3. bookandgolf.comAt launch : • more than 60 golfs to synchronise • 4 different APIs • More than 350 000 slots (starts) to synchronize every 2 hoursAt first we tried to do the synchronization with cron tasks... but...
  4. 4. nginx resque-web:web redis-server:db worker 1 Current worker 2:app worker 3 architecture worker 4
  5. 5. Resque job classclass MyKikoololSleepJob  @queue = :medium  def self.perform(sleep_duration)    sleep sleep_duration  endendResque.enqueue(MyKikoololSleepJob, 10)
  6. 6. Resque workersrake environment resque:work QUEUE=mediumrails@www:~$ ps aux | grep resquerails  8950  sh -c cd /home/rails/www/socianalytics/current &&rake environment resque:work QUEUE=high,medium,low...rails  8951  resque-1.9.10: Forked 9105 at 1302095560               rails  9105  resque-1.9.10: Processing medium since 1302095560      
  7. 7. Resque workers Unix signals• QUIT - Wait for child to finish processing then exit• TERM / INT - Immediately kill child then exit• USR1 - Immediately kill child but dont exit• USR2 - Dont start to process any new jobs• CONT - Start to process new jobs again after a USR2
  8. 8. Resque-web (Sinatra webapp)
  9. 9. God setup - my way!Versioned god config files by instance name, ie :• god / app.god• god / web.god• god / db.god• ...God init.d file found on gist.github.com :)god.conf :GOD_CONFIG=/home/rails/www/bookgolf/current/god/db.godGOD_COMMAND="sudo -u rails /usr/local/bin/god"GOD_LOG=/home/rails/log/god.log
  10. 10. God config# ResqueGod.watch do |w|  w.env = { RAILS_ROOT => rails_root,              RAILS_ENV  => rails_env }  w.name          = "resque-worker"  w.interval      = 30.seconds  w.start         = "cd #{rails_root} && rakeenvironment resque:work QUEUE=high,medium,low"  w.start_grace   = 10.seconds    ...
  11. 11. Workers graceful restartRake task to restart workers, the "graceful"waynamespace :resque do  task :restart_workers => :environment do    pids = Array.new        Resque.workers.each do |worker|      pids << worker.to_s.split(/:/).second ifworker.to_s.include?(Settings.resque.localhost_name)    end        if pids.size > 0      system("kill -QUIT #{pids.join( )}")    end        system("rm /home/rails/.god/pids/resque-worker*.pid")  endend
  12. 12. Workers graceful restartnamespace :resque do  task :restart_workers, :roles => [:app, :db] do    rake "resque:restart_workers"  endend
  13. 13. Resque-web - Final setup• Launched externally with nginx reverse proxy (easy to add http basic auth)• External Ruby config file to change Redis setup, load plugins, etc.     resque-web config/resque-web.rb
  14. 14. Resque-web God monitoring%w{5678}.each do |port|  God.watch do |w|    w.env = { RAILS_ROOT => rails_root,              RAILS_ENV  => rails_env }    w.uid       = "rails"    w.name      = "resque-web"    w.interval  = 30.seconds    w.start     = "cd #{rails_root} && resque-web config/resque-web.rb"    w.start_grace   = 15.seconds    w.start_if do |start|      start.condition(:process_running) do |c|          c.interval = 5.seconds          c.running = false      end    end  endend
  15. 15. ActiveRecord - timeout / stale :(Fast workaround...   ...   ActiveRecord::Base.connection.reconnect!   ...
  16. 16. Resque plugins• defunkt / resque-lockPrevent two workers from working on the sameJob class with same arguments
  17. 17. Resque plugins• jayniz / resque-lonerTIP :Do not forget to :  require "resque-loner"in your Resque-web config.rb file! or... FAIL (cleared jobs wontbe relaunched again!)BAD :Need inheritance in job classes :(... But a fork exists thatdoesnt need it.
  18. 18. Other plugins...Look @ github.com/defunkt/resqueWiki!And remember that with a goodsystem architecture, Resque is just...

×