• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Background Processing in Ruby on Rails
 

Background Processing in Ruby on Rails

on

  • 12,466 views

Background processing is an important tool in web development. Some...

Background processing is an important tool in web development. Some
things just can't optimized enough for the normal request/response cycle
of a web site and need to be run asynchronously. Ruby on Rails doesn't provide
you with any real out of the box solution. There are a lot of external
options available and this part of Rails is lacking any real convention
or standard.

In this presentation, we will talk about how to choose and implement a
solution that fits your needs. We will start with some basic options
using built in Rails tools and we will cover some of the more popular
solutions solutions out there such as BackgrounDRb, Background Job,
Delayed Job, Workling and more.

Rob Mack has been working with Rails professionally on and off since
2005. Rob currently works for VitalSource Technologies.

Statistics

Views

Total Views
12,466
Views on SlideShare
11,401
Embed Views
1,065

Actions

Likes
4
Downloads
111
Comments
0

3 Embeds 1,065

http://blog.robmack.com 984
http://www.slideshare.net 80
http://209.85.229.132 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Background Processing in Ruby on Rails Background Processing in Ruby on Rails Presentation Transcript

    • Background Processing Rob Mack Austin on Rails - April 2009 Thursday, April 30, 2009
    • Why Background Processing? Thursday, April 30, 2009
    • Simple Background Processing Thursday, April 30, 2009
    • script/runner From RAILS_ROOT: • script/runner lib/do_work.rb • script/runner ‘Worker.do_work’ Thursday, April 30, 2009
    • Rake def call_rake(task,options={}) options[:rails_env] = RAILS_ENV args = options.map{|n,v| quot;#{n.to_s.upcase}='#{v}'quot;} system quot;rake #{task} #{args.join(' ')} " end • Episode #127 Railscasts Thursday, April 30, 2009
    • Daemons http://daemons.rubyforge.org/ http://github.com/dougal/daemon_generator/ Thursday, April 30, 2009
    • Generate A Daemon • script/generate daemon my_worker create lib/daemons create script/daemons create lib/daemons/my_worker.rb create lib/daemons/my_worker_ctl create config/daemons.yml Thursday, April 30, 2009
    • Do Some Work $running = true Signal.trap(quot;TERMquot;) do $running = false end while($running) do # Do some work sleep 10 end lib/daemons/my_worker.rb Thursday, April 30, 2009
    • Kick It Off lib/daemons/my_worker_ctl start lib/daemons/my_worker_ctl stop Thursday, April 30, 2009
    • Spawn http://github/tra/spawn Thursday, April 30, 2009
    • Spawn - Do Some Work def my_action flash[:notice] = quot;Processing ... quot; spawn do # do some processing end end Thursday, April 30, 2009
    • Simple Queues Thursday, April 30, 2009
    • ar_mailer setup: gem install ar_mailer ar_sendmail --create-migration ar_sendmail --create-model http://seattlerb.rubyforge.org/ar_mailer/ Thursday, April 30, 2009
    • ar_mailer Schema create_table quot;emailsquot; do |t| t.string quot;fromquot; t.string quot;toquot; t.integer quot;last_send_attemptquot;, :default => 0 t.text quot;mailquot;, :limit => 16777215 t.datetime quot;created_onquot; end Thursday, April 30, 2009
    • ar_mailer Setup environment.rb config.gem 'ar_mailer', :version => '1.3.1', :lib => 'action_mailer/ar_mailer' config.action_mailer.delivery_method = :activerecord Mail Model: class MyMailer < ActionMailer::ARMailer Thursday, April 30, 2009
    • Mail Something MyMailer.deliver_newsletter @recipients Thursday, April 30, 2009
    • The Worker Process ar_sendmail --daemonize --max-age 0 Thursday, April 30, 2009
    • BackgrounDRb http://backgroundrb.rubyforge.org http://github.com/gnufied/backgroundrb Thursday, April 30, 2009
    • BackgrounDRb Setup • sudo gem install chronic packet • script/plugin install git://github.com/ gnufied/backgroundrb.git • rake backgroundrb:setup • rake db:migrate • edit config/backgroundrb.yml • script/generate worker my_worker Thursday, April 30, 2009
    • BackgrounDRb Schema create_table quot;bdrb_job_queuesquot; do |t| t.text quot;argsquot; t.string quot;worker_namequot; t.string quot;worker_methodquot; t.string quot;job_keyquot; t.integer quot;takenquot; t.integer quot;finishedquot; t.integer quot;timeoutquot; t.integer quot;priorityquot; t.datetime quot;submitted_atquot; t.datetime quot;started_atquot; t.datetime quot;finished_atquot; t.datetime quot;archived_atquot; t.string quot;tagquot; t.string quot;submitter_infoquot; t.string quot;runner_infoquot; t.string quot;worker_keyquot; t.datetime quot;scheduled_atquot; end Thursday, April 30, 2009
    • Do Some Work class MyWorker < BackgrounDRb::MetaWorker set_worker_name :my_worker def create(args = nil) # initialization goes here end def do_work(user_id = nil) # do some work end end def my_action flash[:notice] = quot;Processing ...quot; MiddleMan.worker(:my_worker).async_do_work(:arg => @user.id) end Thursday, April 30, 2009
    • Persistent Work def my_action MiddleMan(:my_worker).enq_do_work(:job_key => quot;unique_keyquot;) end Thursday, April 30, 2009
    • Poll Status class MyWorker < BackgrounDRb::MetaWorker set_worker_name :my_worker def create(args = nil) # initialization goes here end def do_work(user_id = nil) # do some work register_status quot;almost donequot; end end Thursday, April 30, 2009
    • Poll Status def my_action @job_key = MiddleMan.new_worker( :worker => :my_worker, :job_key => quot;unique_keyquot; ) MiddleMan.worker(:my_worker, @job_key). do_work(@user.id) end def check_status @status = MiddleMan.worker(:my_worker, @job_key). ask_status end Thursday, April 30, 2009
    • The Worker Process script/backgroundrb start Thursday, April 30, 2009
    • BJ - Background Job http://codeforpeople.rubyforge.org/svn/bj Thursday, April 30, 2009
    • Bj - Setup script/plugin install http:// codeforpeople.rubyforge.org/svn/rails/plugins/bj script/bj setup gem install bj require ‘bj’ bj setup Thursday, April 30, 2009
    • Bj Schema create_table quot;bj_jobquot;, :primary_key => quot;bj_job_idquot; do |t| t.text quot;commandquot; t.text quot;statequot; t.integer quot;priorityquot; t.text quot;tagquot; t.integer quot;is_restartablequot; t.text quot;submitterquot; t.text quot;runnerquot; t.integer quot;pidquot; t.datetime quot;submitted_atquot; t.datetime quot;started_atquot; t.datetime quot;finished_atquot; t.text quot;envquot; t.text quot;stdinquot; t.text quot;stdoutquot; t.text quot;stderrquot; t.integer quot;exit_statusquot; end Thursday, April 30, 2009
    • Bj Schema create_table quot;bj_job_archivequot;, :primary_key => quot;bj_job_archive_idquot; do |t| t.text quot;commandquot; t.text quot;statequot; t.integer quot;priorityquot; t.text quot;tagquot; t.integer quot;is_restartablequot; t.text quot;submitterquot; t.text quot;runnerquot; t.integer quot;pidquot; t.datetime quot;submitted_atquot; t.datetime quot;started_atquot; t.datetime quot;finished_atquot; t.datetime quot;archived_atquot; t.text quot;envquot; t.text quot;stdinquot; t.text quot;stdoutquot; t.text quot;stderrquot; t.integer quot;exit_statusquot; end Thursday, April 30, 2009
    • Do Some Work def my_action flash[:notice] = quot;Processing ...quot; Bj.submit './script/runner /lib/do_some_work.rb' end def my_other_action flash[:notice] = quot;Processing ...quot; Bj.submit 'echo foobar', :tag => 'sample job' end Thursday, April 30, 2009
    • The Worker Process bj run --forever --rails_env=development Thursday, April 30, 2009
    • Delayed Job http://github.com/tobi/delayed_job/ Thursday, April 30, 2009
    • Delayed Job Setup • script/plugin install git://github.com/ tobi/delayed_job.git • create a delayed jobs table Thursday, April 30, 2009
    • Delayed Job Schema create_table quot;delayed_jobsquot; do |t| t.integer quot;priorityquot;, :default => 0 t.integer quot;attemptsquot;, :default => 0 t.text quot;handlerquot; t.string quot;last_errorquot; t.datetime quot;run_atquot; t.datetime quot;locked_atquot; t.datetime quot;failed_atquot; t.string quot;locked_byquot; t.datetime quot;created_atquot; t.datetime quot;updated_atquot; end Thursday, April 30, 2009
    • Do Some Work class MyWorker attr_accessor :user_id def initialize(user) self.user_id = user_id end def perform User.find user_id # do some work end end Thursday, April 30, 2009
    • Do Some Work def my_action flash[:notice] = quot;Processing ...quot; Delayed::Job.enqueue MyWorker.new(@user.id) end def my_action flash[:notice] = quot;Processing ...quot; MyMailer.new(@user.id).send_later(:deliver_newsletter) end Thursday, April 30, 2009
    • The Worker Process rake jobs:work Thursday, April 30, 2009
    • Workling http://github.com/purzelrakete/workling/ Thursday, April 30, 2009
    • Workling Install script/plugin install git://github.com/purzelrakete/ workling.git Thursday, April 30, 2009
    • Do Some Work class MyWorker < Workling::Base def do_something(options) # do some work end end def my_action flash[:notice] = quot;Processing ...quot; MyWorker.async_do_something :user_id => @user.id end Thursday, April 30, 2009
    • The Worker Process with spawn: script/workling_client start with starling: sudo starling -d script/workling_client start Thursday, April 30, 2009
    • Starling http://github.com/starling/starling/ Thursday, April 30, 2009
    • Even More Options • Background Fu - http://github.com/ncr/ background-fu • Roofus Scheduler - http://github.com/ jmettraux/rufus-scheduler • AP4R - http://rubyforge.org/projects/ ap4r/ Thursday, April 30, 2009
    • Advanced Queues Thursday, April 30, 2009
    • Kestrel http://github.com/robey/kestrel/ Thursday, April 30, 2009
    • Rabbit MQ http://www.rabbitmq.com/ Thursday, April 30, 2009
    • Active MQ http://activemq.apache.org/ Thursday, April 30, 2009
    • Amazon SQS http://aws.amazon.com/sqs/ Thursday, April 30, 2009
    • Monitoring • god - http://github.com/mojombo/ god/ • launchd - OS X • monit - http://mmonit.com/monit/ Thursday, April 30, 2009
    • Persistant? Yes No Scaling concerns or fear I enjoy of commitment? managing a process? No Yes No Yes Delayed Job Background Job Spawn Workling Daemons Are you the Kestrel Starling Probably Not Yes next Twitter? Rabbit MQ Thursday, April 30, 2009
    • Thanks rob@robmack.com twitter.com/robmack Thursday, April 30, 2009