The Current State of
Asynchronous
Processing in Ruby
Mathias Meyer, Peritor GmbH

self
• Software-Developer for Peritor GmbH in Berlin
• Code-Reviews, Scaling, Performance-Reviews
and -Tuning, Refactorings
• Maintainer: acts_as_solr, run_later, heaps of stuff
for Capistrano/Webistrano

What?
• Asynchronous Processing
• Just a fancy word for...

“Dude, this request is
getting too long!”

“Okay, let’s move stuff
into the background.”

In other words: What?
• A process puts a job into some queue
• Another process picks up the job and
processes it

Why?
• Requests are taking too long
• Image uploads to S3
• Full-text search updates
• Generating PDFs, reports, etc.
• Sending emails (newsletters, etc.)

Why?
• Your app requires longer running tasks
• Compute daily statistics data
• Create reports
• Data crunching

Why?
• You need to run tasks at a certain time
• Scheduled tasks like invoicing, billing
• Sending out reminder emails

Why?
• Longer tasks
• Are harder to debug and monitor
• Block user and the application

Example
Rails
Client
Upload to
S3
!
Response

Example - Much Better
Client
Rails Worker
Upload to
Response
S3

The simplest Thing that
could possibly work
Thread.new do
AccountMailer.deliver_signup(@user)
end

Done.

Not so fast...
• Problems
• Not easy on resources
• Unreliable
• Not reproducible

A little better...
run_later do
AccountMailer.deliver_signup(@user)
end

run_later
• Borrowed from Merb, available as a Rails-
Plugin
• Uses worker thread and a queue
• Simple solution for simple tasks

What you really want
• Reliable messaging
• Durability
• Scheduling
• Scalable processing
• Not necessarily all at the same time

Options, options
• Messaging Queues
• Polling
• Schedulers
• Oh my!

Protocols, oh my!
• AMQP
• STOMP
• JMS
• XMPP
• RestMS

Message Queues

Message Queues
• Publish/subscribe mechanism
• Usually require more than one new
component in your infrastructure
• Broker middleware
• Subscribers, message listeners

Message Queues
• ActiveMQ
• RabbitMQ
• ActiveMessaging
• Amazon SQS
• Usually more code involved

Pollers
• Polling a database for new jobs
• Simple and domain-specific
• Only one new component
• Usually requires more effort to make them
less prone to errors

Pollers
• Roll Your Own w/ or w/o daemons gem
• delayed_job - adds some infrastructure
• background_job

Pollers - RYO
create_table :jobs do |t|
t.string :klazz, :method
t.string :obj_id
end

Pollers - RYO
class Job < ActiveRecord::Base
def self.schedule(obj, method)
create(:klazz => obj.class.name,
:obj_id => obj.id,
:method => method.to_s)
end
def run
obj = klazz.constantize.find(obj_id)
obj.send(self[:method])
end
end
Job.schedule(@user, :notify_signup)

Pollers - RYO
class JobPoller
def run
loop do
job = Job.find(:first, :lock => true)
next unless job
job.run
job.delete
end
end
end
task :poller do
JobPoller.new.run
end

Pollers - delayed_job
User.find(params[:id]).send_later(:notify_signup)
rake jobs:work

Pollers - delayed_job
class SignupNotifier < Struct.new(:name)
def perform
user = User.find_by_name(name)
user.notify_signup
end
end
Delayed::Job.enqueue SignupNotifier.new(\"david\")

Message Queues, Pollers
• Usually don’t directly support scheduling

Schedulers

Schedulers
• cron, cron, cron (oh, and rake)
• rufus-scheduler
• BackgrounDRb
• Quartz with JRuby (if you’re into that sort of
thing)
• Roll Your Own

rufus-scheduler
scheduler = Rufus::Scheduler.start_new
scheduler.cron '0 22 * * 1‐5' do
Job.new.run
end
scheduler.in '20m' do
puts \"Get a flat white\"
end

Playing with the Big Guys
• Nanite - Jack of all trades
• Uses RabbitMQ, Erlang-based messaging
server, AMQP implementation
• Distributes work across a network of
workers

Nanite
• A self-assembling fabric of Ruby daemons
• Uses mappers and agents
• Mappers route message requests
• Agents handle messages

Nanite
Agent
Agent Mapper
RabbitMQ
Agent Mapper
Agent

Nanite
class Reactor
include Nanite::Actor
expose :react
def react(payload)
\"reacting to message with payload: #{payload}\"
end
end

Nanite
Nanite.request('/reactor/react', 'good acting is reacting') do |r|
p r
end

Nanite
• Agents register themselves, constantly pinging
the mappers
• Mappers remove timed-out agents
• Work distributed based on agent load
• Unfortunately: Poor documentation

Other Stuff
• BackgrounDRb
• AP4R
• job_fu
• spawn
• Starling/Workling
• Daemons
• beanstalkd

Careful now! Pitfalls

Race Conditions

Race Conditions
• Workers try accessing the same data
• Workers update data that is updated heavily
from other layers

Race Conditions
• Make jobs repeatable
• Reduce proneness to errors coming from the
database
• Let your broker take care of that

Queue Congestion

Queue Congestion
http://www.flickr.com/photos/lasgalletas/263909727/

Queue Congestion
• Workers can’t keep up with queue
• More work coming in than being processed

Queue Congestion
• Make it easy to add more workers
• Ensure they don’t steal each other’s work or
do it twice (row-level locking, lock on fields)

Stalled Workers

http://www.flickr.com/photos/freeurmind/2941677586/

Stalled Workers
• Workers stopped processing jobs
• Got stuck on an a particular task
• Choked on an error

Stalled Workers
• Monitor your workers and queues
• Build in error reporting, report exceptions like
in the rest of your code
• Make jobs repeatable
• Use Nanite (self-healing)

Recommendations
• Simple jobs
• Roll Your Own
• delayed_job

Recommendations
• Distributed
• Amazon SQS w/ ActiveMessaging or custom
worker
• Works best on EC2

Recommendations
• Heavy load, scalable
• Nanite/RabbitMQ

The End
• Questions?
• @roidrage
• http://www.paperplanes.de
• http://github.com/mattmatt