● Computer Engineering graduate from Pulchowk
● Runs a Python shop called Thoplo
● Does a lot of Python/Django
● Uses far too many exec statements
● Not a designer
● Some problems take too long to solve within a
● Web applications need to provide an immediate
● Sending emails, processing images, making API
calls, etc. can take time
● So background processes and job queues
● Lots of one-off configuration
● Difficult and time-consuming to set up
● Difficult to get right - lots of moving parts
that can fail
● Time consuming to monitor and maintain
● Needs right kind of storage mechanism with
the right kind of message queues
It’s OVERKILL for MOST web projects!
● RQ (Redis Queue) is a simple Python library
for queueing jobs and processing them in the
background with workers.
● Has a low barrier to entry.
● Has been created as a lightweight alternative
to existing queueing frameworks.
● Can get it up and running in an afternoon
● No configuration file editing -- or very
● Don’t have to configure a message broker --
just uses Redis
A server that lets you associate a data structure
with a key
● Written in C
● Bindings in most popular languages
● Easy to access from Python
● Used extensively for caching in web applications
● I know almost nothing about it.
Monitoring on a
● Django integration with RQ
● Define queues in settings.py
● Add some urls
● Done! You can start queuing jobs
● Enqueue jobs like so:
django_rq.enqueue(func, foo, bar=baz)
● Has a @job decorator like Celery’s @task
● Has management commands for rqworker and
python manage.py rqworker high default low
python manage.py rqscheduler
Break jobs down into atomic packages that don’t
do anything else.
At a point when this doesn’t work for you
anymore, it’s definitely time to upgrade to a
more powerful, more complex tool -- Celery.
(Or just hire someone to do it for you!)