SlideShare a Scribd company logo

Why Task Queues - ComoRichWeb

We start with why you should use task queues. Then we show a few straightforward examples with Python and Celery and Ruby and Resque. Finally, we wrap up with a quick example of a task queue in PHP using Redis. https://github.com/bryanhelmig/phqueue

1 of 30
Download to read offline
Why Task Queues in
Python, Ruby and More!
      ~ a talk by Bryan Helmig
task queue noun ˈtask ˈkyü
a system for parallel execution of discrete
tasks in a non-blocking fashion.
               celery, resque, or home-grown...
broker noun broh-ker
the middle man holding the tasks
(messages) themselves.
               rabbitmq, gearman, redis, etc...
producer noun pruh-doo-ser
the code that places the tasks to be
executed later in the broker.
                        your application code!
consumer noun kuhn-soo-mer
also called the worker, takes tasks from
the broker and perform them.
           usually a daemon under supervision.
imagine this...
your app: posting a new message triggers
“new message” emails to all your friends.
Ad

Recommended

Europython 2011 - Playing tasks with Django & Celery
Europython 2011 - Playing tasks with Django & CeleryEuropython 2011 - Playing tasks with Django & Celery
Europython 2011 - Playing tasks with Django & CeleryMauro Rocco
 
Celery: The Distributed Task Queue
Celery: The Distributed Task QueueCelery: The Distributed Task Queue
Celery: The Distributed Task QueueRichard Leland
 
Advanced task management with Celery
Advanced task management with CeleryAdvanced task management with Celery
Advanced task management with CeleryMahendra M
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with CeleryNicolas Grasset
 
An Introduction to Celery
An Introduction to CeleryAn Introduction to Celery
An Introduction to CeleryIdan Gazit
 
Introduction to Python Celery
Introduction to Python CeleryIntroduction to Python Celery
Introduction to Python CeleryMahendra M
 

More Related Content

What's hot

Data processing with celery and rabbit mq
Data processing with celery and rabbit mqData processing with celery and rabbit mq
Data processing with celery and rabbit mqJeff Peck
 
Fundamental JavaScript [UTC, March 2014]
Fundamental JavaScript [UTC, March 2014]Fundamental JavaScript [UTC, March 2014]
Fundamental JavaScript [UTC, March 2014]Aaron Gustafson
 
Life in a Queue - Using Message Queue with django
Life in a Queue - Using Message Queue with djangoLife in a Queue - Using Message Queue with django
Life in a Queue - Using Message Queue with djangoTareque Hossain
 
Top 10 RxJs Operators in Angular
Top 10 RxJs Operators in Angular Top 10 RxJs Operators in Angular
Top 10 RxJs Operators in Angular Jalpesh Vadgama
 
Introduction to RxJS
Introduction to RxJSIntroduction to RxJS
Introduction to RxJSBrainhub
 
Impact of the New ORM on Your Modules
Impact of the New ORM on Your ModulesImpact of the New ORM on Your Modules
Impact of the New ORM on Your ModulesOdoo
 
Celery - A Distributed Task Queue
Celery - A Distributed Task QueueCelery - A Distributed Task Queue
Celery - A Distributed Task QueueDuy Do
 
New Framework - ORM
New Framework - ORMNew Framework - ORM
New Framework - ORMOdoo
 
Monitoring MySQL with Prometheus and Grafana
Monitoring MySQL with Prometheus and GrafanaMonitoring MySQL with Prometheus and Grafana
Monitoring MySQL with Prometheus and GrafanaJulien Pivotto
 
Observables in Angular
Observables in AngularObservables in Angular
Observables in AngularKnoldus Inc.
 
MongoDB Aggregation
MongoDB Aggregation MongoDB Aggregation
MongoDB Aggregation Amit Ghosh
 

What's hot (20)

Data processing with celery and rabbit mq
Data processing with celery and rabbit mqData processing with celery and rabbit mq
Data processing with celery and rabbit mq
 
Celery with python
Celery with pythonCelery with python
Celery with python
 
Fundamental JavaScript [UTC, March 2014]
Fundamental JavaScript [UTC, March 2014]Fundamental JavaScript [UTC, March 2014]
Fundamental JavaScript [UTC, March 2014]
 
Life in a Queue - Using Message Queue with django
Life in a Queue - Using Message Queue with djangoLife in a Queue - Using Message Queue with django
Life in a Queue - Using Message Queue with django
 
Top 10 RxJs Operators in Angular
Top 10 RxJs Operators in Angular Top 10 RxJs Operators in Angular
Top 10 RxJs Operators in Angular
 
Dapper performance
Dapper performanceDapper performance
Dapper performance
 
Django Celery
Django Celery Django Celery
Django Celery
 
Introduction to RxJS
Introduction to RxJSIntroduction to RxJS
Introduction to RxJS
 
Angular 2 observables
Angular 2 observablesAngular 2 observables
Angular 2 observables
 
Redux Thunk
Redux ThunkRedux Thunk
Redux Thunk
 
Impact of the New ORM on Your Modules
Impact of the New ORM on Your ModulesImpact of the New ORM on Your Modules
Impact of the New ORM on Your Modules
 
Celery - A Distributed Task Queue
Celery - A Distributed Task QueueCelery - A Distributed Task Queue
Celery - A Distributed Task Queue
 
Php tutorial
Php  tutorialPhp  tutorial
Php tutorial
 
New Framework - ORM
New Framework - ORMNew Framework - ORM
New Framework - ORM
 
Introduction to Celery
Introduction to CeleryIntroduction to Celery
Introduction to Celery
 
Monitoring MySQL with Prometheus and Grafana
Monitoring MySQL with Prometheus and GrafanaMonitoring MySQL with Prometheus and Grafana
Monitoring MySQL with Prometheus and Grafana
 
Js: master prototypes
Js: master prototypesJs: master prototypes
Js: master prototypes
 
Express js
Express jsExpress js
Express js
 
Observables in Angular
Observables in AngularObservables in Angular
Observables in Angular
 
MongoDB Aggregation
MongoDB Aggregation MongoDB Aggregation
MongoDB Aggregation
 

Similar to Why Task Queues - ComoRichWeb

Ruby/Rails
Ruby/RailsRuby/Rails
Ruby/Railsrstankov
 
Desarrollando aplicaciones web en minutos
Desarrollando aplicaciones web en minutosDesarrollando aplicaciones web en minutos
Desarrollando aplicaciones web en minutosEdgar Suarez
 
To Batch Or Not To Batch
To Batch Or Not To BatchTo Batch Or Not To Batch
To Batch Or Not To BatchLuca Mearelli
 
Basic Oops concept of PHP
Basic Oops concept of PHPBasic Oops concept of PHP
Basic Oops concept of PHPRohan Sharma
 
Ruby on Rails at PROMPT ISEL '11
Ruby on Rails at PROMPT ISEL '11Ruby on Rails at PROMPT ISEL '11
Ruby on Rails at PROMPT ISEL '11Pedro Cunha
 
Rails 3 Beautiful Code
Rails 3 Beautiful CodeRails 3 Beautiful Code
Rails 3 Beautiful CodeGreggPollack
 
Rails workshop for Java people (September 2015)
Rails workshop for Java people (September 2015)Rails workshop for Java people (September 2015)
Rails workshop for Java people (September 2015)Andre Foeken
 
Don't Repeat Yourself, Repeat Others
Don't Repeat Yourself, Repeat OthersDon't Repeat Yourself, Repeat Others
Don't Repeat Yourself, Repeat OthersJohn Nunemaker
 
REST APIs in Laravel 101
REST APIs in Laravel 101REST APIs in Laravel 101
REST APIs in Laravel 101Samantha Geitz
 
Building Testable PHP Applications
Building Testable PHP ApplicationsBuilding Testable PHP Applications
Building Testable PHP Applicationschartjes
 
Python magicmethods
Python magicmethodsPython magicmethods
Python magicmethodsdreampuf
 
Postobjektové programovanie v Ruby
Postobjektové programovanie v RubyPostobjektové programovanie v Ruby
Postobjektové programovanie v RubyJano Suchal
 

Similar to Why Task Queues - ComoRichWeb (20)

Ruby/Rails
Ruby/RailsRuby/Rails
Ruby/Rails
 
Desarrollando aplicaciones web en minutos
Desarrollando aplicaciones web en minutosDesarrollando aplicaciones web en minutos
Desarrollando aplicaciones web en minutos
 
To Batch Or Not To Batch
To Batch Or Not To BatchTo Batch Or Not To Batch
To Batch Or Not To Batch
 
Why ruby
Why rubyWhy ruby
Why ruby
 
Oops in php
Oops in phpOops in php
Oops in php
 
Basic Oops concept of PHP
Basic Oops concept of PHPBasic Oops concept of PHP
Basic Oops concept of PHP
 
Introduction Php
Introduction PhpIntroduction Php
Introduction Php
 
Ruby on Rails at PROMPT ISEL '11
Ruby on Rails at PROMPT ISEL '11Ruby on Rails at PROMPT ISEL '11
Ruby on Rails at PROMPT ISEL '11
 
Rails 3 Beautiful Code
Rails 3 Beautiful CodeRails 3 Beautiful Code
Rails 3 Beautiful Code
 
SOLID Ruby, SOLID Rails
SOLID Ruby, SOLID RailsSOLID Ruby, SOLID Rails
SOLID Ruby, SOLID Rails
 
Rails workshop for Java people (September 2015)
Rails workshop for Java people (September 2015)Rails workshop for Java people (September 2015)
Rails workshop for Java people (September 2015)
 
Tres Gemas De Ruby
Tres Gemas De RubyTres Gemas De Ruby
Tres Gemas De Ruby
 
lab4_php
lab4_phplab4_php
lab4_php
 
lab4_php
lab4_phplab4_php
lab4_php
 
Don't Repeat Yourself, Repeat Others
Don't Repeat Yourself, Repeat OthersDon't Repeat Yourself, Repeat Others
Don't Repeat Yourself, Repeat Others
 
REST APIs in Laravel 101
REST APIs in Laravel 101REST APIs in Laravel 101
REST APIs in Laravel 101
 
Building Testable PHP Applications
Building Testable PHP ApplicationsBuilding Testable PHP Applications
Building Testable PHP Applications
 
Lecture9_OOPHP_SPring2023.pptx
Lecture9_OOPHP_SPring2023.pptxLecture9_OOPHP_SPring2023.pptx
Lecture9_OOPHP_SPring2023.pptx
 
Python magicmethods
Python magicmethodsPython magicmethods
Python magicmethods
 
Postobjektové programovanie v Ruby
Postobjektové programovanie v RubyPostobjektové programovanie v Ruby
Postobjektové programovanie v Ruby
 

Recently uploaded

"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, GoogleISPMAIndia
 
How AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxHow AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxInfosec
 
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxNeo4j
 
AI Act & Standardization: UNINFO involvement
AI Act & Standardization: UNINFO involvementAI Act & Standardization: UNINFO involvement
AI Act & Standardization: UNINFO involvementMimmo Squillace
 
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...MarcovanHurne2
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build PolandGDSC PJATK
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stackSummit
 
The Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolThe Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolProduct School
 
My Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceMy Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceVijayananda Mohire
 
"AIRe - AI Reliability Engineering", Denys Vasyliev
"AIRe - AI Reliability Engineering", Denys Vasyliev"AIRe - AI Reliability Engineering", Denys Vasyliev
"AIRe - AI Reliability Engineering", Denys VasylievFwdays
 
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...ISPMAIndia
 
10 things that helped me advance my career - PHP UK Conference 2024
10 things that helped me advance my career - PHP UK Conference 202410 things that helped me advance my career - PHP UK Conference 2024
10 things that helped me advance my career - PHP UK Conference 2024Thijs Feryn
 
"Platform Engineering with Development Containers", Igor Fesenko
"Platform Engineering with Development Containers", Igor Fesenko"Platform Engineering with Development Containers", Igor Fesenko
"Platform Engineering with Development Containers", Igor FesenkoFwdays
 
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner,  Challenge Like a VC by former CPO, TripadvisorAct Like an Owner,  Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner, Challenge Like a VC by former CPO, TripadvisorProduct School
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17Ana-Maria Mihalceanu
 
Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?MENGSAYLOEM1
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaISPMAIndia
 
How to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanHow to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanDatabarracks
 
Avoiding Bad Stats and the Benefits of Playing Trivia with Friends: PancakesC...
Avoiding Bad Stats and the Benefits of Playing Trivia with Friends: PancakesC...Avoiding Bad Stats and the Benefits of Playing Trivia with Friends: PancakesC...
Avoiding Bad Stats and the Benefits of Playing Trivia with Friends: PancakesC...Adrian Sanabria
 
Enterprise Architecture As Strategy - Book Review
Enterprise Architecture As Strategy - Book ReviewEnterprise Architecture As Strategy - Book Review
Enterprise Architecture As Strategy - Book ReviewAshraf Fouad
 

Recently uploaded (20)

"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
 
How AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxHow AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptx
 
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
 
AI Act & Standardization: UNINFO involvement
AI Act & Standardization: UNINFO involvementAI Act & Standardization: UNINFO involvement
AI Act & Standardization: UNINFO involvement
 
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build Poland
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stack
 
The Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolThe Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product School
 
My Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceMy Journey towards Artificial Intelligence
My Journey towards Artificial Intelligence
 
"AIRe - AI Reliability Engineering", Denys Vasyliev
"AIRe - AI Reliability Engineering", Denys Vasyliev"AIRe - AI Reliability Engineering", Denys Vasyliev
"AIRe - AI Reliability Engineering", Denys Vasyliev
 
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
 
10 things that helped me advance my career - PHP UK Conference 2024
10 things that helped me advance my career - PHP UK Conference 202410 things that helped me advance my career - PHP UK Conference 2024
10 things that helped me advance my career - PHP UK Conference 2024
 
"Platform Engineering with Development Containers", Igor Fesenko
"Platform Engineering with Development Containers", Igor Fesenko"Platform Engineering with Development Containers", Igor Fesenko
"Platform Engineering with Development Containers", Igor Fesenko
 
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner,  Challenge Like a VC by former CPO, TripadvisorAct Like an Owner,  Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
 
Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?Are Human-generated Demonstrations Necessary for In-context Learning?
Are Human-generated Demonstrations Necessary for In-context Learning?
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
 
How to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanHow to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response Plan
 
Avoiding Bad Stats and the Benefits of Playing Trivia with Friends: PancakesC...
Avoiding Bad Stats and the Benefits of Playing Trivia with Friends: PancakesC...Avoiding Bad Stats and the Benefits of Playing Trivia with Friends: PancakesC...
Avoiding Bad Stats and the Benefits of Playing Trivia with Friends: PancakesC...
 
Enterprise Architecture As Strategy - Book Review
Enterprise Architecture As Strategy - Book ReviewEnterprise Architecture As Strategy - Book Review
Enterprise Architecture As Strategy - Book Review
 

Why Task Queues - ComoRichWeb

  • 1. Why Task Queues in Python, Ruby and More! ~ a talk by Bryan Helmig
  • 2. task queue noun ˈtask ˈkyü a system for parallel execution of discrete tasks in a non-blocking fashion. celery, resque, or home-grown...
  • 3. broker noun broh-ker the middle man holding the tasks (messages) themselves. rabbitmq, gearman, redis, etc...
  • 4. producer noun pruh-doo-ser the code that places the tasks to be executed later in the broker. your application code!
  • 5. consumer noun kuhn-soo-mer also called the worker, takes tasks from the broker and perform them. usually a daemon under supervision.
  • 6. imagine this... your app: posting a new message triggers “new message” emails to all your friends.
  • 7. def new_message(request): user = get_user_or_404(request) message = request.POST.get('message', None) if not message: raise Http404 user.save_new_message(message) for friend in user.friends.all(): friend.send_email(message) return redirect(reverse('dashboard'))
  • 8. def new_message(request): user = get_user_or_404(request) message = request.POST.get('message', None) if not message: raise Http404 user.save_new_message(message) for friend in user.friends.all(): friend.send_email(message) return redirect(reverse('dashboard'))
  • 9. the problem: that works good for like 0-6 friends... but what if you have 100,000? or more? will your user ever get a response?
  • 10. bad idea #1: ignore make your users wait through your long request/response cycle. your users are important, right?
  • 11. bad idea #2: ajax return the page fast with JS code that calls another script in the browser’s background. duplicate calls & http cycles are not cool.
  • 12. bad idea #3: cronjob make a email_friends table with user_id & message column. cron every 5/10 minutes. backlogs will destroy you.
  • 13. good idea: queues the task to potentially email thousands of users is put into a queue to be dealt with later, leaving you to return the response.
  • 14. @task def alert_friends(user_id, message): user = User.objects.get(id=user_id) for friend in user.friends.all(): friend.send_email(message) def new_message(request): user = get_user_or_404(request) message = request.POST.get('message', None) if not message: raise Http404 user.save_new_message(message) alert_friends.delay(user.id, message) return redirect(reverse('dashboard'))
  • 15. RULE #1: adding a task to a queue should be faster than performing the task itself.
  • 16. RULE #2: you should consume tasks faster than you produce them. if not, add more workers.
  • 17. queue libraries: 1. celery (python) earlier... 2. resque (ruby) up next... 3. build your own grand finalé!
  • 18. class MessageSend def self.perform(user_id, message) user = User.find(user_id) user.friends.each do |friend| friend.send_email(message) end end end class MessageController < ActionController::Base def new_message render(file: 'public/404.html', status: 404) unless params[:message] current_user.save_new_message(params[:message]) Resque.enqueue(MessageSend, current_user.id, params[:message]) redirect_to dashboard_url end end
  • 19. let’s build our own! and let’s do it with redis, in php and in less than 50 lines of code! (not including the task code)
  • 20. quick redis aside: so, redis is a key-value store. values can be strings, blobs, lists or hashes. we’ll use lists and RPUSH and LPOP.
  • 21. step #1: task runner i’m envisioning this as a Task class with various methods named after each task. each method accepts an array $params.
  • 22. class Tasks { public function email_friends($params) { $user_id = $params->user_id; $message = $params->message; # get $friends from a db query... $friends = array('paul@example.com', 'john@example.com'); foreach ($friends as $friend) { echo "Fake email ".$friend. " with ".$message."n"; } } }
  • 23. step #2: worker i think an infinite loop with a blocking redis BLPOP will work. then it needs to run the right method on the Task object.
  • 24. include_once 'redis.php'; # sets up $redis, $queue include_once 'tasks.php'; $tasks = new Tasks(); while (true) { # BLPOP will block until it gets an item.. $data = $redis->blpop($queue, 0); $obj = json_decode($data[1]); # grab json... $method = $obj->task_name; $params = $obj->params; # calls the task: Task->$task_name($params)... call_user_func(array($tasks, $method), $params); }
  • 25. step #3: run it! we will need to run the worker: ➜ php worker.php
  • 26. step #4: producer let’s make a handy add_task function for easy placement of tasks into the queue.
  • 27. include_once 'redis.php'; # sets up $redis & $queue function add_task($task_name, $params) { global $redis, $queue; $data = Array('task_name' => $task_name, 'params' => $params); $json = json_encode($data) $redis->rpush($queue, $json); } # an example of our task api... add_task('email_friends', array('user_id' => 1234, 'message' => 'I just bought a car!') );
  • 28. step #5: watch it a command line that runs a worker is standard. use something like supervisord or god to run it & monitor it.
  • 29. things missing: our php example does not: store return results, handle errors, route tasks, degrade gracefully, log activity, etc...