Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Celery


Published on

Introduction to celery task queue using Python, Flask and Redis

Published in: Software
  • Be the first to comment

  • Be the first to like this

Introduction to Celery

  1. 1. Celery The Task Queue
  2. 2. Whatiscelery? ● Celery is a vegetable ● Also used as a spice
  3. 3. Well,NotthatCelery Celery what we want is a task queue
  4. 4. WhattheFishisaTaskQueue? Tasks are handled asynchronously either because they are not initiated by an HTTP request or because they are long- running jobs that would dramatically reduce the performance of an HTTP response.
  5. 5. For example, a web application could poll the Some API every 10 minutes to collect the names of the top 100 customers in a particular service. A task queue would handle invoking code to call the API, process the results and store them in a persistent database for later use.
  6. 6. Again,WhatisCelerythen? Celery is an asynchronous task queue based on distributed message passing.
  7. 7. Distributed?
  8. 8. SomeCode…Yay! from celery import Celery def make_celery(app): celery = Celery(app.import_name, broker=app.config['CELERY_BROKER_URL']) celery.conf.update(app.config) TaskBase = celery.Task class ContextTask(TaskBase): abstract = True def __call__(self, *args, **kwargs): with app.app_context(): return TaskBase.__call__(self, *args, **kwargs) celery.Task = ContextTask return celery
  9. 9. Morecode. from flask import Flask app = Flask(__name__) app.config.update( CELERY_BROKER_URL='redis://localhost:6379', CELERY_RESULT_BACKEND='redis://localhost:6379' ) celery = make_celery(app) @celery.task() def add_together(a, b): return a + b
  10. 10. RunForestRun.. result = add_together.delay(23, 42) result.wait()
  11. 11. SendEmailsAsynchronously?IfeelYouBro. Demo.
  12. 12. FewThingstoPonder. ● Start Simple (1 Worker, 1 Queue) ● Monitor it. (Celery Flower) ● Consume tasks faster than you produce them (Shamal) ● Tweaking for Concurrency ● Keep your tasks clean ● Manage DB transactions
  13. 13. AnyQuestions? Get your lazy ass up and Google it my friend!
  14. 14. ThankYou.