This document discusses using Django and Celery for asynchronous task processing. Some key points: - Celery allows defining and processing asynchronous tasks across multiple machines in a distributed, fault-tolerant way. It supports scheduling via cron-like functionality. - Django-celery provides a Django app that integrates Celery with the Django admin for task scheduling and monitoring. It replaces Celery configuration with Django settings. - Unique tasks ensure only one instance of a task runs at a time across all machines. This is useful for accessing sensitive resources atomically. Custom locking and concurrency handling is implemented. - Centralized logging is implemented by connecting signal handlers to Celery's logging setup. This allows sending all worker logs to a syslog

1. Playing tasks with Django & Celery Mauro Rocco @fireantology

2. About me About Jamendo Jamendo is a community of free, legal and unlimited music published under Creative Commons licenses

3. Free Music for users

4. Popularity and earnings for artists

5. Music licensing and background music at competitive prices for companies I'm a Web Developer

6. Python, Javascript, PHP, Java/Android

7. celery contributor (just one of the hundreds )

8. Jamendo needs Multi-format music encoding

9. Statistics (downloads, listens, reviews, stars, fb likes) on different units

10. Music analysis trough external services

11. Music qualification trough several sources

12. Integration with third part services

13. Common jobs (contract generations, certifications, bills, search index update)

14. Celery Async & Sync processes

15. Concurrency within a box

16. Distributed (across machines)

17. Scheduling (interval, cron, ...)

18. Fault tolerant

19. Subtask, Set of tasks

20. Web monitoring (django-celery and others) “ Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well”

21. AMPQ The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for Message Oriented Middleware.

22. Celery schema

23. Celery worker Is the celery process that execute the tasks

24. Can serve one or multiple queues

25. Have a max number of tasks that can be executed at the same time

26. Can be remotely controlled