Job queues allow long-running background tasks to be processed asynchronously without blocking the user interface. Beanstalkd is an open-source job queue that provides a simple protocol and can distribute jobs across multiple servers. At Practo, Beanstalkd is used to process tasks like sending emails, generating reports, resizing images, and performing analytics in the background. Supervisord manages the Beanstalkd worker processes to ensure tasks are processed reliably and resources are used efficiently.

1. Job Queues Web is more than request and response

2. Who am I Co Founder / CTO Practo.com Practo.com - simple software for doctors 2000+ clinics, 10,000+ doctors, 1M+ patients Cooking PHP for 5 years

3. Job Queues Any long running work which does not require user feedback should be done in background.

4. Order Processing

5. Order Processing

6. Jobs to run in background Sending email Fetching RSS feed periodically Interacting with other application API Complex report generation Log analysis cleaning file system periodically or after certain action.

7. Options out there

8. Beanstalkd Open source C daemon Fast Persistence – bin log file Distributed PHP library – pheanstalk, beanstalk Simple protocol

9. Beanstalkd ~ Memcached Beanstalkd design based on Memcache Philosophy Simple protocol In Memory Queue Distributed servers handled by client

10. Beanstalkd Architecture

11. Beanstalkd client

12. Beanstalkd Worker

13. Beanstalkd Goodness Named Tubes Priority Delayed job Bury Kick

14. How we use it SMS System Image Re sizing PDF Generation Real Time Analytics And a lot more

15. ONE PROCESS TO RULE THEM ALL

16. Supervisord Python Application Manages foreground process Easy to configure - .ini format Robust and Reliable

17. The configuration

18. Load Balance your workers Measure average time taken to complete a job Define threshold of average time Change num_process in supervisor config Reload to launch new workers

19. Practo is Hiring Hacker Love building stuff Solve difficult problems Change the world [email_address]

20. Questions @abhinavlal abhinavlal.in Sildeshare.net/abhinavlal