Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Job_Queues

2,430 views

Published on

Published in: Technology
  • Be the first to comment

Job_Queues

  1. 1. Job Queues Web is more than request and response
  2. 2. Who am I <ul><ul><li>Co Founder / CTO Practo.com </li></ul></ul><ul><ul><li>Practo.com - simple software for doctors </li></ul></ul><ul><ul><li>2000+ clinics, 10,000+ doctors, 1M+ patients </li></ul></ul><ul><ul><li>Cooking PHP for 5 years </li></ul></ul>
  3. 3. Job Queues Any long running work which does not require user feedback should be done in background.
  4. 4. Order Processing
  5. 5. Order Processing
  6. 6. Jobs to run in background <ul><ul><li>Sending email </li></ul></ul><ul><ul><li>Fetching RSS feed periodically </li></ul></ul><ul><ul><li>Interacting with other application API </li></ul></ul><ul><ul><li>Complex report generation </li></ul></ul><ul><ul><li>Log analysis </li></ul></ul><ul><ul><li>cleaning file system </li></ul></ul><ul><ul><li>periodically or after certain action. </li></ul></ul>
  7. 7. Options out there
  8. 8. Beanstalkd <ul><ul><li>Open source C daemon </li></ul></ul><ul><ul><li>Fast </li></ul></ul><ul><ul><li>Persistence – bin log file </li></ul></ul><ul><ul><li>Distributed </li></ul></ul><ul><ul><li>PHP library – pheanstalk, beanstalk </li></ul></ul><ul><ul><li>Simple protocol </li></ul></ul>
  9. 9. Beanstalkd ~ Memcached <ul><ul><li>Beanstalkd design based on Memcache Philosophy </li></ul></ul><ul><ul><li>Simple protocol </li></ul></ul><ul><ul><li>In Memory Queue </li></ul></ul><ul><ul><li>Distributed servers handled by client </li></ul></ul>
  10. 10. Beanstalkd Architecture
  11. 11. Beanstalkd client
  12. 12. Beanstalkd Worker
  13. 13. Beanstalkd Goodness <ul><ul><li>Named Tubes </li></ul></ul><ul><ul><li>Priority </li></ul></ul><ul><ul><li>Delayed job </li></ul></ul><ul><ul><li>Bury </li></ul></ul><ul><ul><li>Kick </li></ul></ul>
  14. 14. How we use it <ul><ul><li>SMS System </li></ul></ul><ul><ul><li>Image Re sizing </li></ul></ul><ul><ul><li>PDF Generation </li></ul></ul><ul><ul><li>Real Time Analytics </li></ul></ul><ul><ul><li>And a lot more </li></ul></ul>
  15. 15. ONE PROCESS TO RULE THEM ALL
  16. 16. Supervisord <ul><ul><li>Python Application </li></ul></ul><ul><ul><li>Manages foreground process </li></ul></ul><ul><ul><li>Easy to configure - .ini format </li></ul></ul><ul><ul><li>Robust and Reliable </li></ul></ul>
  17. 17. The configuration
  18. 18. Load Balance your workers <ul><ul><li>Measure average time taken to complete a job </li></ul></ul><ul><ul><li>Define threshold of average time </li></ul></ul><ul><ul><li>Change num_process in supervisor config </li></ul></ul><ul><ul><li>Reload to launch new workers </li></ul></ul>
  19. 19. Practo is Hiring <ul><ul><li>  Hacker </li></ul></ul><ul><ul><li>Love building stuff </li></ul></ul><ul><ul><li>Solve difficult problems </li></ul></ul><ul><ul><li>Change the world </li></ul></ul><ul><li>[email_address] </li></ul>
  20. 20. Questions @abhinavlal abhinavlal.in Sildeshare.net/abhinavlal

×