Successfully reported this slideshow.

Evented applications with RabbitMQ and CakePHP

8

Share

Upcoming SlideShare
Ch ch-changes cake php2
Ch ch-changes cake php2
Loading in …3
×
1 of 51
1 of 51

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Evented applications with RabbitMQ and CakePHP

  1. 1. Evented Applications with RabbitMQ , Python, and CakePHP
  2. 2. What the wibblefish? All about offloading work, and doing things in ‘realtime’ Make applications that complete tasks in a distributed/scalable fashion. Doing work outside the request/response cycle of a UI/API.
  3. 3. Work to offload Noti cations (SMS, Email, Postal mail). Logging. Import & export. Process image & video. Reporting. Webhooks.
  4. 4. Isn’t cron good enough? Cron tabs are great with one server. Cron doesn’t scale to multiple machines. Cron doesn’t solve the whole problem space.
  5. 5. Problems with Cron Isn’t suitable for ‘real-time’ results. Requires a set schedule. You might spin up processes that do nothing. Or you might not have enough capacity. Parallelizing is a pain.
  6. 6. Queues to the rescue! Queues accept and dispatch events. Events can contain almost anything*. Events can be roughly equal to ‘jobs’. A single event can trigger lots of downstream work.
  7. 7. Architectural benefits Separation of concerns. Collection of mini/small applications. No monolithic applications. Decoupled, and easier to test. Easy to scale. Busy service = more of them.
  8. 8. Queue technologies RabbitMQ ZeroMQ SQS Celery Gearman
  9. 9. RabbitMQ Written in Erlang. Open source and Free. Really Fast. Socket protocol that is has cheap connections. Data persists after a crash.
  10. 10. Queue basics Producers create events. Consumers eat messages, and do work. messages == events.
  11. 11. Queue Basics Producer
  12. 12. Queue Basics Producer
  13. 13. Queue Basics Producer Consumer
  14. 14. Queue Basics Consumer Producer Consumer
  15. 15. Queue Basics Consumer Producer Consumer Consumer
  16. 16. Exchanges Exchanges abstract queues from the producer. Route messages to queues, that are bound by routing keys. Allows consumers to listen to speci c routing keys. Multiple consumers can listen to the same events.
  17. 17. Exchanges Producer
  18. 18. Exchanges Producer
  19. 19. Exchanges page_view Consumer Producer
  20. 20. Exchanges page_view Consumer import Consumer Producer
  21. 21. Exchanges page_view Consumer import Consumer Producer Consumer error, page_view
  22. 22. Topic subscription Consumers can also listen to topics. Which are keys separated by dots. e.g page_view.search, page_view.update Consumer could listen to page_view.*
  23. 23. Producing Can create messages with many tools. PHP, python, node, ruby, etc. Messages can contain any stringy data (JSON)
  24. 24. Data for messages Only string-y things can be stored in messages. JSON encoded data in message. Key to value stored in Redis/Memcache. Keep messages small. RabbitMQ performs better with small messages.
  25. 25. Consuming Use a daemon process. Daemons in PHP can be done, but they kinda suck. Usually use sleep(2) or other workaround. sleep(2) moves you away from ‘real time’.
  26. 26. Aw bugger..
  27. 27. Use a better tool Other tools are much better for making daemons. Python (pika, kombu, sparkplug) NodeJS (node-amqp) Ruby (carrot, ruby-amqp)
  28. 28. Managing mixed applications Many ways to get it done. Avoid duplicating code. Thin daemons + worker processes.
  29. 29. Thin daemons Python/node daemon accepts messages. Delegate to PHP worker processes. Keeps PHP’s fork + die work ow. Re-use application code by calling shells.
  30. 30. Thin daemons PHP
  31. 31. Thin daemons PHP
  32. 32. Thin daemons Py PHP
  33. 33. Thin daemons Py PHP Py
  34. 34. Thin daemons Py PHP Py Py
  35. 35. Thin daemons Py PHP PHP Py Py
  36. 36. Thin daemons Py PHP PHP Py PHP Py
  37. 37. Thin daemons Py PHP PHP Py PHP Py PHP
  38. 38. Demo Time
  39. 39. Overview Use CakeLog to re events. Use CakePHP shell’s for actual work. Thin python daemon for processing messages.
  40. 40. Python + sparkplug Python module for creating queue listeners. Created by a developer at FreshBooks. Ini con guration le de nes behaviour.
  41. 41. What we’ll try: User events logged during application. Log events to the database.
  42. 42. Insert code
  43. 43. Limitations of using shells Daemon and shell processes have to be on the same box. This can be limiting as you scale, depends on how you structure applications.
  44. 44. Solutions Use HTTP. Have thin daemons dispatch back to the front- end application using HTTP. Use Basic/Digest authentication. Use checksums.
  45. 45. HTTP daemons PHP
  46. 46. HTTP daemons PHP
  47. 47. HTTP daemons PHP Py
  48. 48. HTTP daemons Web PHP Py
  49. 49. HTTP daemons Web PHP Py Web
  50. 50. HTTP daemons Web PHP Py Web Web
  51. 51. Thanks; Questions?

×