Unconference presentation from ZendCon 2009.

1. UNCON Message Queues &Offline Processing in PHP ByMarcel Esser Code Samurai, CROSCON Contact +1 202 730 9728 +49 176 785 69729 marcel.esser@croscon.com

2. Offline Processing Explained What is offline processing? Short answer: Running things in the background. Long answer: Offline processing is the act of queuing, prioritizing, dispatching, and executing jobs described by a message packet, received from some initiating process. Practical answer: Do the heavy stuff in a worker thread.

3. Offline Processing Explained Now Later Web Request Dispatcher Message ApplicationData Store Message Dispatcher Worker

4. Offline Processing Explained Now Later Web Request Dispatcher Message ApplicationData Store Message Dispatcher Worker

5. Message Queue Solutions

6. Amazon SQS Proprietary Web Service. 0.01USD/10,000.00 msgs + bw. Dead simple. Handles all locking. Messages up to 8kb in size. Messages expire after 4 days. Many permissions options. Automatically distributed. Runs on a 3rd party. 5 functions (CreateQueue, SendMessage, ReceiveMessage, ChangeMessageVisibility, and DeleteMessage) SOAP or RESTful.

7. gearman Open source C daemon. Socket-based client libraries in several languages. PHP client library is nice and clean. Can be configured for redundancy. Fast. No Persistence. Now with minty-fresh persistence.

8. dropr Open source PHP daemon. “trunk in production” Separate client and poll services. De-centralized PHP-centric Storage and transport are modular. FS and cURL are implemented.

9. Apache ActiveMQ Open source Java daemon. Big. Fairly Java-centric. De-centralized PHP-centric Storage and transport are modular. FS and cURL are implemented. Uses big words like ‘transport connectors’ and ‘network connectors’ and has big XML configuraton files. Just fail. Don’t even ask me.

10. beanstalkd Open source C daemon. Fast. Can be persisted by writing events to a binlog. Can be distributed. Still pretty raw. Client libraries for many languages. Little bit slanted towards Rails.

11. My Homebrew (version 1, PHP) Proprietary PHP library. Pretty fast. Implemented as a memcached linked list. Not even remotely difficult to write. Can scale up pretty well if done with some consideration. Used memcache for locking. I don’t feel comfortable taking PHP out of it’s web role in that way.

12. My Homebrew (version 2, C) Proprietary C daemon. Very, very fast. Non-blocking epoll server. C10K and then some. Used mutexes for locking. Highly specialized for the role; not multi-purpose. Difficult to maintain. Difficult to write (properly). Difficult to deploy.

13. Zend Queue / Zend Job Queue Largely a collection of adaptors to other message queues, plus a job-based queue system. Why? Existing APIs + at command

14. Message Formats

15. Message Formats You need something other than XML and JSON because…?

16. Oh, and btw. In case you forgot…

17. Shameless Plug Hi, my name is Marcel Esser. I work for CROSCON. We do: Bespoke application development Customized service monitoring MyCourt productivity software Info? marcel.esser@croscon.com 202.730.9728