UNCON<br />Message Queues &Offline Processing in PHP<br />ByMarcel Esser<br />Code Samurai, CROSCON<br />Contact<br />+1 2...
Offline Processing Explained<br />What is offline processing?<br />Short answer: Running things in the background.<br />Lo...
Offline Processing Explained<br />Now<br />Later<br />Web Request<br />Dispatcher<br />Message<br />ApplicationData<br />S...
Offline Processing Explained<br />Now<br />Later<br />Web Request<br />Dispatcher<br />Message<br />ApplicationData<br />S...
Message Queue Solutions<br />
Amazon SQS<br />Proprietary Web Service. 0.01USD/10,000.00 msgs + bw.<br />Dead simple.<br />Handles all locking.<br />Mes...
gearman<br />Open source C daemon.<br />Socket-based client libraries in several languages.<br />PHP client library is nic...
dropr<br />Open source PHP daemon.<br />“trunk in production”<br />Separate client and poll services.<br />De-centralized<...
Apache ActiveMQ<br />Open source Java daemon.<br />Big.<br />Fairly Java-centric.<br />De-centralized<br />PHP-centric<br ...
beanstalkd<br />Open source C daemon.<br />Fast.<br />Can be persisted by writing events to a binlog.<br />Can be distribu...
My Homebrew (version 1, PHP)<br />Proprietary PHP library.<br />Pretty fast.<br />Implemented as a memcached linked list.<...
My Homebrew (version 2, C)<br />Proprietary C  daemon.<br />Very, very fast. Non-blocking epoll server. C10K and then some...
Zend Queue / Zend Job Queue<br />Largely a collection of adaptors to other message queues, plus a job-based queue system.<...
Message Formats<br />
Message Formats<br />You need something other than XML and JSON because…?<br />
Oh, and btw.<br />In case you forgot…<br />
Shameless Plug<br />Hi, my name is Marcel Esser. I work for CROSCON.<br />We do: <br />Bespoke application development<br ...
Upcoming SlideShare
Loading in …5
×

Message Queues & Offline Processing with PHP

65,141
-1

Published on

Unconference presentation from ZendCon 2009.

Published in: Technology
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
65,141
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

Message Queues & Offline Processing with PHP

  1. 1. UNCON<br />Message Queues &Offline Processing in PHP<br />ByMarcel Esser<br />Code Samurai, CROSCON<br />Contact<br />+1 202 730 9728<br />+49 176 785 69729<br />marcel.esser@croscon.com<br />
  2. 2. Offline Processing Explained<br />What is offline processing?<br />Short answer: Running things in the background.<br />Long answer: Offline processing is the act of queuing, prioritizing, dispatching, and executing jobs described by a message packet, received from some initiating process.<br />Practical answer: Do the heavy stuff in a worker thread.<br />
  3. 3. Offline Processing Explained<br />Now<br />Later<br />Web Request<br />Dispatcher<br />Message<br />ApplicationData<br />Store<br />Message<br />Dispatcher<br />Worker<br />
  4. 4. Offline Processing Explained<br />Now<br />Later<br />Web Request<br />Dispatcher<br />Message<br />ApplicationData<br />Store<br />Message<br />Dispatcher<br />Worker<br />
  5. 5. Message Queue Solutions<br />
  6. 6. Amazon SQS<br />Proprietary Web Service. 0.01USD/10,000.00 msgs + bw.<br />Dead simple.<br />Handles all locking.<br />Messages up to 8kb in size.<br />Messages expire after 4 days.<br />Many permissions options.<br />Automatically distributed.<br />Runs on a 3rd party.<br />5 functions (CreateQueue, SendMessage, ReceiveMessage, ChangeMessageVisibility, and DeleteMessage)<br />SOAP or RESTful.<br />
  7. 7. gearman<br />Open source C daemon.<br />Socket-based client libraries in several languages.<br />PHP client library is nice and clean.<br />Can be configured for redundancy.<br />Fast.<br />No Persistence. Now with minty-fresh persistence.<br />
  8. 8. dropr<br />Open source PHP daemon.<br />“trunk in production”<br />Separate client and poll services.<br />De-centralized<br />PHP-centric<br />Storage and transport are modular. FS and cURL are implemented.<br />
  9. 9. Apache ActiveMQ<br />Open source Java daemon.<br />Big.<br />Fairly Java-centric.<br />De-centralized<br />PHP-centric<br />Storage and transport are modular. FS and cURL are implemented.<br />Uses big words like ‘transport connectors’ and ‘network connectors’ and has big XML configuraton files.<br />Just fail. Don’t even ask me.<br />
  10. 10. beanstalkd<br />Open source C daemon.<br />Fast.<br />Can be persisted by writing events to a binlog.<br />Can be distributed.<br />Still pretty raw.<br />Client libraries for many languages.<br />Little bit slanted towards Rails.<br />
  11. 11. My Homebrew (version 1, PHP)<br />Proprietary PHP library.<br />Pretty fast.<br />Implemented as a memcached linked list.<br />Not even remotely difficult to write.<br />Can scale up pretty well if done with some consideration.<br />Used memcache for locking.<br />I don’t feel comfortable taking PHP out of it’s web role in that way.<br />
  12. 12. My Homebrew (version 2, C)<br />Proprietary C daemon.<br />Very, very fast. Non-blocking epoll server. C10K and then some.<br />Used mutexes for locking.<br />Highly specialized for the role; not multi-purpose.<br />Difficult to maintain.<br />Difficult to write (properly).<br />Difficult to deploy.<br />
  13. 13. Zend Queue / Zend Job Queue<br />Largely a collection of adaptors to other message queues, plus a job-based queue system.<br />Why? Existing APIs + at command<br />
  14. 14. Message Formats<br />
  15. 15. Message Formats<br />You need something other than XML and JSON because…?<br />
  16. 16. Oh, and btw.<br />In case you forgot…<br />
  17. 17. Shameless Plug<br />Hi, my name is Marcel Esser. I work for CROSCON.<br />We do: <br />Bespoke application development<br />Customized service monitoring<br />MyCourt productivity software<br />Info?<br />marcel.esser@croscon.com<br />202.730.9728<br />

×