Why message queues?We needed a reliable, effective andscalable solution for: ● running asynchronous tasks ● being a common integration point between services (decoupling).
Why message queues?”None of the existing ZODB-based mechanismreally scales. When you are really in need for areal and working queuing mechanism onewould use some external queuing solution (e.g.something AMQP compatible like RabbitMQ).” – Andreas Jung
Advanced Message QueueProtocol (AMQP)Think of it as an highly advancedpost office which can delivermessage to multiple recipientsbased on address.
AMQP jargon 1/2● Message consists of label and payload.● Producer is a program which sends messages to exchange.● Consumer is a program which mostly waits to receive messages.
AMQP jargon 2/2● Exchange receives the messages from producers and pushes them to queues.● Bindings are how the messages get routed from the exchange to a queue.● Queue is a buffer that stores messages.
Real world exampleMoniviestin video publishing service● Plone 4.1 site● Distributed encoder servers (48 CPUs combined)● First version used XML-RPC based communication - lots of error handling code.● New version uses AMQP-messaging.● Very fast and scalable. Recovers from network outages.
Encoder Mediainfomemcached FS RabbitMQ CherryPy upload service Plone
Thank youc.zamqp and examples are available at:● https://github.com/datakurre/collective.zamqp● https://github.com/datakurre/collective.zamqpdemos● https://github.com/datakurre/chatbehavior● https://github.com/datakurre/pubsubannouncementsother relevant links:● https://www.rabbitmq.com/getstarted.html● https://www.amqp.org/about/examples● http://www.manning.com/videla/ (”RabbitMQ in Action”)