safe_bunny: Safe RabbitMQ delivery with local queueing on failure


Published on

safe_bunny is written in erlang, and allows you to safely deliver your messages through RabbitMQ using local queues such as mysql, redis, files, and ets, and will do its best effort to publish the messages through rabbitmq, using confirmation in channels and retrying the delivery periodically

Published in: Software
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

safe_bunny: Safe RabbitMQ delivery with local queueing on failure

  1. 1. Safe Bunny Safe RabbitMQ delivery with local queueing
  2. 2. RabbitMQ: Common Scenario Producer 1 Producer 2 Producer 3 Consumer 1 Consumer 2 Consumer 3 Routing
  3. 3. Safe Bunny: What if something fails? Producer 1 Producer 2 Producer 3 No bindings for the routing key Connectivity issues Temporary issues
  4. 4. Safe Bunny: Safe delivery ● Set channel in confirmation mode ● Sync operation, need to wait for server confirmation or timeout ● How/When to retry the operation? ● How to make messages survive long periods of time?
  5. 5. Safe Bunny: Safe delivery Producer 1 Safe Bunny MySQL Redis File ETS Async, Fire and Forget, no delays ● In case of failure, the message is queued in one of the queue backends. ● If the queue reports an error, the next backend is tried. ● Each queue backend will periodically try to send the message. ● You choose which queues are available, and the order of preference
  6. 6. Safe Bunny: delivery modes ● Direct to MQ: no local queuing: Use safe_bunny:deliver_unsafe/3. This will try to publish a message to the given exchange and routing key, and will not try to queue the message on failure. ● Direct to MQ, local queueing on failure: Use safe_bunny:deliver_safe/3. Like the above, but will queue the message in the fallback queues on failure. Fallback queues are tried in order, according to the producers option of the safe_bunny application. ● Local queuing only: Consumers will try to send the messages to the MQ. "Safest" option (because you queue first). Use safe_bunny:queue/3. One of the queue consumers should pick the new message and try to publish it via rabbitmq.
  7. 7. Safe Bunny: Examples ● safe_bunny:deliver_safe(Exchange, Key, Payload) ● safe_bunny:deliver_unsafe(Exchange, Key, Payload) ● safe_bunny:queue(Exchange, Key, Payload) All arguments are binary(). Where: ● Exchange is the name of the exchange ● Key is the routing key ● Payload is anything you want it to be
  8. 8. ● Concurrency tools: ● Worker Pool: Safe Bunny: Concurrency
  9. 9. ● Jenkins, Coverage, Doc: ● ● Works great with MinionPools :) https://github. com/marcelog/rabbitmq_minionpool ● RabbitMQ confirmation mode: ● RabbitMQ erlang client: html Safe Bunny: Useful links
  10. 10. Thanks!