Agni is a messaging queue built for simplicity, queuing, and priority. It uses RabbitMQ and queues messages asynchronously so services can communicate without slowing down. Agni prioritizes messages and processes them using a Fibonacci heap to ensure important messages are handled quickly. It has been in production use for six months delivering millions of messages daily.

1. AGNIAGNI
MESSAGING. ON FIRE.MESSAGING. ON FIRE.
RickDillon
Yep.⇧Thatdudeisagod.

2. WHY DO WE NEED MESSAGING?WHY DO WE NEED MESSAGING?
Web apps should be fast, so...
do work asynchronously!
And break out cross-cutting concerns into services

3. HOW DO SERVICES COMMUNICATE?HOW DO SERVICES COMMUNICATE?
HTTP, of course.
But then, two problems emerge:
Traffic can be bursty
Systems go down for maintenance

4. So we thought:
“ Wouldn't it be great if all the requests coming in
were queued somewhere? ”

5. So we built Agni.

6. DESIGNDESIGN
Agni is built around three principles:
Simplicity
Queuing
Priority

7. SIMPLICITYSIMPLICITY
One primitive: the queue. Avoid the complexities of AMQP.
Channels
Consumers
Exchanges
Bindings
Routes
Route keys

8. QUEUINGQUEUING
Asynchronous-by-design: just because no one is subscribed to
a queue doesn't mean that messages can't wait for a
consumer.
Durability is baked in.
Flexible architecture: publishers don't need to know about
who is consuming published data.

9. PRIORITYPRIORITY
'Asynchronous' doesn't mean 'not important'. Prioritize the stuff
that needs to be done fast.

10. The Messenger object is used for both publishing and
subscribing.
It's easy to make one.
require 'agni'
m = Agni::Messenger.new('amqp://localhost')

11. Subscribing is a one-liner.
m.subscribe('test_queue') {|m,p| printf p}

12. So is publishing.
1.upto(100).each{|n| m.publish("test#{n}", 'test_queue')}

13. Sending prioritized messages adds only a single parameter.
1.upto(100000).each{|n| m.publish("test#{n}", 'test_queue', n%10)}

14. BEHIND THE SCENESBEHIND THE SCENES
Based on RabbitMQ via
Use ten queues, each with their own AMQP channel
Configure each channel with its own prefetch value
Use an in-memory Fibonacci heap to priortize the incoming
prefetch streams
Process messages off of the heap
ruby-amqp

15. LIMITATIONSLIMITATIONS
Each message is only consumed once (by design)
EventMachine-Fast (5,000 messages/sec)
Supports ten levels of priority

16. STATUSSTATUS
Agni has been in production use at Apartment List for close to
six months, and delivers millions of messages daily.
It is being made available publically for the first time this week.

17. Agni is Open Source, available under the BSD license. Check it
out online at:
Try it out, file bugs, fork it, and send pull requests!
https://github.com/apartmentlist/agni