RabbitMQ And Nanite

8,871 views

Published on

Short presentation on AMQP, RabbitMQ and Nanite.

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,871
On SlideShare
0
From Embeds
0
Number of Embeds
765
Actions
Shares
0
Downloads
248
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

RabbitMQ And Nanite

  1. 1. RabbitMQ and Nanite Scalable Messaging Mathias Meyer, Peritor GmbH
  2. 2. The Problem • A scalable backend to offload asynchronous tasks • Self-configuring • Self-healing • Self-assembling • But not a lot of people have it
  3. 3. The Problem • I’m still learning about the nitty-gritty details
  4. 4. Protocols • JMS • MQ • XMPP • SMTP (yep, for real) • STOMP • RestMS • Wtf?
  5. 5. AMQP!
  6. 6. AMQP • Open standard for messaging middleware • Vendor neutral • Wire level protocol • Mandates interoperatibility • Abundant number of libraries that speak it
  7. 7. AMQP • Supports publish/subscribe and point-to-point messaging, among others
  8. 8. AMQP Components • Broker • Combination of Exchange and a Queue • Exchange routes messages (stateless) based on patterns • Queue buffers messages (stateful) • Bound by Bindings
  9. 9. AMQP Components • Consumer • Producer • Messages (duh!)
  10. 10. RabbitMQ
  11. 11. RabbitMQ • Erlang-based, independent AMQP implementation • Some 6k lines of code • Apache Qpid (Java): 142k lines of code • Also speaks STOMP, SMTP, XMPP and HTTP • Message persistence uses a journal • Wicked fast
  12. 12. RabbitMQ • port install rabbitmq-server • It just works™ • rabbitmqctl • Can be clustered through Erlang’s distributed nodes
  13. 13. Nanite • A self-assembling fabric of Ruby daemons • Uses mappers, agents and actors • Mappers route message requests • Agents handle messages • Actors offer specific services bundled into agents
  14. 14. Nanite • Messages can be simple pushes • Or requests waiting for a response
  15. 15. Nanite Agent Agent Mapper RabbitMQ Agent Mapper Agent
  16. 16. Nanite Agents
  17. 17. Nanite Agents class Reactor include Nanite::Actor expose :react def react(payload) "reacting to message with payload: #{payload}" end end
  18. 18. Nanite Nanite.request('/reactor/react', 'good acting is reacting') do |r| pr end
  19. 19. Nanite • Agents register themselves and advertise their services, constantly pinging the mappers • Mappers remove timed-out agents • Work distributed based on agent load
  20. 20. Nanite • Agents can interoperate with each other during one single message processing • Mapper state can (and should) be stored in redis • Mappers live inside your application or standalone
  21. 21. Intermediate Messages class Reactor include Nanite::Actor expose :react def react(payload) request("/worker/work", "kill all humans", :intermediate_handler => lambda {|res| puts res}) # working, working... yield "I'm almost done" # cleanup the mess "done reacting to message with payload: #{payload}" end end
  22. 22. Intermediate Messages class Worker include Nanite::Actor expose :work def work(payload) Human.all.each do |human| human.destroy yield "Destroyed human #{human.name}" end "Killed all the humans" end end
  23. 23. Nanite Internals • Based on EventMachine and Ruby amqp library (also EventMachine-based) • Everything runs in the EventMachine loop
  24. 24. Nanite Internals • Uses a combination of the following queues to work • Registration queue • Heartbeat queue • Advertise queue • Agent queues
  25. 25. Nanite Internals • Messages can be encrypted and signed between agents and mappers • Sends marshaled Ruby (default),YAML or JSON • The code internals are actually mind-boggling
  26. 26. Nanite Boos • Poor documentation • A not too great test suite
  27. 27. So how do we use it? That really is a topic for another talk, isn’t it?
  28. 28. So how do we use it? That really is a topic for another talk, isn’t it?
  29. 29. Resources • http://skillsmatter.com/podcast/erlang/alexis-richardson- introduction-to-rabbitmq • http://skillsmatter.com/podcast/erlang/rabbitmq-internal- architecture-tony-garnock-jones • http://brainspl.at/articles/2008/10/11/merbcamp-keynote-and- introducing-nanite • http://github.com/ezmobius/nanite • http://www.rabbitmq.com • http://www.amqp.org • http://www.slideshare.net/somic/introduction-to-amqp-messaging- with-rabbitmq

×