Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

RabbitMQ And Nanite

9,079 views

Published on

Short presentation on AMQP, RabbitMQ and Nanite.

Published in: Technology
  • Be the first to comment

RabbitMQ And Nanite

  1. 1. RabbitMQ and Nanite Scalable Messaging Mathias Meyer, Peritor GmbH
  2. 2. The Problem • A scalable backend to offload asynchronous tasks • Self-configuring • Self-healing • Self-assembling • But not a lot of people have it
  3. 3. The Problem • I’m still learning about the nitty-gritty details
  4. 4. Protocols • JMS • MQ • XMPP • SMTP (yep, for real) • STOMP • RestMS • Wtf?
  5. 5. AMQP!
  6. 6. AMQP • Open standard for messaging middleware • Vendor neutral • Wire level protocol • Mandates interoperatibility • Abundant number of libraries that speak it
  7. 7. AMQP • Supports publish/subscribe and point-to-point messaging, among others
  8. 8. AMQP Components • Broker • Combination of Exchange and a Queue • Exchange routes messages (stateless) based on patterns • Queue buffers messages (stateful) • Bound by Bindings
  9. 9. AMQP Components • Consumer • Producer • Messages (duh!)
  10. 10. RabbitMQ
  11. 11. RabbitMQ • Erlang-based, independent AMQP implementation • Some 6k lines of code • Apache Qpid (Java): 142k lines of code • Also speaks STOMP, SMTP, XMPP and HTTP • Message persistence uses a journal • Wicked fast
  12. 12. RabbitMQ • port install rabbitmq-server • It just works™ • rabbitmqctl • Can be clustered through Erlang’s distributed nodes
  13. 13. Nanite • A self-assembling fabric of Ruby daemons • Uses mappers, agents and actors • Mappers route message requests • Agents handle messages • Actors offer specific services bundled into agents
  14. 14. Nanite • Messages can be simple pushes • Or requests waiting for a response
  15. 15. Nanite Agent Agent Mapper RabbitMQ Agent Mapper Agent
  16. 16. Nanite Agents
  17. 17. Nanite Agents class Reactor include Nanite::Actor expose :react def react(payload) "reacting to message with payload: #{payload}" end end
  18. 18. Nanite Nanite.request('/reactor/react', 'good acting is reacting') do |r| pr end
  19. 19. Nanite • Agents register themselves and advertise their services, constantly pinging the mappers • Mappers remove timed-out agents • Work distributed based on agent load
  20. 20. Nanite • Agents can interoperate with each other during one single message processing • Mapper state can (and should) be stored in redis • Mappers live inside your application or standalone
  21. 21. Intermediate Messages class Reactor include Nanite::Actor expose :react def react(payload) request("/worker/work", "kill all humans", :intermediate_handler => lambda {|res| puts res}) # working, working... yield "I'm almost done" # cleanup the mess "done reacting to message with payload: #{payload}" end end
  22. 22. Intermediate Messages class Worker include Nanite::Actor expose :work def work(payload) Human.all.each do |human| human.destroy yield "Destroyed human #{human.name}" end "Killed all the humans" end end
  23. 23. Nanite Internals • Based on EventMachine and Ruby amqp library (also EventMachine-based) • Everything runs in the EventMachine loop
  24. 24. Nanite Internals • Uses a combination of the following queues to work • Registration queue • Heartbeat queue • Advertise queue • Agent queues
  25. 25. Nanite Internals • Messages can be encrypted and signed between agents and mappers • Sends marshaled Ruby (default),YAML or JSON • The code internals are actually mind-boggling
  26. 26. Nanite Boos • Poor documentation • A not too great test suite
  27. 27. So how do we use it? That really is a topic for another talk, isn’t it?
  28. 28. So how do we use it? That really is a topic for another talk, isn’t it?
  29. 29. Resources • http://skillsmatter.com/podcast/erlang/alexis-richardson- introduction-to-rabbitmq • http://skillsmatter.com/podcast/erlang/rabbitmq-internal- architecture-tony-garnock-jones • http://brainspl.at/articles/2008/10/11/merbcamp-keynote-and- introducing-nanite • http://github.com/ezmobius/nanite • http://www.rabbitmq.com • http://www.amqp.org • http://www.slideshare.net/somic/introduction-to-amqp-messaging- with-rabbitmq

×