RabbitMQ And Nanite

RabbitMQ and Nanite Scalable Messaging Mathias Meyer, Peritor GmbH

The Problem • A scalable backend to ofﬂoad asynchronous tasks • Self-conﬁguring • Self-healing • Self-assembling • But not a lot of people have it

The Problem • I’m still learning about the nitty-gritty details

Protocols • JMS • MQ • XMPP • SMTP (yep, for real) • STOMP • RestMS • Wtf?

AMQP • Open standard for messaging middleware • Vendor neutral • Wire level protocol • Mandates interoperatibility • Abundant number of libraries that speak it

AMQP • Supports publish/subscribe and point-to-point messaging, among others

AMQP Components • Broker • Combination of Exchange and a Queue • Exchange routes messages (stateless) based on patterns • Queue buffers messages (stateful) • Bound by Bindings

AMQP Components • Consumer • Producer • Messages (duh!)

RabbitMQ

RabbitMQ • Erlang-based, independent AMQP implementation • Some 6k lines of code • Apache Qpid (Java): 142k lines of code • Also speaks STOMP, SMTP, XMPP and HTTP • Message persistence uses a journal • Wicked fast

RabbitMQ • port install rabbitmq-server • It just works™ • rabbitmqctl • Can be clustered through Erlang’s distributed nodes

Nanite • A self-assembling fabric of Ruby daemons • Uses mappers, agents and actors • Mappers route message requests • Agents handle messages • Actors offer speciﬁc services bundled into agents

Nanite • Messages can be simple pushes • Or requests waiting for a response

Nanite Agent Agent Mapper RabbitMQ Agent Mapper Agent

Nanite Agents

Nanite Agents class Reactor include Nanite::Actor expose :react def react(payload) "reacting to message with payload: #{payload}" end end

Nanite Nanite.request('/reactor/react', 'good acting is reacting') do |r| pr end

Nanite • Agents register themselves and advertise their services, constantly pinging the mappers • Mappers remove timed-out agents • Work distributed based on agent load

Nanite • Agents can interoperate with each other during one single message processing • Mapper state can (and should) be stored in redis • Mappers live inside your application or standalone

Intermediate Messages class Reactor include Nanite::Actor expose :react def react(payload) request("/worker/work", "kill all humans", :intermediate_handler => lambda {|res| puts res}) # working, working... yield "I'm almost done" # cleanup the mess "done reacting to message with payload: #{payload}" end end

Intermediate Messages class Worker include Nanite::Actor expose :work def work(payload) Human.all.each do |human| human.destroy yield "Destroyed human #{human.name}" end "Killed all the humans" end end

Nanite Internals • Based on EventMachine and Ruby amqp library (also EventMachine-based) • Everything runs in the EventMachine loop

Nanite Internals • Uses a combination of the following queues to work • Registration queue • Heartbeat queue • Advertise queue • Agent queues

Nanite Internals • Messages can be encrypted and signed between agents and mappers • Sends marshaled Ruby (default),YAML or JSON • The code internals are actually mind-boggling

Nanite Boos • Poor documentation • A not too great test suite

So how do we use it? That really is a topic for another talk, isn’t it?

Resources • http://skillsmatter.com/podcast/erlang/alexis-richardson- introduction-to-rabbitmq • http://skillsmatter.com/podcast/erlang/rabbitmq-internal- architecture-tony-garnock-jones • http://brainspl.at/articles/2008/10/11/merbcamp-keynote-and- introducing-nanite • http://github.com/ezmobius/nanite • http://www.rabbitmq.com • http://www.amqp.org • http://www.slideshare.net/somic/introduction-to-amqp-messaging- with-rabbitmq

http://www.paperplanes.de	291
http://www.slideshare.net	37
http://links.jetztgrad.net	10
http://feeds.feedburner.com	2
tm-internal://local	1
http://localhost:4000	1
http://translate.googleusercontent.com	1

RabbitMQ And Nanite

by mattmatt on Jul 03, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

7 Embeds 343

Statistics

RabbitMQ And Nanite — Presentation Transcript