Intro to MQ
•
5 likes•760 views
Slides from an intro to MQ talk I gave at a Toronto Ruby Brigade meet up.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (12)
Similar to Intro to MQ
Similar to Intro to MQ (18)
Recently uploaded
Recently uploaded (20)
Intro to MQ
- 1. Intro to Message Queues Rob Paulsen@RobWasHere
- 4. AMQP
- 5. gem 'amqp'
- 6. why
- 18. require 'amqp' def publish(order) json_order = serialize_to_json(order) AMQP.start(:host => SERVER_IP) do MQ.queue('PrintQueue').publish(json_order) end end
- 19. require 'amqp' def subscribe(&msg_callback) AMQP.start(:host => SERVER_IP) do MQ.queue('PrintQueue').subscribe(:ack => true) do |headers, payload| order = deserialize_from_json(payload) headers.ack msg_callback.call(order) end end end
- 32. def subscribe(&msg_callback) AMQP.start(:host => SERVER_IP) do exchange = amq.fanout('OrderExchange') MQ.queue('Terminal123', :exclusive => true) .bind(exchange).subscribe do |headers, payload| order = deserialize_from_json(payload) headers.ack msg_callback.call(order) end end end
- 34. MQ Server Options RabbitMQ OpenAMQ Apache Qpid / Redhat MRG MSMQ (Microsoft) JMS (Java) WebSphere MQ (IBM)
- 35. Notable Gems amqp (asynchronous AMQP) Bunny (synchronous AMQP) Qusion (AMQP for Rails Apps) lots more .. http://rubygems.org/search?query=amqp
- 36. Questions? (feel free to throw tomatoes now if you have to)
Editor's Notes
- both common terms.. refers to facilitates forasynchronous message passing
- Why you’d want to use them..What that involvesAnd how to interact with MQ from ruby
- A quick note . AMPQ is an open standard supported by some of the more popular MQ systems..Going to use it’s concepts and terms to keep things simple, but everything is more or less translatable to other MQs.
- Nice gem for using AMQP asyncronouslybuilt on top of EventMachine..using that in the code examples…
- Why use message queues ..
- You’ve got two applications.. A and B
- And you need to pass some information from one to the other… That’s a pretty common situation..
- things get a more complicated if B takes a while to deal with the request/msg/info….A is going to hit that box several times maybe before B can deal with the first one.
- No problem.. We can just add some sort of queue into B to buffer those messages so it can deal with them when it can.
- It’ll probably be easier if we talk about a slightly less abstract example..Use a photography store like Blacks as an example.. They’ve got a kiosk where people can upload their photos.. Pick their print sizes.. EtcAnd a printer application which is drives the actual printerKiosk needs to pass that info to the printer, but we don’t want the customer to stand there for an hour for their photos .. So we pass it asynchronously.. At this point it doesn’t have to be a message queue per se.. You could roll your own solution.. Maybe you just store a bunch of pending requests in a DB with a date stamp and an iscomplete bit … not the most elegant solution .. but whatever.. It’d work..
- But what happens when you need to add another printer?
- You could consider the kiosk and printer to be a single unit and just add more pairs… Nice thing about that is you don’t have to change any code .. Just deploy another pairBut that’s not great.. If one printer goes down for maintenance .. Or is bogged down with a big order.. This setups not going to give you any load balancing or failover niceness .. And it ties scaling of the kiosk tier to scaling the printer tier.. In reality those will probably need to change at different rates .. So …
- That’s bad .. Not a good idea..
- Another option is to have every kiosk know about every printer ..Maybe each kiosk alternates, or has a primary printer, with a failover .. Its better.. But it puts a lot of responsibility on the kiosk app to manage the overall system.. Does the kiosk check to see if a printer is currently overloaded?How does it handle printers going offline.. EtcAll of that is stuff the kiosk app shouldn’t have to care about..And things are always easier when every piece of a system is as simple as possible.So….
- No good
- A cleaner solution would be to add a message queue between the two tiersEvery kiosk just knows how to publish a new orderEvery printer just knows how to pull orders off the queueBoth apps become much simpler..
- Kiosk ruby code
- Printer ruby code
- So it’s much simpler to write and gives you a lot of extra benefits for free..
- You can add more kiosks.. Or ..
- Or more printers .. And not have to change any code..Or even any configuration..Just plug them into the network and turn them on.. The two tiers can scale independently now very easilyYou also just by the nature of the setup, You get free load balancing and failover behavior. If a printer goes offline.. If its overloaded…
- One thing to note.. Is that we have now introduced a single point of failure into the system.If the broker dies, the entire system is toast. Luckily most of the major MQ systems offer clustering and failover options ..Similar to what you’d expect from a mature database platformSo you can mitigate the risk, but it’s something to think about as you’re structuring the solution…
- So that’s fine .. Adding the message queue help solve a common architectural issue.. Decoupling clients from background processing tasks.. But what else can you do with it…
- Expanding on the photo store example .. Lets say they have some terminals that the staff use .. And they want to have a notification pop up with some details every time an order is sent for printing.
- You might think.. Ok .. I’ll just have them watch the same queue as the printers and everything will work..
- No.. That’s not going to work..that queue is setup to have one and only one consumer retrieve each message.. And once they do they remove it from the queue ..
- Lets take a closer look at the broker..The kiosks aren't actually writing message to a queue, They are passing them to an exchange which then delivers them to the queue In this case that’s a shared queue.. which is to say its independent of any one consumer… But we can more queues if we want (and more exchanges.. But I wont get into that).. And those queues do not need to be shared..
- If each terminal had its own queue… Then the exchange can deliver a copy of each order to each of those queues.. And the kiosk can deal with it as it sees fit..
- Ruby code for terminal..Note: Exclusive = true
- Messages vs RPC callsKeeping messages consumer agnostic makes things easier.
- If you use the AMQP compliant servers (first three)you can get a lot of stuff for free..