Your SlideShare is downloading. ×
0
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Plone, rabbit mq and messaging that just works
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Plone, rabbit mq and messaging that just works

2,378

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,378
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Plone, RabbitMQand messagingthat just worksAsko Soukka & Jukka Ojaniemi
  • 2. Speakers backgroundUniversity of Jyväskyläthe largest Plone user in Finland● 83 Plone sites● 2.5 TB data served through Plone● 700 content editors
  • 3. Demo (serializing writes) RabbitMQ Browser A M1 M1 Browser B Queue Queue Plone M1 M1
  • 4. Why message queues?
  • 5. Why message queues?We needed a reliable, effective andscalable solution for: ● running asynchronous tasks ● being a common integration point between services (decoupling).
  • 6. Why message queues?”None of the existing ZODB-based mechanismreally scales. When you are really in need for areal and working queuing mechanism onewould use some external queuing solution (e.g.something AMQP compatible like RabbitMQ).” – Andreas Jung
  • 7. Advanced Message QueueProtocol (AMQP)Think of it as an highly advancedpost office which can delivermessage to multiple recipientsbased on address.
  • 8. AMQP jargon 1/2● Message consists of label and payload.● Producer is a program which sends messages to exchange.● Consumer is a program which mostly waits to receive messages.
  • 9. AMQP jargon 2/2● Exchange receives the messages from producers and pushes them to queues.● Bindings are how the messages get routed from the exchange to a queue.● Queue is a buffer that stores messages.
  • 10. Basic messagingWork queues M1 Message broker Consumer M2 M2 M2 M1 Exchange M1 M1 Producer Queue (direct) M2 Consumer
  • 11. Basic messagingPublish/Subscribe queues M1 Message broker M1 Consumer Queue M1 Exchange M1 Producer (fanout) M1 Queue M1 Consumer
  • 12. Basic messagingRPC implementation Message broker Exchange M1 M1 RPC queue M1Client/producer M1 R1 Server R1 Reply_to R1 Exchange R1 queue
  • 13. RabbitMQ● Implements AMQP (Advanced Message Queue Protocol) open standard● Written in Erlang ○ made for messaging ○ easy to cluster ○ fast (enough)● Industry tested and reliable
  • 14. Sounds complicated?
  • 15. Example withcollective.zamqpPublish and subscribe announcements M1 Message broker M1 Site B Queue M1 Exchange M1 Site A (fanout) M1 Queue M1 Site C
  • 16. Produce messages withcollective.zamqpfrom zope.component import getUtilityfrom collective.zamqp.interfaces import IProducerproducer = getUtility( IProducer, name="announcer")producer.register() # for transactionproducer.publish("Hello World!")
  • 17. Handle messages withcollective.zamqpfrom five import grokfrom collective.zamqp.interfaces import IMessageArrivedEvent@grok.subscribe(IAnnouncement, IMessageArrivedEvent)def handleMessage(message, event): logger.info(message.body) message.ack()
  • 18. Overview ofcollective.zamqp Message handlers (as event subscribers)Producers Consumers IMessageArrivedEvent stamp messages produce messages Consuming Servers AMQP Broker Connections consume Message broker messages
  • 19. Connections incollective.zamqpbuildout.cfg:[instance]zope-conf-additional = %import collective.zamqp <amqp-broker-connection> connection_id example ... </amqp-broker-connection>
  • 20. Producers incollective.zamqpclass Announcer(Producer): grok.name("announcements") connection_id = "example" exchange = "announcements" exchange_type = "fanout" serializer = "text/plain" durable = False
  • 21. Consuming servers incollective.zamqpbuildout.cfg:zope-conf-additional = %import collective.zamqp .... <amqp-consuming-server> connection_id example site_id Plone </amqp-consuming-server>
  • 22. Consumers incollective.zamqpclass Announcements(Consumer): grok.name("announcements") connection_id = "example" exchange = "announcements" exchange_type = "fanout" queue = "" # anonymous temp. queue durable = False marker = IAnnouncement
  • 23. Message handlers incollective.zamqp@grok.subscribe(IAnnouncement, IMessageArrivedEvent)def handleMessage(message, event): logger.info(message.body) message.ack()
  • 24. Overview ofcollective.zamqp Message handlers (as event subscribers)Producers Consumers IMessageArrivedEvent stamp messages produce messages Consuming Servers AMQP Broker Connections consume Message broker messages
  • 25. Its so simple! It really is!
  • 26. Real world exampleMoniviestin video publishing service● Plone 4.1 site● Distributed encoder servers (48 CPUs combined)● First version used XML-RPC based communication - lots of error handling code.● New version uses AMQP-messaging.● Very fast and scalable. Recovers from network outages.
  • 27. Encoder Mediainfomemcached FS RabbitMQ CherryPy upload service Plone
  • 28. Thank youc.zamqp and examples are available at:● https://github.com/datakurre/collective.zamqp● https://github.com/datakurre/collective.zamqpdemos● https://github.com/datakurre/chatbehavior● https://github.com/datakurre/pubsubannouncementsother relevant links:● https://www.rabbitmq.com/getstarted.html● https://www.amqp.org/about/examples● http://www.manning.com/videla/ (”RabbitMQ in Action”)

×