Your SlideShare is downloading. ×
0
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
RabbitMQ + CouchDB = Awesome
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

RabbitMQ + CouchDB = Awesome

17,130

Published on

Slides from a talk I gave at erlounge Wellington about a workflow system using RabbitMQ and CouchDB.

Slides from a talk I gave at erlounge Wellington about a workflow system using RabbitMQ and CouchDB.

Published in: Technology, Economy & Finance
2 Comments
22 Likes
Statistics
Notes
  • sure you can, you only have to write all the message passing on your own. i did that before using rabbit and feel sad about all the time i spent to get the message passing right. if you write in erlang that is done for you in the VM and you can do it right away. if you write in anything else, a message queue is a huge benefit :-)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I don't get it, why can't you do all of this without RabbitMQ?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
17,130
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
247
Comments
2
Likes
22
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. one system to rule them all
  • 2. the challenge
  • 3. • write a backend that scales well • handle various frontends (web, iPhone, ...) • runs “somewhere in the cloud” • easy to maintain and extend
  • 4. the solution
  • 5. RabbitMQ +CouchDB =Awesome
  • 6. Part I
  • 7. RabbitMQ • AMQP based Message Queue • lots of additional transports (XMPP, HTTP, STOMP, ...) • erlang based
  • 8. frontend frontend frontend daemon RabbitMQ daemon daemon daemon
  • 9. • every functionality gets a backed daemon • every daemon has a queue associated • communication via a JSON message that gets passed around
  • 10. the JSON message
  • 11. { "response": { "expiry": 710, "ts": "1234425918", "name": "Lenz Gschwendtner", "last4": 0, "cc_type": "master" }, "data": { "options": { "billing_id": "E16003D4-B312-BEA23D9AB789" }, "command": "billing_info" }, "meta": { "lang": "en", "reply_to": "B312-E16003D4-BEA23D9AB789", "platform": "iwmn" } }
  • 12. works great for ... • RPC style communication • if the flow through the system is static • fast transactions
  • 13. gets fiddly when ... • there are asynchronous callbacks from outside • message flow can change based on various factors • tasks can take days to complete
  • 14. Part II
  • 15. workflows
  • 16. ... again ...
  • 17. what we have ... • various frontends • backend daemons • JSON messages
  • 18. what we need • a way to describe message flow • a easy flexible way to write and update it
  • 19. CouchDB
  • 20. ... to the rescue
  • 21. workflow • really only a path definition through backends • some sort of condition management • a backend definition
  • 22. { "_id": "f7a4408898e5...a05b4181045", "_rev": "3-633060011", "name": "billing_info", "type": "workflow", "user": "iwmn", "path": [ "logger", "billing", "billing_iwmn_cc" ] }
  • 23. ... and the backend definition
  • 24. { "_id": "33adf2e3efc3...f423929f057ac", "_rev": "3-3507820969", "conn": "rpc", "type": "worker", "name": "billing_iwmn_cc", "location": "billing.cc", "description": "the CC interface", "command": "billing_verify_cc" }
  • 25. now we need a workflow daemon
  • 26. frontend frontend frontend daemon RabbitMQ workflow daemon daemon daemon
  • 27. where are we now? • we can call backends as before • we can call workflows via the workflow queue • we can manage workflows in CouchDB
  • 28. Part III
  • 29. asynchronous callbacks
  • 30. problem • we start a workflow • we call some external API • the external API somehow notifies us (Mail, Callback URL, ...) about progress • we need to continue in the workflow
  • 31. we need a persistent message store
  • 32. { "_id": "22cd37afd06c82129adb665d3150f570", "_rev": "3-2669449653", "status": "done", "last_update": 1252296819, "name": "E16003D4-F8DB-11DD-B312-BEA23D9AB789", "message": { "response": { "expiry": 710, "name": "Lenz Gschwendtner", "last4": 0, "cc_type": "master" }, "data": { "options": { "billing_id": "E16003D4-F8DB-11DD-B312-BEA23D9AB789" }, "command": "billing_info" }, "meta": { "lang": "de", "name": "E16003D4-F8DB-11DD-B312-BEA23D9AB789", "orig_reply_to": "D234ADE8-9B64-11DE-A92C-E5EF11C6FBDB", "reply_to": "workflow", "user": "iwmn", "log": [ "logger", "billing", "billing.iwmn.info" ], "platform": "iwmn" } }, "created": 1252296815, "type": "bill" }
  • 33. one more problem how do we correlate messages
  • 34. ... with some CouchDB awesome ...
  • 35. • the backend daemon for the callback polls a view in CouchDB • we can search for any value in the request we sent • all we need is a view that searches for the very value this external API uses as correlation ID
  • 36. external frontend frontend frontend callback daemon RabbitMQ workflow daemon daemon daemon
  • 37. me Lenz Gschwendtner CTO iWantMyName @norbu09
  • 38. credits • RabbitMQ • CouchDB • Erlang and Perl
  • 39. more credits • http://www.flickr.com/photos/djpd • http://www.flickr.com/photos/lilyapp • http://www.flickr.com/photos/b-tal • http://www.flickr.com/photos/mikebaird • http://www.flickr.com/photos/eyetwist

×