RabbitMQ & Hutch

1,795
-1

Published on

Talk I gave at LRUG about RabbitMQ, how we're using it at GoCardless and the some tools we built - Hutch and AMQPC

Hutch is an opinionated library for RabbitMQ which makes using it for comms in SOA super simple

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,795
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
20
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

RabbitMQ & Hutch

  1. 1. RabbitMQ & Hutch Pete Hamilton - GoCardless
  2. 2. About Me • Developer at GoCardless • CompSci Student at ICL • Github: https://github.com/petehamilton • Blog: pete-hamilton.co.uk • Twitter: @peterejhamilton
  3. 3. SOA Communication A (very) brief overview Awesome Service
  4. 4. SOA Communication A (very) brief overview Awesomer Services
  5. 5. SOA Communication A (very) brief overview Awesomer Services
  6. 6. SOA Communication A (very) brief overview ? Client - Server? Message Queues?
  7. 7. RabbitMQ
  8. 8. RabbitMQ Why? • Nice Learning Curve • Reliable • Active Community • Polyglot • Scalable • Proven itself in battle production
  9. 9. RabbitMQ Why? • Payments are pretty serious stuff • Async = Potential problems • Network Failure? • Consumer Crashes? • Duplication/redelivery? • RabbitMQ helps with confirms, acknowledgements, mandatory flags
  10. 10. RabbitMQ Routing P X C1 C2 Q1 Q2
  11. 11. Fanout Exchange RabbitMQ Routing P X C1 C2 Q1 Q2
  12. 12. Direct Exchange RabbitMQ Routing P X C1 C2 type=direct firefly star_trek big_bang_theory Q1 Q2
  13. 13. Topic Exchange RabbitMQ Routing P X C1 C2 type=topic tv.scifi.* tv.comedy.* C3 tv.# Q1 Q2 Q3 Sample Message Routing Keys: - tv.comedy.big_bang_theory - tv.scifi.firefly - tv.scifi.star_trek
  14. 14. RabbitMQ • Very Flexible, but... • Which data structure do you use? • How do you serialise your data? • Which fields do you require in your messages? • How should messages be routed? fan out? direct? topically? • How do new nodes set themselves up? • What form should “Boiler Plate” code take?
  15. 15. A system for processing RabbitMQ messages in Ruby
  16. 16. • Our Conventions (Warning: Opinions!) • Topic Exchange Routing • Serialisation w/ JSON • Message IDs and Timestamps • Simple to use • Configurable • Specific ruby files or full Rails apps
  17. 17. Publisher
  18. 18. Consumer
  19. 19. https://github.com/gocardless/hutch
  20. 20. RabbitMQ Scaling • Good guide to clustering online • Potential for issues with multiple nodes
  21. 21. RabbitMQ Scaling • Good guide to clustering online • Potential for issues with multiple nodes S S M C1
  22. 22. RabbitMQ Scaling • Good guide to clustering online • Potential for issues with multiple nodes S S M x C1
  23. 23. RabbitMQ Scaling • Good guide to clustering online • Potential for issues with multiple nodes S S M x C1
  24. 24. RabbitMQ Scaling • Good guide to clustering online • Potential for issues with multiple nodes S S M C1
  25. 25. RabbitMQ Scaling • Good guide to clustering online • Potential for issues with multiple nodes S S M C1 x
  26. 26. RabbitMQ Scaling • Good guide to clustering online • Potential for issues with multiple nodes M S M C1 x
  27. 27. RabbitMQ Scaling • Good guide to clustering online • Potential for issues with multiple nodes • Clients exist for load testing, but... • We wanted ++ parallelism • Built “AMQPC”
  28. 28. AMQPC • Written in Go • Producer & Consumer modes • Traffic Modes • One off burst • Intermittent burst • Consistent load
  29. 29. AMQPC
  30. 30. https://github.com/gocardless/amqpc
  31. 31. Useful Links • Hutch - www.github.com/gocardless/hutch • AMQPC - www.github.com/gocardless/amqpc • RabbitMQ - http://www.rabbitmq.com/getstarted.html
  32. 32. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×