Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Keynote: Idiomatic RabbitMQ - Gavin M Roy

37 views

Published on

Watch the full lecture on YouTube: https://www.youtube.com/watch?v=8Zf4zTatQYA&list=PLDUzG2yLXrU4Lz33ZzSdHyfqdHJ8Zum5A

In this talk Gavin will cover AMQP in the context of its use in RabbitMQ with a premise that if you don't fully understand AMQP, you don't understand RabbitMQ. Gavin will discuss the by-directional RPC behaviour, connections, channels, objects, methods, and properties.

--

The first RabbitMQ Summit connected RabbitMQ users and developers from around the world in London on November 12, 2018. Learn what's happening in and around RabbitMQ, and how top companies utilize RabbitMQ to power their services.

https://www.rabbitmqsummit.com

RabbitMQ Summit was organized by:
- Erlang Solutions, offering world-leading RabbitMQ Consultancy, Support, Health Checks & Tuning solutions https://www.erlang-solutions.com/
- CloudAMQP, offering fully managed RabbitMQ clusters https://www.cloudamqp.com

RabbitMQ Summit 2018 was sponsored by the following companies.

Platinum sponsors:
Pivotal
LShift

Gold sponors:
Trifork
AWS

Silver sponsor:
Cogin Queue Explorer

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Keynote: Idiomatic RabbitMQ - Gavin M Roy

  1. 1. Idiomatic RabbitMQ Gavin M. Roy VP of Architecture AWeber Communications Twitter: @Crad
  2. 2. To grok RabbitMQ is to know AMQP 0-9-1
  3. 3. Beginner Questions How do I … What is a …
  4. 4. Next Level Questions Why do / does / is …
  5. 5. RabbitMQ’s Identity Capabilities and behaviors
  6. 6. Advanced Message Queueing Model Exchange
 Receives and route messages Message Queue
 Store messages until they are consumed Binding
 Defines the relationship between the exchange and queue X QueueBindingExchange
  7. 7. Advanced Message Queueing Protocol Client Server Protocol Header Connection.Start Connection.StartOk Connection.Secure Connection.SecureOk Connection.Tune Connection.TuneOk Connection.Open Connection.OpenOk AMQP 0-8 to 0-9-1
 RabbitMQ’s foundation
  8. 8. AMQP Classes and Methods Connection
 Start, StartOk, Close … Channel
 Open, Close, Flow … Exchange
 Declare, Bind, Unbind, Delete … Queue
 Declare, Bind, Purge … Basic
 Publish, Consume, Get, Recover … TX
 Select, Commit, Rollback … Confirm
 Select, SelectOk Exceptions
 Arguments to Channel.Close
  9. 9. AMQP Framing 0 335 0xce1 Frame Payload The frame header is comprised of three parts, the frame type, channel number and the frame size The frame payload varies depending on the frame type There is a single byte marker to specify the end of the frame
  10. 10. Message Publishing Anatomy The Basic.Publish method frame tells RabbitMQ that a client is going to publish a message and that it should expect a Header frame with Basic.Properties in it The content header frame carries the message properties and tells RabbitMQ how big the message body is so that it can process the correct number of body frames that are going to be sent. A single message published may contain multiple body frames, depending on the size of the content and the maximum frame size setting 1 41 0xce1 Basic.Publish 1 82 0xce2 Content Header 1 55 0xce3 Body Frame Type, Channel and Byte Size Frame Type, Channel and Byte Size Frame Type, Channel and Byte Size
  11. 11. Basic.Publish Frame The first two fields of the method frame payload carry the Class and Method ID, numeric values that represent the RPC command. The next part of the method frame carries the first argument for the method, in this case, the name of the exchange to publish in to The routing key value is the next argument and provides information for the exchange to route the message into the appropriate queue or queues. Exchange Name Routing Key Value Mandatory Flag The mandatory flag tells RabbitMQ that it must be able to route the message or it should send a Basic.Return frame indicating the message could not be routed. Basic Publish 1 41 0xce1 Method Frame
  12. 12. Content Header Frame The first field in a content header is the body size, however this value is not considered a property of the message The flag values specify which properties are set The first property of the message, the content-type is specified The app_id property is also specified 55 application/json Test 1144, 200 1014206880 The timestamp property is specified, but is actually carried as the binary packed value The property delivery-mode with a value of 1 tells RabbitMQ to only keep the message in memory when publishing to a queue 1 45 0xce2 Content Header Frame
  13. 13. Body Frame(s) 1 55 0xce3 Body {"foo": "bar", "baz": "qux", "quux", "corge": "grault"} The message body is opaque to the AMQP protocol and is not decoded, inspected, or evaluated by RabbitMQ
  14. 14. Message Publishing How do you answer publishing questions?
  15. 15. Persisted M essages Transactions H A Q ueues Publisherconfirm s N o guarantees N otification on failureAlternate exchanges H A Q ueues w /Transactions
  16. 16. Receiving Messages Questioning how messages are delivered
  17. 17. G etting M essages C onsum ing and using Transactions C onsum e w ith Acknow ledgem ents C onsum ing w ith "N o Ack M ode"enabled C onsum ing w ith Acknow ledgem ents and Q oS > 1
  18. 18. How will you use RabbitMQ? Leverage the AMQ Model or Do-It Yourself
  19. 19. Use RabbitMQ and DYI Contract and content in the message body
  20. 20. Use the AMQ Model Message Properties and Routing
  21. 21. AMQ Model with RPC C / PC / P P / C C / PResponse Queue X X Worker Queue Message Properties content-encoding app-id content-type delivery-modecorrelation-id expiration message-id headers priority user-id reply-to type timestamp cluster-id
  22. 22. AMQ Model with Message Firehose Message Properties content-encoding app-id content-type delivery-modecorrelation-id expiration message-id headers priority user-id reply-to type timestamp cluster-id X Image Hashing Facial Detection RPC Worker P image.new.profile User Directory Materialization image.new.* Auditing Worker image.*.profile image.# Image Removal image.delete.* image.new.profile
  23. 23. Common User Problems • Not understanding the bidirectional nature of AMQP • Heartbeats? • Throttling Messages • How to use channels? • Slow performance due to poor choices
  24. 24. AMQP Problems • Currently 30 items in the 0-9-1 errata • Logic Errors • Marshaling incompatibilities between 
 0-9 && 0-9-1 • Behavior differences between 0-8 && 0-9-1 • Authentication Failure Notifications • Client responsibilities
  25. 25. RabbitMQ Extensions Building on top of the AMQ Model and Protocol
  26. 26. Federation Exchanges Queues Downstream Broker / ClusterUpstream Broker / Cluster P X Upstream Queue XC / P Downstream Broker / ClusterUpstream Broker / Cluster P X Upstream Queue C / P Downstream Queue C
  27. 27. Shovels Exchanges Queues Destination Broker / ClusterSource Broker / Cluster P X Source Queue C / P Destination Queue Destination Broker / ClusterSource Broker / Cluster P X Source Queue XC / P
  28. 28. Protocol Plugins Client AMQP 1.0 Server Plugin Client MQTT Server Plugin Client STOMP Server Plugin Internal AMQP 0-9-1 Client Internal AMQP 0-9-1 Client Internal AMQP 0-9-1 Client
  29. 29. AMQ Model & Protocol Extensions • Queue Mirroring (HA Queues) • Publisher Confirmations (Confirm.Select) • Connection.Blocked and Connection.Unblocked • Consumer Cancellation Notifications (Basic.Cancel behavior enhancement) • Negative Acknowledgements (Basic.Nack) • Exchange to Exchange Bindings (Exchange.Bind && Exchange.Unbind) • Direct Reply-To
  30. 30. AMQ Model & Protocol Extensions (Continued) • Authentication Failure Notifications • Alternate Exchanges • Dead Letter Exchanges • Sender Selected Distribution (CC & BCC in Headers Property) • Time-to-live for messages and queues • Maximum Queue Lengths • Priorities (different implementation than specified)
  31. 31. Architectural Impact of Embracing AMQP • Simplicity • Extensibility • Robustness • Flexibility
  32. 32. Use of Message Properties Robustness by describing content
 Content-Encoding, Content-Type, and Type Traceability by identifying messages
 Correlation-ID and Message-ID Better debugging by identifying sender 
 and time of publishing
 App-ID, Timestamp, and User-ID Message Properties content-encoding app-id content-type delivery-modecorrelation-id expiration message-id headers priority user-id reply-to type timestamp cluster-id
  33. 33. Message Flexibility Same message, multiple simple consumer types
  34. 34. Questions? Follow me on Twitter: @Crad

×