Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Red dwarf&rabbit

435 views

Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Red dwarf&rabbit

  1. 1. How RedDwarf Uses RabbitMQ Hong Yuan HPCloud OaaS & DaaS
  2. 2. RabbitMQ Basics Refresh Queue  Store messages in order Exchange  Routing table to map routing keys with queues Routing Key  String/pattern used to look up queues to talk Channel  Exchange + Queue + Routing Key
  3. 3. RedDwarf RPC the Messaging Library RedDwarf RPC library wraps up RabbitMQ messaging modules and provides 2 + 1 messaging models RPC.cast  Asynchronous call with no waiting for response  One-way from API Server to Smart Agent RPC.call  Synchronous call waiting for response  Round trip from API Server to Smart Agent RPC.listen  Asynchronous call with no waiting for response  One-way from API Server to Smart Agent  Smart Agent cannot use RPC.cast as it is independent of RedDwarf code base
  4. 4. Asynchronous Call (rpc.cast) On the API Server side, a MQ publisher is instantiated to send the message to a topic exchange  Exchange name = “nova”  Routing key = hostname On the Smart Agent side, the message is fetched by a MQ consumer and passed to the Worker in charge of the task  Exchange name = “nova”  Routing key = hostname
  5. 5. Asynchronous Call (rpc.cast) Hostname = Routing Key “host1”API Server 1 SmartAgent = Exchange “host1” Routing Key Hostname = Name =API Server 2 = “host2” “Nova” “host2” SmartAgent Type = “topic” Routing Key = Hostname =API Server N “hostN” “hostN” SmartAgent
  6. 6. Synchronous Call (rpc.call) RedDwarf API, rpc.call()  Send and Wait  Response required  Two-way between API and Smart Agent One Topic Publisher instantiated to send the message request to MQ; Meanwhile, one Direct Consumer is instantiated to wait for the response message. On the other end, the message is consumed by one Topic Consumer dictated by the routing key (such as Instance ID or hostname) Once the task is completed, one Direct Publisher is allocated to send the response message to MQ One Direct Consumer dictated by the routing key (such as msg_id) on the rpc.call Sender side consumes the response message. The direct exchange and queue identifiers are determined by a UUID generator, and are marshaled in the request message The direct channel’s life-cycle is limited to the message delivery
  7. 7. Synchronous Call (rpc.call) Routing Key = Hostname =API Server 1 “host1” Topic “host1” Message Exchange Routing Key SmartAgentUUID =1234 = Name = “host2” “Nova” Hostname = Routing Key “host2”API Server 2 = SmartAgent “hostN” Direct Hostname = Exchange Routing Key “hostN”API Server N = SmartAgent Name = “1234” “1234”
  8. 8. Phone-Home Call from SmartAgent Asynchronous call from Smart Agent to API Server HPCS extension to RedDwarf RPC library Phone-Home as new use cases  API Server has no idea how long to complete instance creation  Need Smart Agent to automatically phone home for updating the instance state when it’s done Background listener to passively waiting on MQ for Smart Agent’s phone home messages A direct consumer is instantiated at the API Sever at its starting time
  9. 9. Phone-Home Call (rpc.listen) Hostname = “host1”API Server 1 SmartAgent Exchange Routing Key Hostname =API Server 2 = Name = “host2” “phonehome” “phonehome” SmartAgent Type = “direct” Hostname =API Server N “hostN” SmartAgent
  10. 10. Topic Publisher Topic Publisher used for both rpc.call() and rpc.cast() Instantiated and used to push a message to the RabbitMQ Every publisher connects always to the same topic-based exchange Life-cycle is limited to the message delivery
  11. 11. Topic Consumer• Used to receive messages from RabbitMQ sent by both rpc.call and rpc.cast• Instantiated by Agent instance and exists throughout Agent instance’s life-cycle• Connects to the same topic-based exchange either via a shared queue or via a unique exclusive queue• Every Agent has two topic consumers – One for rpc.cast and it connects to a shared queue – One for rpc.call and it connects to a unique queue• Used by Agent to invoke the appropriate action based on the request coming in
  12. 12. Direct Consumer Used for receiving the response message from Agent Comes to life only when rpc.call operation is executed Instantiated and used to receive response message from the queuing system Every consumer connects to a unique direct-based exchange via a unique exclusive queue named by message UUID Life-cycle of the queue limited to the message delivery The exchange and queue identifiers are determined by a UUID generator, and are marshaled in the message sent by the Topic Publisher (only rpc.call operations)
  13. 13. Direct Publisher• Used by Agent Server to return the message required by the request operation• Comes to life only during rpc.call operations• Connects to a direct-based exchange whose identity is dictated by the incoming message UUID• Life-cycle is limited to the message delivery
  14. 14. Topic Exchange• On RabbitMQ Server• The Exchange is a routing table that exists in the context of a virtual host (the multi- tenancy mechanism provided by RabbitMQ); its type (such as topic vs. direct) determines the routing policy; a RabbitMQ node will have only one topic-based exchange for every topic in Nova.

×