Downloaded 59 times
Uploaded byMajdee Zoabi
Message queue architecture
This document discusses message queue architecture. It defines message queues as providing asynchronous communication where the sender and receiver do not need to interact at the same time. Messages are stored in the queue until retrieved. It describes how message queues work with producers publishing messages to an exchange which routes them to queues until handled by consumers. A message broker is described as mediating communication between applications by routing and transforming messages. Examples of message queue systems like ActiveMQ, RabbitMQ and SQS are provided. The document contrasts using message queues versus web services, noting how message queues allow for decoupling and ensure message delivery in failure scenarios.
More Related Content
Similar to Message queue architecture
![谷歌留痕技术教程[ 𝙩𝙤𝙥 𝟮𝟯𝟯. 𝙘 𝙤𝙢 ]](https://cdn.slidesharecdn.com/ss_thumbnails/top233-260130173900-2eb784f9-thumbnail.jpg?width=640&height=640&fit=bounds)
![20260201 [FOSDEM] gomodjail - library sandboxing for Go modules.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/20260201fosdemgomodjail-librarysandboxingforgomodules-260201225659-76609ec4-thumbnail.jpg?width=640&height=640&fit=bounds)
Message queue architecture
- 1. Message Queue Architecture ByMajdee Zoabi
- 2. Agenda • Definition • DoI need Message Queue? • How Does it work? • What is Message Broker? • Uses For A Message Queue • Examples of Queue Systems • Message Queue instead of Web Services • References
- 3. Definition Message queues providean asynchronous communications protocol, meaning that the sender and receiver of the message do not need to interact with the message queue at the same time. Messages placed onto the queue are stored until the recipient retrieves them.
- 4. Do I needMessage Queues ? • Message Queues are useful in certain situation. • General guidelines: • Does your application take a time to generate a response? • Are you using a lot of cron jobs that is not time sensitive, to process data in the background? • Do you wish you could distribute the processing of the data generated by your application among many servers?
- 5.
- 6. How does itwork ? • A system administrator installs and configures message-queueing software. Or can deployed as application during the installation process. • The producer publishes a message to the exchange. • The exchange receives the message and is now responsible for the routing of the message. • The exchange routes the message into the queues. • The messages stay in the queue until they are handled by a consumer. • The consumer handles the message.
- 7. What is aMessage Broker A message broker is an architectural pattern for message validation, transformation and routing. It mediates communication amongst applications, minimizing the mutual awareness that applications should have of each other in order to be able to exchange messages, effectively implementing decoupling.
- 8. Routing Types –based on RabbitMQ • Direct: A message goes to the queue(s) whose binding key exactly matches the routing key of the message.
- 9. Routing Types (Continue) • Fan-out: The fan-out copies and routes a received message to all queues that are bound to it regardless of routing keys or pattern matching.
- 10. Routing Types (Continue) • Topic: Route messages to queues based on wildcard matches between the routing key and something called the routing pattern specified by the queue binding. Binding Key Queue quick.orange.rabbit lazy.brown.fox lazy.orange.elephant
- 11. Uses For AMessage Queue Decoupling Redundancy
- 12. Uses For AMessage Queue Scalability Resiliency
- 13. Uses For AMessage Queue Ordering Guarantees Buffering
- 14. Uses For AMessage Queue Understanding Data Flow Asynchronous Communication
- 15. Examples of QueueSystems • ActiveMQ ( most popular) • RabbitMQ • SQS (Amazon Simple Queue Service) • ZeroMQ • IronMQ • HornetQ • Apollo • QPID
- 18. Message Queue insteadof Web Services Scenario Web Service Message Queue The Server Fails The Client must take responsibility to handle the error. The queue persist the message (optionally, even if the machine shutdown). When the server is working again The client is responsible of resending it. It receives the pending message. When the server gives a response to the call and the client fails The operation is lost***. if the client didn't acknowledge the response the message is persisted.
- 19. References • https://msdn.microsoft.com/en-us/library/ff648849.aspx • https://blog.iron.io/top-10-uses-for-message-queue/ •https://www.slideshare.net/tarequeh/life-in-a-queue-using-message- queue-with-django/32 • https://app.pluralsight.com/player?name=rabbitmq-by-example- m0&clip=0&course=rabbitmq-by-example&author=stephen-haunts
- 20.
Editor's Notes
- #12 Decoupling: By introducing a layer in between processes, message queues create an implicit, data-based interface that both processes implement. This allows you to extend and modify these processes independently, by simply ensuring they adhere to the same interface requirements. Redundancy: Sometimes processes fail when processing data. Unless that data is persisted, it’s lost forever. Queues mitigate this by persisting data until it has been fully processed.
- #13 Scalability Because message queues decouple your processes, it’s easy to scale up the rate with which messages are added to the queue or processed; simply add another process. Resiliency When part of your architecture fails, it doesn’t need to take the entire system down with it. Message queues decouple processes, so if a process that is processing messages from the queue fails, messages can still be added to the queue to be processed when the system recovers. The redundancy provided by message queues guarantees that a message will be processed eventually, so long as a process is reading the queue.
- #14 Ordering Guarantees: In a lot of situations, the order with which data is processed is important. Message queues are inherently ordered, and capable of providing guarantees that data will be processed in a specific order. Buffering: In any non-trivial system, there are going to be components that require different processing times. For example, it takes less time to upload an image than it does to apply a filter to it. Message queues help these tasks operate at peak efficiency by offering a buffer layer–the process writing to the queue can write as fast as it’s able to, instead of being constrained by the readiness of the process reading from the queue. This buffer helps control and optimize the speed with which data flows through your system.
- #15 Understanding Data Flow: In a distributed system, getting an overall sense of how long user actions take to complete and why is a huge problem. Message queues, through the rate with which they are processed, help to easily identify under-performing processes or areas where the data flow is not optimal. Asynchronous Communication: A lot of times, you don’t want to or need to process a message immediately. Message queues enable asynchronous processing, which allows you to put a message on the queue without processing it immediately. Queue up as many messages as you like, then process them at your leisure.