Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Messaging for the cloud

398 views

Published on

For successful implementation of distributed systems, flexible and scalable messaging layer is one of the most important components. Setting a static messaging infrastructure and provisioning it manually doesn’t fit well in the cloud-centric development model most organisations are adopting lately. The EnMaase project (http://enmasse.io/) provides an open source solution for deploying your own messaging infrastructure in the cloud. It’s based on proven standards and technologies like AMQP and Kubernetes and provides all the features you’d need, from multi-tenancy to simple management and monitoring. This session will cover EnMaase project in details, providing details on the architecture, messaging concepts supported and ways to set and configure it.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Messaging for the cloud

  1. 1. Messaging for the cloud Building a scalable messaging service on Kubernetes and OpenShift Dejan Bosanac Senior Software Engineer, Red Hat @dejanb 1
  2. 2. 2 Outline ● What is messaging? ● How to scale messaging in general ● EnMasse messaging cloud platform
  3. 3. 3 ● Sending messages ○ Internally in distributed systems ○ Externally between systems ● Communication at the application level ● Messages go from sender/producer to receiver/consumer ○ Asynchronously ○ Time decoupling What is messaging?
  4. 4. 4 Basic features Request-response Publish-subscribe Competing-consumers
  5. 5. 5 Basic features At most once At least once Exactly once X X
  6. 6. 6 Brokered messaging Store and forward Producer Broker Consumer Send message Accepted Send message Accepted
  7. 7. 7 Direct Messaging Router Producer Router Consumer Send message Accepted Send message Accepted
  8. 8. 8 ● Store and Forward ○ Queue ○ Topic ● Direct ○ Anycast ○ Multicast (Broadcast) Addressing semantics
  9. 9. 9 Basic idea R R R P C B B
  10. 10. 10 Messaging in the cloud
  11. 11. 11 ● Freedom of choice ○ On-premise or in the cloud ○ Ability to choose which cloud ○ Open Standards protocols allows users to choose client freely ● Migrating from one to the other can be complex Cloud provider limitations
  12. 12. 12 ● Open source cloud messaging running on Kubernetes and OpenShift ● enmasse.io ● github.com/enmasseproject/enmasse EnMasse Messaging-as-a-Service
  13. 13. 13 ● Support for store-and-forward and direct messaging mechanisms ● Multiple communication patterns: request-response, publish-subscribe and competing consumers ● Scale and elasticity of message brokers ● AMQP 1.0 and MQTT support ● Simple setup, management and monitoring ● Multitenancy: manage multiple independent instances ● Deploy on-premise or in the cloud EnMasse Features
  14. 14. OpenShift console 14
  15. 15. Messaging console 15 ●
  16. 16. 16 Address { "apiVersion": "v3", "kind": "Address", "metadata": { "name": "myqueue" }, "spec": { "store_and_forward": true, "multicast": false, "flavor": "vanilla-queue" } }
  17. 17. 17 ● Queue ○ store-and-forward = true ○ multicast = false ● Topic ○ store-and-forward = true ○ multicast = true ● Anycast ○ store-and-forward = false ○ multicast = false ● Broadcast ○ store-and-forward = false ○ multicast = true Address types
  18. 18. { "apiVersion": "v3", "kind": "Flavor", "metadata": { "name": "vanilla-queue" }, "spec": { "type": "queue", "Description": "Simple in-memory queue", "templateName": "queue-inmemory", "templateParameters": { "BROKER_IMAGE": "mybroker:latest", "STORAGE_CAPACITY": "8Gi" } } } 18 Flavor
  19. 19. 19 ● Persistence ○ In memory ○ Persisted ● Scaling ○ Single broker ○ Pooled ● HA Flavor examples
  20. 20. 20 Architecture
  21. 21. MQTT support 21 ● MQTT gateway ○ Handles connections with remote MQTT clients ○ Bridges MQTT - AMQP protocols ● MQTT lwt ○ Provides the “will testament” feature ○ In charge to recover & send the “will” if client dies
  22. 22. 22 Scaling (routers and brokers)
  23. 23. 23 Scaling routers R R R R A
  24. 24. 24 Scaling routers (#2) R R R R A
  25. 25. Scaling brokers 25 a a b b Router network
  26. 26. Adding brokers 26 a a b b Router network a
  27. 27. Removing brokers 27 a a b b Router network a
  28. 28. 28 Monitoring
  29. 29. 29 ● Authentication and authorization ● Service broker API ● HTTP(S) ● Broker address space ○ Message grouping ○ Distributed transactions ○ Message ordering ● Multiple flavors ○ Apache Kafka? ● ... Future In progress/TODO
  30. 30. Resources 30 ● EnMasse - http://enmasse.io ● ActiveMQ Artemis - https://activemq.apache.org/artemis/ ● Qpid Dispatch Router - http://qpid.apache.org/components/dispatch-router/
  31. 31. Thank you ! Questions ? @dejanb

×