Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Messaging-as-a-Service
Building a scalable messaging service on Kubernetes
and OpenShift
Ulf Lilleengen
Senior Software En...
2
Outline
● Background
● Messaging as a service
● Platform scalability
● Platform configuration
● Platform CI
● Platform i...
3
● Sending messages
○ Internally in distributed systems
○ Externally between systems
● Communication at the application l...
4
Basic features
Request-response Publish-subscribe Competing-consumers
5
Basic features
At most once At least once Exactly once
X X
6
Interoperability
Standards
7
Messaging in the cloud
8
● Freedom of choice
○ On-premise or in the cloud
○ Ability to choose which cloud
○ Open Standards protocols allows users...
9
● Open source cloud messaging running on Kubernetes and OpenShift
● enmasse.io
● github.com/enmasseproject/enmasse
EnMas...
1
0
● Multiple communication patterns: request-response, publish-subscribe and competing
consumers
● Support for store-and...
1
1
● Authentication and authorization
● Service broker API
● HTTP(S)
● Message grouping
● Distributed transactions
● Mess...
1
2
Basic idea
R
R
R
P C
B
B
1
3
Routing vs “Broking”
Broker
Producer Broker Consumer
Send message
Accepted
Send message
Accepted
1
4
Routing vs “Broking”
Router
Producer Router Consumer
Send message
Accepted
Send message
Accepted
1
5
Architecture
1
6
Scaling (routers and brokers)
1
7
Scaling routers
R
R
R
R
A
1
8
Scaling routers (#2)
R
R
R
R
A
Scaling brokers
1
9
a a b b
Router network
Adding brokers
2
0
a a b b
Router network
a
Removing brokers
2
1
a a b b
Router network
a
2
2
Configuration management
2
3
Configuration distribution
2
4
Configuration interface
{
"apiVersion": "v3",
"kind": "Address",
"metadata": {
"name": "myqueue"
},
"spec": {
"store_a...
{
"apiVersion": "v3",
"kind": "Flavor",
"metadata": {
"name": "vanilla-queue"
},
"spec": {
"type": "queue",
"Description":...
2
6
Continuous integration
Continuous integration
2
7
Component build pipeline
build test
Build docker
image & push
success
failure
trigger
systemtes...
2
8
User interface
OpenShift console
2
9
Messaging console
3
0
3
1
Monitoring
3
2
DEMO
3
3
Questions?
MQTT support
3
4
● MQTT gateway
○ Handles connections with remote MQTT clients
○ Bridges MQTT - AMQP protocols
● MQTT lwt
...
Upcoming SlideShare
Loading in …5
×

Messaging-as-a-Service Rivieradev 2017

932 views

Published on

Messaging as a service presentation from Rivieradev 2017

Published in: Software
  • Be the first to comment

Messaging-as-a-Service Rivieradev 2017

  1. 1. Messaging-as-a-Service Building a scalable messaging service on Kubernetes and OpenShift Ulf Lilleengen Senior Software Engineer, Red Hat @lulf 1
  2. 2. 2 Outline ● Background ● Messaging as a service ● Platform scalability ● Platform configuration ● Platform CI ● Platform interface
  3. 3. 3 ● Sending messages ○ Internally in distributed systems ○ Externally between systems ● Communication at the application level ● Messages go from sender/producer to receiver/consumer ○ Asynchronously ○ Time decoupling What is messaging?
  4. 4. 4 Basic features Request-response Publish-subscribe Competing-consumers
  5. 5. 5 Basic features At most once At least once Exactly once X X
  6. 6. 6 Interoperability Standards
  7. 7. 7 Messaging in the cloud
  8. 8. 8 ● Freedom of choice ○ On-premise or in the cloud ○ Ability to choose which cloud ○ Open Standards protocols allows users to choose client freely ● Migrating from one to the other can be complex Cloud provider limitations
  9. 9. 9 ● Open source cloud messaging running on Kubernetes and OpenShift ● enmasse.io ● github.com/enmasseproject/enmasse EnMasse Messaging-as-a-Service
  10. 10. 1 0 ● Multiple communication patterns: request-response, publish-subscribe and competing consumers ● Support for store-and-forward and direct messaging mechanisms ● Scale and elasticity of message brokers ● AMQP 1.0 and MQTT support ● Simple setup, management and monitoring ● Multitenancy: manage multiple independent instances ● Deploy on-premise or in the cloud EnMasse Features
  11. 11. 1 1 ● Authentication and authorization ● Service broker API ● HTTP(S) ● Message grouping ● Distributed transactions ● Message ordering ● Multiple flavors ○ Apache Kafka? ● ... EnMasse In progress/TODO
  12. 12. 1 2 Basic idea R R R P C B B
  13. 13. 1 3 Routing vs “Broking” Broker Producer Broker Consumer Send message Accepted Send message Accepted
  14. 14. 1 4 Routing vs “Broking” Router Producer Router Consumer Send message Accepted Send message Accepted
  15. 15. 1 5 Architecture
  16. 16. 1 6 Scaling (routers and brokers)
  17. 17. 1 7 Scaling routers R R R R A
  18. 18. 1 8 Scaling routers (#2) R R R R A
  19. 19. Scaling brokers 1 9 a a b b Router network
  20. 20. Adding brokers 2 0 a a b b Router network a
  21. 21. Removing brokers 2 1 a a b b Router network a
  22. 22. 2 2 Configuration management
  23. 23. 2 3 Configuration distribution
  24. 24. 2 4 Configuration interface { "apiVersion": "v3", "kind": "Address", "metadata": { "name": "myqueue" }, "spec": { "store_and_forward": true, "multicast": false, "flavor": "vanilla-queue" } }
  25. 25. { "apiVersion": "v3", "kind": "Flavor", "metadata": { "name": "vanilla-queue" }, "spec": { "type": "queue", "Description": "Simple in-memory queue", "templateName": "queue-inmemory", "templateParameters": {} } } 2 5 Configuration interface
  26. 26. 2 6 Continuous integration
  27. 27. Continuous integration 2 7 Component build pipeline build test Build docker image & push success failure trigger systemtests
  28. 28. 2 8 User interface
  29. 29. OpenShift console 2 9
  30. 30. Messaging console 3 0
  31. 31. 3 1 Monitoring
  32. 32. 3 2 DEMO
  33. 33. 3 3 Questions?
  34. 34. MQTT support 3 4 ● MQTT gateway ○ Handles connections with remote MQTT clients ○ Bridges MQTT - AMQP protocols ● MQTT lwt ○ Provides the “will testament” feature ○ In charge to recover & send the “will” if client dies

×