Utilising messaging in cloud deployments isn't straightforward, particularly if you want to take advantage of auto scaling. This talk covers the general problems of scaling for cloud deployments, and messaging for faster inter-service communication for Microservices

1. Messaging for the Cloud and
Microservices
Rob Davies
2015

2. RED HAT | Fabric8 Rocks!2
Rob Davies
• Director of Middleware Engineering
for xPaaS
• Over 20 years experience of
developing large scale solutions for
telcos and finance
• Creator of ActiveMQ and ServiceMix
• Committer on open source projects,
including fabric8, Apache Camel and
other stuff …

3. RED HAT | Fabric8 Rocks!3
All singing, all dancing scalable
messaging for the cloud
AND …
Messaging for Microservices

4. RED HAT | Fabric8 Rocks!4
Why do we use messaging ?
• Robustness to change
• Time independence
• Hide latency
• Event driven
• Platform and language
independence
Broker
Message in
Message out after
enrichment by Process B
Process C
Process B
Process A
Queue:Foo
Queue:Bar

5. RED HAT | Fabric8 Rocks!5
Traditional Enterprise Message brokers
• Designed to support
many different
messaging patterns
• highly available
• Support clustering
• Support store and
forward
• But – are usually
very static in nature

6. RED HAT | Fabric8 Rocks!6
Core Messaging Patterns …

7. RED HAT | Fabric8 Rocks!7
Requestor
Core Message Pattern: Request/Reply
Servic
e

8. RED HAT | Fabric8 Rocks!8
Producer
Core Message Pattern: Queue
Consumer
Consumer
Consumer

9. RED HAT | Fabric8 Rocks!9
Core Message Pattern: Publish/Subscribe
Consumer
Consumer
Consumer
Producer

10. RED HAT | Fabric8 Rocks!10
Message Channels and Filters

11. RED HAT | Fabric8 Rocks!11
Message Routing: Selectors
Producer Destination
Consumer
Color = red
Consumer
Color =
blue

12. RED HAT | Fabric8 Rocks!12
Message Routing: Destination Wildcards
• * matches a subject
• > matches sub-tree
Topic:
BAR.BEER.WHITE
Topic:
BAR.WINE.WHITE
Producer
Consumer
Topic:BAR.WINE.WHITE
Consumer
Topic:BAR.BEER.WHITE
Consumer
Topic:BAR.*.WHITE
Consumer
Topic:BAR.>

13. RED HAT | Fabric8 Rocks!13
Message Groups
Producer
Consumer
Consumer
Consumer
Queue:Prices
Message message = session.createTextMessage(“Hi DWP”);
message.setStringProperty("JMSXGroupID", ”JavaRocks");
producer.send(message);

14. RED HAT | Fabric8 Rocks!14
Scaling Messaging –
the traditional way …

15. RED HAT | Fabric8 Rocks!15
Client Scaling:
Message Bus

16. RED HAT | Fabric8 Rocks!16
Client Scaling:
Message Bus

17. RED HAT | Fabric8 Rocks!17
Broker Scaling:
Message BusMessage
throughput
requirement exceeds
broker capacity

18. RED HAT | Fabric8 Rocks!18
Broker Scaling:
Message Bus Message Bus

19. RED HAT | Fabric8 Rocks!19
Broker Scaling:
Message Bus Message Bus

20. RED HAT | Fabric8 Rocks!20
Broker Scaling:
Message Bus Message Bus
Message BusMessage Bus

21. RED HAT | Fabric8 Rocks!21
Broker Scaling: Problems
Message Bus Message Bus
Message BusMessage Bus
• Diminished
returns
• Focused
Overload
• Stranded
Capacity

22. RED HAT | Fabric8 Rocks!22
How do we scale Messaging for the
Cloud ?

23. RED HAT | Fabric8 Rocks!23
Requirements for Messaging as a Service
• Needs to support many thousands of clients
• Flexible, brokers need to be spun up and down, based on demand
• Client connections may need to be multiplexed, to decrease the
load on individual message brokers
• Popular messaging protocols support
• Flexible routing needs to be supported

24. RED HAT | Fabric8 Rocks!24
What contenders are already out
there ?

25. RED HAT | Fabric8 Rocks!25
ActiveMQ
HornetQ
RabbitMQ
Proprietary Enterprise Messaging

26. RED HAT | Fabric8 Rocks!26
Apache Kafka
• Publish/Subscribe messaging – rethought as a distributed
commit log
• Designed for fast data ingestion of logs
• Scalable – designed to allow a single cluster to act as a
messaging platform, allowing for elastic expansion with no
downtime
• Messages are persisted to disk, and replicated to prevent
data loss
• Allows partitioning of data streams across cluster of
machines
• Created at Linkedin – donated to the ASF

27. RED HAT | Fabric8 Rocks!27
Apache Kafka Architecture
ZooKeeper BrokerBrokerBroker Broker
Producer Producer
Consumer Consumer

28. RED HAT | Fabric8 Rocks!28
Apache Kafka Performance
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations
From: Kafka: A distributed Messaging System for log processing, Jun Rao

29. RED HAT | Fabric8 Rocks!29
Apache Kafka – Differences with a Message
Broker
• Distributed commit log – broker maintains no state
• Niche – so works really well for a particular set of use
cases
• Proprietary
• Written in Scala
• Consumers have to maintain state

30. RED HAT | Fabric8 Rocks!30
Kubernetes Helps

31. RED HAT | Fabric8 Rocks!31
Fabric8 MQ – not a message broker – but a
scalable messaging system
Many concurrent
connections, one out
Flexible
Protocols
Multiplex
Connections
Vert.x
Core
Embedded Camel,
Integration with APIMan
OpenWire,
STOMP
MQTT
AMQP
WebSockets – all done
asynchronously
Scaling Connections
NOT a problem
Destination
Sharding

32. RED HAT | Fabric8 Rocks!32
Fabric8 MQ Independently scalable:
Node
Pod
ActiveMQ
Broker
AMQ
Replication
Controller
Node
Pod
ActiveMQ
Broker
Node
Pod
ActiveMQ
Broker
Fabric8MQ
Replication
Controller
Vert.x
Vert.x
Vert.x
Multiplexer
Multiplexer
Multiplexer
Fabric8MQ
Vert.x
Vert.x
Vert.x
Multiplexer
Multiplexer
Multiplexer
Fabric8MQ
Vert.x
Vert.x
Vert.x
Multiplexer
Multiplexer
Multiplexer
Fabric8MQ

33. RED HAT | Fabric8 Rocks!33
Fabric8 MQ Message Flow:
Protocol
Conversion
Camel
Routing
API
Management Multiplexer
Destination
Sharding
Broker
Control

34. RED HAT | Fabric8 Rocks!34
Qpid Dispatch Router
BrokerBrokerBroker Broker
Producer ProducerConsumer Consumer
Router Router Router Router
AMQP
1.0

35. RED HAT | Fabric8 Rocks!35
Multiplexing
• Brokers performs better the lower
the number of connections – less
contention
• IoT applications have long lived
connections, but small amount of
traffic
• All configurable – you can decide
how fine or coarse grained you want
multiplexing
• Dispatch Router does this
automatically

36. RED HAT | Fabric8 Rocks!36
Destination Sharding: Benefits
• There is an overhead associated with a
Destination – restrict the number of
Destinations per Broker to improve
performance
• Co-locate producers and consumers of a
Destination to the same broker reduces
latency and improves overall performance
• Increased scalability for your messaging
solution
• Dispatch Router does this automatically

37. RED HAT | Fabric8 Rocks!37
Red Hat Messaging – how we plan
to tackle messaging for the cloud
…

38. RED HAT | Fabric8 Rocks!38
What we have today
 Separate clients tied to
broker- specific protocols
 Limited
integration between brokers
& overlapping capabilities
MRG-M (Apache Qpid C++ broker)
•AMQP, high performance, routing, HA
•No longer sold but actively supported
M. MQ 6.x (Apache ActiveMQ 5.x Java
broker)
• Multi-protocol JMS-oriented broker
• Large user community, rich feature set
EAP HornetQ JMS Messaging Broker (HornetQ GitHub
project)
•Multi-protocol JMS-oriented broker
•JMS 2.0, best-in-class SpecJMS performance
MRG-M
3.1
A-MQ
6.x
HornetQ
EAP6

39. RED HAT | Fabric8 Rocks!39
Clients
A-MQ
Where we’re going
• Consolidate to a single messaging product, known
as A-MQ
• Consists of three components: Broker,
Interconnect & Clients
• Developed in open, community-based, upstream
projects and
leveraging open, standards-based protocols
Broker
Interconnect

40. RED HAT | Fabric8 Rocks!40
Where we’re going
M. MQ Broker
• Full-featured, high-performance, multi-protocol
enterprise broker
• HA through replication and failover
M. MQ Interconnect Router
• Large-scale, secure, reliable, and manageable
messaging networks
• Use with or without broker. HA through redundant
routing.
M. MQ Clients
• Ubiquitous, standards-based messaging clients for all
common platforms and programming languages
A-
MQ
A-MQ
Broker
Interconnect
Clients

41. RED HAT | Fabric8 Rocks!41
Where we’re going
• AMQP (Advanced Message Queuing
Protocol)
• Rich semantics, multiplexing, flow control,
extensible
• International standard – ISO/IEC ISO
19464
• MQTT (Message Queuing Telemetry
Transport)
• Lightweight “first mile” telemetry
• OASIS Standard; International
standardization in progress
• STOMP (Simple Text-Oriented
Messaging Protocol)
• WebSocket encapsulation
A-MQ
Broker
Interconnect
Clients

42. RED HAT | Fabric8 Rocks!42
Messaging for Microservices

43. RED HAT | Fabric8 Rocks!43
Reliable, Fast Communication between Services

44. RED HAT | Fabric8 Rocks!44
Functional Decomposition

45. RED HAT | Fabric8 Rocks!45
Micromessaging
• Whilst external services should
often be http – internal services
do not need to be.
• Using a messaging system can
improve performance and
scalability
• Standards are important –
immutable infrastructure leads to
disposable infrastructure. Apply
that all assets in your design.

46. RED HAT | Fabric8 Rocks!46
Micromessaging: inter-service communications
Broker
Service A
Service A
Service A
Service C
Service A
Service B
Service A
Service F
Service A
Service E
Service A
Service Drequest
reply
subscribe
subscribe
publishpublish

47. RED HAT | Fabric8 Rocks!47
Micromessaging: Control Plane
BrokerController
Service A
Service F
Service A
Service A
subscribe
subscribe
publish
{Start Processing}
{Stop Processing}

48. RED HAT | Fabric8 Rocks!48
Micromessaging: Stream Processing
Broker
publish
Event Listener
SMS
Event Listener
Event Listener
HTTP Post
Event Listener
HTTP Post
Event Listener
HTTP Post
Event Listener
HTTP Post
Event Listener
HTTP Post
SMTP
publish
publish
Event
Processing
Engine

49. RED HAT | Fabric8 Rocks!49
Micromessaging anti-patterns
• Any pattern that is transactional in nature
e.g. Transactional Client
• Any pattern that relies on Message Persistence
Durable Subscriber, Guaranteed Delivery
• Any pattern where the message system inspects, or
selectively routes messages
Message Filter, Content Based Routing, Content Enricher

50. RED HAT | Fabric8 Rocks!50
Micromessaging: what to use ?
Qpid Dispatch
Router

51. RED HAT | Fabric8 Rocks!51

52. RED HAT | Fabric8 Rocks!52
Demo:
Variance
Collector
Compare Interprocess
communication:
HTTP vs Messaging

53. RED HAT | Fabric8 Rocks!53
Standard Deviation Processor:
public void process(Exchange exchange) throws Exception {
String message = exchange.getIn().getBody(String.class);
ObjectMapper objectMapper = new ObjectMapper();
TypeFactory typeFactory = objectMapper.getTypeFactory();
List<Double> values = objectMapper.readValue(message,
typeFactory.constructCollectionType(List.class, Double.class));
SummaryStatistics summaryStatistics = new SummaryStatistics();
List<Double> list = new ObjectMapper().readValue(message, List.class);
for (Double value : list) {
summaryStatistics.addValue(value);
}
String stdDev = Double.toString(summaryStatistics.getStandardDeviation());
exchange.getOut().setBody(stdDev);
}

54. RED HAT | Fabric8 Rocks!54
Standard Deviation Route:
@ContextName("stddevCamel")
public class StdDevHTTP extends RouteBuilder {
@Inject
StdDevProcessor processor;
@Override
public void configure() throws Exception {
from("jetty:http://0.0.0.0:8183/std-dev").doTry()
.process(processor)
.doCatch(Throwable.class)
.setHeader(Exchange.HTTP_RESPONSE_CODE, constant(500))
.setBody(constant("{"error" : "Service failed"}"))
.end();
}
}

55. RED HAT | Fabric8 Rocks!55
HTTP Calculator
@Override
public void configure() throws Exception {
onException(Throwable.class).maximumRedeliveries(-1).delay(5000);
from("direct:start")
.multicast()
.parallelProcessing().timeout(10000).to(stdDevService,
varianceService)
.end().setHeader("name", constant("HTTP")).to(collectorService);
}

56. RED HAT | Fabric8 Rocks!56
HTTP Calculator
@Inject
@Uri("netty4-http:http://{{service:collector-http:localhost:8184}}/results/http")
private Endpoint collectorService;
@Inject
@Uri("netty4-http:http://{{service:variance-http:localhost:8182}}/variance")
private Endpoint varianceService;
@Inject
@Uri("netty4-http:http://{{service:std-dev-http:localhost:8183}}/std-dev")
private Endpoint stdDevService

57. RED HAT | Fabric8 Rocks!57
Messaging Calculator
Endpoint jmsSender = getContext().getEndpoint("jms:topic:"
+ CALCULATION_TOPIC
+ "?preserveMessageQos=true"
+ "&replyTo=" + RESULT_QUEUE
+"&replyToType=Exclusive"
+ "&asyncConsumer=true"
+ "&asyncStartListener=true"
+ "&concurrentConsumers=10");

58. RED HAT | Fabric8 Rocks!58
Messaging Calculator
from("jms:queue:”+RESULT_QUEUE).aggregate(header(CORRELATION_HEADER),
new AggregationStrategy() {
@Override
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
if (oldExchange == null) {
return newExchange;
}
String oldBody = oldExchange.getIn().getBody(String.class);
String newBody = newExchange.getIn().getBody(String.class);
oldExchange.getIn().setBody(oldBody + "+" + newBody);
return oldExchange;
}
}).completionSize(NUMBER_OF_SERVICES).completionTimeout(2000)
.setHeader("name", constant("MSG")).to(collectorService);

59. RED HAT | Fabric8 Rocks!59
The Collector:
@ContextName("collectorCamel")
public class CollectorHTTP extends RouteBuilder {
@Override
public void configure() throws Exception {
getContext().addRoutePolicyFactory(new MetricsRoutePolicyFactory());
from("jetty:http://0.0.0.0:8184/results/http”).setId("HTTP");
from("jetty:http://0.0.0.0:8184/results/msg”).setId("MSG");
}
}

60. RED HAT | Fabric8 Rocks!60
Links:Links:
http://camel.apache.orghttp://camel.apache.org
http://fabric8.iohttp://fabric8.io
https://github.com/rajdavies/microservices-exampleshttps://github.com/rajdavies/microservices-examples