Presentation by Rob Davies (Red Hat) delivered at the London JBoss USer Group event on the 10th of July 2013.
See the video here: http://youtu.be/VrJyKLAXVTg
As part of the FuseSource acquisition, Red Hat now supports Apache ActiveMQ as the recently released Red Hat JBoss A-MQ product.
ActiveMQ is the most widely used message-oriented middleware that uses messaging to connect remote applications written in Java, C/C++, Python, Perl, Ruby, and many other languages. ActiveMQ is standards based and supports messaging protocols such as AMQP 1.0, WebSockets, Stomp, OpenWire, and MQTT.
In this session, Rob Davies will discuss the product’s features and functionality, and demonstrate connectivity from a microprocessor to web sockets using JBoss A-MQ.
London JBUG - Connecting Applications Everywhere with JBoss A-MQ
1. 1
redhatRob Davies
Connec(ng
Applica(ons
Everywhere
with
JBoss
A-‐MQ
Rob Davies,
Technical Director,
Fuse Engineering,
Red Hat
July 2013
2. 2
redhatRob Davies
2
Rob Davies
Technical Director, Red Hat -
h"p://www.redhat.com
■ Software projects:
■ Apache ActiveMQ,
■ Apache Camel
■ Apache ServiceMix
■ On
Expert
Group
for
JSR
343:
JMS
2.0
■ Co-‐author
of
AcCveMQ
in
AcCon:
3. 3
redhatRob Davies
History
of
Fuse
open
source
Development
2004
2005
2006
2007
2008
2009
2010
2011
2012
ActiveMQ created
May 2004
The CodeHaus
ServiceMix created
May 2005
Camel created
March 2007
IONA acquire
April 2007
Karaf created
Progress acquire
October 2008
FuseSource
created
October 2010
The Apache Software Foundation
LogicBlaze Formed
May 2005
CXF
August 2006
Fabric created
February 2011
Fuse Forge And
ASF
Red Hat
acquire
Sept 2012
4. 4
redhatRob Davies
Why
use
Message-‐Oriented
Middleware?
• Robustness
to
change
• Time
Independence
• LocaCon
Independence
• Hide
Latency
• Scalability
• Event
driven
• Simplicity
• Configurable
Quality
of
Service
• PlaRorm
and
Language
IntegraCon
• Fault
tolerant
5. 5
redhatRob Davies
5
What is Apache ActiveMQ ?
• Top Level Apache Software Foundation Project
• Wildly popular, high performance, reliable message broker
• Connects to nearly everything
• Native Java, C/C++, .Net,
• AMQP 1.0, MQTT 3.1, STOMP (1.0-1.2) and
OpenWire
• STOMP enables Ruby, JS, Perl, Python, PHP,ActionScript
…
• Embedded and standalone deployment options
• The key component of JBoss A-MQ!
7. 7
redhatRob Davies
Message
Channels
and
RouIng
• Message
Channels
• Named
communica(on
between
interested
par(es
• JMS
calls
them
‘Des(na(ons’
• Can
“tune-‐in”
to
mul(ple
channels
using
wildcards
• Can
fine-‐tune
message
consump(on
with
selectors
• Can
route
a
message
based
on
content
8. 8
redhatRob Davies
Message
Channels
=
JMS
DesInaIons
Producer
Broker
Consumer
Consumer
Consumer
Destinatio
n
WIDGET
Destinatio
n
ORDER
15. 15
redhatRob Davies
Message
Groups
• Like
Exclusive
Consumers
–
but
in
parallel
• Guaranteed
ordering
of
related
messages
across
a
Queue
• But
–
load
balancing
of
messages
across
mul(ple
consumers
• All
messages
with
the
same
JMSXGroupID
go
to
the
same
consumer
• How
you
group
messages
is
down
to
the
applica(on’s
producer
• To
explicitly
close
a
group,
set
the
JMSXGroupSeq
to
-‐1
15
17. 17
redhatRob Davies
JBoss
A-‐MQ
–
Synchronous
sends
Set alwaysSyncSend on the ActiveMQConnectionFactory
You can set:
sendTimeout on the ActiveMQMessageProducer
Synchronous send: JMS Client
producer.send(session.createTextMessage("Hello"), new AsyncCallback() {
public void onSuccess() {}
public void onException(JMSException exception) {
exception.printStackTrace();
}
});
Performance Tip – use callbacks …
18. 18
redhatRob Davies
Deploying
JBoss
A-‐MQ
• AcIveMQ
Can
run
standalone
or
embedded
• As
a
standalone,
or
part
of
a
highly
available
message
broker
cluster
• Embedded
–
easy
to
use
an
en(re
broker
in
JUnit
tests
(no
need
to
Mock)
• In
Tomcat,
deployed
as
a
war
• In
JEE
Server
–
either
co-‐locate
–
or
use
client
with
JCA
• Ac(veMQ
distribu(ons
contain
a
rar
for
this
purpose
18
20. 20
redhatRob Davies
Message
Delivery
Mode
Messages
can
be
persisted
–
allowing
for
decoupled
applicaIons
For
Queues,
any
message
delivered
as
PERSISTENT
must
be
stored
on
long
term
storage
before
being
delivered
to
a
consumer
For
Topics,
a
message
delivered
as
PERSISTENT
will
only
be
stored
if
there
exists
a
durable
subscriber
NON-‐PERSISTENT
messages
may
also
be
stored
on
disk
if
the
memory
limits
of
the
broker
have
been
reached
20
21. 21
redhatRob Davies
Message
stores
supported
by
JBoss
A-‐MQ
■ SQL
(JDBC)
■ SQL
(JDBC)
with
journal
■ AMQ
Message
Store
■ Kaha
■ KahaDB
■ LevelDB
21
Slow but
popular
Don’t use
(deprecated)
The “current”
default
The “best yet”
22. 22
redhatRob Davies
LevelDB
Store
vs
KahaDB
• Fewer
index
entries
per
message
than
KahaDB
• Faster
recovery
when
a
broker
restarts
• LevelDB
index
out-‐perform
Btree
index
at
sequen(al
access
.
• LevelDB
indexes
support
concurrent
read
access.
• Pauseless
data
log
file
garbage
collec(on
cycles.
• Fewer
IOPS
to
load
stored
messages.
• It
exposes
it's
status
via
JMX
for
monitoring
25. 25
redhatRob Davies
OpenWire
• Advantages:
• Fast
–
op(mized
for
Ac(veMQ
• client
failover
• automa(c
reconnect
• Client
load
balancing
• Flow
control
• Many
advanced
features
25
Disadvantages:
• Not a recognized
standard
• Only Java, C/C++/.Net
http://activemq.apache.org/openwire.html
26. 26
redhatRob Davies
MQTT
h`p://mq`.org
Advantages:
• M2M/”Internet
of
Things”
transport
• Proposed
as
an
OASIS
standard
• Extremely
light
weight
• Growing
support
from
vendors
and
OS
products
• WebSphereMQ
• Ac(veMQ
+
Apollo
• Mosquilo
• RabbitMQ
26
Disadvantages:
• 3.1 does not support
Queues
• Advanced features not
standard
• Flow control
• Failover etc.
27. 27
redhatRob Davies
AMQP
–
see
www.amqp.org
Advantages:
• AMQP
1.0
OASIS
standard
• Proposed
as
an
OASIS
standard
• Commodi(zes
the
Broker
27
Disadvantages:
• One size doesn’t really fit all
• Currently no plans for IBM or
Tibco to adopt it
28. 28
redhatRob Davies
STOMP
–
h`p://stomp.github.com
Advantages:
• Easy
to
use
text
based
protocol
• Can
use
telnet
as
a
client
• Defacto
standard:
• Ac(veMQ
+
Apollo
• HornetQ
• RabbitMQ
• PocoMQ
• StompServer
• OpenMQ
• Many
more
…
28
Disadvantages:
• Not as fast as binary
formats
29. 29
redhatRob Davies
But
there’s
more:
WebSockets
STOMP
is
a
natural
wire
protocol
for
WebSockets
And
so
is
MQTT!
29
ActiveMQ
Client
(Browser)
WebSocket
Connector
Stomp
Converter
ActiveMQ
(Backend)
TCP
(WebSocket)
HTTP(S)
Request/Response
35. 35
redhatRob Davies
• Extreme
reliability
–
but
not
as
fast
• Recommended
if
already
using
an
enterprise
database
• No
restric(on
on
number
of
slaves
• Simple
configura(on
• Configurable
lockKeepAlivePeriod
JDBC Master Slave
36. 36
redhatRob Davies
Broker
Slave
Shared
File
System
Master/Slave
Broker
Master
ActiveMQ Client
Broker
Slave
File System
Larry’sRemovals
37. 37
redhatRob Davies
• Recommended
if
you
have
a
SAN,
or
DRDB
• No
restric(on
on
number
of
slaves
• Simple
configura(on
• Ensure
file
locking
works
–
and
(mes
out
–
NFSv4
good!
Shared File System M/S
38. 38
redhatRob Davies
Local File System Local File SystemLocal File System
Broker
Slave
Broker
Master
Broker
Slave
ZooKeeper
Cluster
Replicated LevelDB
Master Slave
40. 40
redhatRob Davies
• Requires
a
HA
ZooKeeper
Cluster
• No
Single
Point
of
Failure
• Dynamic
number
of
slaves
• Simple
configura(on
• Sync
or
Async
Replica(on
Replicated LevelDB
Master Slave
41. 41
redhatRob Davies
• Link
Brokers
together
• Use
Store
and
Forward
• Are
uni-‐direc(onal
by
default
• All
Des(na(ons
are
global
Network of Brokers
42. 42
redhatRob Davies
Apache
AcIveMQ
–
Broker
Topologies
Store and Forward
ActiveMQ
Local broker
ActiveMQ
Remote broker
46. 46
redhatRob Davies
Broker
Networks
(Store
and
Forward)
47. 47
redhatRob Davies
Network
of
Brokers
:
Geographically
Dispersed
48. 48
redhatRob Davies
Network
of
Brokers
:
Network
with
Master/Slave
49. 49
redhatRob Davies
Scaling
Networks
of
Brokers
• Brokers
share
rou(ng
informa(on
across
networks
• All
des(na(ons
are
considered
Global
• This
is
really
convenient
• Though
it
starts
to
get
problema(c
with
1000’s
of
brokers
• However
–
we
can
do
filtering
to
shape
the
traffic
across
networks
• But
this
involves
a
lot
of
manual
configura(on
49
50. 50
redhatRob Davies
• Brokers
share
rou(ng
informa(on
across
networks
• All
des(na(ons
are
considered
Global
• This
is
really
convenient
• Though
it
starts
to
get
problema(c
with
1000’s
of
brokers
• However
–
we
can
do
filtering
to
shape
the
traffic
across
networks
• But
this
involves
a
lot
of
manual
configura(on
50
Time to talk about
Apache Camel …
Scaling
Networks
of
Brokers
51. 51
redhatRob Davies
AcIveMQ
with
embedded
Camel
Service
Consumer
Consumer
ActiveMQ Broker
Flexible and is faster J
52. 52
redhatRob Davies
Scaling
Networks
–
use
Apache
Camel
• Allows
for
non-‐chaly
networks
to
be
established
• Rou(ng
informa(on
can
be
externalized
to
the
broker
• Successfully
used
in
produc(on
for
1000’s
of
brokers
today
52
53. 53
redhatRob Davies
More problems
with Large
Deployments
54. 54
redhatRob Davies
Problems
–
Deploying
and
maintenance
Main
problems
• Installing
brokers
on
mul(ple
hosts
o ssh,
untar,
set
directories
and
environment
• Sesng
configura(on
manually
for
every
broker
o copying
xml
config,
tweaking,
tes(ng
• Upda(ng
configura(on
across
cluster
• Upgrading
brokers
It’s a tedious and error-prone process
55. 55
redhatRob Davies
Problems
–
TradiIonal
best-‐pracIce
Ips
• Keep
XML
as
a
template
and
configure
node-‐specific
details
through
proper(es
• Keep
configura(on
in
SVC
system
(git,
svn,
…)
• Keep
configura(on
separate
from
installa(on
for
easier
upgrades
56. 56
redhatRob Davies
Problems
-‐
Clients
• Topology
is
very
“sta(c”
• Clients
need
to
be
aware
of
topology
• Clients
need
to
know
broker
loca(ons
• Changes
are
not
easy
as
clients
need
to
be
updated
• Adding
new
resources
(brokers)
requires
client
updates
• Not
suitable
for
“cloud”
deployments
Fuse Fabric makes deployments more “elastic”
59. 59
redhatRob Davies
FuseMQ
features
• mq-‐base
profile
• Defines
OSGi
features
and
bundles
to
be
installed
• Defines
basic
broker
sesngs
• mq-‐create
command
• Helper
command
for
crea(ng
brokers
• Creates
an
new
profile
based
on
mq-‐base
• Op(onally
creates
new
containers
• Assigns
the
profile
to
containers
(essen(ally
starts
the
broker)
FuseMQ = ActiveMQ deployed in Apache Karaf +
60. 60
redhatRob Davies
JBoss
A-‐MQ
A small-footprint, high-performance, open source messaging platform
61. 61
redhatRob Davies
JBoss
A-‐MQ
–
Value
ProposiIon
Value Proposition:
– Easier to configure, monitor,
manage and maintain: Small IT
footprint
– Multiple deployment options –
centralized, distributed,
embedded: Small IT footprint
– Proven and robust platform: Based
on popular Apache ActiveMQ,
Standards-based reliable
messaging & High-performance
– Multiple connectivity options: Multi-
standards and multi-platform
support
– Real-time and reliable integration:
Supports messaging paradigms:
Pub/Sub, Point-2-Point, Store and
forward
New in 6.0:
AMQP 1.0 support (Tech Preview)
JBoss Developer Studio to include
Fuse IDE
JBoss Operations Network + Fabric
Management Console
Rebranding, repackaging
62. 62
redhatRob Davies
JBoss
A-‐MQ
-‐
Benefits
Breadth
of
connecIvity
–
cross-‐language
client
support
and
mul(-‐protocol
support
provides
wide
range
of
connec(vity
from
browser-‐based
clients
to
mobile
devices
Proven
reliability
and
support
–
proven
track
record
of
suppor(ng
mission-‐cri(cal
applica(ons,
and
backed
by
enterprise-‐class
services
and
tools
Small
IT
footprint
–
high-‐performance
and
reliable
messaging
with
small
footprint
and
minimal
maintenance
Multiple deployment options – Flexible configuration allows multiple deployment
options
No
license
fees
–
try
before
you
buy,
and
deploy
widely
without
incurring
exorbitant
costs
63. 63
redhatRob Davies
So where is JBoss A-MQ
used ?
65. 65
redhatRob Davies
Next
GeneraIon
Air
Traffic
Control
…
6
NextGen Applications
FTI IP Backbone
En Route
Controllers
Terminal
Controllers
Non-FAA Users
(e.g., Airlines, DoD
DHS, etc.)
FAA
Command Center
SWIM Enterprise Infrastructure
66. 66
redhatRob Davies
Well
not
exactly
in
space
–
but
close
6
67. 67
redhatRob Davies
And
helping
run
the
biggest
science
experiment
–
ever!
6
68. 68
redhatRob Davies
Btw
-‐
Why
is
IntegraIon
and
Messaging
Important
?
Head Office
Enterprises
Need to
know
Everything!
69. 69
redhatRob Davies
There
are
more
things
to
Integrate
with!
70. 70
redhatRob Davies
The
Internet
of
Things
• Uniquely
iden(fiable
objects
and
their
virtual
representa(ons
in
an
internet-‐like
structure
• Originally
defined
by
Kevin
Ashton,
co-‐founder
of
Auto-‐ID
center
at
MIT
• Today
its
meaning
has
evolved
to
encompass
a
world
where
physical
objects
are
integrated
into
the
informa(on
network,
and
where
physical
objects
par(cipate
in
business
processes.
71. 71
redhatRob Davies
Machine
to
Machine
• M2M
involves
collec(on
and
transmission
of
event
level
data
(used
to
be
called
telemetry)
from
intelligent
devices
•
Machines
can
interact
over
the
net
–
and/or
alached
networks
–
by
wired
and
wireless
networks
• Payloads
are
typically
minimal
(temperature,
pressure,
loca(on,
metering
etc.)
72. 72
redhatRob Davies
Internet
of
Things
Time
Connected
Devices
on
the
internet
2010
2000
2020
1
Billion
50
Billion
10
Billion
Ubiquitous
posiConing
–
locaCng
people
and
everyday
objects
RFID
tags
for
Inventory
control
Advanced
sensors,
Enterprises
fully
connected:
integraCon
of
everything
Geo-‐located
devices
Microcontrollers
internet
enabled
Vending
machines,
lone
workers,
POS,
vehicles,
planes,
tolls,
power
systems,
transport,
telemedicine,
home
appliances,
power
systems,
etc.
73. 73
redhatRob Davies
M2M
Examples
Smart
Energy
–
•
Smart
Grid
uses
smart
meters
for
monitoring
energy
uses
by
premises
for
billing,
substaCon
and
transmission
line
monitoring,
energy
producCon
(renewables)
Inventory
Control
–
•
smart
labels
and
RFID
tags
–
connected
scanners
pass
informaCon
upstream
to
servers
for
monitoring
In-‐Vehicle
Systems
• Maintenance
informaCon,
global
posiConing,
‘black-‐box’
(speed
and
driving
habits),
wireless
payments
for
tolls/gas
Environmental
Monitoring
•
Weather
sta(ons
(temperature,
air-‐pressure),
ocean
monitors,
earth
movements,
volcanic
ac(vity
crop
yields
etc.
Livestock
monitoring
• Cow
herds,
monitored
for
opCmum
Cme
for
milking,
sheep
herds
for
locaCon
74. 74
redhatRob Davies
M2M
Topology
Gateway
Hadoop
Servers
Analytics
Gateway
Internet
Mesh Network
Mobile wireless
75. 75
redhatRob Davies
Power
ConsumpIon!
• Balery
life
for
mobiles,
power
consump(on
for
smart
devices
is
an
important
considera(on.
• Facebook
use
MQTT
for
real-‐(me
updates
–
efficiency
was
one
of
the
things
considered
• MQTT
is
more
efficient
at
establishing
connec(ons,
sends
informa(on
more
reliably,
and
consumes
less
power
to
transmit
data
than
HTTP.
76. 76
redhatRob Davies
Deluge
of
Data
• Billions
of
devices
send
informa(on
will
require
storage
and
processing
of
terabytes
or
even
petabytes
of
data.
• Big
Data
Inges(on:
Processing
of
vast
amounts
of
data
business
generates
to
make
good
decisions
is
only
half
the
problem.
Inges(ng
that
data
from
all
areas
of
the
enterprise
will
require
TBs
of
data
to
be
ingested
in
highly
reliable
and
scalable
way.
77. 77
redhatRob Davies
Need
an
Eco
system
of
technologies
Dev
• Integra(on
is
gesng
harder
• Tradi(onal
message
brokers
are
key
to
connect
“everything”
• Mul(ple
protocols
need
to
be
supported
• Camel
can
provide
integra(on
at
the
edges
–
and
the
data
center
• Technologies
like
Fuse
Fabric
provide
centralized
provisioning
and
management
• Need
to
mix
protocols,
brokers
and
“message
routers”
to
scale
FuseMQ
Clusters
Arrival
Airport 1
79. 79
redhatRob Davies
Messaging
through
the
Internet
of
Things
ActiveMQ
AMQP/MQTT
routers
AMQP/MQTT
routers
AMQP/MQTT
routers
MQTT
MQTT-‐S
AMQP
AMQP
AMQP
MQTT
MQTT
websocket
s
81. 81
redhatRob Davies
What
is
Arduino
?
• Open
Source
Hardware
microcontroller
• Cheap
and
easily
available.
• Open
Source
Souware.
• Widespread:
many
projects.
• Extra
HW
(shields)
available
(e.g.
Ethernet
shield)
82. 82
redhatRob Davies
USB
7-12 v
3 v GRD 5 v Analog Input Pins
Digital Input/Output Pins
Pins with ~ are PWM
[Analog Output]
GRD
Transmitter/Receiver
Serial Connection
Microcontroller ATmega328
Operating Voltage 5V
Input Voltage (recommended)7-12V
Input Voltage (limits)6-20V
Digital I/O Pins 14
(of which 6 provide PWM output)
Analog Input Pins 6
DC Current per I/O Pin 40 mA
DC Current for 3.3V Pin 50 mA
83. 83
redhatRob Davies
Arduino
Programming
• 3
types
of
memory
on
Arduino
• Flash
memory
(program
space)
–
32
Kb
• SRAM
–
used
by
sketches
–
2
Kb
• EPROM
–
long
term
memory
–
1
Kb
• Programming
language
based
on
Wiring:
C/C++
library
designed
to
make
input/output
easier.
Programs
are
called
sketches.
• Arduino
has
a
simple
IDE,
available
on
Windows,
Linux
and
Mac