Camel oneactivemq posta-final

Christian Posta
Christian PostaGlobal Field CTO at Solo.io
CamelOne 2013	

June 10-11 2013	

Boston, MA	

Apache ActiveMQ
Internals	

Christian Posta	

Senior Consultant and Architect	

June 10, 2013	

1
CamelOne 2013	

CamelOne	

2	

Why are You Here?	

•  Use ActiveMQ?	

•  What is ActiveMQ?	

•  Dug into the code?	

•  Interested in contributing back?	

•  Understand configuration better
CamelOne 2013	

CamelOne	

3	

Goals of Presentation	

•  Give back to community	

•  Deliver understanding about architecture	

•  Brief intro to ActiveMQ	

•  Overview of architecture	

•  Dive into major subcomponents
CamelOne 2013	

CamelOne	

4	

Who am I?	

•  Senior Consultant and Architect at Red Hat (formerly
FuseSource)	

•  Committer on ActiveMQ	

•  Author: Essential Camel Components DZone Refcard	

•  Christian Posta	

•  Blog: http://christianposta.com/blog	

•  Email: ceposta@apache.org	

•  Twitter: @christianposta
CamelOne 2013	

CamelOne	

5	

What is ActiveMQ?	

•  OpenSource messaging broker	

•  Apache v2 licensed	

•  Very active community	

•  Used at top companies	

•  5.8.0 latest release (in JBoss A-MQ 6.0)
5.9.0 on its way
CamelOne 2013	

CamelOne	

6	

Features	

•  High performance	

•  High availability	

•  Light-weight	

•  Multi-protocol	

•  JMS compliant	

•  Supported in production
CamelOne 2013	

CamelOne	

7	

When to use ActiveMQ?	

•  Asynchronous communication	

•  Reliable message passing	

•  Loose coupling	

•  Heterogeneous integration	

•  Real-time data
CamelOne 2013	

CamelOne	

8	

This is ActiveMQ…
CamelOne 2013	

CamelOne	

9	

How it’s built	

•  Java project (supports Java 7!)	

•  Maven project	

•  Divided into client/broker and feature modules	

•  Unit tests in activemq-unit-tests	

•  mvn	
  clean	
  package	
  –Dtest=false	
  
CamelOne 2013	

CamelOne	

10	

ActiveMQ Source	

•  SVN at Apache: https://svn.apache.org/repos/asf/
activemq/trunk/	

•  Git at Apache:
http://git.apache.org	

•  Github:
https://github.com/apache/activemq	

•  Use git-svn and point directly to Apache SVN!
CamelOne 2013	

CamelOne	

11	

Major Subcomponents	

•  Transports and transport connectors	

•  Broker core	

•  Persistence adapters	

•  Network of brokers / clustering
CamelOne 2013	

CamelOne	

12	

Architecture	

Broker CoreTransport
Transport
Transport
JMX
Network
Connectors
Security
Store
Adapter
CamelOne 2013	

CamelOne	

13	

Transports
and 
Transport Connectors
CamelOne 2013	

CamelOne	

14	

Transport Connector	

•  Let data in or out	

•  Think of as “SocketServer” (which it is J)	

•  Broker side	

•  Highly configurable through URI	

•  TransportConnector vs Transport
CamelOne 2013	

CamelOne	

15	

Example Configuration	

•  Broker side:

transportConnector	
  uri=“tcp://0.0.0.0:61616”	
  /	
  
•  Broker side tuning:

transportConnector	
  uri=tcp://0.0.0.0:0?transport.option=true”/	
  
•  Client side:

ConnectionFactory(tcp://0.0.0.0:61616?soTimeout=xx”)	
  
CamelOne 2013	

CamelOne	

16	

Out of the box	

•  TCP, NIO	

•  UDP	

•  SSL, SSL+NIO	

•  VM	

•  HTTP	

•  WebSockets
CamelOne 2013	

CamelOne	

17	

Wire Formats	

•  Part of transport	

•  Encode/marshall	

•  Decode/unmarshall	

•  Creates Java objects	

Wire
Broker
Transport
Wire
Format
Socket
CamelOne 2013	

CamelOne	

18	

Wire Formats	

•  AMQP 1.0	

•  OpenWire v1- v10	

•  STOMP 1.0, 1.1, 1.2	

•  MQTT 3.1
CamelOne 2013	

CamelOne	

19	

Filters, Listeners	

•  TransportFilter – decorator pattern to add specific
functionality, implements Transport,
TransportListener	

•  TransportListener – callback for staged handling of
incoming frame objects, on the way to broker	

•  Examples:
Stomp, inactivity heartbeat, negotiation, logging,
mutex, etc
CamelOne 2013	

CamelOne	

20	

Transports Architecture	

Wire Broker
Transport
Listener
Transport
Filter
Transport
Wire
Format
Transport
Filter
Transport
Filter
Transport
Listener
Transport
Listener
Transport
Listener
CamelOne 2013	

CamelOne	

21	

Broker Core
CamelOne 2013	

CamelOne	

22	

Broker Core	

•  Regions, layers of brokers	

•  Memory management	

•  Message cursors	

•  Threading
CamelOne 2013	

CamelOne	

23	

Region Broker	

•  Core of the broker	

•  Create consumers	

•  Create producers	

•  Create destinations	

•  Send messages to
correct destination	

RegionBroker
Router
Topic Region
TempTopic
Region
Queue
Region
TempQueue
Region
Connection
States
Destinations ClientIDs
CamelOne 2013	

CamelOne	

24	

Broker Filters	

•  Decorate RegionBroker	

•  How internal features like advisories and scheduling
are implemented	

•  Basis for broker plugins	

•  http://activemq.apache.org/interceptors.html	

•  http://activemq.apache.org/developing-plugins.html
CamelOne 2013	

CamelOne	

25	

Broker Filters	

SchedulerBroker
TransactionBroker
AdvisoryBroker
RegionBroker
Router
Send message
Add Consumer
Add Connection
CamelOne 2013	

CamelOne	

26	

System Usage	

•  Allocate resources for the broker	

•  Persistent, Non-Persistent, in Memory Cache	

systemUsage	
  
	
  	
  	
  	
  systemUsage	
  
	
  	
  	
  	
  	
  	
  	
  	
  memoryUsage	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  memoryUsage	
  limit=64	
  mb/	
  
	
  	
  	
  	
  	
  	
  	
  	
  /memoryUsage	
  
	
  	
  	
  	
  	
  	
  	
  	
  storeUsage	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  storeUsage	
  limit=100	
  gb/	
  
	
  	
  	
  	
  	
  	
  	
  	
  /storeUsage	
  
	
  	
  	
  	
  	
  	
  	
  	
  tempUsage	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  tempUsage	
  limit=50	
  gb/	
  
	
  	
  	
  	
  	
  	
  	
  	
  /tempUsage	
  
	
  	
  	
  	
  /systemUsage	
  
/systemUsage	
  
	
  
CamelOne 2013	

CamelOne	

27	

Memory Usage	

JVM Heap Space
Messages (MemoryUsage)
Queue Queue Queue Queue
ActiveMQ
Objects
Index
Cache
Threads
16 MB 16 MB 16 MB
256 MB
CamelOne 2013	

CamelOne	

28	

Message Cursors	

•  Buffers for tracking position	

•  Consumer Subscriptions	

•  VMPendingMessageCursor	

•  FilePendingMessageCursor	

•  Store Cursors + PrefetchSubscriptions
CamelOne 2013	

CamelOne	

29	

Store Cursors	

http://activemq.apache.org/message-cursors.html
CamelOne 2013	

CamelOne	

30	

Store Cursors	

http://activemq.apache.org/message-cursors.html
CamelOne 2013	

CamelOne	

31	

Threading	

•  Producer connections	

•  Dispatching within destinations	

•  Consumer connections	

•  Store write threads	

•  Ancillary broker tasks (expiration, producer flow
control, scheduled messages, etc)
CamelOne 2013	

CamelOne	

32	

Threads	

TCP
TCP
Queue Dispatch Thread
Consumer Transport Thread
Producer Transport Thread
Queue
Consumer
Connection
Subscription
Producer
Connection
CamelOne 2013	

CamelOne	

33	

TCP
TCP
Consumer Transport Thread
Producer Transport Thread
Queue
Consumer
Connection
Subscription
Producer
Connection
policyEntry	
  queue=“”	
  optimizedDispatched=“true”	
  	
  
CamelOne 2013	

CamelOne	

34	

TCP
TCP
Producer Transport Thread
Queue
Consumer
Connection
Subscription
Producer
Connection
policyEntry	
  queue=“”	
  optimizedDispatched=“true”	
  	
  
tcp://host:port?jms.dispatchAsync=false	
  
CamelOne 2013	

CamelOne	

35	

Persistence Adapter
CamelOne 2013	

CamelOne	

36	

Persistence Adapters	

•  Add reliability to message delivery	

•  Optimized databases	

•  Pluggable implementations	

•  Out of the box: 	

•  KahaDB	

•  LevelDB	

•  JDBC
CamelOne 2013	

CamelOne	

37	

Persistence Adapter	

Locker
Queue Store Topic Store
Journal
Index Redo
File System / RDBMS
CamelOne 2013	

CamelOne	

38	

Lockers	

•  Used for master election	

•  SharedFileLocker	

•  DatabaseLocker, LeaseDatabaseLocker	

•  ZooKeeper based for replicated LevelDB
new in upcoming 5.9.0, not available yet 	

•  Your own!
CamelOne 2013	

CamelOne	

39	

Message Stores	

•  Topic vs Queue stores	

•  Difference use cases	

•  Storage vs broadcast	

•  Relatively cheap	

Topic Store
Topic Store
Topic Store
Journal
Index Redo
File System
Queue Store
Queue Store
Queue Store
CamelOne 2013	

CamelOne	

40	

Store implementation	

•  Journal	

•  Index	

•  Redo/write ahead log	

Index Journal
Redo Log
X X X X
CamelOne 2013	

CamelOne	

41	

Network of Brokers
CamelOne 2013	

CamelOne	

42	

Network of Brokers	

•  Connect brokers together	

•  Achieve scale	

•  Store and forward	

Broker
Message
Broker
CamelOne 2013	

CamelOne	

43	

Networking components	

•  Network Connector	

•  Discovery Agents	

•  Bridge	

•  Advisory messages/static destination filters	

•  Proxy Consumers
CamelOne 2013	

CamelOne	

44	

Advisory Messages	

•  Advisory Topics	

•  Internal events
ActiveMQ.Advisory.Connection
ActiveMQ.Advisory.Consumer	

•  Event driven	

•  Replay upon first connect	

•  Necessary for Dynamic Demand Forwarding
CamelOne 2013	

CamelOne	

45	

Dynamic Demand Forwarding	

Broker
Message
Broker
Producer
Message
Consumer
Message
establish demand
forward message
CamelOne 2013	

CamelOne	

46	

Bridging in Detail	

Broker A
Broker B
Forward Msg
Advisories
Queue
C
Queue
Producer Consumer
Bridge
C
CamelOne 2013	

CamelOne	

47	

Wrap Up	

•  Checkout the ActiveMQ wiki:
http://activemq.apache.org	

•  Mailing list:
users@activemq.apache.org	

•  Book: ActiveMQ in Action
http://www.manning.com/snyder/	

•  Blog
http://christianposta.com/blog
CamelOne 2013	

CamelOne	

48	

Questions?
1 of 48

More Related Content

Similar to Camel oneactivemq posta-final

AMF Flash and .NETAMF Flash and .NET
AMF Flash and .NETYaniv Uriel
727 views23 slides

Similar to Camel oneactivemq posta-final(20)

Monitoring Akka with Kamon 1.0Monitoring Akka with Kamon 1.0
Monitoring Akka with Kamon 1.0
Steffen Gebert2.4K views
Red Hat Open Day JBoss FuseRed Hat Open Day JBoss Fuse
Red Hat Open Day JBoss Fuse
Adrian Gigante3.3K views
Apache Camel IntroductionApache Camel Introduction
Apache Camel Introduction
Claus Ibsen938 views
AMF Flash and .NETAMF Flash and .NET
AMF Flash and .NET
Yaniv Uriel727 views
Picking a  message queuePicking a  message queue
Picking a message queue
Vladislav Kirshtein540 views
Mom those things v1 Mom those things v1
Mom those things v1
von gosling2.2K views
Mule overviewMule overview
Mule overview
sssatya114 views
mule-overview.pptmule-overview.ppt
mule-overview.ppt
NalamalpuBhakthavats3 views
Real world Scala hAkking NLJUG JFall 2011Real world Scala hAkking NLJUG JFall 2011
Real world Scala hAkking NLJUG JFall 2011
Raymond Roestenburg1.2K views
EIP In PracticeEIP In Practice
EIP In Practice
Bruce Snyder3.4K views
An introduction to Apache CamelAn introduction to Apache Camel
An introduction to Apache Camel
Kapil Kumar759 views
Websphere mq series admin trainingWebsphere mq series admin training
Websphere mq series admin training
FuturePoint Technologies330 views
Mule overviewMule overview
Mule overview
ppts123456101 views
Messaging for IoTMessaging for IoT
Messaging for IoT
dejanb1.8K views
Apache Camel K - FredericiaApache Camel K - Fredericia
Apache Camel K - Fredericia
Claus Ibsen751 views

More from Christian Posta(20)

Compliance and Zero Trust Ambient MeshCompliance and Zero Trust Ambient Mesh
Compliance and Zero Trust Ambient Mesh
Christian Posta143 views
Cilium + Istio with Gloo MeshCilium + Istio with Gloo Mesh
Cilium + Istio with Gloo Mesh
Christian Posta596 views
Multi-cluster service mesh with GlooMeshMulti-cluster service mesh with GlooMesh
Multi-cluster service mesh with GlooMesh
Christian Posta1.2K views
Chaos Debugging for MicroservicesChaos Debugging for Microservices
Chaos Debugging for Microservices
Christian Posta694 views
Intro Istio and what's new Istio 1.1Intro Istio and what's new Istio 1.1
Intro Istio and what's new Istio 1.1
Christian Posta852 views
PHX DevOps Days: Service Mesh LandscapePHX DevOps Days: Service Mesh Landscape
PHX DevOps Days: Service Mesh Landscape
Christian Posta736 views
Intro to KnativeIntro to Knative
Intro to Knative
Christian Posta1.9K views
API World: The service-mesh landscapeAPI World: The service-mesh landscape
API World: The service-mesh landscape
Christian Posta851 views

Recently uploaded(20)

Java Platform Approach 1.0 - Picnic MeetupJava Platform Approach 1.0 - Picnic Meetup
Java Platform Approach 1.0 - Picnic Meetup
Rick Ossendrijver24 views
ChatGPT and AI for Web DevelopersChatGPT and AI for Web Developers
ChatGPT and AI for Web Developers
Maximiliano Firtman161 views
CXL at OCPCXL at OCP
CXL at OCP
CXL Forum203 views

Camel oneactivemq posta-final