1.
CamelOne 2013 June 10-11 2013 Boston, MA Apache ActiveMQInternals Christian Posta Senior Consultant and Architect June 10, 2013 1

2.
CamelOne 2013 CamelOne 2 Why are You Here? • Use ActiveMQ? • What is ActiveMQ? • Dug into the code? • Interested in contributing back? • Understand configuration better

3.
CamelOne 2013 CamelOne 3 Goals of Presentation • Give back to community • Deliver understanding about architecture • Brief intro to ActiveMQ • Overview of architecture • Dive into major subcomponents

4.
CamelOne 2013 CamelOne 4 Who am I? • Senior Consultant and Architect at Red Hat (formerlyFuseSource) • Committer on ActiveMQ • Author: Essential Camel Components DZone Refcard • Christian Posta • Blog: http://christianposta.com/blog • Email: ceposta@apache.org • Twitter: @christianposta

5.
CamelOne 2013 CamelOne 5 What is ActiveMQ? • OpenSource messaging broker • Apache v2 licensed • Very active community • Used at top companies • 5.8.0 latest release (in JBoss A-MQ 6.0)5.9.0 on its way

6.
CamelOne 2013 CamelOne 6 Features • High performance • High availability • Light-weight • Multi-protocol • JMS compliant • Supported in production

7.
CamelOne 2013 CamelOne 7 When to use ActiveMQ? • Asynchronous communication • Reliable message passing • Loose coupling • Heterogeneous integration • Real-time data

8.
CamelOne 2013 CamelOne 8 This is ActiveMQ…

9.
CamelOne 2013 CamelOne 9 How it’s built • Java project (supports Java 7!) • Maven project • Divided into client/broker and feature modules • Unit tests in activemq-unit-tests • mvn clean package –Dtest=false

10.
CamelOne 2013 CamelOne 10 ActiveMQ Source • SVN at Apache: https://svn.apache.org/repos/asf/activemq/trunk/ • Git at Apache:http://git.apache.org • Github:https://github.com/apache/activemq • Use git-svn and point directly to Apache SVN!

11.
CamelOne 2013 CamelOne 11 Major Subcomponents • Transports and transport connectors • Broker core • Persistence adapters • Network of brokers / clustering

12.
CamelOne 2013 CamelOne 12 Architecture Broker CoreTransportTransportTransportJMXNetworkConnectorsSecurityStoreAdapter

13.
CamelOne 2013 CamelOne 13 Transportsand Transport Connectors

14.
CamelOne 2013 CamelOne 14 Transport Connector • Let data in or out • Think of as “SocketServer” (which it is J) • Broker side • Highly configurable through URI • TransportConnector vs Transport

15.
CamelOne 2013 CamelOne 15 Example Configuration • Broker side:transportConnector uri=“tcp://0.0.0.0:61616” / • Broker side tuning:transportConnector uri=tcp://0.0.0.0:0?transport.option=true”/ • Client side:ConnectionFactory(tcp://0.0.0.0:61616?soTimeout=xx”)

16.
CamelOne 2013 CamelOne 16 Out of the box • TCP, NIO • UDP • SSL, SSL+NIO • VM • HTTP • WebSockets

17.
CamelOne 2013 CamelOne 17 Wire Formats • Part of transport • Encode/marshall • Decode/unmarshall • Creates Java objects WireBrokerTransportWireFormatSocket

18.
CamelOne 2013 CamelOne 18 Wire Formats • AMQP 1.0 • OpenWire v1- v10 • STOMP 1.0, 1.1, 1.2 • MQTT 3.1

19.
CamelOne 2013 CamelOne 19 Filters, Listeners • TransportFilter – decorator pattern to add specificfunctionality, implements Transport,TransportListener • TransportListener – callback for staged handling ofincoming frame objects, on the way to broker • Examples:Stomp, inactivity heartbeat, negotiation, logging,mutex, etc

20.
CamelOne 2013 CamelOne 20 Transports Architecture Wire BrokerTransportListenerTransportFilterTransportWireFormatTransportFilterTransportFilterTransportListenerTransportListenerTransportListener

21.
CamelOne 2013 CamelOne 21 Broker Core

22.
CamelOne 2013 CamelOne 22 Broker Core • Regions, layers of brokers • Memory management • Message cursors • Threading

23.
CamelOne 2013 CamelOne 23 Region Broker • Core of the broker • Create consumers • Create producers • Create destinations • Send messages tocorrect destination RegionBrokerRouterTopic RegionTempTopicRegionQueueRegionTempQueueRegionConnectionStatesDestinations ClientIDs

24.
CamelOne 2013 CamelOne 24 Broker Filters • Decorate RegionBroker • How internal features like advisories and schedulingare implemented • Basis for broker plugins • http://activemq.apache.org/interceptors.html • http://activemq.apache.org/developing-plugins.html

25.
CamelOne 2013 CamelOne 25 Broker Filters SchedulerBrokerTransactionBrokerAdvisoryBrokerRegionBrokerRouterSend messageAdd ConsumerAdd Connection

26.
CamelOne 2013 CamelOne 26 System Usage • Allocate resources for the broker • Persistent, Non-Persistent, in Memory Cache systemUsage systemUsage memoryUsage memoryUsage limit=64 mb/ /memoryUsage storeUsage storeUsage limit=100 gb/ /storeUsage tempUsage tempUsage limit=50 gb/ /tempUsage /systemUsage /systemUsage

27.
CamelOne 2013 CamelOne 27 Memory Usage JVM Heap SpaceMessages (MemoryUsage)Queue Queue Queue QueueActiveMQObjectsIndexCacheThreads16 MB 16 MB 16 MB256 MB

28.
CamelOne 2013 CamelOne 28 Message Cursors • Buffers for tracking position • Consumer Subscriptions • VMPendingMessageCursor • FilePendingMessageCursor • Store Cursors + PrefetchSubscriptions

29.
CamelOne 2013 CamelOne 29 Store Cursors http://activemq.apache.org/message-cursors.html

30.
CamelOne 2013 CamelOne 30 Store Cursors http://activemq.apache.org/message-cursors.html

31.
CamelOne 2013 CamelOne 31 Threading • Producer connections • Dispatching within destinations • Consumer connections • Store write threads • Ancillary broker tasks (expiration, producer flowcontrol, scheduled messages, etc)

32.
CamelOne 2013 CamelOne 32 Threads TCPTCPQueue Dispatch ThreadConsumer Transport ThreadProducer Transport ThreadQueueConsumerConnectionSubscriptionProducerConnection

33.
CamelOne 2013 CamelOne 33 TCPTCPConsumer Transport ThreadProducer Transport ThreadQueueConsumerConnectionSubscriptionProducerConnectionpolicyEntry queue=“” optimizedDispatched=“true”

34.
CamelOne 2013 CamelOne 34 TCPTCPProducer Transport ThreadQueueConsumerConnectionSubscriptionProducerConnectionpolicyEntry queue=“” optimizedDispatched=“true” tcp://host:port?jms.dispatchAsync=false

35.
CamelOne 2013 CamelOne 35 Persistence Adapter

36.
CamelOne 2013 CamelOne 36 Persistence Adapters • Add reliability to message delivery • Optimized databases • Pluggable implementations • Out of the box: • KahaDB • LevelDB • JDBC

37.
CamelOne 2013 CamelOne 37 Persistence Adapter LockerQueue Store Topic StoreJournalIndex RedoFile System / RDBMS

38.
CamelOne 2013 CamelOne 38 Lockers • Used for master election • SharedFileLocker • DatabaseLocker, LeaseDatabaseLocker • ZooKeeper based for replicated LevelDBnew in upcoming 5.9.0, not available yet • Your own!

39.
CamelOne 2013 CamelOne 39 Message Stores • Topic vs Queue stores • Difference use cases • Storage vs broadcast • Relatively cheap Topic StoreTopic StoreTopic StoreJournalIndex RedoFile SystemQueue StoreQueue StoreQueue Store

40.
CamelOne 2013 CamelOne 40 Store implementation • Journal • Index • Redo/write ahead log Index JournalRedo LogX X X X

41.
CamelOne 2013 CamelOne 41 Network of Brokers

42.
CamelOne 2013 CamelOne 42 Network of Brokers • Connect brokers together • Achieve scale • Store and forward BrokerMessageBroker

43.
CamelOne 2013 CamelOne 43 Networking components • Network Connector • Discovery Agents • Bridge • Advisory messages/static destination filters • Proxy Consumers

44.
CamelOne 2013 CamelOne 44 Advisory Messages • Advisory Topics • Internal eventsActiveMQ.Advisory.ConnectionActiveMQ.Advisory.Consumer • Event driven • Replay upon first connect • Necessary for Dynamic Demand Forwarding

45.
CamelOne 2013 CamelOne 45 Dynamic Demand Forwarding BrokerMessageBrokerProducerMessageConsumerMessageestablish demandforward message

46.
CamelOne 2013 CamelOne 46 Bridging in Detail Broker ABroker BForward MsgAdvisoriesQueueCQueueProducer ConsumerBridgeC

47.
CamelOne 2013 CamelOne 47 Wrap Up • Checkout the ActiveMQ wiki:http://activemq.apache.org • Mailing list:users@activemq.apache.org • Book: ActiveMQ in Actionhttp://www.manning.com/snyder/ • Bloghttp://christianposta.com/blog

48.
CamelOne 2013 CamelOne 48 Questions?