Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
CamelOne 2013	June 10-11 2013	Boston, MA	Apache ActiveMQInternals	Christian Posta	Senior Consultant and Architect	June 10,...
CamelOne 2013	CamelOne	2	Why are You Here?	•  Use ActiveMQ?	•  What is ActiveMQ?	•  Dug into the code?	•  Interested in co...
CamelOne 2013	CamelOne	3	Goals of Presentation	•  Give back to community	•  Deliver understanding about architecture	•  Br...
CamelOne 2013	CamelOne	4	Who am I?	•  Senior Consultant and Architect at Red Hat (formerlyFuseSource)	•  Committer on Acti...
CamelOne 2013	CamelOne	5	What is ActiveMQ?	•  OpenSource messaging broker	•  Apache v2 licensed	•  Very active community	•...
CamelOne 2013	CamelOne	6	Features	•  High performance	•  High availability	•  Light-weight	•  Multi-protocol	•  JMS compli...
CamelOne 2013	CamelOne	7	When to use ActiveMQ?	•  Asynchronous communication	•  Reliable message passing	•  Loose coupling...
CamelOne 2013	CamelOne	8	This is ActiveMQ…
CamelOne 2013	CamelOne	9	How it’s built	•  Java project (supports Java 7!)	•  Maven project	•  Divided into client/broker ...
CamelOne 2013	CamelOne	10	ActiveMQ Source	•  SVN at Apache: https://svn.apache.org/repos/asf/activemq/trunk/	•  Git at Apa...
CamelOne 2013	CamelOne	11	Major Subcomponents	•  Transports and transport connectors	•  Broker core	•  Persistence adapter...
CamelOne 2013	CamelOne	12	Architecture	Broker CoreTransportTransportTransportJMXNetworkConnectorsSecurityStoreAdapter
CamelOne 2013	CamelOne	13	Transportsand Transport Connectors
CamelOne 2013	CamelOne	14	Transport Connector	•  Let data in or out	•  Think of as “SocketServer” (which it is J)	•  Brok...
CamelOne 2013	CamelOne	15	Example Configuration	•  Broker side:transportConnector	  uri=“tcp://0.0.0.0:61616”	  /	  •  Brok...
CamelOne 2013	CamelOne	16	Out of the box	•  TCP, NIO	•  UDP	•  SSL, SSL+NIO	•  VM	•  HTTP	•  WebSockets
CamelOne 2013	CamelOne	17	Wire Formats	•  Part of transport	•  Encode/marshall	•  Decode/unmarshall	•  Creates Java object...
CamelOne 2013	CamelOne	18	Wire Formats	•  AMQP 1.0	•  OpenWire v1- v10	•  STOMP 1.0, 1.1, 1.2	•  MQTT 3.1
CamelOne 2013	CamelOne	19	Filters, Listeners	•  TransportFilter – decorator pattern to add specificfunctionality, implement...
CamelOne 2013	CamelOne	20	Transports Architecture	Wire BrokerTransportListenerTransportFilterTransportWireFormatTransportF...
CamelOne 2013	CamelOne	21	Broker Core
CamelOne 2013	CamelOne	22	Broker Core	•  Regions, layers of brokers	•  Memory management	•  Message cursors	•  Threading
CamelOne 2013	CamelOne	23	Region Broker	•  Core of the broker	•  Create consumers	•  Create producers	•  Create destinatio...
CamelOne 2013	CamelOne	24	Broker Filters	•  Decorate RegionBroker	•  How internal features like advisories and schedulinga...
CamelOne 2013	CamelOne	25	Broker Filters	SchedulerBrokerTransactionBrokerAdvisoryBrokerRegionBrokerRouterSend messageAdd C...
CamelOne 2013	CamelOne	26	System Usage	•  Allocate resources for the broker	•  Persistent, Non-Persistent, in Memory Cache...
CamelOne 2013	CamelOne	27	Memory Usage	JVM Heap SpaceMessages (MemoryUsage)Queue Queue Queue QueueActiveMQObjectsIndexCach...
CamelOne 2013	CamelOne	28	Message Cursors	•  Buffers for tracking position	•  Consumer Subscriptions	•  VMPendingMessageCu...
CamelOne 2013	CamelOne	29	Store Cursors	http://activemq.apache.org/message-cursors.html
CamelOne 2013	CamelOne	30	Store Cursors	http://activemq.apache.org/message-cursors.html
CamelOne 2013	CamelOne	31	Threading	•  Producer connections	•  Dispatching within destinations	•  Consumer connections	•  ...
CamelOne 2013	CamelOne	32	Threads	TCPTCPQueue Dispatch ThreadConsumer Transport ThreadProducer Transport ThreadQueueConsum...
CamelOne 2013	CamelOne	33	TCPTCPConsumer Transport ThreadProducer Transport ThreadQueueConsumerConnectionSubscriptionProdu...
CamelOne 2013	CamelOne	34	TCPTCPProducer Transport ThreadQueueConsumerConnectionSubscriptionProducerConnectionpolicyEntry	...
CamelOne 2013	CamelOne	35	Persistence Adapter
CamelOne 2013	CamelOne	36	Persistence Adapters	•  Add reliability to message delivery	•  Optimized databases	•  Pluggable ...
CamelOne 2013	CamelOne	37	Persistence Adapter	LockerQueue Store Topic StoreJournalIndex RedoFile System / RDBMS
CamelOne 2013	CamelOne	38	Lockers	•  Used for master election	•  SharedFileLocker	•  DatabaseLocker, LeaseDatabaseLocker	•...
CamelOne 2013	CamelOne	39	Message Stores	•  Topic vs Queue stores	•  Difference use cases	•  Storage vs broadcast	•  Relat...
CamelOne 2013	CamelOne	40	Store implementation	•  Journal	•  Index	•  Redo/write ahead log	Index JournalRedo LogX X X X
CamelOne 2013	CamelOne	41	Network of Brokers
CamelOne 2013	CamelOne	42	Network of Brokers	•  Connect brokers together	•  Achieve scale	•  Store and forward	BrokerMessa...
CamelOne 2013	CamelOne	43	Networking components	•  Network Connector	•  Discovery Agents	•  Bridge	•  Advisory messages/st...
CamelOne 2013	CamelOne	44	Advisory Messages	•  Advisory Topics	•  Internal eventsActiveMQ.Advisory.ConnectionActiveMQ.Advi...
CamelOne 2013	CamelOne	45	Dynamic Demand Forwarding	BrokerMessageBrokerProducerMessageConsumerMessageestablish demandforwa...
CamelOne 2013	CamelOne	46	Bridging in Detail	Broker ABroker BForward MsgAdvisoriesQueueCQueueProducer ConsumerBridgeC
CamelOne 2013	CamelOne	47	Wrap Up	•  Checkout the ActiveMQ wiki:http://activemq.apache.org	•  Mailing list:users@activemq....
CamelOne 2013	CamelOne	48	Questions?
Upcoming SlideShare
Loading in …5
×

Camel oneactivemq posta-final

1,625 views

Published on

Published in: Technology
  • Be the first to comment

Camel oneactivemq posta-final

  1. 1. CamelOne 2013 June 10-11 2013 Boston, MA Apache ActiveMQInternals Christian Posta Senior Consultant and Architect June 10, 2013 1
  2. 2. CamelOne 2013 CamelOne 2 Why are You Here? •  Use ActiveMQ? •  What is ActiveMQ? •  Dug into the code? •  Interested in contributing back? •  Understand configuration better
  3. 3. CamelOne 2013 CamelOne 3 Goals of Presentation •  Give back to community •  Deliver understanding about architecture •  Brief intro to ActiveMQ •  Overview of architecture •  Dive into major subcomponents
  4. 4. CamelOne 2013 CamelOne 4 Who am I? •  Senior Consultant and Architect at Red Hat (formerlyFuseSource) •  Committer on ActiveMQ •  Author: Essential Camel Components DZone Refcard •  Christian Posta •  Blog: http://christianposta.com/blog •  Email: ceposta@apache.org •  Twitter: @christianposta
  5. 5. CamelOne 2013 CamelOne 5 What is ActiveMQ? •  OpenSource messaging broker •  Apache v2 licensed •  Very active community •  Used at top companies •  5.8.0 latest release (in JBoss A-MQ 6.0)5.9.0 on its way
  6. 6. CamelOne 2013 CamelOne 6 Features •  High performance •  High availability •  Light-weight •  Multi-protocol •  JMS compliant •  Supported in production
  7. 7. CamelOne 2013 CamelOne 7 When to use ActiveMQ? •  Asynchronous communication •  Reliable message passing •  Loose coupling •  Heterogeneous integration •  Real-time data
  8. 8. CamelOne 2013 CamelOne 8 This is ActiveMQ…
  9. 9. CamelOne 2013 CamelOne 9 How it’s built •  Java project (supports Java 7!) •  Maven project •  Divided into client/broker and feature modules •  Unit tests in activemq-unit-tests •  mvn  clean  package  –Dtest=false  
  10. 10. CamelOne 2013 CamelOne 10 ActiveMQ Source •  SVN at Apache: https://svn.apache.org/repos/asf/activemq/trunk/ •  Git at Apache:http://git.apache.org •  Github:https://github.com/apache/activemq •  Use git-svn and point directly to Apache SVN!
  11. 11. CamelOne 2013 CamelOne 11 Major Subcomponents •  Transports and transport connectors •  Broker core •  Persistence adapters •  Network of brokers / clustering
  12. 12. CamelOne 2013 CamelOne 12 Architecture Broker CoreTransportTransportTransportJMXNetworkConnectorsSecurityStoreAdapter
  13. 13. CamelOne 2013 CamelOne 13 Transportsand Transport Connectors
  14. 14. CamelOne 2013 CamelOne 14 Transport Connector •  Let data in or out •  Think of as “SocketServer” (which it is J) •  Broker side •  Highly configurable through URI •  TransportConnector vs Transport
  15. 15. CamelOne 2013 CamelOne 15 Example Configuration •  Broker side:transportConnector  uri=“tcp://0.0.0.0:61616”  /  •  Broker side tuning:transportConnector  uri=tcp://0.0.0.0:0?transport.option=true”/  •  Client side:ConnectionFactory(tcp://0.0.0.0:61616?soTimeout=xx”)  
  16. 16. CamelOne 2013 CamelOne 16 Out of the box •  TCP, NIO •  UDP •  SSL, SSL+NIO •  VM •  HTTP •  WebSockets
  17. 17. CamelOne 2013 CamelOne 17 Wire Formats •  Part of transport •  Encode/marshall •  Decode/unmarshall •  Creates Java objects WireBrokerTransportWireFormatSocket
  18. 18. CamelOne 2013 CamelOne 18 Wire Formats •  AMQP 1.0 •  OpenWire v1- v10 •  STOMP 1.0, 1.1, 1.2 •  MQTT 3.1
  19. 19. CamelOne 2013 CamelOne 19 Filters, Listeners •  TransportFilter – decorator pattern to add specificfunctionality, implements Transport,TransportListener •  TransportListener – callback for staged handling ofincoming frame objects, on the way to broker •  Examples:Stomp, inactivity heartbeat, negotiation, logging,mutex, etc
  20. 20. CamelOne 2013 CamelOne 20 Transports Architecture Wire BrokerTransportListenerTransportFilterTransportWireFormatTransportFilterTransportFilterTransportListenerTransportListenerTransportListener
  21. 21. CamelOne 2013 CamelOne 21 Broker Core
  22. 22. CamelOne 2013 CamelOne 22 Broker Core •  Regions, layers of brokers •  Memory management •  Message cursors •  Threading
  23. 23. CamelOne 2013 CamelOne 23 Region Broker •  Core of the broker •  Create consumers •  Create producers •  Create destinations •  Send messages tocorrect destination RegionBrokerRouterTopic RegionTempTopicRegionQueueRegionTempQueueRegionConnectionStatesDestinations ClientIDs
  24. 24. CamelOne 2013 CamelOne 24 Broker Filters •  Decorate RegionBroker •  How internal features like advisories and schedulingare implemented •  Basis for broker plugins •  http://activemq.apache.org/interceptors.html •  http://activemq.apache.org/developing-plugins.html
  25. 25. CamelOne 2013 CamelOne 25 Broker Filters SchedulerBrokerTransactionBrokerAdvisoryBrokerRegionBrokerRouterSend messageAdd ConsumerAdd Connection
  26. 26. CamelOne 2013 CamelOne 26 System Usage •  Allocate resources for the broker •  Persistent, Non-Persistent, in Memory Cache systemUsage          systemUsage                  memoryUsage                          memoryUsage  limit=64  mb/                  /memoryUsage                  storeUsage                          storeUsage  limit=100  gb/                  /storeUsage                  tempUsage                          tempUsage  limit=50  gb/                  /tempUsage          /systemUsage  /systemUsage    
  27. 27. CamelOne 2013 CamelOne 27 Memory Usage JVM Heap SpaceMessages (MemoryUsage)Queue Queue Queue QueueActiveMQObjectsIndexCacheThreads16 MB 16 MB 16 MB256 MB
  28. 28. CamelOne 2013 CamelOne 28 Message Cursors •  Buffers for tracking position •  Consumer Subscriptions •  VMPendingMessageCursor •  FilePendingMessageCursor •  Store Cursors + PrefetchSubscriptions
  29. 29. CamelOne 2013 CamelOne 29 Store Cursors http://activemq.apache.org/message-cursors.html
  30. 30. CamelOne 2013 CamelOne 30 Store Cursors http://activemq.apache.org/message-cursors.html
  31. 31. CamelOne 2013 CamelOne 31 Threading •  Producer connections •  Dispatching within destinations •  Consumer connections •  Store write threads •  Ancillary broker tasks (expiration, producer flowcontrol, scheduled messages, etc)
  32. 32. CamelOne 2013 CamelOne 32 Threads TCPTCPQueue Dispatch ThreadConsumer Transport ThreadProducer Transport ThreadQueueConsumerConnectionSubscriptionProducerConnection
  33. 33. CamelOne 2013 CamelOne 33 TCPTCPConsumer Transport ThreadProducer Transport ThreadQueueConsumerConnectionSubscriptionProducerConnectionpolicyEntry  queue=“”  optimizedDispatched=“true”    
  34. 34. CamelOne 2013 CamelOne 34 TCPTCPProducer Transport ThreadQueueConsumerConnectionSubscriptionProducerConnectionpolicyEntry  queue=“”  optimizedDispatched=“true”    tcp://host:port?jms.dispatchAsync=false  
  35. 35. CamelOne 2013 CamelOne 35 Persistence Adapter
  36. 36. CamelOne 2013 CamelOne 36 Persistence Adapters •  Add reliability to message delivery •  Optimized databases •  Pluggable implementations •  Out of the box: •  KahaDB •  LevelDB •  JDBC
  37. 37. CamelOne 2013 CamelOne 37 Persistence Adapter LockerQueue Store Topic StoreJournalIndex RedoFile System / RDBMS
  38. 38. CamelOne 2013 CamelOne 38 Lockers •  Used for master election •  SharedFileLocker •  DatabaseLocker, LeaseDatabaseLocker •  ZooKeeper based for replicated LevelDBnew in upcoming 5.9.0, not available yet •  Your own!
  39. 39. CamelOne 2013 CamelOne 39 Message Stores •  Topic vs Queue stores •  Difference use cases •  Storage vs broadcast •  Relatively cheap Topic StoreTopic StoreTopic StoreJournalIndex RedoFile SystemQueue StoreQueue StoreQueue Store
  40. 40. CamelOne 2013 CamelOne 40 Store implementation •  Journal •  Index •  Redo/write ahead log Index JournalRedo LogX X X X
  41. 41. CamelOne 2013 CamelOne 41 Network of Brokers
  42. 42. CamelOne 2013 CamelOne 42 Network of Brokers •  Connect brokers together •  Achieve scale •  Store and forward BrokerMessageBroker
  43. 43. CamelOne 2013 CamelOne 43 Networking components •  Network Connector •  Discovery Agents •  Bridge •  Advisory messages/static destination filters •  Proxy Consumers
  44. 44. CamelOne 2013 CamelOne 44 Advisory Messages •  Advisory Topics •  Internal eventsActiveMQ.Advisory.ConnectionActiveMQ.Advisory.Consumer •  Event driven •  Replay upon first connect •  Necessary for Dynamic Demand Forwarding
  45. 45. CamelOne 2013 CamelOne 45 Dynamic Demand Forwarding BrokerMessageBrokerProducerMessageConsumerMessageestablish demandforward message
  46. 46. CamelOne 2013 CamelOne 46 Bridging in Detail Broker ABroker BForward MsgAdvisoriesQueueCQueueProducer ConsumerBridgeC
  47. 47. CamelOne 2013 CamelOne 47 Wrap Up •  Checkout the ActiveMQ wiki:http://activemq.apache.org •  Mailing list:users@activemq.apache.org •  Book: ActiveMQ in Actionhttp://www.manning.com/snyder/ •  Bloghttp://christianposta.com/blog
  48. 48. CamelOne 2013 CamelOne 48 Questions?

×