Your SlideShare is downloading. ×
0
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
ActiveMQ In Action - ApacheCon 2011
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

ActiveMQ In Action - ApacheCon 2011

5,550

Published on

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,550
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
124
Comments
0
Likes
7
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. ActiveMQ In Action Common Problems and Solutions Bruce Snyder VMware, Inc. bsnyder@vmware.comWednesday, November 16, 11
  • 2. Common Questions • Should I create my JMS clients from scratch? • How do I manage connections efficiently? • How do I consume only certain messages? • Why is ActiveMQ locking up or freezing? • Bonus: Do I need a network of brokers? • Bonus: Should I use a master/slave configuration?Wednesday, November 16, 11
  • 3. Should I create JMS clients from scratch? 3Wednesday, November 16, 11
  • 4. Should I create JMS clients from scratch? • Question: – Would you create a HTTP client from scratch? – Would you create a SMTP client from scratch? • Answer: – Sometimes, but mostly no • Solution: – Use Spring JMS 4Wednesday, November 16, 11
  • 5. Spring JMS 5Wednesday, November 16, 11
  • 6. Spring JMS • JMS Template – Send and receive messages synchronously • Message Listener Container – Receive messages asynchronously – Message-Driven POJOs (MDPs) 6Wednesday, November 16, 11
  • 7. Spring JMS • JMS Template – Send and receive messages synchronously • Message Listener Container – Receive messages asynchronously – Message-Driven POJOs (MDPs) 7Wednesday, November 16, 11
  • 8. How do I manage connections efficiently? 8Wednesday, November 16, 11
  • 9. How do I manage connections efficiently? • JMS connections are expensive to constantly create and destroy • Create a group that never closes, i.e., pooling • Solutions: – ActiveMQ PooledConnectionFactory – Spring CachingConnectionFactory 9Wednesday, November 16, 11
  • 10. ActiveMQ PooledConnectionFactory • Based on Apache Commons Pool – Generic object pooling framework from the ASF • Highly configurable – Instantiate your own custom GenericObjectPool • Could be improved – Upon hitting pool limit, grow the pool instead of blocking – Throw exception when the pool is exhausted • Caches JMS Sessions and MessageProducers 10Wednesday, November 16, 11
  • 11. Spring CachingConnectionFactory • Based on Spring SingleConnectionFactory – Ignores calls to Connection.close() • Caches JMS Sessions and MessageProducers • By default only one session is cached – Increase the sessionCacheSize property! • Consumers are not closed until Session is closed – NOTE: Cache strategy uses the JMS selector as a key 11Wednesday, November 16, 11
  • 12. Quick Tip: Monitor the Broker • Q: How can I monitor the message broker while I am developing? • A: Manually via JMX • A: Use the web console (backed by JMX) • A: Use the advisory messages – Especially powerful – http://activemq.apache.org/advisory-message.html 12Wednesday, November 16, 11
  • 13. How do I consume only certain messages? 13Wednesday, November 16, 11
  • 14. How do I consume only certain messages? • ActiveMQ is for sending and receiving events • ActiveMQ is NOT a message store • Solutions: – Use message selectors – Correct application design 14Wednesday, November 16, 11
  • 15. JMS Selectors • Allows a client to filter messages from a destination • Filters message headers only, not payload • Conditional expressions using a subset of SQL • Provide boolean evaluation of message headers Booleans TRUE/FALSE; numbers such as 5, -10, +34; numbers with decimal Literals or scientific notation such as 43.3E7, +10.5239 Identifiers A header field AND, OR, LIKE, BETWEEN, =, <>, <, >, Operators <=, =>, +, =, *, /, IS NULL, IS NOT NULL 15Wednesday, November 16, 11
  • 16. JMS Selector Examples // Select messages with a header named symbol whose value is APPL String selector = "symbol = ‘APPL’"; // Create a consumer that only receives messages about Apple Computer stock MessageConsumer consumer = session.createConsumer(someDestination, selector); // Select messages with a header named symbol whose value is APPL // and with a header named price that is greater than the previous price String selector = "symbol = ‘APPL’ AND price > " + getPreviousPrice(); // Create a consumer that only receives messages about Apple Computer stock // that has increased in price MessageConsumer consumer = session.createConsumer(someDestination, selector); 16Wednesday, November 16, 11
  • 17. JMS Selectors • Very powerful, but like a sharp knife • Applied to every message on a destination – Can cause unnecessary overhead 17Wednesday, November 16, 11
  • 18. Correct Application Design • ActiveMQ is for sending and receiving events • ActiveMQ is NOT a message store • Phase one, consume the messages – Lightweight processing • Phase two, conduct further processing – Heavyweight processing • I.e., a proper service-oriented architecture 18Wednesday, November 16, 11
  • 19. Quick Tip: Clearing the DLQ • Q: Is there a way to automatically clear messages in the DLQ? • A: Use the DiscardingDLQ plugin or create a custom consumer <broker brokerName="myBroker” ...> <plugins> <discardingDLQBrokerPlugin dropAll="true" dropTemporaryTopics="true" dropTemporaryQueues="true" /> </plugins> </broker> <broker brokerName="myBroker” ...> <plugins> <discardingDLQBrokerPlugin dropOnly="TEST.FOO.[1-9] EXAMPLE.TOPIC" reportInverval="5000" /> </plugins> </broker> 19Wednesday, November 16, 11
  • 20. Why is ActiveMQ is locking up or freezing? 20Wednesday, November 16, 11
  • 21. Why is ActiveMQ is locking up or freezing? • JVM memory • Broker memory • Prefetch limit • Producer flow control • Message cursors 21Wednesday, November 16, 11
  • 22. JVM Memory • ActiveMQ start script – In 5.4.x, JVM is given 256mb of memory (min and max) • You may need to increase this! 22Wednesday, November 16, 11
  • 23. Broker Memory • ActiveMQ controls how much memory it can use • Will not automatically use all the JVM memory • Configurable but commented out by default 23Wednesday, November 16, 11
  • 24. Broker Memory Example <broker brokerName="myBroker” ...> ... <systemUsage> <systemUsage> <memoryUsage> <memoryUsage limit="64 mb” /> </memoryUsage> <storeUsage> <storeUsage limit="100 gb" /> </storeUsage> <tempUsage> <tempUsage limit="10 gb" /> </tempUsage> </systemUsage> </systemUsage> ... </broker> 24Wednesday, November 16, 11
  • 25. Prefetch Limit • Prevents a consumer from being flooded with messages • Applied on a per client basis • Incorrect prefetch limit + slow consumer = messages remain in a queue unconsumed • Results in some consumers being starved of messages • NOTE: Be careful with connection pools 25Wednesday, November 16, 11
  • 26. Prefetch Limit Example ... <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" p:brokerURL="tcp://localhost:61616" p:prefetchPolicy-ref="prefetchPolicy"/> <bean id="prefetchPolicy" class="org.apache.activemq.ActiveMQPrefetchPolicy" p:queuePrefetch="1" /> ... 26Wednesday, November 16, 11
  • 27. Producer Flow Control • Prevents producer from flooding broker • If memory exceeds limit, a producer will be paused • NOTE: This setting is enabled by default 27Wednesday, November 16, 11
  • 28. Broker Memory Example <broker brokerName="myBroker” ...> ... <destinationPolicy> <policyMap> <policyEntries> <policyEntry topic=">" producerFlowControl="true" memoryLimit="10mb"> <pendingSubscriberPolicy> <vmCursor /> </pendingSubscriberPolicy> </policyEntry> <policyEntry queue=">" producerFlowControl="true" memoryLimit="10mb"> </policyEntry> </policyEntries> </policyMap> </destinationPolicy> ... </broker> 28Wednesday, November 16, 11
  • 29. Message Cursors • Only so many messages can be held in memory • Message cursors provide a configurable message paging • Two types of cursors – VM cursors • Holds only message reference in memory – File cursors • Flushes both message and reference to disk • http://activemq.apache.org/how-do-i-configure-activemq-to- hold-100s-of-millions-of-queue-messages-.html 29Wednesday, November 16, 11
  • 30. Message Cursor Example <broker brokerName="myBroker” ...> ... <destinationPolicy> <policyMap> <policyEntries> <policyEntry topic=">" producerFlowControl="true" memoryLimit="10mb"> <pendingSubscriberPolicy> <vmCursor /> </pendingSubscriberPolicy> </policyEntry> <policyEntry queue=">" producerFlowControl="true" memoryLimit="10mb"> <pendingQueuePolicy> <fileQueueCursor /> </pendingQueuePolicy> </policyEntry> </policyEntries> </policyMap> </destinationPolicy> ... </broker> 30Wednesday, November 16, 11
  • 31. Quick Tip: Rebalancing Clients • Q: After restarting one of my message brokers, how can I force clients to connect to that broker? • A: Enforce a deterministic lifetime on a connection by setting the expiryTimeout on the pool of connections <bean id="connectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" <property name="brokerURL” value="tcp://localhost:61616" /> <property name="expiryTimeout” value="10000" /> <property name="connectionFactory"> <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" p:brokerURL="tcp://localhost:61616" /> </property> </bean> 31Wednesday, November 16, 11
  • 32. Thank You! Q&AWednesday, November 16, 11
  • 33. Bonus Material! #apachecorn 33Wednesday, November 16, 11
  • 34. Do I need a network of brokers? 34Wednesday, November 16, 11
  • 35. Do I need a network of brokers? • What is a network of brokers? – Clustered ActiveMQ instances • How are they clustered? – They pass messages between broker instances – Send a message to one broker, consume the message from a different broker • Where might this be useful? – Situations where a centralized broker is not suitable • How does this work? – Using store and forward 35Wednesday, November 16, 11
  • 36. Store and Forward 36Wednesday, November 16, 11
  • 37. Topology Example 37Wednesday, November 16, 11
  • 38. Topology Example 38Wednesday, November 16, 11
  • 39. Topology Example 39Wednesday, November 16, 11
  • 40. Topology Example 40Wednesday, November 16, 11
  • 41. Topology Example 41Wednesday, November 16, 11
  • 42. Should I use a master/slave config? 42Wednesday, November 16, 11
  • 43. Should I use a master/slave config? • What is a master/slave configuration? – It helps to provide high availability for ActiveMQ • What does that mean? – ActiveMQ brokers are configured for warm failover – If one broker fails or becomes unreachable, another one takes over • Where might this be useful? – In situations that need highly available message brokers • How does this work? – Depends on the type of master/slave 43Wednesday, November 16, 11
  • 44. Types of Master/Slave • Shared nothing master/slave • Shared storage master/slave – Shared database – Shared file system 44Wednesday, November 16, 11
  • 45. Shared Nothing Master/Slave • Sometimes called pure master/slave • Uses a fully replicated data store – Does not depend on database or file system • Slave broker consumes all message states from the Master broker (messages, acks, tx states) • Slave does not start any networking or transport connectors • Master broker will only respond to client when a message exchange has been successfully passed to the slave broker 45Wednesday, November 16, 11
  • 46. Shared Nothing Master/Slave • If the master fails, the slave optionally has two modes of operation: 1. Start up all it’s network and transport connectors • All clients connected to failed Master resume on Slave 2. Close down completely • Slave is simply used to duplicate state from Master • Clients should use failover transport: failover://(tcp://masterhost:61616, tcp://slavehost:61616)?randomize=false 46Wednesday, November 16, 11
  • 47. Shared Database Master/Slave • Uses tables in a relational database to store data • No restriction on the number of brokers • Simple configuration (JDBC URL) • Clustered database mitigates single point of failure • One master selected at random • Clients should use failover transport: failover://(tcp://masterhost:61616, tcp://slavehost:61616)?randomize=false 47Wednesday, November 16, 11
  • 48. Shared File System Master/Slave • Utilizes a directory on a shared file system to store data • No restriction on number of brokers • Simple configuration (point to the data dir) • Shared file system mitigates single point of failure • One master selected at random • Clients should use failover transport: failover://(tcp://masterhost:61616, tcp://slavehost:61616)?randomize=false 48Wednesday, November 16, 11
  • 49. Should I use a master/slave config? • Are you trying to provide high availability? – Then, yes • Which one should I use? – It depends on your situation 49Wednesday, November 16, 11

×