Last Updated: Jan. 2014
Senior Software Engineer
Hasitha Abeykoon
WSO2 Message
Broker 2.2.0
Ishara Premadasa
Software Engi...
2
About the Presenter(s)
๏ Hasitha Hiranya Abeykoon,
Senior Software Engineer,
WSO2.
๏ Email: hasithah@wso2.com
๏ Blog: ht...
3
About WSO2
๏ Global enterprise, founded in
2005 by acknowledged leaders in
XML, web services technologies,
standards and...
4
What WSO2 delivers
Outline
1. A Brief Introduction to WSO2 Message Broker
2. What is new in MB 2.2.0
3. Example Use Cases
4. Pre-recorded dem...
Why Messaging?
๏ Enhanced reliability
๏ E.g. via persistence, transactions
๏ Decoupling
๏ Space
๏ Time
๏ Synchronization
6
Basic Messaging Models
๏ Messaging is implemented with a broker (or brokers in
the middle)
๏ Participants send messages, a...
Queues
8
• A queue is a destination that contains messages sent from a producer
that await delivery to one consumer.
• Mes...
Topics
9
• Topics are used to send messages to one or more consumers. Producers publish
messages to a topic and each consu...
Durable Topics
10
• Broker collects messages while subscriber is offline – like a queue
• Broker distributes the messages ...
AMQP, JMS and WSO2 MB
๏ JMS (Java Message Service)
๏ A specification that define a standard API for java programmer to per...
How WSO2 MB is Different?
๏ Use Apache Cassandra as the storage for messages
those require persistence
Apache Cassandra is...
Why WSO2 MB is Different?
๏ Failover with no message lost – Clustering + Failover
capability brings this feature
๏ Ability...
Why WSO2 MB is Different?
๏ Interoperability with many languages / platforms via
AMQP clients for Java, .Net, C, C++, PHP,...
WSO2 MB 2.2.0 Release
๏ A lot of bug fixes with respect to WSO2 2.1.0
๏ Fixed a lot of clustering related issues with edge...
WSO2 MB 2.2.0 Release
๏ New features added to WSO2 MB management console
UI
๏ View active queue subscriptions in whole MB ...
WSO2 MB 2.2.0 Release
๏ Now it is even more easier to deploy WSO2 MB
๏ Tested ability to handle up to 3 million message bu...
Dead Letter Channel
18
• If the subscriber
• Rejected the message
• Did not ack the sever within a configured time
• Could...
Dead Letter Channel contd..
๏ Able to view DLC messages in the Management Console
(Queue vise display).
๏ You can perform ...
Flow Control – Backpressure
๏ Every system has its limits.
๏ What happens to the broker cluster if a message burst
continu...
MB Profile Support
๏ You can start WSO2 MB purely as a Apache Cassandra
server.
๏ All configurations files of Cassandra ar...
Message Selectors
๏ A Subscriber might be interested in particular set of
messages having certain properties.
๏ It only li...
Other tit-bits
Improved SSL Support for MB 2.2.0
๏ The wire level connection between client-server can be
encrypted using ...
A Few Example Use cases
๏ WSO2 Message Broker is part of Messaging Middleware
๏ Rather than using WSO2 MB alone, most of t...
Asynchronous Messaging
๏ Timely decoupled (transactions - inter departmental job
submitting and processing)
25
University ...
Broker Cluster with Failover
26
MB1 MB2 MB3
Connection = <mb1,mb2,mb3>
Application
MB1
MB Cluster
Cassandra
Cluster
Reliable Messaging
๏ Message store processor pattern with WSO2 ESB
27
Web
Application
Enterprise Service Bus
Msg Store Msg...
Reliable Messaging contd..
๏ Want to reliably store information to database?
๏ Message order is guaranteed
๏ Message relia...
Reliable Messaging contd..
๏ Message Store Configuration for MB
<messageStore xmlns="http://ws.apache.org/ns/synapse"
clas...
Throttling – SAP/ESB/MB
๏ Same as above setup for reliable messaging, but
addresses a different concern.
๏ If messages are...
A Short Recoded Demo
31
Future Work
๏ Support for Message Queuing Telemetry Transport
(MQTT)
๏ Dynamic queue support in WSO2 MB
๏ CQL support inte...
Papers Published
๏ Perera H.S, Wickramarachchi C, Jayasinghe S.D,
Weeravarana S, *“Andes: a highly scalable persistent
mes...
Conclusion
๏ WSO2 MB 2.2.0 comes out as a more stabilized and
improved AMQP message broker with several high
demand featur...
35
More Information !
๏ WSO2 MB product download page:
http://wso2.com/products/message-broker/
๏ WSO2 MB documentation:
h...
36
Business Model
Contact us !
Upcoming SlideShare
Loading in …5
×

WSO2 Product Release webinar - WSO2 Message Broker 2.2.0

1,476 views

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,476
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
50
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

WSO2 Product Release webinar - WSO2 Message Broker 2.2.0

  1. 1. Last Updated: Jan. 2014 Senior Software Engineer Hasitha Abeykoon WSO2 Message Broker 2.2.0 Ishara Premadasa Software Engineer
  2. 2. 2 About the Presenter(s) ๏ Hasitha Hiranya Abeykoon, Senior Software Engineer, WSO2. ๏ Email: hasithah@wso2.com ๏ Blog: http://abeykoon.blogspot.com/ ๏ Ishara Premadasa, Software Engineer, WSO2. ๏ Email: ishara@wso2.com ๏ Blog: http://isharapremadasa.blogspot.com
  3. 3. 3 About WSO2 ๏ Global enterprise, founded in 2005 by acknowledged leaders in XML, web services technologies, standards and open source ๏ Provides only open source platform-as-a-service for private, public and hybrid cloud deployments ๏ All WSO2 products are 100% open source and released under the Apache License Version 2.0. ๏ Is an Active Member of OASIS, Cloud Security Alliance, OSGi Alliance, AMQP Working Group, OpenID Foundation and W3C. ๏ Driven by Innovation ๏ Launched first open source API Management solution in 2012 ๏ Launched App Factory in 2Q 2013 ๏ Launched Enterprise Store and first open source Mobile solution in 4Q 2013
  4. 4. 4 What WSO2 delivers
  5. 5. Outline 1. A Brief Introduction to WSO2 Message Broker 2. What is new in MB 2.2.0 3. Example Use Cases 4. Pre-recorded demo (ESB + MB Integration using Message Store Processor) 5. Conclusion and future work 5
  6. 6. Why Messaging? ๏ Enhanced reliability ๏ E.g. via persistence, transactions ๏ Decoupling ๏ Space ๏ Time ๏ Synchronization 6
  7. 7. Basic Messaging Models ๏ Messaging is implemented with a broker (or brokers in the middle) ๏ Participants send messages, and broker delivers them to recipients ๏ There are two main models ๏ Queues - A message is delivered only once to a single consumer. ๏ Publish/Subscribe: Broadcast a message to many message consumers 7
  8. 8. Queues 8 • A queue is a destination that contains messages sent from a producer that await delivery to one consumer. • Messages are delivered in the sent order. • A message is removed from the queue once it has been acknowledged as received by the consumer.
  9. 9. Topics 9 • Topics are used to send messages to one or more consumers. Producers publish messages to a topic and each consumer subscribed to the topic receives a copy of message • Brokers matches events (messages) and delivers to all interested parties • In this one-to-many messaging scenario, producers are also referred to as publishers and consumers as subscribers
  10. 10. Durable Topics 10 • Broker collects messages while subscriber is offline – like a queue • Broker distributes the messages to all interested parties – like a topic • Each subscriber has its own internal queue until it unsubscribes
  11. 11. AMQP, JMS and WSO2 MB ๏ JMS (Java Message Service) ๏ A specification that define a standard API for java programmer to perform messaging by interacting with a message broker ๏ It does not define the message format or how java API interacts with the message broker ๏ AMQP (Advanced Message Queuing Protocol) ๏ Open standard for passing business messages between applications or organizations. ๏ AMQP let different systems (e.g. .NET and Java) to interact with each other by agreeing on the message format at the wire level just like Web Services. ๏ WSO2 MB (Broker) ๏ Implements and supports JMS API using AMQP protocol for messaging (All the same, we are not limited to JMS) 11
  12. 12. How WSO2 MB is Different? ๏ Use Apache Cassandra as the storage for messages those require persistence Apache Cassandra is an open source distributed database management system. It is designed to handle very large amounts of data spread out across many commodity servers while providing a highly available service with no single point of failure. It has very fast search schemes built in. Thus Cassandra is a good choice for WSO2 MB with clustering support. ๏ Scaling – WSO2 MB can be clustered adding nodes to meet business requirements. Nodes can be dynamically added and removed ๏ Supports Distributed Queuing with clustering enabled 12
  13. 13. Why WSO2 MB is Different? ๏ Failover with no message lost – Clustering + Failover capability brings this feature ๏ Ability to handle large message transfers in real time As Cassandra is our storage, messages are written fast to store and we create only one copy of message while delivery ๏ High Availability and Fault Tolerance No single point of failure with MB cluster + Cassandra cluster + Zookeeper Cluster. All nodes are vertically and horizontally scalable. Cassandra keep replicates of messages. ๏ Interoperability with many languages / platforms via AMQP clients for Java, .Net, C, C++, PHP, Ruby, Erlang and more13
  14. 14. Why WSO2 MB is Different? ๏ Interoperability with many languages / platforms via AMQP clients for Java, .Net, C, C++, PHP, Ruby, Erlang and more We use AMQP internally. Thus any AMQP client is theoretically compliant ๏ Fair effort to scale in all 3 dimensions (reliability, performance and resources) 14
  15. 15. WSO2 MB 2.2.0 Release ๏ A lot of bug fixes with respect to WSO2 2.1.0 ๏ Fixed a lot of clustering related issues with edge cases. ๏ Stabilization of message delivery with high message concurrency. ๏ Concurrent issues arises with a Cassandra cluster. ๏ Four new features ๏ Dead Letter Channel implementation ๏ Publisher Flow control implementation ๏ MB profile support for Cassandra and Zookeeper ๏ JMS Message selectors support which was missing in 2.1.0 release 15
  16. 16. WSO2 MB 2.2.0 Release ๏ New features added to WSO2 MB management console UI ๏ View active queue subscriptions in whole MB cluster ๏ View active topic subscriptions in whole MB cluster ๏ View inactive durable topic subscriptions specifically. ๏ Ability to unsubscribe inactive durable topic subscriptions with UI. ๏ Ability to deal with messages at Dead Letter Channel. ๏ Improved reliability with deleting a queue (remove queue from whole cluster). ๏ Improved reliability with message count displayed for queues. ๏ Improved security and role based access model for queues and topics 16
  17. 17. WSO2 MB 2.2.0 Release ๏ Now it is even more easier to deploy WSO2 MB ๏ Tested ability to handle up to 3 million message burst from concurrent producers with simultaneous concurrent subscribers for queues. ๏ Improved logging – trace level logs are now separated for easy debugging. 17
  18. 18. Dead Letter Channel 18 • If the subscriber • Rejected the message • Did not ack the sever within a configured time • Could not receive message due to connection fail Server will try to resend the message. • MB server will do this for a maximum number of configured times. • After that it will route message to DLC.
  19. 19. Dead Letter Channel contd.. ๏ Able to view DLC messages in the Management Console (Queue vise display). ๏ You can perform following actions on DLC messages ๏ Delete (discard mal-formatted message) ๏ Restore message to the queue (fixed the message subscribe Back End. Now ready to go) ๏ Re-route the message to any other existing queue in MB cluster (allow some other subscriber in the system to handle the message) 19
  20. 20. Flow Control – Backpressure ๏ Every system has its limits. ๏ What happens to the broker cluster if a message burst continued to come and they need to be persisted…!! ๏ WSO2 MB 2.2.0 comes with following flow control mechanisms. 1. Throttling based on global memory threshold 2. Throttling based on message count per connection 20
  21. 21. MB Profile Support ๏ You can start WSO2 MB purely as a Apache Cassandra server. ๏ All configurations files of Cassandra are there ๏ Port is configurable with an offset ๏ Logs are written to Carbon Log ๏ Cassandra tools are also packed inside MB ๏ You can start WSO2 MB purely as a Apache Zookeeper server. ๏ To create a complete broker cluster you only need a WSO2 MB distribution. 21
  22. 22. Message Selectors ๏ A Subscriber might be interested in particular set of messages having certain properties. ๏ It only likes to consume those messages. ๏ When subscription is make to the broker, it can specify the message properties as a query. 22 queueReceiver= queueSession.createReceiver(queue,"(S1_double > 3.7 OR JMSType = 'TWO') AND Number > 20")
  23. 23. Other tit-bits Improved SSL Support for MB 2.2.0 ๏ The wire level connection between client-server can be encrypted using SSL to provide maximum protection. ๏ Supported in both standalone and clustered mode with failover as well. ๏ Easily configurable at the client side using AMQP connection url. 23
  24. 24. A Few Example Use cases ๏ WSO2 Message Broker is part of Messaging Middleware ๏ Rather than using WSO2 MB alone, most of the time for an integrated platform it sits in the middle of messaging clients where message delivery reliability is needed, or some message throttling is needed or some asynchronous message processing is OK. ๏ Following is some real world use cases where Message Broker comes into play. 24
  25. 25. Asynchronous Messaging ๏ Timely decoupled (transactions - inter departmental job submitting and processing) 25 University Grant Commission UniversityMessaging Middleware Web App Messaging
  26. 26. Broker Cluster with Failover 26 MB1 MB2 MB3 Connection = <mb1,mb2,mb3> Application MB1 MB Cluster Cassandra Cluster
  27. 27. Reliable Messaging ๏ Message store processor pattern with WSO2 ESB 27 Web Application Enterprise Service Bus Msg Store Msg Processor Data Services Server Oracle DB WSO2 MB Message Queue HTTP to JMS JMS to HTTP
  28. 28. Reliable Messaging contd.. ๏ Want to reliably store information to database? ๏ Message order is guaranteed ๏ Message reliability is guaranteed ๏ No message duplication is possible 28
  29. 29. Reliable Messaging contd.. ๏ Message Store Configuration for MB <messageStore xmlns="http://ws.apache.org/ns/synapse" class="org.wso2.carbon.message.store.persistence.jms.JMSMessageStore" name="JMSMS"> <parameter name="java.naming.factory.initial">org.wso2.andes.jndi.PropertiesFileInitialContextFactory</parameter> <parameter name="java.naming.provider.url">repository/conf/jndi.properties</parameter> </messageStore> ๏ Message Processor Configuration for MB <messageProcessor class="org.apache.synapse.message.processors.forward.ScheduledMessageForwardingProcessor" name="Processor1" messageStore="JMSMS"> <parameter name="max.delivery.attempts">4</parameter> <parameter name="interval">4000</parameter> </messageProcessor> 29
  30. 30. Throttling – SAP/ESB/MB ๏ Same as above setup for reliable messaging, but addresses a different concern. ๏ If messages are coming out from SAP very fast, we directly store it in a WSO2 MB queue via message store. ๏ Message processor can work slowly and process the messages with a tolerable speed. ๏ Alternatively, can use JMS proxies to send and receive messages as JMS. ๏ Doable with an ESB cluster as well. 30
  31. 31. A Short Recoded Demo 31
  32. 32. Future Work ๏ Support for Message Queuing Telemetry Transport (MQTT) ๏ Dynamic queue support in WSO2 MB ๏ CQL support integration over Hector to interact with Cassandra storage ๏ Support message priority levels 32
  33. 33. Papers Published ๏ Perera H.S, Wickramarachchi C, Jayasinghe S.D, Weeravarana S, *“Andes: a highly scalable persistent messaging system”+, Web Services (ICWS), 2012 IEEE 19th International Conference, Honolulu, HI 33
  34. 34. Conclusion ๏ WSO2 MB 2.2.0 comes out as a more stabilized and improved AMQP message broker with several high demand features like DLC and Flow Control implementation etc. The new version also includes a lot of bug fixes which makes it robust enough to reliably handle a bulk amount of messages with high concurrency in a clustered environment 34
  35. 35. 35 More Information ! ๏ WSO2 MB product download page: http://wso2.com/products/message-broker/ ๏ WSO2 MB documentation: https://docs.wso2.org/display/MB211/WSO2+Messag e+Broker+Documentation ๏ WSO2 MB 2.2.0 cluster guide: https://docs.wso2.org/display/CLUSTER420/Clustering +Message+Broker
  36. 36. 36 Business Model
  37. 37. Contact us !

×