Using OpenMQ
           Linda Schneider
           Technical Lead
           Sun Microsystems,
           Inc.
What will be covered ?
• An introduction to OpenMQ.
• A Customer example.
• Basic customer requirements.
• Building a piec...
What is OpenMQ ?



 Copyright Sun Microsystems Inc. All Rights Reserved.   [3]
Overview
• Allows heterogenous applications to reliability and
  asynchronously pass data between each other.
• Open Sourc...
Overview (cont.)
• Enterprise level quality (>8 years in development)
• Open Source since JavaOne 2006
• Available as a su...
What this looks like ....




 A set of loosely coupled applications talking through
  OpenMQ apis

          Copyright Su...
Ok – maybe a little about JMS
Some basic concepts of JMS:
• Producers and Consumers
• Destinations
• Consumer Types
• Reli...
What are Producers and Consumers?
• Producers
 > Create messages which are given to the “provider” (e.g.
   OpenMQ) to del...
So, what are destinations ?
• Named buckets to hold groups of messages.
• Messages are sent to “destinations”
 > Topics: e...
What are the types of Consumers ?
• Receivers are delivered messages from a Queue
• Subscribers are delivered messages fro...
And what does reliability mean ?
• Messages can be set to be persistent or non-
  persistent
 > Persistent messages are st...
Tell Me More
• Developer and User discussion forums
• Stable builds with product releases
• Early access, promoted builds ...
Using OpenMQ



Copyright Sun Microsystems Inc. All Rights Reserved.   [13]
An example
To mimic problems faced in designing applications, an
example:
• Represents a complex system with loosely
  con...
Our Example: Santa Claus, Inc
Why, you ask ?

• Even if you don't believe in Santa Claus, you must
  still understand that...
Overall System Requirements
Santa Claus, Inc. software applications need do the
  following:
• Handle gift selection and d...
What are we doing:
• Focusing on handling christmas gift processing
• Steps to design it include:
   > Determining the hig...
Defining the High Level
Operation of the System


   Copyright Sun Microsystems Inc. All Rights Reserved.   [18]
What do we need to do ?




       Copyright Sun Microsystems Inc. All Rights Reserved.   [19]
What do we need to do ? (cont.)




       Copyright Sun Microsystems Inc. All Rights Reserved.   [20]
How can Santa Do it ?




       Copyright Sun Microsystems Inc. All Rights Reserved.   [21]
What are the
    destinations ?


Copyright Sun Microsystems Inc. All Rights Reserved.   [22]
The Child Queue




      Copyright Sun Microsystems Inc. All Rights Reserved.   [23]
The Naughty/Nice Queues




      Copyright Sun Microsystems Inc. All Rights Reserved.   [24]
The Wrap Queue




      Copyright Sun Microsystems Inc. All Rights Reserved.   [25]
The Stuff to Pack Queue




       Copyright Sun Microsystems Inc. All Rights Reserved.   [26]
Select-a-gift Queues




       Copyright Sun Microsystems Inc. All Rights Reserved.   [27]
Topic LogChild




       Copyright Sun Microsystems Inc. All Rights Reserved.   [28]
A quick overview to
  design patterns


 Copyright Sun Microsystems Inc. All Rights Reserved.   [29]
Some basic design patterns:
Pattern                 Description
                        Message is sent to another applica...
More things to think about:
• Use persistent messages if it can not afford to be lost
• Use non-persistent messages for:
 ...
Processing Queue Child
• Conduit: many producers to one queue
• Persistent: would be time consuming to lose message




  ...
Processing Naughty and Nice
• Step Pattern: one step of it
• Naughty Queue: Non-
  Persistent
   > its OK if a child who i...
Processing Nice
• Step Pattern: more steps of it
• Multiple resources so XA




            Copyright Sun Microsystems Inc...
Processing Nice (Select a gift)
• Request/Reply Pattern
• Non-persistent
• Action repeated on failure




            Copy...
The Wrap Queue
• Step operation
• Persistent: end of an expensive set of steps that they
  don't want to repeat




      ...
The Log Child Topic
• Broadcast Pattern
• Persistent because santa wants his database
  accurate




           Copyright ...
In this example:
• The batch pattern was not used
   > Santa does use It for processing HR updates for the
     elves
• Th...
Performance Requirements
• 22 billion kids
• 364 days for preparation (since christmas is taken)
   > 31,526,000 seconds
•...
Performance Requirements (cont)
• Naughty Kids use
   > 1 Persistent queue (child)
   > 1 Non-persistent queue (naughty)
 ...
The cold hard requirements
• Messages:
  > Child: 100 msgs/second (persistent)
  > Naughty: 40 msgs/second (non-persistent...
Some Sample Code



 Copyright Sun Microsystems Inc. All Rights Reserved.   [42]
Sending the “Child” message
 public void doPost(HttpServletRequest request,
   HttpServletResponse response)       throws ...
Processing the “nice” queue
 public void onMessage(Message inMessage) {
     TextMessage msg = null;
     try {
         /...
Processing the “nice” queue (step 2)
 public String getListItem(String childid)
           throws Exception {
      QueueC...
Processing the “nice” queue (step 3)
 public String pack(String item, String child_id)
     throws JMSException
 {
     Qu...
You'll need to fill in the
          rest


   Copyright Sun Microsystems Inc. All Rights Reserved.   [47]
More Information



Copyright Sun Microsystems Inc. All Rights Reserved.   [48]
OpenMQ -- More Information
•Visit the product webpage
>http://sun.com/software/products/message_queue
•Join the Open Messa...
Related Information
•Java Composite Application Platform Suite
>http://sun.com/software/javaenterprisesystem/javacaps/
•Ja...
Thank You!


Using OpenMQ
Linda Schneider
linda.schneider@sun.com
Upcoming SlideShare
Loading in...5
×

Building With Open Mq V2

2,544

Published on

Upgraded version of the "Building with MQ" presentation by Linda Schneider

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,544
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
33
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Building With Open Mq V2

  1. 1. Using OpenMQ Linda Schneider Technical Lead Sun Microsystems, Inc.
  2. 2. What will be covered ? • An introduction to OpenMQ. • A Customer example. • Basic customer requirements. • Building a piece of the infrastructure. • Warnings: > No in depth coverage > Assumes basic JMS knowledge Copyright Sun Microsystems Inc. All Rights Reserved. [2]
  3. 3. What is OpenMQ ? Copyright Sun Microsystems Inc. All Rights Reserved. [3]
  4. 4. Overview • Allows heterogenous applications to reliability and asynchronously pass data between each other. • Open Source Java Message Service (JMS) implementation (+ additions) • Default Messaging Provider for Glassfish • Useful on its own for standalone JMS applications Copyright Sun Microsystems Inc. All Rights Reserved. [4]
  5. 5. Overview (cont.) • Enterprise level quality (>8 years in development) • Open Source since JavaOne 2006 • Available as a supported product: Sun Java System Message Queue (SJSMQ) Copyright Sun Microsystems Inc. All Rights Reserved. [5]
  6. 6. What this looks like .... A set of loosely coupled applications talking through OpenMQ apis Copyright Sun Microsystems Inc. All Rights Reserved. [6]
  7. 7. Ok – maybe a little about JMS Some basic concepts of JMS: • Producers and Consumers • Destinations • Consumer Types • Reliability Copyright Sun Microsystems Inc. All Rights Reserved. [7]
  8. 8. What are Producers and Consumers? • Producers > Create messages which are given to the “provider” (e.g. OpenMQ) to deliver. > Once the message has been handed off to the provider it goes off and does other processing. • Consumers > Are sent messages from the “provider” to process > Notify the provider when they are done with the message. Copyright Sun Microsystems Inc. All Rights Reserved. [8]
  9. 9. So, what are destinations ? • Named buckets to hold groups of messages. • Messages are sent to “destinations” > Topics: each messages goes to all consumers > Queues: a message goes to only one consumer Copyright Sun Microsystems Inc. All Rights Reserved. [9]
  10. 10. What are the types of Consumers ? • Receivers are delivered messages from a Queue • Subscribers are delivered messages from a Topic > Normal subscribers only receive messages sent while they are running > Messages for durable subscribers are maintained when the subscriber is not running Copyright Sun Microsystems Inc. All Rights Reserved. [10]
  11. 11. And what does reliability mean ? • Messages can be set to be persistent or non- persistent > Persistent messages are stored > Non-persistent messages are not and could be lost if a server crashed • Messages can be transacted or non-transacted > Transactions guarantees that a set of messages will be processed or not processed (not partially processed) if a server fails > XA transactions allow other resources (e.g. databases) to be processed in a group with messages. [Requires an Application Server] Copyright Sun Microsystems Inc. All Rights Reserved. [11]
  12. 12. Tell Me More • Developer and User discussion forums • Stable builds with product releases • Early access, promoted builds available > New features, and fixes • Dual license support (GPL v2 and CDDL) • Open source version of Java MQ is available from http://mq.dev.java.net Copyright Sun Microsystems Inc. All Rights Reserved. [12]
  13. 13. Using OpenMQ Copyright Sun Microsystems Inc. All Rights Reserved. [13]
  14. 14. An example To mimic problems faced in designing applications, an example: • Represents a complex system with loosely connected applications • Utilizes various types of messaging • Is easy to understand • Is at least minimally interesting Copyright Sun Microsystems Inc. All Rights Reserved. [14]
  15. 15. Our Example: Santa Claus, Inc Why, you ask ? • Even if you don't believe in Santa Claus, you must still understand that delivering all those presents would be a daunting task • And while its not Christmas which comes but once a year, but requires year round planning and preparation. • Just because Santa Claus lives at the North Pole, doesn't imply he can't use technology Copyright Sun Microsystems Inc. All Rights Reserved. [15]
  16. 16. Overall System Requirements Santa Claus, Inc. software applications need do the following: • Handle gift selection and delivery • Manage resources e.g. > gifts > reindeer > Elves • Track general status information > how many days before Christmas > etc. Copyright Sun Microsystems Inc. All Rights Reserved. [16]
  17. 17. What are we doing: • Focusing on handling christmas gift processing • Steps to design it include: > Determining the high level operation > Coming up with the name and type of destinations > Determining models used for the messaging > Determining load characteristics > Looking at code for some components Copyright Sun Microsystems Inc. All Rights Reserved. [17]
  18. 18. Defining the High Level Operation of the System Copyright Sun Microsystems Inc. All Rights Reserved. [18]
  19. 19. What do we need to do ? Copyright Sun Microsystems Inc. All Rights Reserved. [19]
  20. 20. What do we need to do ? (cont.) Copyright Sun Microsystems Inc. All Rights Reserved. [20]
  21. 21. How can Santa Do it ? Copyright Sun Microsystems Inc. All Rights Reserved. [21]
  22. 22. What are the destinations ? Copyright Sun Microsystems Inc. All Rights Reserved. [22]
  23. 23. The Child Queue Copyright Sun Microsystems Inc. All Rights Reserved. [23]
  24. 24. The Naughty/Nice Queues Copyright Sun Microsystems Inc. All Rights Reserved. [24]
  25. 25. The Wrap Queue Copyright Sun Microsystems Inc. All Rights Reserved. [25]
  26. 26. The Stuff to Pack Queue Copyright Sun Microsystems Inc. All Rights Reserved. [26]
  27. 27. Select-a-gift Queues Copyright Sun Microsystems Inc. All Rights Reserved. [27]
  28. 28. Topic LogChild Copyright Sun Microsystems Inc. All Rights Reserved. [28]
  29. 29. A quick overview to design patterns Copyright Sun Microsystems Inc. All Rights Reserved. [29]
  30. 30. Some basic design patterns: Pattern Description Message is sent to another application Request/Reply who sends back a response Messages go through several iterations, the message is persisted at key points where processing it again would be Step Operations expensive Broadcast One message goes to many consumers Multiple consumers send messages to a Conduit single destination Batch Messages are processed in a chunk Messages must be processed within a short period of time (e.g. under an hour) Time Critical/Sensitive and can not be lost Copyright Sun Microsystems Inc. All Rights Reserved. [30]
  31. 31. More things to think about: • Use persistent messages if it can not afford to be lost • Use non-persistent messages for: > Non-critical step messages (when it can be repeated > Request/Reply > Anytime a message can be lost on a server crash • Use durables for Topics when it may need to be retrieved later • Use normal or XA transactions when multiple things must process together: > XA if it includes other resources like databases Copyright Sun Microsystems Inc. All Rights Reserved. [31]
  32. 32. Processing Queue Child • Conduit: many producers to one queue • Persistent: would be time consuming to lose message Copyright Sun Microsystems Inc. All Rights Reserved. [32]
  33. 33. Processing Naughty and Nice • Step Pattern: one step of it • Naughty Queue: Non- Persistent > its OK if a child who is bad misses their coal • Nice Queue: Persistent. > They must get their present. Copyright Sun Microsystems Inc. All Rights Reserved. [33]
  34. 34. Processing Nice • Step Pattern: more steps of it • Multiple resources so XA Copyright Sun Microsystems Inc. All Rights Reserved. [34]
  35. 35. Processing Nice (Select a gift) • Request/Reply Pattern • Non-persistent • Action repeated on failure Copyright Sun Microsystems Inc. All Rights Reserved. [35]
  36. 36. The Wrap Queue • Step operation • Persistent: end of an expensive set of steps that they don't want to repeat Copyright Sun Microsystems Inc. All Rights Reserved. [36]
  37. 37. The Log Child Topic • Broadcast Pattern • Persistent because santa wants his database accurate Copyright Sun Microsystems Inc. All Rights Reserved. [37]
  38. 38. In this example: • The batch pattern was not used > Santa does use It for processing HR updates for the elves • The time sensitive/critical data pattern was not used: > Santa does use it during present delivery on christmas eve to track where he is • Because he has no time sensitive/critical data, reliability is important however data availability isn't for gift processing Copyright Sun Microsystems Inc. All Rights Reserved. [38]
  39. 39. Performance Requirements • 22 billion kids • 364 days for preparation (since christmas is taken) > 31,526,000 seconds • 70 children/second must be processed • Assume 60% are “nice” • Assume 40% downtime to cover outages and normal processing (so goal is approx 100 kids/second) Copyright Sun Microsystems Inc. All Rights Reserved. [39]
  40. 40. Performance Requirements (cont) • Naughty Kids use > 1 Persistent queue (child) > 1 Non-persistent queue (naughty) > 1 Persistent Topic (log child) • Nice Kids Use: > Persistent Queue (child) > Persistent Queue (nice) > 2 Non-Persistent Queues (Inventory request and reply queues) > Non-persistent queue (Wrap) > 1 Persistent Topic (log child) Copyright Sun Microsystems Inc. All Rights Reserved. [40]
  41. 41. The cold hard requirements • Messages: > Child: 100 msgs/second (persistent) > Naughty: 40 msgs/second (non-persistent) > Log Child: 100msgs/second (persistent) > Nice: 60 msgs/second (persistent in XA transaction) > Inventory request/reply: 60 msgs/second *2 (non- persistent) > Wrap: 60 msgs/second (persistent) • TOTALS: > Persistent: 380 msgs/second > Non-persistent: 160 msgs/second Copyright Sun Microsystems Inc. All Rights Reserved. [41]
  42. 42. Some Sample Code Copyright Sun Microsystems Inc. All Rights Reserved. [42]
  43. 43. Sending the “Child” message public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { // retrieve initial context (ic) QueueConnectionFactory qcf = (QueueConnectionFactory) ic.lookup(quot;MyConnectionFactoryquot;); Queue destQueue = (Queue)ic.lookup(quot;Childquot;); QueueConnection connection = qcf.createQueueConnection(); try { QueueSession session = connection.createQueueSession( False, Session.AUTO_ACKNOWLEDGE); QueueSender sender = session.createSender(destQueue); MapMessage msg = session.createMapMessage(); // Set each item msg.setString(“firstname”, request.getParameter(“firstname”)); // … retrieve other properties …; sender.send(msg); } finally { connection.close(); } } Copyright Sun Microsystems Inc. All Rights Reserved. [43]
  44. 44. Processing the “nice” queue public void onMessage(Message inMessage) { TextMessage msg = null; try { //Message is of type text and has a unique child id msg = (TextMessage) inMessage; String id = msg.getText(); String[] list = db.getList(id); // makes SQL call String item = null; if (list == null) { // no list, send request String item = getListItem(); //next slide } else { item = list[0]; } //update inventory db.updateInventory(item, id);//makes SQL call // put on packing list pack(item, id); } catch (Exception e) { // things went wrong roll back mdc.setRollbackOnly(); } } Copyright Sun Microsystems Inc. All Rights Reserved. [44]
  45. 45. Processing the “nice” queue (step 2) public String getListItem(String childid) throws Exception { QueueConnectionFactory factory = jndiContext.lookup(“MyQueueFactory”); QueueConnection qc = factory.createQueueConnection(); qc.start(); QueueSession session = qc.createSession(true, Session.AUTO_ACKNOWLEDGED); Queue q = session.createQueue(“RandomPresent”); Queue reply = session.createTemporaryQueue(); // get sender and receiver QueueSender sender = session.createSender(q); QueueReceiver receiver = session.createReceiver(q); //send message and wait TextMessage m = session.createTextMessage(childid); m.setJMSReplyTo(reply); //send the message sender.send(m); TextMessage back = (TextMessage) receiver.receive(60*1000); // wait a minute if (back == null) { didn't get anything throw new Exception(“Nothing”); return back.getText(); } Copyright Sun Microsystems Inc. All Rights Reserved. [45]
  46. 46. Processing the “nice” queue (step 3) public String pack(String item, String child_id) throws JMSException { QueueConnectionFactory factory = jndiContext.lookup(“MyQueueFactory”); QueueConnection qc = factory.createQueueConnection(); QueueSession session = qc.createSession(true, Session.AUTO_ACKNOWLEDGED);); Queue q = session.createQueue(“Pack”); // get sender QueueSender sender = session.createSender(q); //send message MapMessage m = session.createMapMessage(childid); m.setString(“child_id”, child_id); m.setString(“present”, item); sender.send(m); } Copyright Sun Microsystems Inc. All Rights Reserved. [46]
  47. 47. You'll need to fill in the rest Copyright Sun Microsystems Inc. All Rights Reserved. [47]
  48. 48. More Information Copyright Sun Microsystems Inc. All Rights Reserved. [48]
  49. 49. OpenMQ -- More Information •Visit the product webpage >http://sun.com/software/products/message_queue •Join the Open Message Queue project >https://mq.dev.java.net •Browse the product documentation >http://docs.sun.com/app/docs/coll/1307.3 •Take the free technical training >http://www.sun.com/training/catalog/courses/WMT-SMQ-1491.xml Copyright Sun Microsystems Inc. All Rights Reserved. [49]
  50. 50. Related Information •Java Composite Application Platform Suite >http://sun.com/software/javaenterprisesystem/javacaps/ •Java System Identity Manager >http://sun.com/software/products/identity •Project GlassFish >https://glassfish.dev.java.net/ •The Aquarium, A community forum >http://blogs.sun.com/theaquarium/ Copyright Sun Microsystems Inc. All Rights Reserved. [50]
  51. 51. Thank You! Using OpenMQ Linda Schneider linda.schneider@sun.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×