Better Living Through Messaging -
Leveraging the HornetQ Message
        Broker at Shopzilla

        Mark Lui ( Merchant Services)
                    and
           Josh Long (Inventory)



                     - 1-
What is Messaging




                    - 2-
What is Messaging?

  Store and Forward
  Enables asynchronous communication
    • Reduces latency for information transfer




                                      - 3-
What is Messaging?

  Loosely coupled distributed communication (Data or Command)
  Exchange of messages between software components
  Two forms
    • Point-to-Point (Queue)
    • Publish/Subscribe (Topic)




                                  - 4-
Enterprise Application Integration

  Integration of data and services between applications
  Communication between information silos
  Approaches
    • File Transfer
    • Shared Database
    • Remote Procedure Call
    • Messaging




                                - 5-
Integration Patterns

  Patterns
    • Data Synchronization
    • Workflow
    • Portal




                             - 6-
Enterprise Application Integration

•   Approaches integration using messaging
•   Enables Event Driven Architecture
•   Multiple users of information




                                        - 7-
Message Queue Features

  Reliable
  Fast

   …
  Secure
  Duplicate message detection
  Flow control – Slow/Fast consumer handling
  Wild card destinations
  Durable subscription
  Scheduled Delivery



                                 - 8-
Message Queue Features

  Redelivery handling
    • DLQ/Discard
    • Delay redelivery

  Message Ordering on Topics
  Client side load balancing options
    • Random
    • Round Robin

  Message prioritization




                                 - 9-
Introducing HornetQ

  New ultra high performance enterprise grade messaging system
   from JBoss
  Apache V2 License
  HornetQ 2.0.GA obtained scores up to 307% higher than
   previously published SPECjms2007 benchmark results
  “Throughput of 800K small non persistent messages per second
   on a single 4 x 2.5GHz core server” – Tim Fox




                              - 10 -
HornetQ – Secret Sauce

  LibAIO
    • Linux Kernel 2.6 Subsystem
    • Able to submit I/O requests without waiting for completion

  HornetQ uses an append-only journal using LibAIO
  ActiveMQ 6 will support LibAIO 




                                      - 11 -
ActiveMQ Issues

  Difficult to configuration correctly
  Prone to hang
  Poor persistence options
    • KahaDB – better scalability, shorter recovery time
    • AMQ Message Store – faster
    • JDBC database – too slow




                                      - 12 -
High Availability - About Resiliency

  Active/Passive replication
  HornetQ Example




                                - 13 -
High Availability - About Resiliency

  Live Server




                           - 14 -
High Availability - About Resiliency

  Backup Server




                           - 15 -
Clustering - About Scalability

  Client side load balancing




                                - 16 -
About performance

  Offer Bidding Service Performance Data




                              - 17 -
Spring Framework

  Connection Options




                        - 18 -
Spring Framework

  Alternative Connection Options




                               - 19 -
Spring Framework

  JMS Template




                   - 20 -
Spring Framework

  Message Driven POJOs : MessageListener




                            - 21 -
Spring Framework

  Message Driven POJOs : MessageListener




                            - 22 -
Spring Framework

  Message Driven POJOs : SessionAwareMessageListener




                            - 23 -
Spring Framework

  Message Driven POJOs : SessionAwareMessageListener




                            - 24 -
Spring Framework

  Spring Integration




                        - 25 -
Spring Framework

  Spring Integration




                        - 26 -
Messaging at Shopzilla

  Merchant Service
    • Offer Bidding Service – Using to scale

  Inventory
    • Offer Service
      Dampening request to Coherence

      Dampening request to Oracle for write behind




                                           - 27 -
Merchant Service’s Order Bidding Service

  Demo




                         - 28 -
Inventory’s Offer Service

  Demo




                            - 29 -
HornetQ – Lessons Learned

  Use the right Connection Factory
  Use Spring 2.5.6 or 3.0
  For High Availability disable Spring reconnection logic
  Be aware of the High Availability restart sequence
  Paging is turned off by default
  JMX is your friend




                                 - 30 -
Possible Future Opportunities

  SLA compliance
  Unified messaging
  Real-time, event driven architectures




                                - 31 -
Conclusion

  Q and A?
  Don't forget to tip your waiter!




                                      - 32 -

Messaging sz

  • 1.
    Better Living ThroughMessaging - Leveraging the HornetQ Message Broker at Shopzilla Mark Lui ( Merchant Services) and Josh Long (Inventory) - 1-
  • 2.
  • 3.
    What is Messaging?  Store and Forward  Enables asynchronous communication • Reduces latency for information transfer - 3-
  • 4.
    What is Messaging?  Loosely coupled distributed communication (Data or Command)  Exchange of messages between software components  Two forms • Point-to-Point (Queue) • Publish/Subscribe (Topic) - 4-
  • 5.
    Enterprise Application Integration  Integration of data and services between applications  Communication between information silos  Approaches • File Transfer • Shared Database • Remote Procedure Call • Messaging - 5-
  • 6.
    Integration Patterns Patterns • Data Synchronization • Workflow • Portal - 6-
  • 7.
    Enterprise Application Integration • Approaches integration using messaging • Enables Event Driven Architecture • Multiple users of information - 7-
  • 8.
    Message Queue Features  Reliable  Fast …  Secure  Duplicate message detection  Flow control – Slow/Fast consumer handling  Wild card destinations  Durable subscription  Scheduled Delivery - 8-
  • 9.
    Message Queue Features  Redelivery handling • DLQ/Discard • Delay redelivery  Message Ordering on Topics  Client side load balancing options • Random • Round Robin  Message prioritization - 9-
  • 10.
    Introducing HornetQ New ultra high performance enterprise grade messaging system from JBoss  Apache V2 License  HornetQ 2.0.GA obtained scores up to 307% higher than previously published SPECjms2007 benchmark results  “Throughput of 800K small non persistent messages per second on a single 4 x 2.5GHz core server” – Tim Fox - 10 -
  • 11.
    HornetQ – SecretSauce  LibAIO • Linux Kernel 2.6 Subsystem • Able to submit I/O requests without waiting for completion  HornetQ uses an append-only journal using LibAIO  ActiveMQ 6 will support LibAIO  - 11 -
  • 12.
    ActiveMQ Issues Difficult to configuration correctly  Prone to hang  Poor persistence options • KahaDB – better scalability, shorter recovery time • AMQ Message Store – faster • JDBC database – too slow - 12 -
  • 13.
    High Availability -About Resiliency  Active/Passive replication  HornetQ Example - 13 -
  • 14.
    High Availability -About Resiliency  Live Server - 14 -
  • 15.
    High Availability -About Resiliency  Backup Server - 15 -
  • 16.
    Clustering - AboutScalability  Client side load balancing - 16 -
  • 17.
    About performance Offer Bidding Service Performance Data - 17 -
  • 18.
    Spring Framework Connection Options - 18 -
  • 19.
    Spring Framework Alternative Connection Options - 19 -
  • 20.
    Spring Framework JMS Template - 20 -
  • 21.
    Spring Framework Message Driven POJOs : MessageListener - 21 -
  • 22.
    Spring Framework Message Driven POJOs : MessageListener - 22 -
  • 23.
    Spring Framework Message Driven POJOs : SessionAwareMessageListener - 23 -
  • 24.
    Spring Framework Message Driven POJOs : SessionAwareMessageListener - 24 -
  • 25.
    Spring Framework Spring Integration - 25 -
  • 26.
    Spring Framework Spring Integration - 26 -
  • 27.
    Messaging at Shopzilla  Merchant Service • Offer Bidding Service – Using to scale  Inventory • Offer Service  Dampening request to Coherence  Dampening request to Oracle for write behind - 27 -
  • 28.
    Merchant Service’s OrderBidding Service  Demo - 28 -
  • 29.
  • 30.
    HornetQ – LessonsLearned  Use the right Connection Factory  Use Spring 2.5.6 or 3.0  For High Availability disable Spring reconnection logic  Be aware of the High Availability restart sequence  Paging is turned off by default  JMX is your friend - 30 -
  • 31.
    Possible Future Opportunities  SLA compliance  Unified messaging  Real-time, event driven architectures - 31 -
  • 32.
    Conclusion  Qand A?  Don't forget to tip your waiter! - 32 -