Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Connecting applicationswitha mq

Overview of JBoss A-MQ

  • Login to see the comments

Connecting applicationswitha mq

  1. 1. Connecting Applications with Red Hat JBoss A-MQ Rob Davies Technical Director, Fuse September 2014
  2. 2. Rob Davies – Red Hat • Director, Engineering xPaaS • Technical Director of Fuse Engineering • Over 20 years experience of developing large scale solutions for telcos and finance • Creator of ActiveMQ and ServiceMix • Committer on open source projects, including Apache Camel … 2 RED HAT | Fuse Rocks! 2
  3. 3. Why use Message-Oriented Middleware ? • Robustness to change • Time Independence • Location Independence • Hide Latency • Scalability • Event driven • Simplicity • Configurable Quality of Service • Platform and Language Integration • Fault tolerant Process A Queue:Foo Process B Process 3 RED HAT | Fuse Rocks! 3 Broker Message in Message out after enrichment by Process B C Queue:Bar
  4. 4. • Top Level Apache Software Foundation Project • Wildly popular, high performance, reliable message broker • Connects to nearly everything •Native Java, C/C++, .Net, •AMQP 1.0, MQTT 3.1, STOMP (1.0-1.2) and OpenWire •STOMP enables Ruby, JS, Perl, Python, PHP, ActionScript … • Embedded and standalone deployment options • The key component of JBoss A-MQ! 4 RED HAT | Fuse Rocks! 4 What is Apache ActiveMQ ?
  5. 5. Messaging: The Basics 5 RED HAT | Fuse Rocks!
  6. 6. Message Channels and Routing • Message Channels Named communication between interested parties JMS calls them ‘Destinations’ • Can “tune-in” to multiple channels using wildcards • Can fine-tune message consumption with selectors • Can route a message based on content 6 RED HAT | Fuse Rocks! 6
  7. 7. Message Channels = JMS Destinations 7 RED HAT | Fuse Rocks! 7 Broker Consumer Consumer Destination WIDGET Destination ORDER Producer Consumer
  8. 8. Point-to-Point channel = JMS Queue 8 RED HAT | Fuse Rocks! 8 Consumer Producer Queue Consumer Consumer
  9. 9. Publish/Subscribe channel = JMS Topic 9 RED HAT | Fuse Rocks! 9 Producer Consumer Consumer Consumer Topic
  10. 10. Message Routing: Selectors 10 RED HAT | Fuse Rocks! 10 Producer Destination Consume Colorr = blue Consume Colorr = red
  11. 11. Message Routing: Destination Wildcards • * matches a subject • > matches sub-tree 11 RED HAT | Fuse Rocks! 11 Topic: BAR.BEER.WHITE Topic: BAR.WINE.WHITE Producer Consumer Topic:BAR.> Consumer Topic:BAR.*.WHITE Consumer Topic:BAR.WINE.WHITE Consumer Topic:BAR.BEER.WHITE
  12. 12. Message Delivery Mode • Messages can be persisted – allowing for decoupled applications • For Queues, any message delivered as PERSISTENT must be stored on long term storage before being delivered to a consumer • For Topics, a message delivered as PERSISTENT will only be stored if there exists a durable subscriber • NON-PERSISTENT messages may also be stored on disk if the memory limits of the broker have been reached 12 RED HAT | Fuse Rocks! 12
  13. 13. Messaging: Features every message broker should have … 13 RED HAT | Fuse Rocks!
  14. 14. Exclusive Consumers 14 RED HAT | Fuse Rocks! 14 Consumer Consumer Consumer Producer Queue
  15. 15. Message Groups • Like Exclusive Consumers – but in parallel • Guaranteed ordering of related messages across a Queue • But – load balancing of messages across multiple consumers • All messages with the same JMSXGroupID go to the same consumer • How you group messages is down to the application’s producer • To explicitly close a group, set the JMSXGroupSeq to -1 15 RED HAT | Fuse Rocks! 15
  16. 16. Message Groups Code example 16 RED HAT | Fuse Rocks! 16 //Starting a Group … Message message = session.createTextMessage("<foo>hi DevNation</foo>"); message.setStringProperty("JMSXGroupID", ”RHT_NYSE"); producer.send(message); //Close a Group … Message message = session.createTextMessage("<foo>bye DevNation</foo>"); message.setStringProperty("JMSXGroupID", ”RHT_NYSE"); message.setStringProperty("JMSXGroupSeq", -1); producer.send(message);
  17. 17. Message Groups 17 RED HAT | Fuse Rocks! 17 Producer Consumer Consumer Consumer Queue:Prices
  18. 18. Deployment • ActiveMQ Can run standalone or embedded • As a standalone, or part of a highly available message broker cluster • Embedded – easy to use an entire broker in JUnit tests (no need to Mock) • In Tomcat, deployed as a war • In JEE Server – either co-locate – or use client with JCA ActiveMQ distributions contain a rar for this purpose 18 RED HAT | Fuse Rocks! 18
  19. 19. JBoss A-MQ 19 RED HAT | Fuse Rocks! 19
  20. 20. Messaging: Persistence 20 RED HAT | Fuse Rocks!
  21. 21. Different Storage Options • SQL (JDBC) • KahaDB • LevelDB 21 RED HAT | Fuse Rocks! 21 Slow but popular The “current” default The “best yet”
  22. 22. ActiveMQ Message Storage 22 22 RED HAT | Fuse Rocks!
  23. 23. LevelDB vs KahaDB • Fewer index entries per message than KahaDB • Faster recovery when a broker restarts • LevelDB index out-perform Btree index at sequential access • LevelDB indexes support concurrent read access • Pause-less data log file garbage collection cycles • Fewer IOPS to load stored messages • It exposes it's status via JMX for monitoring 23 RED HAT | Fuse Rocks! 23
  24. 24. LevelDB vs KahaDB (cont.) 24 RED HAT | Fuse Rocks! 24
  25. 25. Messaging: Protocols 25 RED HAT | Fuse Rocks!
  26. 26. OpenWire Advantages: Disadvantages: 26 RED HAT | Fuse Rocks! 26 • Fast – optimized for ActiveMQ • client failover • automatic reconnect • Client load balancing • Flow control • Many advanced features • Not a recognized standard • Only Java, C/C++/.Net
  27. 27. MQTT Advantages: Disadvantages: 27 RED HAT | Fuse Rocks! 27 • “light weight” telemetry transport • Standard out of OASIS • Adoption by a large cross-section, from Facebook (Messenger) to embedded devices • Version 3.x only supports publish/subscribe • No advance features, such as flow control
  28. 28. AMQP Advantages: Disadvantages: 28 RED HAT | Fuse Rocks! 28 • AMQP 1.0 OSASIS standard • Feature rich messaging protocol • C/C++/Java clients and more in development • Currently, only client to broker supported in ActiveMQ
  29. 29. STOMP Advantages: Disadvantages: 29 RED HAT | Fuse Rocks! 29 • Easy to implement • Defacto standard: • ActiveMQ + Apollo • HornetQ • RabbitMQ • PocoMQ • StompServer • OpenMQ • Many more … • Not as fast a binary formats
  30. 30. Websockets • STOMP is a natural wire protocol for WebSockets • And so is MQTT! HTTP(S) Request/Response 30 RED HAT | Fuse Rocks! 30 ActiveMQ Client (Browser) WebSocket Connector Stomp Converter ActiveMQ (Backend) TCP (WebSocket)
  31. 31. Configuring protocols 31 RED HAT | Fuse Rocks! 31 <transportConnector uri = “tcp://0.0.0.0:61616”/> <transportConnector uri = “nio://0.0.0.0:61615”/> <transportConnector uri = “stomp+ssl://0.0.0.0:61617”/> <transportConnector uri = “mqtt+nio+ssl://0.0.0.0:1883”/> <transportConnector uri = “amqp+nio+ssl://0.0.0.0:5672”/> <transportConnector uri = “wss://0.0.0.0:61614”/>
  32. 32. High Availability 32 RED HAT | Fuse Rocks!
  33. 33. Master/Slave • Shared Message Store • Replicated LevelDB Master/Slave 33 RED HAT | Fuse Rocks! 33
  34. 34. Messaging: High Availability 34 RED HAT | Fuse Rocks!
  35. 35. Failover Supported by both Java and C++ OpenWire clients ActiveMQ Cluster ActiveMQ Client 35 RED HAT | Fuse Rocks!
  36. 36. High Availability with shared storage ActiveMQ Cluster ActiveMQ Client 36 RED HAT | Fuse Rocks!
  37. 37. Replicated LevelDB 37 RED HAT | Fuse Rocks! Registry (ZooKeeper) ActiveMQ Client
  38. 38. Messaging: Store and Forward (Networks) 38 RED HAT | Fuse Rocks!
  39. 39. Networks of Brokers Link Brokers together Use Store and Forward Are uni-directional by default All Destinations are global 39 RED HAT | Fuse Rocks! 39
  40. 40. Networks of Brokers 40 RED HAT | Fuse Rocks! 40
  41. 41. Store and Forward 41 RED HAT | Fuse Rocks! 41
  42. 42. Bi-directional network 42 RED HAT | Fuse Rocks! 42
  43. 43. Bi-directional Configuration <networkConnectors> <networkConnector name="backoffice" uri="static://(tcp://backoffice:61617)” duplex="true"> </networkConnector> </networkConnectors> 43 RED HAT | Fuse Rocks! 43 <networkConnectors> <networkConnector name="backoffice" uri="static://(tcp://backoffice:61617)” duplex="true"> </networkConnector> </networkConnectors>
  44. 44. Reducing cross-talk <networkConnectors> <networkConnector uri="static: (tcp://remote:61617) "/> <dynamicallyIncludedDestinations> <queue physicalName=”free.food.>"/> <queue physicalName=”free.beer.>"/> <topic physicalName=”cricket.scores.>"/> </dynamicallyIncludedDestinations> </networkConnectors> 44 RED HAT | Fuse Rocks! 44 <networkConnectors> <networkConnector uri="static: (tcp://remote:61617) "/> <dynamicallyIncludedDestinations> <queue physicalName=”free.food.>"/> <queue physicalName=”free.beer.>"/> <topic physicalName=”cricket.scores.>"/> </dynamicallyIncludedDestinations> </networkConnectors>
  45. 45. Scaling Networks 45 RED HAT | Fuse Rocks! 45 • Brokers share routing information across networks • All destinations are considered Global – easy to use • Though it starts to get problematic with 1000’s of brokers • However – we can do filtering to shape the traffic across networks • But this involves a lot of manual configuration Time to talk about Apache Camel …
  46. 46. ActiveMQ and Apache Camel 46 RED HAT | Fuse Rocks! 46 S e rvice C o nsum e r C o nsum e r A ctiveM Q B ro ke r
  47. 47. ActiveMQ Broker 47 RED HAT | Fuse Rocks! 47 Topics Queues Message Store JMX JaaS Advisor y schedul er
  48. 48. ActiveMQ Broker 48 RED HAT | Fuse Rocks! 48 Topics Queues Message Store JMX JaaS Advisor y schedul er Broker
  49. 49. ActiveMQ with embedded Camel <beans> <broker brokerName="testBroker" xmlns="http://activemq.apache.org/schema/core"> <transportConnectors> <transportConnector uri="tcp://localhost:61616"/> </transportConnectors> </broker> <import resource="camel.xml"/> </beans> 49 49 RED HAT | Fuse Rocks!
  50. 50. Setup Camel Context in the usual way <camelContext errorHandlerRef="errorHandler" xmlns=http://camel.apache.org/schema/spring> <route> <from uri="activemq:queue:test.queue"/> <choice> <when> <xpath>$foo = 'bar'</xpath> <to uri="activemq:topic:topic.bar"/> </when> <when> <xpath>$foo = 'cheese'</xpath> <to uri="activemq:topic:topic.cheese"/> </when> <otherwise> <to uri="activemq:topic:topic.all"/> </otherwise> </choice> </route> </camelContext> 50 50 RED HAT | Fuse Rocks!
  51. 51. Scaling Networks with Camel 51 RED HAT | Fuse Rocks! 51 • Allows for non-chatty networks to be established • Routing information can be externalized to the broker • Successfully used in production for 1000’s of brokers today
  52. 52. Messaging: More Problems with large deployments 52 RED HAT | Fuse Rocks!
  53. 53. Deployment and Maintenance 53 RED HAT | Fuse Rocks! 53 Main problems Installing brokers on multiple hosts ssh, untar, set directories and environment Setting configuration manually for every broker copying xml config, tweaking, testing Updating configuration across cluster Upgrading brokers
  54. 54. Best practices for ActiveMQ only 54 RED HAT | Fuse Rocks! 54 • Keep XML as a template and configure node-specific details through properties • Keep configuration in SVC system (git, svn, …) • Keep configuration separate from installation for easier upgrades
  55. 55. Clients 55 RED HAT | Fuse Rocks! 55 • Topology is very “static” • Clients need to be aware of topology • Clients need to know broker locations • Changes are not easy as clients need to be updated • Adding new resources (brokers) requires client updates • Not suitable for “cloud” deployments
  56. 56. Fabric8 to the rescue! 56 RED HAT | Fuse Rocks!
  57. 57. Messaging: Use cases 57 RED HAT | Fuse Rocks!
  58. 58. Not space - but close 58 RED HAT | Fuse Rocks! 58
  59. 59. Running the biggest machine on the planet 59 RED HAT | Fuse Rocks! 59
  60. 60. Messaging: The Internet of Things 60 RED HAT | Fuse Rocks!
  61. 61. The Internet of Things (IoT) 61 RED HAT | Fuse Rocks! 61 • Uniquely identifiable objects and their virtual representations in an internet-like structure • Originally defined by Kevin Ashton, co-founder of Auto-ID center at MIT • Today its meaning has evolved to encompass a world where physical objects are integrated into the information network, and where physical objects participate in business processes.
  62. 62. Telemetry 62 RED HAT | Fuse Rocks! 62 • M2M involves collection and transmission of event level data (used to be called telemetry) from intelligent devices • Machines can interact over the net – and/or attached networks – by wired and wireless networks • Payloads are typically minimal (temperature, pressure, location, metering etc.)
  63. 63. IoT predicted Growth 50 Billion 10 Billion RFID tags for Inventory control 1 Billion Ubiquitous positioning – locating people and everyday objects Geo-located devices Microcontrollers internet enabled 63 RED HAT | Fuse Rocks! 63 Vending machines, lone workers, POS, vehicles, planes, tolls, power systems, transport, telemedicine, home appliances, power systems, etc. Time Connected Devices on the internet Advanced sensors, Enterprises fully connected: integration of everything 2000 2010 2020
  64. 64. Collect IOT information Messaging Platform (Multi-protocol, Multi-platform) JBoss A-MQ Mobile wireless Medical Devices Industrial Machines Distribution/Deli very Vehicles Energy Monitoring Equipment Sensors MQTT AMQP 64 RED HAT | Fuse Rocks! Aggregate IOT data reliably Real-time notification Lightweight, embeddable messaging
  65. 65. Integrate IOT information Messaging Platform (Multi-protocol, Multi-platform) JBoss A-MQ IOT data Enterprise Service Bus (Mediate, Transform, Route, Connect) JBoss Fuse 65 RED HAT | Fuse Rocks! SaaS/Cloud application s Enterprise application s CRM application Unlock information, services, processes Lightweight, embeddable, enterprise integration pattern based platform
  66. 66. IOT – Red Hat JBoss solution 66 RED HAT | Fuse Rocks!
  67. 67. Messaging: IoT Demo 67 RED HAT | Fuse Rocks!
  68. 68. What is Arduino ? 68 RED HAT | Fuse Rocks! 68 • Open Source Hardware microcontroller • Cheap and easily available. • Open Source Software. • Widespread: many projects. • Extra HW (shields) available
  69. 69. USB 7-12 v 3 v GRD5 v GRD 69 69 RED HAT | Fuse Rocks! Transmitter/Receiver Serial Connection Microcontroller ATmega328 Operating Voltage 5V Input Voltage (recommended)7-12V Input Voltage (limits)6- 20V Digital I/O Pins 14 (of which 6 provide PWM output) Analog Input Pins 6 DC Current per I/O Pin 40 mA DC Current for 3.3V Pin 50 mA Digital Input/Output Pins Pins with ~ are PWM [Analog Output] Analog Input Pins
  70. 70. Arduino Programming 70 RED HAT | Fuse Rocks! 70 • 3 types of memory on Arduino Flash memory (program space) – 32 Kb SRAM – used by sketches – 2 Kb EPROM – long term memory – 1 Kb • Programming language based on Wiring: C/C++ library designed to make input/output easier. Programs are called sketches. • Arduino has a simple IDE, available on Windows, Linux and Mac
  71. 71. Arduino MQTT to WebSockets 71 RED HAT | Fuse Rocks! 71 Arduino AMQP MQTT ActiveMQ MQTT/AMQP Gateway ActiveMQ AMQP/WebSockets Broker HTML 5/WebSockets
  72. 72. QQuueessttiioonnss aanndd DDiissccuussssiioonn 72 RED HAT | Fuse Rocks!

×