• Save
Embedded Java and MQTT
Upcoming SlideShare
Loading in...5
×
 

Embedded Java and MQTT

on

  • 2,038 views

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/15sQGei. ...

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/15sQGei.

Peter Niblett explains what MQTT is and how it compares with HTTP, showing how to program to it in Java and Eclipse Paho, and reporting on the current MQTT standardization status at OASIS. Filmed at qconnewyork.com.

Peter Niblett is an IBM Senior Technical Staff Member, responsible for the architecture and design of IBM's Messaging products. Peter was one of the original designers of the Java Message Service (JMS) programming interface, and chaired the OASIS Technical Committee that developed the Web Services Notification standard.

Statistics

Views

Total Views
2,038
Views on SlideShare
2,037
Embed Views
1

Actions

Likes
10
Downloads
0
Comments
0

1 Embed 1

http://localhost 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Embedded Java and MQTT Embedded Java and MQTT Presentation Transcript

  • 1 © 2013 IBM Corporation MQTT and Java Peter Niblett IBM Senior Technical Staff Member OASIS MQTT Technical Committee Eclipse M2M Working Group
  • InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /embedded-java-mqtt
  • Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  • 22 © 2013 IBM Corporation Mobile is a Prelude to the “Internet of Things”
  • 3 © 2013 IBM Corporation Agenda What is MQTT? – What is it for? – How does it compare with HTTP? – What's its relationship with JMS and WebSocket? Details of MQTT – the protocol – Protocol features – Example data flows Developing with MQTT – What you need – Java API walkthrough
  • 44 © 2013 IBM Corporation An Open Approach to Connectivity for Mobile, M2M and IoT MQTT is a lightweight publish/subscribe protocol with reliable bi-directional message delivery Lossy or Constrained Network Lossy or Constrained Network Monitoring & Analytics Server Real-World Aware Business Processing High volumes of data/events IT Systems In this arena, open source and standards are essential 1999 Invented by Dr. Andy Stanford-Clark (IBM), Arlen Nipper (now Cirrus Link Solutions) 1999 Invented by Dr. Andy Stanford-Clark (IBM), Arlen Nipper (now Cirrus Link Solutions) 2011 - Eclipse PAHO MQTT open source project 2011 - Eclipse PAHO MQTT open source project 2004 MQTT.org open community2004 MQTT.org open community 2013 – MQTT Technical Committee formed 2013 – MQTT Technical Committee formed Cimetrics, Cisco, Eclipse, dc-Square, Eurotech, IBM, INETCO Landis & Gyr, LSI, Kaazing, M2Mi, Red Hat, Solace, Telit Comms, Software AG, TIBCO, WSO2 Evolution of an open technology
  • 55 © 2013 IBM Corporation Sensors ActuatorsEmbedded Controllers Sense and ControlVisualise and Respond Intelligence and Analytics Messaging server Edge Gateways Mobile Web Systems of Record BigData Sense Data/Alert Respond ControlM2M What’s MQTT for? MQTT
  • 6 © 2013 IBM Corporation Volume (cost) of data being transmitted (especially in M2M with limited data plans) Power consumption (battery powered devices) Responsiveness (near-real time delivery of information) Reliable delivery over fragile connections Security and privacy Scalability Some challenges facing Mobile and M2M developers
  • 7 © 2013 IBM Corporation Original MQTT design principles ■ Expect and cater for frequent network disruption – built for low bandwidth, high latency, unreliable, high cost networks ■ Expect that client applications may have very limited resources available. ■ Minimal the on-the-wire footprint. ■ Publish/subscribe messaging paradigm as required by the majority of SCADA and sensor applications. ■ Provide traditional messaging qualities of service where the environment allows. ■ Publish and standardize the protocol for ease of adoption by device vendors and third-party client software.
  • 8 © 2013 IBM Corporation Central Systems ● Monitoring - temp, pressure... ● Control - valves… 4000 devices integrated, need to add 8000 more BUT: • Satellite network saturated due to polling of device • VALMET system CPU at 100% • Other applications needed access to data ("SCADA prison") Proprietary polling protocol Billing Maintenance SCADA low-bandwidth, expensive comms Real Life - Pipeline integration challenges
  • 9 © 2013 IBM Corporation Central Systems Billing Maintenance SCADA low-bandwidth, expensive comms Scalability for whole pipeline! Network traffic much lower - events pushed to/from devices and report by exception Network cost reduced Lower CPU utilization Broken out of the SCADA prison – data accessible to other applications Message Broker pub sub transformation Enterprise MessagingMQTT 20 Field Devices to 1 Concentrator Enterprise to physical world solution with MQTT
  • 1010 © 2013 IBM Corporation Industry Examples of use Automotive Connected Cars Healthcare Medical Devices Home monitoring Transport & Supply Chain Asset tracking Ticketing Oil and Gas Pipeline management Energy & Utilities Smart metering and demand management Manufacturing Process Control Finance Customer alerting Telecomms Generic push notification M2M services Mobile apps Facebook Messenger Some areas where MQTT is being used
  • 11 © 2013 IBM Corporation And Home Hackers as well
  • 1212 © 2013 IBM Corporation MQTT serverMQTT serverMQTT serverMQTT server Reliable asynchronous transactions User submits a transaction. One or more responses may come back over time. MQTT provides reliability and store/forward of requests and responses if needed – reducing the amount of application code Continuous update of real-time information Server-side data is “streamed” to the device and used to update the UI. In most cases this is only required when the app is in the foreground Small MQTT header size reduces battery consumption and network traffic. One->many publish/subscribe reduces load on application Push Notification Sending alert or other informational message to the device. The app may or may not be running at the time. Avoidance of polling reduces battery consumption and network traffic. Store/forward of important notifications if app/device is not contactable Collection of data from device Data sent to the server coming either from User Interface, of from onboard sensors or from devices attached to the phone Small MQTT header size reduces battery consumption and network traffic. Store/forward of messages. One->many publish/subscribe Mobile Usage patterns MQTT
  • 13 © 2013 IBM Corporation • The HTTP standard revolutionized how we consume data ‒ A single simple model: Send a request, read the response ‒ Available via any tablet, laptop, phone, PC etc. ‒ Good for requesting data from a known source • However, HTTP wasn’t designed for mobile use • MQTT provides support for transactions and an event-oriented paradigm, engineered around wireless: Reliably and securely completing mobile business transactions over unreliable networks Pushing information over unreliable networks Transmitting information one to many Listening for events whenever they happen Distributing minimal packets of data in huge volumes Why isn't HTTP enough?
  • 14 © 2013 IBM Corporation MQTT compared with... Java Message Service (JMS) – JMS is a fully-featured messaging API designed for Enterprise apps. It contains additional features that aren't included in MQTT • Rich set of message headers • Explicit Map and Stream message body types • Rollback transactional capabilities • Explicit point-point domain – JMS and MQTT can inter-operate via a shared topic space (IBM implementations do this).  WebSocket – Provides basic (socket like) bidirectional comms to web apps. – MQTT can run over a WebSocket to provide reliable delivery and publish/subscribe to HTML5 apps in a manner that • Reduces the coding required by the application developer • Is inter-operable with other channels and applications
  • 1515 © 2013 IBM Corporation A producer publishes a message (publication) on a topic (subject) A consumer subscribes (makes a subscription) for messages on a topic (subject) A message server matches publications to subscriptions If none of them match the message is discarded If one or more matches the message is delivered to each matching consumer Publish / Subscribe has three important characteristics: 1. It decouples message senders and receivers, allowing for more flexible applications 2. It can take a single message and distribute it to many consumers 3. This collection of consumers can change over time, and vary based on the nature of the message. Publish / Subscribe Messaging (One to Many)
  • 16 © 2013 IBM Corporation MQTT Topics All subscriptions are to a topic space All messages are published to an individual topic Topic names are hierarchical – Levels separated by “/” – Single-level wildcards “+” can appear anywhere in the topic string – Multi-level wildcards “#” must appear at the end of the string • Wildcards must be next to a separator • Can't use wildcards when publishing MQTT topic names can be 64KB long Fruit Grape Red White Fruit/# Fruit/Grape/+ Fruit/+/Red
  • 17 © 2013 IBM Corporation Agenda What is MQTT? – What is it for? – How does it compare with HTTP? – What's its relationship with JMS and WebSocket? Details of MQTT – the protocol – Protocol features – Example data flows Developing with MQTT – What you need – Java API walkthrough
  • 18 © 2013 IBM Corporation MQTT Protocol Deep Dive - Headers MQTT protocol flows: – Fixed header (2 bytes) – Variable header (optional, length varies) – Message payload (optional, length encoded, up to 256MB) Fixed header indicates the Message Packet type, the length of the payload and Quality of Service Variable header contents depends on the Message Packet type – Message ID, Topic name and so on. Fixed Variable Payload
  • 19 © 2013 IBM Corporation A 2 byte flow? What can we do with that?! Each bit in each byte is important! Disconnect and pings only need the fixed header Remaining length allows for a 256MB payload – Use 1 byte for up to 127 bytes, – 2 bytes for up to 16383 bytes – Max. 4 bytes = 256MB
  • 20 © 2013 IBM Corporation MQTT API Flows Most flows have a corresponding acknowledgment Connect – Can restart a previous session – Can specify a “Last Will and Testament” message and topic Subscribes can specify multiple topics Publish flows – Flow depends on QoS level – Sent from client → server to publish a message, or – Server → client to send messages Connection Management CONNECT CONNACK DISCONNECT PINGREQ PINGRESP Subscription Management SUBSCRIBE SUBACK UNSUBSCRIBE UNSUBACK Message Delivery PUBLISH PUBACK PUBREC PUBREL PUBCOMP
  • 21 © 2013 IBM Corporation Qualities of Service QoS 0: At most once delivery (non- persistent) – No retry semantics are defined in the protocol. – The message arrives either once or not at all. QoS 1: At least once delivery (persistent, dups possible) – Client sends message with Message ID in the message header – Server acknowledges with a PUBACK control message – Message resent with a DUP bit set If the PUBACK message is not seen QoS 2: Exactly once delivery (persistent) – Uses additional flows to ensure that message is not duplicated – Server acknowledges with a PUBREC control message – Client releases message with a PUBREL control message – Server acknowledges completion with a PUBCOMP control message
  • 22 © 2013 IBM Corporation MQTT Keep Alive Protocol includes support for client and server to detect failed connections – At connection time, a keep alive can be specified If client does not send a PINGREQ request to the server, the server assumes the connection has failed. If client does not receive a PINGRESP after the keep alive time has passed after sending the request, the connection is failed. Maximum keep alive interval of 18 hours. – Can specify a value of 0 to disable keep alive ??
  • 23 © 2013 IBM Corporation MQTT Last Will & Testament During connection, a Will message and topic can be specified – Abnormal disconnections will cause WMQ to publish the message – Clean disconnects will not cause the message to publish Can set the message as retained – Message is published to a subscriber when registering Useful to report the connection status of the client – Will message is a retained “down” – Upon connecting, client publishes a retained “up” message.
  • 24 © 2013 IBM Corporation The life of a MQTT client MQTT serverCONNECT SUBACK SUBSCRIBE PINGREQ CONNACK PINGRESP Has a subscriber connected on a topic Is connected, and is awaiting messages Is the connection still active? Yes!
  • 25 © 2013 IBM Corporation The life of a MQTT client (2) MQTT Server PUBLISH PUBLISH PUBREC Send some important messages Send some low importance messages PUBREL PUBCOMP DISCONNECTI'm done!
  • 26 © 2013 IBM Corporation Agenda What is MQTT? – What is it for? – How does it compare with HTTP? – What's its relationship with JMS and WebSocket? Details of MQTT – the protocol – Protocol features – Example data flows Developing with MQTT – What you need – Java API walkthrough
  • 27 © 2013 IBM Corporation MQTT server options  Commercial ‒ IBM WebSphere MQ ‒ IBM IBM MessageSight (Appliance) ‒ Software AG my-Channels Nirvana ‒ dcSquare HiveMQ  Open Source or Free download ‒ Mosquitto (mosquitto.org) ‒ RSMB (IBM developerWorks) ‒ ActiveMQ (Apache) ‒ RabbitMQ (vmWare) ‒ mqtt.js , eMQTT (GitHub)  Web-hosted ‒ Eclipse Paho (implemented using Mosquitto) ‒ Everywhere Device Cloud ‒ m2m.io
  • 28 © 2013 IBM Corporation MQTT Clients and APIs The published wire format is useful when implementing very lightweight sensors However, a prebuilt client library simplifies MQTT programming – No need to know the bits and bytes flowing over the wire – No need to code thread management for background processing – Makes code more maintainable Open Source clients available in Eclipse Paho project – Java, C and JavaScript Clients for other languages are available, see mqtt.org/software – C++, Delphi, Erlang, Lua, .Net, Objective-C, PERL, PHP, Python, Ruby
  • 29 © 2013 IBM Corporation Eclipse Paho Java APIs for MQTT MqttClient - “Synchronous” or “blocking” API – This was the original API – Some processing is done on background threads, but most calls block until their processing is complete – Slightly simpler to program to than the Asynch API MqttAsyncClient – New “Asynchronous” API – Better fit to asynchronous environments, e.g. Android – All significant processing is done on background threads The synchronous client is actually implemented as a thin layer on top of the Asynchronous one Both interfaces are included in the same Jar file – org.eclipse.paho.client.mqttv3.jar
  • 30 © 2013 IBM Corporation The synchronous Java API  MqttClient object is the starting point  Incoming messages are received via callbacks  Messages can be published synchronously or asynchronously  Local persistence stores used to for QoS 1 and 2 – Persistence store implementation can be replaced MqttClient subscribe MqttConnectOptions MqttCallback MqttClientPersistence MqttTopic setCallBack MqttMessage connect getTopic publish publish
  • 31 © 2013 IBM Corporation Synchronous API : Connecting  The first step is to create an MqttClient object. This represents the connection of an application to the MQTT server – Specify details of the server (in a URI format), a ClientID and optionally a persistence implementation – The ClientID must be unique for the server that it is connecting to  Then specify connection options, and connect. In this example: – Keep alive of 480 seconds – A retained publication Will message of QoS 1
  • 32 © 2013 IBM Corporation Synchronous API : Sending a Message  The Message object contains a payload and a few message properties, such as the QoS level. – The payload is always a byte array, though this might be an encoded string  Messages can be sent synchronously, using the MqttClient object – This call does not complete until all acknowledgements (if any) have been received:  Messages can be sent asynchronously, using an MqttTopic object: Both forms are overloaded so you don't have to create an MqttMessage object first
  • 33 © 2013 IBM Corporation Synchronous API : Subscriptions After connecting, subscribe by providing the topic string: You can subscribe to multiple topics at the same time, and provide QoS levels: Messages will then be delivered to the messageArrived callback Call unsubscribe to stop getting messages. Again, you can optionally provide a list of containing multiple topics:
  • 34 © 2013 IBM Corporation Asynchronous MQTT API To use the new non-blocking API, you use an MqttAsyncClient object instead of using MqttClient – In this class the methods for connect, disconnect, publish, subscribe and unsubscribe all operate asynchronously – As you will see, it's quite easy to achieve the effect of a blocking call using the new MqttAsyncClient. We therefore recommend that you use MqttAsyncClient in place of MqttClient You construct the MqttAsyncClient and set callbacks just like you do for the MqttClient:
  • 35 © 2013 IBM Corporation Asynchronous MQTT API  MqttAsyncClient provides two asynchronous invocation styles  In the first style, each call looks like its MqttClient counterpart except that it returns a token, which you can then use to track completion of the operation: – You can register a listener callback against the token which will get called when the operation completes: – You can block on the token waiting for its completion, which in effect makes the call into a blocking call:  In the second style you supply the listener callback as a parameter on the method invocation itself.
  • 36 © 2013 IBM Corporation Asynchronous API: Listener callbacks  The ActionListener contains two callback methods, one for Success and one for Failure:  Applications may pass application-specific context in the iMqttToken which can then be used by the callback method implementations
  • 37 © 2013 IBM Corporation Receiving messages – both APIs Messages are delivered to the app via a callback mechanism – Callbacks are also used to indicate when the connection has broken, and when an asynchronous publish has has completed Note that the messageArrived callback is scoped to the MqttClient or MqttAsyncClient object and is driven whenever a message arrives on any of its subscriptions.
  • 38 © 2013 IBM Corporation Further notes To resume a previous session, set the clean session option to false when connecting – You must also use the same client identifier that was used last time If you set cleanSession to true, you still need to provide a clientID, there's a convenience method to generate a random clientID for this case Client-side state is persisted using the initial URI and clientID as its key. – The latest version of the MQTT client has the capability to connect to a server which has a different URI, using the same client state. This is for HA configurations where you want to failover to a server which has a different IP address
  • 39 © 2013 IBM Corporation MQTT Messaging optimized for mobile, smart sensors and telemetry devices Enables intelligent decision-making based on remote real-world events Management of static or moving assets, people, locations Simple APIs for Java, JavaScript and other languages reduce the burden for application developers An open protocol with Industry leadership & mindshare • MQTT Protocol and client code contributed to open source • see MQTT.org and Eclipse Paho • Open licence allows development communities to provide further client code & device support • 16+ MQTT servers and 40+ MQTT clients • Standardisation process is under way at OASIS Summary
  • 40 © 2013 IBM Corporation Useful Links MQTT information – http://mqtt.org MQTT 3.1 Specification – http://www.ibm.com/developerworks/webservices/library/ws-mqtt/index.html RSMB (server implementation) – http://www.alphaworks.ibm.com/tech/rsmb/ Eclipse Paho project – http://www.eclipse.org/paho Eclipse M2M Industry Working Group – http://wiki.eclipse.org/Machine-to-Machine OASIS MQTT Technical Committee – https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=mqtt MQTT: the Smarter Planet Protocol – http://andypiper.co.uk/2010/08/05/mqtt-the-smarter-planet-protocol
  • Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/embedded -java-mqtt