Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lightweight Messaging for a Connected Planet


Published on

Video online at

The Internet of Things is one of the buzz phrases of the moment - the idea that there are a of a lot of devices around, from tiny sensors and RFID tags, through smartphones, GPS location-aware devices, laptops, and embedded systems. They are nearly all natively "online" to some extent, and most will have an Internet address of their own (even if the connections are not always super-high bandwidth, always-on, or reliable). So, they are becoming interconnected, either directly to one another across local networks, or indirectly via clouds. These devices typically have enough computing power to at least gather and transmit data, and some of them have enough to respond to requests to modify their behaviour. Many of these devices run Linux!

So, with the lightweight nature of the systems involved, what about a lightweight but reliable messaging protocol that can help to connect them? Well, there's one available, it's called MQTT (MQ Telemetry Transport, see, and it's increasingly being adopted for some very cool projects like, for example, mosquitto ( This presentation will look at why lightweight reliable messaging is useful; what MQTT is and how to use it (including examples in a number of languages); and how it can be used in a variety of systems from home automation solutions and Arduino-based gadgets all the way through to city-wide transport systems, and even used to bridge up to messaging systems used by large enterprises.

Published in: Technology
  • Be the first to comment

Lightweight Messaging for a Connected Planet

  1. 1. Lightweight Messagingfor a Connected Planet Andy Piper WebSphere Messaging Community Lead, IBM
  2. 2. Whats this all about?
  3. 3. The Internet of ThingsTrillions of smartdevices instrumentour world today Interconnecting these smart devices creates a Central Nervous System
  4. 4. Building a Smarter Planet + + =
  5. 5. The Networked Enterprise Connectivity for Applications Head Offices, Transport WebSphere MQ Data centres REST/HTTP Intelligent Cloud Computing WS* Mission-Critical Connectivity & Intelligence Messages WebSphere MQ Regional Offices Stores, Outlets Files WebSphere MQ File Transfer Edition Interconnected Transform, WebSphere Message Broker Enrich, WebSphere ESB Mediate DataPower WebSphere Sensor Events Catalog WebSphere Service Registry & Repository Intelligence WebSphere Business Events Instrumented Remote Systems and Devices Cognos. ILOG, SPSS InfoSphere StreamsConnectivity for Smart Devices Other InfoSphere and Tivoli productsTransport MQ Telemetry HTTP Multicast Actuators Embedded Controllers Sensors Power meters, weather data Tag printers, status lights, LoadMillions of Filtering of duplicate read events, Store- based HVAC & lighting controls, SCADA sensors, pressure, volume, generation, HVAC and lighting, RFID readers, Motion detectors… Valves, switches and pumps…devices Industrial Network Gateways (SCADA)
  6. 6. MQTT =MQ Telemetry Transport
  7. 7. Design principles■ Publish/subscribe messaging paradigm (useful for the majority of sensor applications)■ Minimise the on-the-wire footprint.■ Expect and cater for frequent network disruption – built for low bandwidth, high latency, unreliable, high cost networks■ Expect that client applications may have very limited processing resources available.■ Provide traditional messaging qualities of service where the environment allows.■ Publish the protocol royalty-free, for ease of adoption by device vendors and third- party software developers.
  8. 8. Key facts■ Low complexity and footprint■ Simple publish/subscribe messaging semantics  Asynchronous (“push”) delivery of messages to applications  Simple verbs: connect, publish, (un)subscribe, disconnect Minimised on-the-wire format  Plain byte array message payload  No application message headers  Protocol compressed into bit-wise headers and variable length fields  Smallest possible packet size is 2 bytes■ In-built constructs to support loss of contact between client and server  “Last will and testament” to publish a message if the client goes offline  Stateful “roll-forward” semantics and “durable” subscriptions
  9. 9. What aboutthat HTTP thing?
  10. 10. Good point.Heres a (very) quick comparison.
  11. 11. Data-centricityMQTT is agnostic of data content and transferssimple byte arrays, making drip-feeds ofupdating information trivial.HTTP is (basically) document-centric.
  12. 12. SimplicityMQTT has few methods(publish/subscribe/unsubscribe) and is quick tolearn.HTTP can be complex (although it is often well-understood) - there are a multitude of returncodes and methods. REST is a great principle but not always the bestfor simple data applications(POST/PUT/GET/DELETE? er what?)
  13. 13. Light on the networkThe smallest possible packet size for an MQTTmessage is 2 bytes. The protocol was optimised from the start forunreliable, low-bandwidth, expensive, high-latency networks.HTTP is relatively verbose - lots of "chatter" in aPOST
  14. 14. Easy distribution of dataMQTT distributes 1-to-none, 1-to-1 or 1-to-nvia the publish/subscribe mechanism→ very efficientHTTP is point-to-point (can bemediated/clustered but no distributionmechanism). To distribute to multiple receivers alarge number of POSTs may be required.
  15. 15. Lightweight Stack (CPU/Mem)MQTT has been trivially implemented on tiny tolarger platforms in very small libraries[IBM ref implementation = ~80Kb for full broker]HTTP (often with associated XML or JSONlibraries for SOAP and REST etc) can be relativelylarge on top of OS network librariesPlus... even if the client is small, considerwhether it is really necessary to run an HTTPserver on every device
  16. 16. Variable Quality-of-ServiceMQTT supports fire-and-forget or fire-and-confirm (aka QoS 0/1/2)HTTP has no retry / confirmation / attempt atonce-only delivery. It is basically brittle, i.e. retryneeds to be written in at the application level.Applications must also handle timeouts.
  17. 17. Sounds like theres a place for both! Where is MQTT in use?
  18. 18. • Simple• Lightweight (CPU,Mem,**Net)• Data-centric• Distributes data (pub/sub)• Range of QoS → strong developer community
  19. 19. “strong developer community” huh...
  20. 20. Ok, to be fair, I haveno knowledge of theirphysical strength, butthey are all awesome...
  21. 21. Home automation
  22. 22. Gardening “It all started with the seemingly simple question – “How can I water the garden without leaving my laptop/phone/sofa using tech?”” - Dan Fish
  23. 23. TV, Android, burglar “control”
  24. 24. Mind-controlled Taxis b “Kevin already had the headset hooked up to MQTT, so it would be trivial to use my Arduino MQTT library to get them all talking.” - Nick OLeary
  25. 25. Flashing Arduino-controlled ducks “Now, you may wonder why I would want 20 rubber ducks to flash when my phone goes off.... There is no scientific or technical reason in itself. I just had a Mini Cooper’s worth of rubber ducks sitting around, unemployed.” - Chris Phillips
  26. 26. This sounds moderatelyinteresting (and fun) Lemme at it!
  27. 27. Seriously.Weve been waiting for 25 slides already. CODE, DAMMIT
  28. 28. The IBM way•• Download• Unzip• Run nohup ./broker >> /dev/null &• Play with C client utils, code, and be merry!• Available for Linux IA32, IA64 kernel 2.6.8+; Linux on IBM System z; Linux for ARM XScale, kernel 2.0.0+ (Crossbow Stargate or Eurotech Viper); Windows XP; Mac OS X Leopard; Unslung (Linksys NSLU2) – Binary only, request other platforms from IBM
  29. 29. Alternatively...•• On e.g. Ubuntu: sudo add-apt-repository ppa:mosquitto-dev/mosquitto- ppa && sudo apt-get update && sudo apt-get install mosquitto (optional: mosquitto-pub, mosquitto-sub, python- mosquitto)• Runs as a daemon; IPv4/IPv6-capable• C++ and Python examples available• Packaged for Ubuntu, Fedora, RHEL, OpenSuSE, CentOS, Debian, Mandriva; Windows - binary; OS X – binary (homebrew compile via github package); source tarball; dev version in bitbucket
  30. 30. Client APIs• C• C++• Java (your route to Android and Kindle goodness)• .NET (yes, Mono too...)• Delphi • Perl• Python • PHP • Ruby • Erlang • Arduino • mbed
  31. 31. Show us the code!public void sendAMessage() throws MqttException { MqttProperties mqttProps = new MqttProperties(); Create a connection using the mqttProps.setCleanStart( true ); connection factory, this time MqttClient client = MqttClientFactory. INSTANCE. for a clean starting client createMqttClient("testClient", “tcp://localhost:1883”, mqttProps); Register the class as a listener and client.registerCallback(this); connect to the broker client.connect(); client.publish(“abc/123”, new MqttPayload((“Hello World!”).getBytes(),0), (byte) 2, false); client.disconnect(); Publish a message to the} given topic and disconnectpublic void publishArrived (String topicName, MqttPayload payload, byte qos, boolean retained, int msgId) { System.out.println(“Got it!”);} On receipt of a publication, simply print out a message on the console to say we received it
  32. 32. Moar code plz#!/usr/bin/pythonimport pynotifyimport mosquitto Setup a callback – post a# define what happens after connection desktop notification with message topic and payloaddef on_connect(rc): when a publication arrives print "Connected"# On receipt of a message create a pynotification and show itdef on_message(msg): n = pynotify.Notification (msg.topic, msg.payload) ()# create a client connectionmqttc = mosquitto.Mosquitto("python_sub")# define the callbacksmqttc.on_message = on_messagemqttc.on_connect = on_connect Set a client id (“python_sub”), connect to localhost, and# connect subscribe on topic “test”mqttc.connect("localhost", 1883, 60, True)# subscribe to topic testmqttc.subscribe("test", 2)# keep connected to brokerwhile mqttc.loop() == 0: pass
  33. 33. Community• (including wiki)• rsmb forum at IBM alphaWorks• #mqtt on freenode• mosquitto project on launchpad•• many bloggers, developers, etc...
  34. 34. More random-but-cool schtuffs• File sync over MQTT?• Desktop notifications and• Web thermometers• Digital-to-analogue readouts• CEIT @ UQ research projects• LEGO microscope control
  35. 35. Elsewhere in theworld, exciting things were occurring...
  36. 36. Integrating to the EnterpriseWebSphere MQ Telemetry Sensors DIRECT MQTT CLIENT WebSphere Mobile, smart meters MQ 7.0.1 MQXR Applications BRIDGERich clients MQTT BRIDGErequiring buffering, WebSphere MQ Telemetryremote management Daemon for Devices DIRECTcapabilities, or MQTT CLIENT Microbrokeradvanced data handling (Lotus Expeditor, WebSphere Sensor Events)
  37. 37. Smarter HealthcareMedical organization created a remote pace-maker monitoringsolution to provide better patient care Client Pains  Physicians needed better monitoring of cardiac patients  Improve efficiency of checkups  Meet healthcare data capture standards Enables higher level of patient care and peace of mind Improves administrative efficiency and maintenance Helps conform to standards and ease integration of data
  38. 38. Smarter EnergyUtility company developing an Intelligent Utility Network offering foroptimizing load on electricity grids Business Partner  Needs robust middleware technology to connect to remote smart meters  Needs to be able to rapidly scale solution nationally Able to offer daily energy savings of 15-20% Enables utilities to reduce peaks and avoid punitive charges Helps save electricity through better peak load management
  39. 39. KTHXBAI! Andy Piper @andypiper
  40. 40. Thanks!!• Roger Light @ralight (mosquitto awesomeness)• Nick OLeary @knolleary (Arduino/MQTT awesomeness – images from Flickr – and general uber-awesomeness)• Chris Yeoh @ckbyeoh (home hacking awesomeness)• Benjamin Hardill @hardillb (TV hacking awesomeness)• Chris Phillips @cminion (Rubber Duck awesomeness)• Oliver Smith @chemicaloliver (lots of webby awesomeness)• Dan Fish @ossmedicine (garden awesomeness)• Donna Benjamin @kattekrab (Inkscape awesomeness)