Real World Applications of MQTT

Real World Applications of MQTT
Manoj Gudi
Ex-R.A. IIT Bombay — CTO — Focus Analytics
27th August 2017
Manoj Gudi Ex-R.A. IIT Bombay — CTO — Focus Analytics
Real World MQTT 1 / 45

Content & Scope
What is MQTT?
MQTT Characteristics and Features
Who are we and what do we do?
Real world use for indoor positioning- I
Real world use for indoor positioning- II
Code samples

What is MQTT..
MQ Telemetry Transport: lightweight pub/sub messaging
protocol
Designed for constrained devices (low bandwidth — high
latency — unreliable networks)
Built to minimize network bw and device resource
requirements
Oﬀers various reliability levels of message delivery
MQTT v3.1.1 is OASIS standard (2014)
Initially developed by IBM and Eurotech for M2M
communication for embedded devices

..What is MQTT
Pub/Sub model
Publishers: sensors like temperature sensors
Brokers
Open source: Mosquitto(Eclipse Project), EMQ(Apache v2
license)
Enterprise services: HiveMQ, Azure IoT Hub, IBM Websphere
etc.
Subscribers: entities which consume data, mobile phones etc.
A MQTT client can be publisher or a subscriber or both
Topics allow clients to exchange information with deﬁnedManoj Gudi Ex-R.A. IIT Bombay — CTO — Focus Analytics

Why MQTT?
Designed App with killer features

Why MQTT?
By the time it hits production

MQTT Characteristics
Network model:
Lightweight: 2 byte header overhead per packet
Three main components: data producer(publisher) — data
consumer (subscriber) — queue like structure (topics)
MQTT requires TCP/IP Stack (i.e. way to deliver packets in
order reliably). Can’t be used over UDP.
MQTT-SN for sensor networks, doesn’t require TCP/IP stack.
Works directly over transport layer (like ZigBee)
Suitable for asynchronous communication model with events
(messages)

MQTT Message format..
Message format:
2 byte ﬁxed-length header
Optional message-speciﬁc variable length header and message
payload
Very lean message format. Tries to save and reuse bytes as
much as possible.

..MQTT Message format..
Message Type (Description/Values):
0: Reserved 5: PUBREL 10: UNSUBSCRIBE
1: CONNECT 6: PUBCOMP 11: UNSUBACK
2: PUBLISH 7: SUBSCRIBE 12: PINGREQ
3: PUBACK 8: SUBACK 13: PINGRESP
4: PUBREC 9: UNSUBSCRIBE 14: DISCONNECT
15: RESERVED
DUP: Duplicate ﬂag(==1) to indicate re-delivery of an earlier
PUBLISH packet(as of 3.1.1)
QoS Level: Possible values are 0, 1, 2
Remaining Length: encodes(length(variable length header) +
length(payload))

..MQTT Message format..
RETAIN:
For a PUBLISH message instructing server to keep the
message for this topic
Last known good value.

..MQTT Message format

MQTT QoS levels..
MQTT provides 3 levels of QoS (Quality of Service)
Although uses TCP/IP, but data loss can still occur if network
is disrupted (lost messages)
Automatic retransmission and delivery guarantees for certain
QoS by MQTT

MQTT QoS 0
At most once delivery
Minimal guarantee. Receiver doesn’t acknowledge reception
of message
Sender doesn’t store message for redelivery
Same guarantee as underlying TCP protocol
Also has the least overhead in comparison

MQTT QoS 1
At least once delivery
QoS1 guarantees that a message will be delivered at least
once to receiver.
Duplicate messages maybe received (identiﬁed by DUP = 1)
Sender stores the message until it gets PUBACK
PUBLISH and PUBACK associated by packetId
Re-transmission of packet by sender if PUBACK not receivedManoj Gudi Ex-R.A. IIT Bombay — CTO — Focus Analytics

MQTT QoS 2
Exactly once delivery
Highest QoS, guarantees each message is received only once
Slowest QoS because of two ﬂows between sender-receiver.
Critical to your application to receive all messages exactly once

..MQTT QoS levels
Net-Net:
QoS0: when you have stable connection, once a while packet
drop is OK.
QoS1: when you need to get every message and your app can
handle duplicates
QoS2: when it’s critical to your app to receive all messages
exactly once.
QoS levels deﬁned when creating a connection by the client.
QoS Downgrade possible if subscribing client has lower QoS
then publishing client
What if publisher is QoS1 and subscriber is QoS2?

Message Persistence..
Non Persistent Session:
Deﬁned by client to broker in CONNECT packet

..Message Persistence..
Persistent topics:
Broker doesn’t unsubscribe the client on disconnection

..Message Persistence
What’s stored in a session?
All messages in a QoS 1,2 flow
PUBACK while client was offline
All received QoS2 messages, which are not yet confirmed to
the client
Session identified uniquely to a client by clientID in
CONNECT
A client identifies a stored session by session flag in
CONNACK
Useful if client is in patchy networks and has to consume all
packets
Max number of messages usually limited by memory of
broker/client
Mosquitto can write persistent messages to file, reload when
broker restarts

Topics
A UTF-8 string used by broker to filter messages for each
client
Hierarchical. Can contain one or more topic levels
Supports wildcards
A ”+” is single-level wildcard
myhome/groundfloor/+/temperature
A ”#” is multilevel-level wildcard (Must be at end)
myhome/groundfloor/#
There can be null sub-topics myhome//kitchen/

Other Features
Last Will and Testament: used to notify other clients by
broker when a client is disconnected abruptly
MQTT supports TLS, but you lose low overhead beneﬁts
For a long-living TCP connections with MQTT, TLS overhead
negligible
MQTT broker typically runs at 1883(TCP) and 8883(TLS)
Free test broker at https://test.mosquitto.org/

Who are we?

What do we do?
Outdoor: GPS — Indoors: ?
Directly lifted from marketing pitch deck
I didn’t make this
I don’t know why there is a magnifying glass

We work with

Indoor Positioning in a Large Warehouse..

..Indoor Positioning in a Large Warehouse..
Challenges
Warehouse usually situated in very remote places
Metal mesh ﬂoors (Weak cellular network)
Operates 24X7
XXL: One misplaced item in inventory takes about a month to
ﬁnd
Idea is to able to quickly trace people movements in realtime
and retrospectively.
We need to precisely position a warehouse worker up to a foot
level.
With users expecting faster delivery times, misplaced
inventory problem is critical

Solution # 1
Upside: Very fast and 110% accurate!
Downside: Very complicated and available only in Sirsa(now
in Rohtak)

UWB (Ultra Wide Band) based positioning system:
Low energy, high-bandwidth communication over a large
spectrum(3.1 to 10.6 GHz)
Minimum 3 Anchors(known position) and 1 Tag(position to be
determined)
Leverages Time of Flight (TOF) method to come up with
position of tag
Accuracy 1 foot and less susceptible to interferenceManoj Gudi Ex-R.A. IIT Bombay — CTO — Focus Analytics

UWB (Ultra Wide Band) based positioning system:
Accuracy 1 foot and less susceptible to interference
Anchors can talk up to 60M distance in LOS, spread across the
entire warehouse section
Each worker carries a tag with their inventory tracking gadget

..Indoor Positioning in a Large Warehouse

Indoor Positioning in Retail Spaces..
A simple android SDK. Integrate, and done.
We send you location of your users. On phone and if required
on your server

..Indoor Positioning in Retail Spaces..

..Indoor Positioning in Retail Spaces
Whenever a big client comes on board, with in 24 hrs 30-40%
of their users come on board as well.
A chunk of the location data comes when users are in non
retail places
What if there’s a way where other users coarsely predict other
users?

Small Confession
That network traﬃc graph you saw was of a VM, running
HAProxy

Location Prediction of Users in a distributed way..
This is in a prototype stage.
The idea is to leverage large number of mobile phones in same
area help predict each other while
Avg 140 smart phone per sq km in India. Density increases in
city.
Save us $$$ on network and server costs
It should work even when our prediction engine is down

..Location Prediction of Users in a distributed way..

..Location Prediction of Users in a distributed way

Enough of talk, show me some code!
def getDataAndPubish (sensorData , client , ):
"""
Get sensor data and publish to broker
"""
topic = "client/warehouse/f1/s1"
sensorData["timestamp"] = int(time.time () * 1000)
client.publish(topic ,
json.dumps(sensorData), qos=1, retain=False)

def onConnect(client , userData , msg):
’’’
subscribe when (CONNACK)
’’’
print("Successfully Connected")
client.subscribe("client/warehouse /#", qos =1)
def onMessage(client , userData , msg):
"""
call locationEngine main and get prediction
"""
anchorCoordinates = userData
print("Got Message ..")
data = json.loads(msg.payload.decode("utf -8")
### DO SECRET STUFF

def main ():
anchorCoordinates = [[1, 12, 123], [32, 123, 32],
# Get MQTT connection paramers
brokerIP = "localhost"
brokerPort = 1883
defaultTimeout = 60
client = mqtt.Client ()
# Pass baseInstance and session tuple to callbacks
client.on_connect = onConnect
client.on_message = onMessage
# Set anchorCoordinates
client.user_data_set( anchorCoordinates )
client.connect(brokerIP , brokerPort , defaultTimeou
client.loop_forever ()

Some caveats
If implementing on MQTT on battery constrained devices,
select keep-alive intervals carefully
Too short keep-alive can impact battery proﬁle because of
PINGREQ/PINGRESP
Although Mosquitto (MQTT broker) allows horizontal scaling
between two brokers by bridging. It can be tricky if you need
more brokers or a cluster with Mosquitto

References/Links
Eclipse Paho Project www.eclipse.org/paho
Hive MQTT Essentials
http://www.hivemq.com/mqtt-essentials/
IoT For Tiny Devices (MQTT-SN), Stefan Roth, Zuehlke
Power profiling MQTT on android:
http://stephendnicholas.com/posts/power-profiling-mqtt-on-
android
Mosquitto Bridging http://www.steves-internet-
guide.com/mosquitto-bridge-configuration/
An Introduction to MQTT, A Protocol for M2M and IoT
Applications. Peter R. Egli INDIGOO.COM
Mosquitto broker docs mosquitto.org

This seems cool, can I join?
Sure!
manoj@getfocus.in — +91-9920909145
careers@getfocus.in
Thanks!

Real World Applications of MQTT

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Real World Applications of MQTT

Similar to Real World Applications of MQTT (20)

Recently uploaded

Recently uploaded (20)

Real World Applications of MQTT