The document discusses the MQTT protocol, focusing on the message format, connection establishment, client identification, and security features. It outlines how MQTT clients communicate with brokers, the importance of unique client identifiers, and the role of clean sessions in message delivery. Additionally, it addresses authentication methods and features like the Last Will and Testament and keep-alive mechanisms for maintaining connections.

1. Message Queuing Telemetry Transport (MQTT)
Message Format
(ID & other parameters)
Khamdamboy Urunov, a Ph.D. student.
Special Communication Research Center.,
Graduate School of Financial Information
Security., Kookmin University
Seoul, South Korea

2. MQTT Connection
2
 The MQTT protocol is based on top of TCP/IP and both client and broker need to have
a TCP/IP stack.
 The connection is initiated through a client sending a CONNECT message to the broker.
 The broker response with a CONNACK and a status code.
 Once the connection is established, the broker will keep it open as long as the client doesn’t
send a disconnect command or it looses the connection.
http://www.hivemq.com/blog/mqtt-essentials-part-3-client-broker-connection-establishment

3. 3
MQTT connection through a NAT
 in order to translate from a private network address (like 192.168.x.x, 10.0.x.x) to a public facing
one.
 MQTT client is doing the first step by sending a CONNECT message
 there is no problem at all with clients behind a NAT
• because the broker has a public address and the connection will be kept open to allow
sending and receiving message bidirectional after the initial CONNECT.
http://www.hivemq.com/blog/mqtt-essentials-part-3-client-broker-connection-establishment
It is a common use case that MQTT clients are behind routers, which are using network
address translation (NAT)

4. MQTT connection message flow
4http://www.sharetechnote.com/html/IoT/App_Protocol_MQTT.html

5. Client initiates connection with the CONNECT
message
5
So let’s look at the MQTT CONNECT command message. As already mentioned this is
sent from the client to the broker to initiate a connection.
 if the CONNECT message is malformed (according to the MQTT spec)
• or it takes too long from opening a network socket to sending it,
• the broker will close the connection.
 this is a reasonable behavior to avoid that malicious clients can slow down the broker.
 a good-natured client will send a connect message with the following content among
other things
http://www.hivemq.com/blog/mqtt-essentials-part-3-client-broker-connection-establishment

6. MQTT Client ID
6
The client identifier (short ClientId) is an identifier of each MQTT client connecting to a MQTT
broker.
 As the word identifier already suggests, it should be unique per broker.
 The broker uses it for identifying the client and the current state of the client.
 If you don’t need a state to be hold by the broker, in MQTT 3.1.1 (current standard) it is also
possible to send an empty ClientId, which results in a connection without any state.
 A condition is that clean session is true, otherwise the connection will be rejected.
https://www.eclipse.org/paho/files/javadoc/org/eclipse/paho/client/mqttv3/MqttClient.html#generateClientId()
o clean session is true = 1
o clean session is false =0
o clean session is transient
o clean session is durable

7. https://www.ibm.com/support/knowledgecenter/SSFKSJ_7.1.0/com.ibm.mq.doc/tt60310_.htm 7
Client ID (1)
What is The MQTT client identifier?
 the client identifier is a 23 byte string that identifies an MQTTclient
 each identifier must be unique to only one connected client at a time
 the identifier must contain only characters valid in a queue manager
name
What is The MQTT client identifier parameter(s)?
 within these constraints, you are able to use any identification string
How MQTT client identifier for constraint environment ?
What is MQTT WebSphere ?
 the client identifier is used in the administration of a WebSphere MQ TT
system
 it is used by WebSphere MQ as a queue manager name, to identify a
destination.

8. 8
1. How does the customer identify the device, and how do you correlate that identification with the
server that is typically connected to the client?
2. Do you have to consult a database that maps each device to a client identifier and to a server?
3. Does the name of the device identify which server it is attached to?
When you browse through MQTT client connections using WebSphere MQ Explorer, each
connection is labeled with the client identifier.
Client ID (2)
What is happen?
example, a device has malfunctioned, and you are notified, perhaps by a customer ringing a help desk.

9. 9
1. Do you need to look up a table to map a client identifier to a physical device?
2. Does the client identifier identify a particular device, a user, or an application running at the
client?
3. If a replaces a faulty device with a new one, does the new device have the same identifier as
the old device?
4. Do you allocate a new identifier?
Client ID (3)
What is happen?
If you change a physical device, but keep the same identifier, outstanding publications and
active subscriptions are automatically transferred to the new device.

10. 10
 as well as a system for generating unique identifiers,
 you must have a reliable process for setting the identifier on the client.
 perhaps the client device is a "black-box", with no user interface.
Client ID (4)
 you might create a client identifier from the 48 bit device MAC address,
 to keep the identifier short and unique.
 if transmission size is not a critical issue,
 you might use the remaining 17 bytes to make the address easier to administer.
How do you ensure that client identifiers are unique?
Do you manufacture the device with a client identifier - such as using its MAC
address?

11. Client ID (5)
11
https://www.eclipse.org/paho/files/javadoc/org/eclipse/paho/client/mqttv3/MqttClient.html#generateClientId()
 a convenience method is provided
to generate a random client id that
should satisfy this criteria -
generateClientId().
 as the client identifier is used by
the server to identify a client when it
reconnects,
 the client must use the same
identifier between connections if
durable subscriptions or reliable
delivery of messages is required.
 a client identifier (clientId) must
be specified and be less that 65535
characters.
 it must be unique across all clients
connecting to the same server.
 the clientId is used by the server
to store data related to the client,
 it is important that the clientId
remain the same when connecting to
a server if durable subscriptions or
reliable messaging are required.

12. https://www.eclipse.org/paho/files/javadoc/org/eclipse/paho/client/mqttv3/MqttClient.html#generateClientId() 12
o generateClientId
public static String generateClientId()
 returns a randomly generated client identifier based on the current user's login name and the
system time.
 when cleanSession is set to false, an application must ensure it uses the same client identifier
 when it reconnects to the server to resume state and maintain assured message delivery.
Client ID (6)

13. 13
Clean Session
 The clean session flag indicates the broker, whether the client wants to establish a persistent
session or not.
 A persistent session (CleanSession is false) means, that the broker will store all subscriptions
for the client and also all missed messages, when subscribing with Quality of Service (QoS) 1 or 2.
 If clean session is set to true, the broker won’t store anything for the client and will also purge
all information from a previous persistent session.
http://www.hivemq.com/blog/mqtt-essentials-part-3-client-broker-connection-establishment
Client ID (7)

14. MQTT Security Fundamentals: Authentication with
Username and Password
14http://www.hivemq.com/blog/mqtt-security-fundamentals-authentication-username-password
• authentication is part of the transport and application level security in MQTT
• on the transport level TLS can guarantee authentication of the client to the server using
client certificates and of the server to the client validating the server certificate.
• on the application level the MQTT protocol provides username and password for
authentication.
• various broker implementations add different mechanisms on top of that.

15. 15
MQTT authentication with username/password
 Therefore a client has the possibility to send a username and password when
connecting to an MQTT broker
When it comes to authentication in MQTT the protocol itself provides username and
password fields in the CONNECT message.

16. 16
Username & Password flag
 MQTT allows to send a username and password
for authenticating the client and also authorization.
 However, the password is sent in plaintext,
• if it isn’t encrypted or hashed by
implementation
• or TLS is used underneath.
 We highly recommend to use username and
password together with a secure transport of it.
 In brokers like HiveMQ it is also possible to
authenticate clients with an SSL (secure sockets
layer) certificate, so no username and password is
needed.
MQTT message format (cont…)
http://slides.com/disk91/mqtt#/13

17. MQTT Will
• When a client connects to a broker, it may inform the broker that it has a will.
• This is a message that it wishes the broker to send when the client disconnects
unexpectedly.
• The will message has a topic, QoS and retain status just the same as any other message.
17
Will
Topic QoS Retain

18. 18
Example: Subscribers receive last known temperature value from the temperature data
topic. RETAIN=1 indicates to subscriber B that the message may have been published
some time ago
MQTT message format
Will Retain: If set to 1 indicates to server that it should retain a Will message for the
client which is published in case the client disconnects unexpectedly.

19. MQTT Last Will and Testament (1)
19
The Last Will and Testament feature is used in MQTT to notify other clients about an ungracefully
disconnected client.
 MQTT is often used in scenarios were unreliable networks are very common
 it is assumed that some clients will disconnect ungracefully from time to time
 because they lost the connection
 the battery is empty or any other imaginable case.
 if a connected client has disconnected gracefully (which means with a
MQTT DISCONNECT message)
 or not, in order to take appropriate action.

20. 20
When will a broker send the LWT message?
MQTT Last Will and Testament (2)
According to the MQTT 3.1.1 specification the broker will distribute the LWT of a client in
the following cases:
• An I/O error or network failure is detected by the server.
• The client fails to communicate within the Keep Alive time.
• The client closes the network connection without sending a DISCONNECT packet
first.
• The server closes the network connection because of a protocol error.
For example after a client has connected to a broker, it will send a retained message to the
topic client1/status with the payload “online“.
 When connecting to the broker, the client sets the LWT message on the same topic to
the payload “offline” and marks this LWT message as a retained message.
 If the client now disconnects ungracefully, the broker will publish the retained
message with the content “offline“.
 This pattern allows for other clients to observe the status of the client on a single topic
and due to the retained message even newly connected client now immediately the current
status.

21. MQTT Message Keep Alive
• The keep alive is a time interval, the clients commits to by sending regular PING
Request messages to the broker.
• The broker response with PING Response and this mechanism will allow both sides
to determine if the other one is still alive and reachable.
• That are basically all information that are necessary to connect to a MQTT broker
from a MQTT client.
• Often each individual library will have additional options, which can be configured
• Protocol includes support for client and server to detect failed connections
– At connection time, a keep alive can be specified
• Maximum keep alive interval of 18 hours
– Can specify a value of 0 to disable keep alive
21

22. 22
CONNACK message format:
MQTT message format
CONNACK 2 Connect Acknowledgment

23. 23
 when using the built-in username/password authentication
 the MQTT broker will evaluate the credential based on the implemented authentication
mechanism
• (more on that in the next post) and
• return one of the following return codes (a full list of all return codes can be found in
the MQTT Essential Part 3: Establishing an MQTT connection)
MQTT message format (cont..)
 When setting username and password on
the client,
 it will be sent to the broker in clear text.
 this would allow eavesdropping by an
attacker and is an easy way of obtaining the
credentials.
 The only way to guarantee a completely
secure transmission of username and password
is to use transport encryption.

24. 24
MQTT message format (cont…)
PUBLISH 3 Publish message
Byte 2 (At least one byte) contains the Remaining Length field. The fields are described in the following sections.
All data values are in big-endian order: higher order bytes precede lower order bytes. A 16-bit word is presented
on the wire as Most Significant Byte (MSB), followed by Least Significant Byte (LSB).

25. Publish (1)
25
Payload
This is the actual content of the message. MQTT is totally data-agnostic, it’s possible to
send images, texts in any encoding, encrypted data and virtually every data in binary.
Packet Identifier
The packet identifier is a unique identifier between client and broker to identify a
message in a message flow.
 This is only relevant for QoS greater than zero.
 Setting this MQTT internal identifier is the responsibility of the client library and/or
the broker.

26. MQTT Protocol Length Field Encoding
26
The length of the remaining length field is between 1 and 4 bytes depending on the
payload size (the actual user message).
MQTT was designed for devices with very limited capabilities such as battery-driven sensor
nodes and wireless devices.
 this implies that the protocol needs to be very efficient
• low protocol overhead
• any excess byte transmitted over a wireless link
• would consume precious battery capacity
http://indigoo.com/petersblog/?p=263

27. 27
MQTT message format (cont…)
PUBACK 4 Publish Acknowledgment
PUBREC 5 Publish Received (assured delivery part1)

28. 28
MQTT message format (cont…)
PUBREL 6 Publish Release (assured delivery part 2)
PUBCOMP 7 Publish Complete (assured delivery part 3 )

29. 29
MQTT message format (cont…)
SUBSCRIBE 8 Client Subscribe request

30. 30
MQTT message format (cont…)
SUBACK 9 Subscribe Acknowledgment

31. 31
MQTT message format (cont…)
UNSUBSCRIBE 10 Client Unsubscribe request

32. 32
MQTT message format (cont…)
UNSUBACK 11 Unsubscribe Acknowledgment
PINGREC 12 PING Request
PINGREST 13 PING Response
DISCONNECT 14 Client is Disconnecting
http://www.slideshare.net/PeterREgli/mq-telemetry-transport

33. MQTT example
33
< CONNECT ACK>

34. MQTT example
34
< PUBLISH >

35. MQTT example
35
< SUBSCRIBE >

36. • Thank you
• hamdamboy.urunov@gmail.com
36