Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Internet of Things - protocols review (MeetUp Wireless & Networks, Poznań 21.02.2018)


Published on

Internet of Things - protocols review
MQTT, HTTP/REST, DDS - differences and properties

Published in: Engineering
  • Be the first to comment

Internet of Things - protocols review (MeetUp Wireless & Networks, Poznań 21.02.2018)

  1. 1. Internet Of Things Protocols Review Marcin Bielak Software Engineer Open Source contributor . | , _---_ / -_ .' `. _- __ : .---. : __ : / : -" `. | | .' "- / |`-'| ' ]"-_[ ` ]"-_[ "*" o | | +~11001~+ | +~010001~+ | | | | | | | | +~11001011~+~bieli
  2. 2. Agenda - Introduction - IoT protocols and standards - MQTT - HTTP/REST - MQTT vs. HTTP/REST - DDS - MQTT vs. DDS - IoT architectures - Implementations & cloud dev. IoT Protocols review
  3. 3. Hello world - Who am I ? - Open Source contributor* & user - Open Hardware fanatic - Software Engineer & DevOps - Golang / Python / Java - GNU Linux - Sci-fi fanIoT Protocols review *) ,
  4. 4. - IoT Protocols review Welcome industry 4.0
  5. 5. Everything is “connected” IoT Protocols review
  6. 6. IoT Protocols review IoT sensors
  7. 7. General reasons - why IoT ? - Connect both inanimate and living things - Use sensors for data collection (data streams) - Change what types of item communicate over an IP Network - Benefits for today: - Faster knowledge/stats sharing with M2M - Better quality in control and automation - Cost Savings (Machine Learning, Analytics) IoT Protocols review
  8. 8. IoT Protocols review
  9. 9. Communication protocol... - ..., a defined set of rules and regulations that determine how data is transmitted in telecommunications and computer networking - machines languages in ‘Machine to Machine’ (M2M) world IoT Protocols review
  10. 10. Protocols are everywhere IoT Protocols review protocols
  11. 11. Typical protocols for internet and industrial field ● DNS ● FTP ● HTTP / HTTPS ● IRC ● NNTP ● POP3 ● SMTP ● SSH ● Telnet ● IMAP ● modbus ● profibus ● M-Bus ● S-Bus ● Foundation Fieldbus ● BACnet ● LonWorks ● CAN bus IoT Protocols review
  12. 12. MQTT - IoT protocol IoT Protocols review
  13. 13. MQTT - general information - Message Queuing Telemetry Transport (ver. 3.1.1) - publish – subscribe messaging pattern - requires a message broker - asynchronous messages model (events) - works on top of the TCP/IP protocol (possible TLS) - lightweight, efficient and simple implementation - Open Sourced with standards (40+ client libraries)IoT Protocols review
  14. 14. MQTT - design concept IoT Protocols review
  15. 15. MQTT - core elements IoT Protocols review
  16. 16. MQTT - topic wildcards IoT Protocols review
  17. 17. MQTT - message format IoT Protocols review (up to 250 MB)
  18. 18. MQTT - fixed header fields IoT Protocols review
  19. 19. MQTT - Quality of Service Even though TCP/IP provides guaranteed data delivery, data loss can still occur if a TCP connection breaks down and message in transmit are lost. Therefore MQTT adds 3 quality of service levels on top of TCP. IoT Protocols review
  20. 20. QoS for reliable messaging IoT Protocols review MQTT Broker
  21. 21. MQTT QoS - examples QoS level 0 (best effort) - temperature sensor data is regularly published - lost of an individual value is not critical for application QoS level 1 (quaranteed to arrive, but duplicated are possible) - door sensor senses the door state - it’s important how door state change (close->open, open->close) - states are published losslessly to subscribers QoS level 2 (exactly once) - application where duplicated events could lead to incorrect actions, e.g. sounding an alarm as a reaction to an event recived by the message IoT Protocols review
  22. 22. MQTT client comparision IoT IoT Protocols review
  23. 23. MQTT toolbox IoT Protocols review mqtt-spy - is probably one of the most advanced open source utilities for publishing and monitoring activity on MQTT topics. It is aimed at two groups of users: - innovators who need a tool for creating IoT prototypes or integration projects - power users who need an advanced utility for their work environments mosquitto - an open source MQTT broker (from Eclipse project) - implements the MQTT protocol versions 3.1 and 3.1.1. - is lightweight and is suitable for use on all devices from low power single board computers to full servers paho - MQTT client library implementrations
  24. 24. HTTP + RESTful IoT Protocols review
  25. 25. HTTP + RESTful - The Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, and hypermedia information systems.[1] HTTP is the foundation of data communication for the World Wide Web. - - REpresentational State Transfer (REST), or RESTful, web services provide interoperability between computer systems on the Internet. - - created by the internet community with global companies support - supported by all vendors, companies, organizations (wide range) - based on RFC (Request for Comments) - client/server libs exists in all (popular) programming lang. - stateful, cached, scalable, session, extensible and secureIoT Protocols review
  26. 26. HTTP - design concept IoT Protocols review
  27. 27. HTTP - typical flow IoT Protocols review 1) Open a TCP connection. The TCP connection will be used to send a request, or several, and receive an answer. 2) Send an HTTP message in ASCII. HTTP messages (before HTTP/2) are human-readable. GET / HTTP/1.1 Host: Accept-Language: en 3) Read the response sent by the server: HTTP/1.1 200 OK Date: Sat, 09 Oct 2010 14:28:02 GMT Server: Apache Last-Modified: Tue, 01 Dec 2009 20:18:22 GMT ETag: "51142bc1-7449-479b075b2891b" Accept-Ranges: bytes Content-Length: 29769 Content-Type: text/html <!DOCTYPE html... (here comes the 29769 bytes of the requested web page) 4) Close or reuse the connection for further requests.
  28. 28. HTTP Messages IoT Protocols review REQUEST RESPONSE
  29. 29. Uniform Resource Identifier scheme:[//[user[:password]@]host[:port]][/path][?query][#fragment] IoT Protocols review hierarchical part ┌───────────────────┴─────────────────────┐ authority path ┌───────────────┴───────────────┐┌───┴────┐ abc:// └┬┘ └───────┬───────┘ └────┬────┘ └┬┘ └───┬───┘ └──┬──┘ scheme user information host port query fragment urn:example:mammal:monotreme:echidna └┬┘ └──────────────┬───────────────┘ scheme path ,
  30. 30. HTTP - status codes IoT Protocols review - 1xx: Information. The request was received by the server, continue processing. - 2xx: Successful. The request was received, understood, accepted, and processed by the server. - 3xx: Redirection. Additional action is required to complete the request. - 4xx: Client Error. There was a problem with the request received by the server. - 5xx: Server Error. The request was received, understood, and accepted by the server, however, a server error occurred in processing the request.
  31. 31. HTTP - request methods IoT Protocols review - OPTIONS - GET - HEAD - POST - PUT - DELETE - TRACE - CONNECT - PATCH ,
  32. 32. HTTP - classifying methods IoT Protocols review
  33. 33. HTTP - types of headers fields IoT Protocols review - Authentication - Caching - Client hints - Conditionals - Connection management - Content negotiation - Controls - Cookies - CORS - Downloads - Proxies - Redirects - Request context - Response context - Security - Transfer coding - Server-sent events - WebSockets - …
  34. 34. HTTP still evolves IoT Protocols review
  35. 35. HTTP 1.1 vs. HTTP/2 IoT Protocols review no pipelining vs. pipelining (better performance) optional secure vs. secure by design
  36. 36. REST API - definition IoT Protocols review - REST is a simple way to organize interactions between independent systems (based on HTTP protocol). - Optimal usage of HTTP with few architectural constraints: - Client-server architecture - Statelessness - Cacheability - Layered system - Uniform interface - Code on demand (optional) ,
  37. 37. REST API - design concept IoT Protocols review ,
  38. 38. REST API - design concept IoT Protocols review
  39. 39. REST API - endpoints IoT Protocols review
  40. 40. REST - data representations IoT Protocols review JSON { "_type": "vm", "name": "A virtual machine", "memory": 1024, "cpu": { "cores": 4, "speed": 3600 }, "boot": { "devices": ["cdrom", "harddisk"] } } , YAML !vm name: A virtual machine memory: 1024 cpu: cores: 4 speed: 3600 boot: devices: - cdrom - harddisk XML <vm xmlns:xs=" ma"> <name type="xs:string">My VM</name> <memory type="xs:int">1024</memory> <cpu> <cores type="xs:int">4</cores> <speed type="xs:int">3600</speed> </cpu> <boot> <devices type="xs:list"> <device type="xs:string">cdrom</device> <device type="xs:string">harddisk</device> </devices> </boot> </vm>
  41. 41. Relationship between URL and HTTP methods IoT Protocols review
  42. 42. REST IoT solutions (hubs) IoT Protocols review - The REST APIs for IoT Hub offer programmatic access to the device, messaging, and job services, as well as the resource provider, in IoT Hub. - Typical subjects for IoT hubs: - Device Identities - Device Twins - Device Messaging - Jobs - Resource Provider - Common error codes
  43. 43. Request: GET https://<IoT hub domain name>/devices?api-version=2016-11-14 Response: - status code: 200 - type: Device[] - description: OK Media Types: "application/json", "text/json" Device[] - collections of Device resource Device - has dedicated properties like: deviceId (string), deviceStatus (disabled | enabled : string), cloudToDeviceMessageCount (integer) REST - GET all devices IoT Protocols review
  44. 44. Request: PUT https://<IoT hub domain name>/jobs/v2/{jobid} Request Body: - Media Types: "application/json", "text/json" - required: jobId (string) - required: type (string - enum: schedule | Device | Method | Update | Twin) - optional: queryCondition (string) Response: - status code: 201 - description: Created Media Types: "application/json", "text/json" REST - PUT create job IoT Protocols review
  45. 45. Request: PATCH https://<IoT domain name>/twins/{id}?api-version=2016-11-14 Request Body: - Media Types: "application/json", "text/json" - required: DeviceId (string) - required: IsConnected (boolean) - optional: Version (integer) Response: - status code: 200 - type: DeviceTwinInfo - description: Created + Media Types: "application/json", "text/json" REST - PATCH update twin IoT Protocols review
  46. 46. Request: DELETE https://<IoT domain>/devices/{id}?api-version=2016-11-14 Response: - status code: 204 - description: NoContent Media Types: "application/json", "text/json" REST - DELETE device IoT Protocols review
  47. 47. Request: PUT https://<IoT hub domain name>/iot/core/api/v1/capabilities Request Headers: - Content-Type : application/json - Authorization : Basic <basic_auth_token_using_user_and_password> Request Body: - {"name" : "weather_property_set", "properties" : [{ "name" : "temperature", "dataType" : "float", "unitOfMeasure" : "°C" },{ "name" : "humidity", "dataType" : "float", "unitOfMeasure" : "%" } ]} Response: - status code: 201 Created REST - POST create sensor IoT Protocols review
  48. 48. Text based HTTP sniffer: HTTP/REST toolbox IoT Protocols review
  49. 49. MQTT - simple protocol and message centric - has pretty short specification - public - subscribe (default port 1883) - light (small, it’s binary with 2Byte header) - ideal for exchanging data between low power machines - implemented data security - has Quality Of Service - avg. libraries size: 30KB C, 100KB Java - data distribution: 1 to 0/1/N MQTT vs. HTTP/REST IoT Protocols review , HTTP/REST - complex protocol and document centric - specifications are much longer - client - server (default port 80 or 8080) - text based protocol (large, it is in ASCII format) - standard communication protocol for sharing data and information - security only in HTTPS - no Quality Of Service - avg. libraries size: Large - data distribution: one to one only
  50. 50. MQTT vs. HTTP/REST - life IoT Protocols review According to measurements in 3G networks, throughput of MQTT is 93 times faster than HTTP’s. HTTP is worthy, well tested and extendable. But MQTT is more suitable when it is referred to IoT development.
  51. 51. DDS from standard to protocol IoT Protocols review
  52. 52. DDS - basics IoT Protocols review - "The first open international middleware standard directly addressing publish-subscribe communications for real-time and embedded systems." - DDS addresses the needs of applications like financial trading, air-traffic control, smart grid management, and other big data applications. - The standard is used in applications such as smartphone operating systems, transportation systems and vehicles, software-defined radio, and by healthcare providers.
  53. 53. MoM - middleware concept Message-oriented middleware (MOM) is software or hardware infrastructure supporting sending and receiving messages between distributed systems. MOM allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems and network protocols. A MOM system with built-in intelligence can transform messages en route to match the requirements of the sender or of the recipient. IoT Protocols review
  54. 54. DDS - interoperability The DDS interoperability demonstration used scenarios such as: - Basic connectivity to network using IP - Discovery of publishers and subscribers - Quality of service (QoS) Compatibility between requester and offerer - Delay-tolerant networking - Multiple topics and instances of topics - Exclusive ownerships of topics - Content filtering of topic data including time and geographicIoT Protocols review
  55. 55. Data-Centricity or Message-Centricity? - In a message-centric system the focus is on delivery of the message itself regardless of the data payload it contains and the infrastructure's role is to ensure that messages get to their intended recipients. - In a data-centric system the focus is on user defined data (the data model). The unit of exchange in this type of system is a data value. The middleware understands the context of the data and ensures that all interested subscribers have a correct and consistent view of the data. This is similar in concept to a database that can provide a global view (data space) of the data and can manage its access. IoT Protocols review Global data space
  56. 56. - DDS supports a decentralized broker-less architecture to enable seamless data sharing between producers and consumers. - DDS is based on the idea of a virtual “global data space” where producers write to the data space and consumers read from the data space. - A data model consisting of named topics, their user defined data types and associated QoS is used to by the DDS infrastructure to control how data is shared. - DDS connects producers to consumers over the data bus DDS Data Bus Architecture IoT Protocols review DDS Data Bus Architecture
  57. 57. DDS toolbox IoT Protocols review - DDS community code examples (free) - - CoreDX DDS Spy (commercial) - - it can process data from industry standard network capture files (e.g. Wireshark)
  58. 58. MQTT - simple publish-subscribe protocol - messages delivery with Quality of Service is the most important - communication by one point: Message Broker - Message Broker is the SPoF - data messages routed by topics - for collecting data from few systems with sensors network - usage in telemetry: device to server, data center, ITcloud - centralized & server based analytics, business logic and integration MQTT vs. DDS IoT Protocols review , DDS - data-centric standard - data sharing (visiblility for consumers) is the most important - communication is peer-to-peer - decentralized (no SPoF’s) - distributed, manage tiny devices - reliable, high-performance architecture for connecting many systems with different networks - usage in Intelligent Systems: within and between devices, dedicated systems, real-time cloud, analytics, biz logic & integration distributed, embedded, at edge
  59. 59. MQTT vs. DDS requirements IoT Protocols review
  60. 60. MQTT vs. DDS - QoS differences IoT Protocols review
  61. 61. IoT protocols war ! HTTP AMQP DDS STOMP CoAP MQTT XMPP Thread Bluetooth Zigbee Z-Wave 6LowPAN WiFi Sigfox Neul LoRaWANIoT Protocols review
  62. 62. IoT protocols distribution Rather than trying to fit all of the IoT Protocols on top of existing architecture models like OSI Model, we have broken the protocols into the following layers to provide some level of organization: 1. Infrastructure (ex: 6LowPAN, IPv4/IPv6, RPL, UDP, DTLS) 2. Identification (ex: EPC, uCode, IPv6, URIs) 3. Comms / Transport (ex: NFC, Wifi, Bluetooth, LoRaWAN, NB-IoT) 4. Discovery (ex: Physical Web, mDNS, DNS-SD, HyperCat) 5. Data Protocols (ex: MQTT, CoAP, AMQP, Websocket, DDS, HTTP) 6. Device Management (ex: TR-069, OMA-DM) 7. Semantic (ex: JSON-LD, Web Thing Model, SensorML) 8. Multi-layer Frameworks (ex: Alljoyn, IoTivity, Weave, Homekit, Thread) IoT Protocols review
  63. 63. IoT architectures - domains - platform - gateways - integrations - clouds IoT Protocols review
  64. 64. IoT architectures (domains) IoT Protocols review
  65. 65. IoT architectures (platform) IoT Protocols review
  66. 66. Gateway IoT Protocols review
  67. 67. Integrate DDS into the Broker/ESB IoT Protocols review
  68. 68. Dual Clouds IoT Protocols review
  69. 69. IoT implementation steps STEP-1: Look at the use cases that are relevant to the domain or industry you operate in. STEP-2: Think of ways to enhance those use cases to increase your Reach, Accuracy, Customer Satisfaction, Retention or Cross-sell opportunities. STEP-3: Analyze the impact and compatibility of existing Processes and Systems. STEP-4: Identify the IoT platform and Skills required to implement and carry out maintenance activities. STEP-5: Identify the “Things” required and technical aspects like protocols. IoT Protocols review
  70. 70. 1. - my private tech. blog 2. IoT Standards and Protocols 3. Mesh vs star topology diagram from slides, slides 4. Understanding IoT cloud 5. Key Standarization Activities 6. MQTT telemetry transport 7. Power Profiling: HTTPS Long Polling vs. MQTT with SSL, on Android References