Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An IOT gateway Architecture using an MQTT bundle transport & LTP Convergence Layer

1,161 views

Published on

In the last 30 years, messaging middleware software (MOM) has been offering store and forward services by decoupling senders (publishers) and receivers (subscribers) while offering various levels of QOS and transport protocol options. As hardware options became multi-core and with the aid of developments in Service Oriented Architecture (SOA), Event Driven Architecture (EDA) and cloud computing, a plethora of MOM products have emerged most of them based on the popular publish-subscribe paradigm. This has also led to standardisation efforts both at the API level (JMS) or the transport protocol level (MQTT, AMQP) and an ever increasing developer user base.

The majority of today’s IOT architectures involve a gateway node that bridges sensor/field networks with the internet. This is typically done either by using the consumer internet subscription or by leveraging new long range WAN technology such as SigFox and LoRa. In the majority of real world cases, the gateways use advanced software to buffer / support back pressure, an unavoidable result of the network diversity and intermittent connectivity of IOT and mobile nodes.

In an effort to find a killer application for DTN that is immediately useful and based on technology available today, we have researched an IOT gateway architecture that acts as a bi-directional convergence layer between MQTT events and RFC 5050 Bundle protocol packets. This allows existing MQTT based sensor applications to leverage DTN services without the need to make any code modifications. At the same time, the approach allows the definition of alternate DTN based routing paths between the gateway and the accompanying data ingestion cloud services, without being restricted from the typically tiny routing table size. Geographically dispersed such gateways can now also provide routing services for each other by further converging over Licklider Transmission Protocol (LTP) for all or a subset of the messaging namespace payloads. Finally the cost of data ingestion from occasionally connected field networks is greatly reduced due to the simplicity of developing applications for them.

  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

An IOT gateway Architecture using an MQTT bundle transport & LTP Convergence Layer

  1. 1. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks BIT 5th Annual World Congress Emerging InfoTech 2016 An IOT gateway Architecture using a MQTT bundle transport & LTP Convergence Layer 11/11/16 1 By Alex Kritikos
  2. 2. . Senior director R&D Universal Messaging Software AG UK . OASIS MQTT TC member Email: alex.kritikos@softwareag.com Twitter: @krital LinkedIn: https://uk.linkedin.com/in/alexkritikos ALEX KRITIKOS About the speaker 2 Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  3. 3. Growing fast Internet of (Earth) Things 3 11/11/16 Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  4. 4. DTN based Solar System Internet 4 11/11/16 Internet of (Space) Things Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  5. 5. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks Earth & Space IOT research What if we combined them? • Link layer agnostic propagation (bundles) • MQTT encapsulation: payloads to local subscribers 5 11/11/16 • DTN encapsulation: payloads are routed progressively via intermediary nodes • Full MQTT & DTN stack on each node • Lower latency and higher throughput
  6. 6. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks DTN to MQTT Convergence Layer • /temperature => dtn://node/temperature • /consumption => dtn://node/consumption • /ac/actions => dtn://node/ac/actions • MQTT topic => DTN EID 6 11/11/16 • bi directional • MQTT publish DTN receive • DTN send MQTT subscribe
  7. 7. 7 11/11/16 • MQTT integration with DTN • Hardware sharing = cost reduction • Software sharing = cost reduction • MQTT bridging • via DTN BP • Via Licklider Transmission Protocol (LTP) DTN to MQTT Value Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  8. 8. MQTT PUBLISH BP TRAFFIC MQTT SUBSCRIBE 8 11/11/16 DTN to MQTT integration Hardware and Software Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  9. 9. MQTT ROUTING via DTN Best effort MQTT Bridging via bundles 9 11/11/16 Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  10. 10. 10 11/11/16 MQTT ROUTING via DTN Assured MQTT Bridging via custody transfer bundles Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  11. 11. • Schedules best effort or assured data propagation • Decouples local fanout from bridging propagation 11 11/11/16 • Events are routed progressively via intermediary nodes • Results in lower latency and higher throughput MQTT ROUTING via DTN Thoughts Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  12. 12. 12 11/11/16 MQTT ROUTING via LTP Bridging MQTT brokers via LTP UDPCL Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  13. 13. • Unidirectional, Connectionless, Efficient flow • Fragmentation support • Flow control via LTP sessions • Multi stream (red, green) assured data propagation • Link layer agnostic 13 11/11/16 MQTT ROUTING via LTP Thoughts Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  14. 14. Interested? Contact me • To find out more • To collaborate 14 11/11/16 Earth & Space IOT research What if we combined them? Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks
  15. 15. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks References • Advances in Delay-tolerant Networks (DTNs) – Architecture and Enhanced Performance By Woodhead Publishing 2015 11/11/16 15 • Brenda Lyons, NASA NDBS 2016 • Appendix A: What is IOT? • Appendix B: What is DTN? • Appendix C: What is MQTT? • Further Research A: DTN routing via MQTT • Further Research B: High availability with MQTT • Further Research C: High availability with MQTT & DTN
  16. 16. Appendix A Never heard of Internet of Things? Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 1611/11/16 • Read about • History and status • M2M Comparison • Market & estimates • Example architectures
  17. 17. A IOT & M2M Comparison… • M2M came first during the telecom era • M2M: proprietary, closed, private machine communication (e.g. CallerID) • IOT is much more than machine communication Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 1711/11/16
  18. 18. A IOT & M2M Comparison…(2) • IOT: devices connect to a larger network, usually via a gateway, are cloud enabled, can interop, enables device physical reach (e.g. Samsung SmartThings) • IOT – M2M common: machine comms Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 1811/11/16
  19. 19. IOT Market Estimates… • 2020: • 25-30 billion Connected devices • Today: • 6.4-9 billion Connected devices !This excludes phones, tablets, PCs! Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 1911/11/16
  20. 20. IOT IMPORTANCE Status… • Ongoing transport standardization efforts • MQTT, COAP, AMQP, Websocket, Node • Enormous telco traffic challenge • Global M2M traffic:1 to 6.3 Exabytes / month by 2020 SOURCE • Real time streaming analytics • Security nightmare • 1TB DDOS by hacked CCTVs • PCs, PVRs, routers, smart fridge • MCUs + PKI = slow • Privacy concerns Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 2011/11/16
  21. 21. IOT Example Smart Home… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 21 11/11/16
  22. 22. IOT Example Smart Home… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 22 11/11/16 Why cant all things use WiFi? • If mains powered, sometimes they do • WiFi may not reach your boiler room • A lot of battery powered – low energy sensors • WiFi lacks ad-hoc network features (e.g. mesh) Things tend to be in their own sensor/field net • Based on low power RF (e.g. zigbee, zwave) • Needs a gateway to bridge to local IP network
  23. 23. Smart Home IOT Architecture GSM/DSL/ FIBRE HTTP MQTT AMQP Ethernet / WiFi HTTP MQTT AMQP Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 23 11/11/16 Users interact with things via a device …
  24. 24. Smart Home IOT Architecture (2) GSM/DSL/ FIBRE HTTP MQTT AMQP RF ZigBee ZWave BLE Ethernet / WiFi HTTP MQTT AMQP Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 2411/11/16 Things interact with environment via gateway…
  25. 25. Smart Home IOT Architecture (3) GSM/DSL/ FIBRE HTTP MQTT AMQP RF ZigBee ZWave BLE Ethernet / WiFi HTTP MQTT AMQP Cloud Platform Services Cloud services add value & remote access
  26. 26. Smart Home IOT Architecture (4) GSM/DSL/ FIBRE HTTP MQTT AMQP RF ZigBee ZWave BLE Ethernet / WiFi HTTP MQTT AMQP Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 2611/11/16
  27. 27. Appendix B Never heard of DTN? Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 2711/11/16 • Read about • history and status • Protocol overview • TCP/IP Comparison • Terrestrial uses
  28. 28. DTN History… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 2811/11/16 initiative to develop an “Interplanetary Internet” 1998 NASA starts Delay / Disruption Tolerant Networking 1998 NASA leads worldwide DTN development 2005 IOAG Recommends Solar System Internetwork (SSI) 2010 IOAG Suggests DTN a core service of SSI 2010 DTN is being standardized by IETF DTN is being standardized by CCSDS
  29. 29. DTN Solar System Internet… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 2911/11/16
  30. 30. DTN • The (DTN) protocol suite • Extends the terrestrial Internet capabilities • Works • in challenged communication environments • over unidirectional/asymmetric links • Tolerates • frequent disruptions • long delays / high error rates • Provides • assured delivery of data • automatic store-and-forward • Can run over IP or as a full Internetworking protocol Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3011/11/16 Overview
  31. 31. DTN Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3111/11/16 Overview (2)
  32. 32. DTN Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3211/11/16 TCP/IP Comparison
  33. 33. DTN IP: Must wait for complete path Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3311/11/16 TCP/IP Comparison (2)
  34. 34. DTN TCP/IP Comparison (3) DTN: Incremental progress without complete path Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3411/11/16
  35. 35. DTN DTN decouples data propagation from link layer characteristics • Can be used to propagate data not needed in real time (e.g. wearables, home automation etc) • Introduces some node storage requirements • Use cases • sea-going vessels • outdoor explorers • wildlife research • vehicular networks • disaster area networks Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3511/11/16 Terrestrial use
  36. 36. Appendix C Never heard of MQTT? Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3611/11/16 • Read about • History and status • Protocol overview • Role in IOT architectures
  37. 37. MQTT History… Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3711/11/16 IBM & Eurotech develop 1990s IBM & Eurotech Open Source 2012 Software AG implements 2013 OASIS MQTT TC Created 2013 Many others join MQTT becomes OASIS standard 2014 MQTT becomes ISO/IEC 20922 2016
  38. 38. MQTT Description • MQTT is • Is an open, simple, lightweight publish- subscribe binary messaging protocol • Is easy to implement (15 protocol commands) • has a fixed 2 byte over-the-wire overhead header • Has 256MB max payload • runs on top of TCP/TLS/WS/WSS stacks Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3811/11/16
  39. 39. MQTT Description (2) • MQTT offers 3 quality of service levels: • QOS 0: At most once (Fire and forget) • QOS 1: At least once (ACKed delivery) • QOS 2: Exactly once (ACKed / No duplicates) Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 3911/11/16
  40. 40. MQTT Description (3) • Information on software/hardware that uses MQTT: http://ow.ly/wHm2e • The Eclipse Paho project offers some great MQTT clients in minimal C, regular C, C++, Java, Javascript, Python, Lua, Go, Objective- C: http://ow.ly/wHl8c Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4011/11/16
  41. 41. MQTT in IOT Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4111/11/16
  42. 42. MQTT in IOT (2) Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4211/11/16
  43. 43. MQTT in IOT (3) MQTT BRIDGE MQTT PUB & SUB MQTT SUB Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4311/11/16
  44. 44. Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4411/11/16 Further Research A DTN routing via MQTT
  45. 45. DTN ROUTING via MQTT One to many Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4511/11/16
  46. 46. DTN ROUTING via MQTT Many to one Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4611/11/16
  47. 47. DTN ROUTING via MQTT Thoughts • Unicast routing focus • Data sharing importance: • Vehicular networks • Disaster area networks • Delegates shared custodian to MQTT broker • Can use various MQTT QOS Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4711/11/16
  48. 48. Further Research B High availability with MQTT Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4811/11/16
  49. 49. IOT with MQTT HA architecture MQTT PUB & BRIDGE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 4911/11/16
  50. 50. IOT with MQTT HA active – active cluster architecture MQTT PUB PROPRIETRY QUORUM REPLICATE SYNC COORDINATE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5011/11/16
  51. 51. IOT with MQTT HA active – active: master failure MQTT PUB PROPRIETRY QUORUM REPLICATE SYNC COORDINATE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5111/11/16
  52. 52. IOT with MQTT HA active – active: publisher failover MQTT PUB PROPRIETRY QUORUM REPLICATE SYNC COORDINATE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5211/11/16
  53. 53. IOT with MQTT HA active – active: node rejoin MQTT PUB PROPRIETRY QUORUM REPLICATE SYNC COORDINATE Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5311/11/16
  54. 54. IOT with MQTT HA active – active: multiple streams MQTT PUB STREAM 1 MQTT PUB STREAM 2 MQTT PUB STREAM 3 Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5411/11/16
  55. 55. Further Research C High availability with MQTT & DTN Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5511/11/16
  56. 56. MQTT HA via DTN Eventual consistency … Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5611/11/16
  57. 57. MQTT HA via DTN Master failure Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5711/11/16
  58. 58. MQTT HA via DTN Partial sync while offline Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5811/11/16
  59. 59. MQTT HA via DTN Fast node cluster rejoin Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 5911/11/16
  60. 60. MQTT HA via DTN Thoughts • Schedules best effort or assured data propagation • Decouples local fanout from slave propagation • Bundles continue to flow while broker offline • Allows fast cluster recovery Alex Kritikos - Thoughts on the Future Development of Ad hoc and Wireless Sensors Networks 6011/11/16

×