The document discusses the role of a middle layer between IoT devices and cloud computing resources. It presents several alternatives for the middle layer, including IoT gateways, edge/fog computing, and multi-level architectures. The optimal approach depends on the use case. For large-scale applications, a multi-level architecture with components at the device, edge, and cloud layers will likely be necessary. The middle layer poses challenges around data processing, communication standards, and extending cloud models to support IoT applications.

1. IoT to Cloud: Middle Layer
(e.g. Gateways, Hubs, Fog, Edge Computing)
Bob Marcus
Co-Chair NIST Big Data PWG
robert.marcus@et-strategies.com
Caveat: This is a rough first cut and will be revised extensively!
Tuesday, May 17, 16

2. Key Points on Middle Layer IoT to Cloud - Initial Thoughts
• There are several alternatives for interfacing device level IoT to large-scale elastic
computing resources (i.e. Cloud)
• These alternatives include IoT gateways to Cloud, IoT gateways to edge processing (e.g.
Fog Computing, and multilevel IoT interfaces to edge processing and edge processing
interfaces to Clouds.
• The architectural approach will depend on the Use Case
• For large scale applications and system of systems (e.g. Smart City), it seems that the
multilevel architecture will be necessary
• Converting Cloud analytics into actuator control commands through a middle layer is a
challenge that will be discussed in the slide set “Control of CPS Systems”.
http://www.slideshare.net/bobmarcus/control-in-cyberphysical-systems
• Data and communication standard formats, protocols, and APIs will be necessary for
interoperability between IoT and Clouds. See the slide set on “CPS Standards” at
http://www.slideshare.net/bobmarcus/standards-and-open-source-for-big-data-cloud-and-iot
Tuesday, May 17, 16

3. Outline of Slides
・Terminology
・CPS Architecture Alternatives
・Middle Layer Architectures
Tuesday, May 17, 16

4. Terminology
Tuesday, May 17, 16

5. Terminology
Cloud - One or more data centers supporting large scale storage and processing
Things - Devices that interact with physical world (e.g. sensors, actuators)
Internet of Things - Devices and local cyber components connected to networks
Analytics - Processing of data to understand past events and support future decisions
Big Data - Data that requires parallel or distributed processing for analytics
Middle Layer - Cyber components between devices and Clouds
Cloud Facing Sublayer - Middle Layer components interfacing with Cloud
Device Facing Sublayer - Middle Layer components interfacing with Devices in IoT
Tuesday, May 17, 16

6. Terminology continued
・Middle Layer - Cyber components between devices and Cloud Data Centers
IoT Device Gateways - Connect devices to network nodes and/or data centers
Edge/Fog Computing - Distributes some of the resources and services of computation,
communication, control, and storage away from Cloud and closer to devices and gateways
IoT Cloud API Gateways - Connects Cloud with intermediate network node and/or device gateways
Hubs - Manage devices, collect device data, process device data, and distribute data and messages
Hub Locations - Hubs can be located in the IoT, Fog and/or Cloud
Hub Roles - Key differentiator for architectures is Hubs locations, capabilities, and interactions
Hub Capabilities - Hubs can have different capabilities based on location
Hub Sizes - There may be many small IoT hubs, fewer larger Fog Hubs, and several very large Cloud Hubs
Hub Standardizations - Standardized Hub interfaces and capabilities will enable open architectures
Tuesday, May 17, 16

7. CPS Architecture Alternatives
Tuesday, May 17, 16

8. IoT to Cloud
Clouds
Gateways
Centralized Processing with Large Data Flows
Tuesday, May 17, 16

9. Fog Computing with no Cloud
Hub Sublayer
Device Facing Sublayers
Decentralized Processing with Reduced Data Flows
Tuesday, May 17, 16

10. Middle Layer Between IoT Devices and Clouds (most scalable)
Clouds
Centralized Processing as needed with Decentralized Processing to Reduce Data Flows
Fog Layers
Tuesday, May 17, 16

11. Middle Layer Architectures
Tuesday, May 17, 16

12. From http://www.slideshare.net/IoTBruce/the-iot-food-chain-picking-the-right-dining-partner-is-important-with-dean-freeman-of-gartner/28
IoT Platform Functionality from Gartner
Tuesday, May 17, 16

13. Middle Layers Between Sensors and Clouds from ChainLink Research
From http://www.chainlinkresearch.com/research/detail.cfm?guid=E1C1B535-3048-79ED-9939-6956607B0CED
Tuesday, May 17, 16

14. Device to Cloud Networks from Datafloq
From https://datafloq.com/read/fog-computing-vital-successful-internet-of-things/1166
Tuesday, May 17, 16

15. IoT Architecture from Cloud Standards Customer Council
From http://www.cloud-council.org/deliverables/CSCC-Cloud-Customer-Architecture-for-IoT.pdf
Tuesday, May 17, 16

16. DeviceHive D-Bus connects Multiple Devices to DeviceHive Cloud
From docs.devicehive.com/docs/iot-toolkit-overview
Tuesday, May 17, 16

17. Middle Layer from MIMOS Berhard
From http://www.slideshare.net/mazlan1/internet-of-things-iot-we-are-at-the-tip-of-an-iceberg/
Tuesday, May 17, 16

18. Middle Layer from Black Duck
From http://www.slideshare.net/blackducksoftware/io-t-and-open-source
Tuesday, May 17, 16

19. Multiple Layers for CPS from Devices to Cloud by Bob Marcus
Tuesday, May 17, 16

20. Open Source in the IoT Stack from Black Duck
From http://www.slideshare.net/blackducksoftware/io-t-and-open-source
Tuesday, May 17, 16

21. Open Source Roles + Technologies in the IoT Stack from Black Duck
From http://www.slideshare.net/blackducksoftware/io-t-and-open-source
Tuesday, May 17, 16

22. EU OpenIoT Architecture
From http://www.slideshare.net/jpcik/xgsn-openiot-summerschool
Tuesday, May 17, 16

23. AnotherView of EU OpenIoT Architecture
From http://www.amazon.com/Interoperability-Open-Source-Solutions-Internet-Things/dp/3319165453
Tuesday, May 17, 16

24. Publish/Subscribe Middleware for EU OpenIoT Architecture
From http://www.amazon.com/Interoperability-Open-Source-Solutions-Internet-Things/dp/3319165453
Tuesday, May 17, 16

25. Ayla PaaS IoT Cloud
From https://www.aylanetworks.com/products/iot-platform
Tuesday, May 17, 16

26. Device to Cloud Architecture using MQTT and OGC Standards
From http://www.opengeospatial.org/blog/23155453
SWE = Sensor Web Enablement
Tuesday, May 17, 16

27. IoT Categories and Open Source Business from Black Duck
From http://www.slideshare.net/blackducksoftware/io-t-and-open-source
Tuesday, May 17, 16

28. Propagator and Integrator Middle Layer from Francis de Costa
From https://www.capgemini.com/blog/capping-it-off/2014/02/things-on-the-internet-to-the-internet-of-things
Tuesday, May 17, 16

29. From http://www.mdpi.com/1424-8220/15/4/7172/pdf
Network Elements in Cyber-Physical Systems
Tuesday, May 17, 16

30. Discovery and Broker Middle Layer from Geoffrey Fox
From http://www.slideshare.net/Foxsden/high-performance-processing-of-streaming-data
Tuesday, May 17, 16

31. Enterprise Data Hub from Cloudera
From http://www.cloudera.com/resources/recordedwebinar/enterprise-data-hub--the-next-big-thing-in-big-data.html
Tuesday, May 17, 16

32. 2001 Internet Dispatcher from Rogue Wave Software
From http://www.slideshare.net/bobmarcus/2001-talk-on-possible-research-collaboration-in-boulder
Tuesday, May 17, 16

33. Middle Layer is the Challenge
• Multiple sublayers + Cloud needed for large-scale Smart X applications
• Must connect to all Devices
• Must connect to Clouds when necessary
• Must extend Cloud and Big Data Processing Models
• Middle Layer interconnections must support needed communication
• Nodes must have significant capabilities e.g for reducing data flows
• Security, Privacy, System Management must be supported
Tuesday, May 17, 16

34. Example: Smart City Architecture with Middle Layer from U of RI
From http://dl.acm.org/citation.cfm?id=2818898&CFID=733103700&CFTOKEN=57270032 (AMS Digital Library)
Tuesday, May 17, 16

35. Connecting Middle Layer to Devices
• Need to manage devices and device connections
• Collect, process and forward data from devices
• Send commands to devices
• Connect to device networks and Internet
• Report status of devices to other sublayers
Tuesday, May 17, 16

36. Connecting Middle Layer to Cloud
• Need to transmit processed data from devices to Cloud
• Need to transmit device status information to Cloud
• Need to transmit commands from Cloud to lower sublayers
• Need to respond to queries from Cloud
• Need to be configurable from Cloud
Tuesday, May 17, 16

37. Extending Cloud Processing Models
• Extending Cloud delivery models
• Services provided through interface to Cloud
• Service delivery can be IaaS, PaaS, or SaaS
• Client must specify data sources and actuators of interest
• Cloud uses and/or supplies appropriate Middle Layer resources for delivery model
• Extending Cloud deployment models
• Middle Layer resources may be public, shared, private, or hybrid
• Cloud and device resources can also be public, shared, private, or hybrid
• Client must specify deployment model for Middle Layer and devices
• Cloud delivers Middle Layer and device resources based on client specification
Tuesday, May 17, 16

38. Sensor Clouds from the University of Indiana
From http://grids.ucs.indiana.edu/ptliupages/publications/SensorCloudsWhitePaper.pdf
Tuesday, May 17, 16

39. IoT, Big Data, and Cloud Delivery Models
From http://www.slideshare.net/sridharv11/cloud-iot-bigdata
Tuesday, May 17, 16

40. IoT Driving Adoption of PaaS Delivery Models
From http://www.forbes.com/sites/mikekavis/2016/03/03/iot-will-drive-paas-adoption-just-not-the-way-youd-expect/
Tuesday, May 17, 16

41. Big Data Processing Models
• Middle Layer must handle volume, velocity, variety, and variability of data
• Volume: May require filtering/aggregation of data to reduce network traffic
• Velocity: Middle Layer processing may be necessary for real-time performance
• Variety: Diverse types of data will require transforming and integration
• Variability: Elastic use of Middle Layer resources may be necessary
• Stream processing analytics can be performed in Middle Layer
• Interactive querying could be supported by Middle Layer
Tuesday, May 17, 16

42. Examples of Extending Cloud and Big Data Processing Models
IaaS - General Electric’s Predix
http://cloudtweaks.com/2015/08/general-electrics-predix-cloud-building-iot-applications/
PaaS - Advantech’s Wise and Carriot Platform
http://www.advantech.com/embedded-boards-design-in-services/wisepaas
https://www.carriots.com/what-is-carriots
SaaS - Salesforce IoT Cloud
http://www.salesforce.com/iot-cloud/
Cloud Deployment Models extended to IoT (Book on “IoT in the Cloud”)
http://tinyurl.com/jgtuq85
Stream Analytics for IoT
http://www.sas.com/content/dam/SAS/en_gb/doc/other1/events/sasforum/slides/day2/pjone-internet-things.pdf
https://azure.microsoft.com/en-us/services/stream-analytics/
Edge Computing Analytics - Overview and vendor list
https://www.abiresearch.com/market-research/product/1021642-edge-analytics-in-iot/
MapReduce on the IoT
http://www.sciencedirect.com/science/article/pii/S1877050915024357/pdf?
md5=cee2551e9af4f07baa63f1265f3e1512&pid=1-s2.0-S1877050915024357-main.pdf
IBM Project Compose
http://ibmresearchnews.blogspot.ca/2014/02/connected-living-new-applications-for.html
Tuesday, May 17, 16

43. From http://www.ic.unicamp.br/~bit/mo809/seminarios/Marcio-Fog/suporte/Fog%20Computing-%20A%20Platform%20for%20Internet%20of%20Things%20and%20Analytics.pdf
Example of Multilevel Analytics from U of Campinas in Brazil
Tuesday, May 17, 16

44. IoT to Cloud through Distributed Broker from U of Indiana
From http://grids.ucs.indiana.edu/ptliupages/publications/cts_2012_paper-2.pdf
Tuesday, May 17, 16

45. From www.slideshare.net/honbozhou/unified-middleware-for-internet-of-things
Cloud of Things from HonBo Zhou
Tuesday, May 17, 16

46. A Cisco Fog Computing to Cloud Reference Architecture
From http://blogs.cisco.com/innovation/openfog-consortium-announcement
Tuesday, May 17, 16

47. Fog/Edge to Cloud Computing Overview from Cisco
From ubiquity.acm.org/article.cfm?id=2822875
Tuesday, May 17, 16

48. Internal Interconnections within the Middle Layer
• Distribution of data and status from lower sublayer
• Transmitting commands and configuration from upper sublayer
• Relaying messages, data, commands, and status
• Support messaging, middleware, and data format standardization
• Managed network connections
Tuesday, May 17, 16

49. Node Capabilities
• Importing, caching, transforming, and distributing data
• Analytics based on data received and cached
• Creating and transmitting commands based on analytics
• Managing network and node status information
• Receiving, executing, and relaying external commands
• Responding to queries
Tuesday, May 17, 16

50. Security
System Management
Message
Processing
Data
Processing
Data
Storage
Coordinator
Data,
Status,
Alerts
Commands,
Queries
Peer
Messages
Peer
Data
Data,
Status,
Alerts
Commands,
Queries
Possible Middle Layer Node from Bob Marcus
Tuesday, May 17, 16

51. References
Book on “The Internet of Things in the Cloud: A Middleware Perspective”
https://www.crcpress.com/The-Internet-of-Things-in-the-Cloud-A-Middleware-Perspective/Zhou/9781439892992
Book on “Rethinking The Internet of Things”
http://www.amazon.com/Rethinking-Internet-Things-Connecting-Everything-ebook/dp/B00EBCGHSW
IoTCloud from the University of Indiana
http://www.hindawi.com/journals/js/2015/468047/
http://iotcloud.github.io/
Fog Computing Overview
http://arxiv.org/ftp/arxiv/papers/1601/1601.02752.pdf
Tuesday, May 17, 16