Describes extensions of current NIST Reference Architectures and Frameworks that are needed to handle CPS System of Systems Use Cases (e.g. Smart Grid, Smart City). These extensions include Data Hubs and CPS Hubs.

1. Reference Architecture for
CPS Systems of Systems (SoS)
using Data Hubs and CPS Hubs
Bob Marcus
Co-Chair NIST Big Data PWG
robert.marcus@et-strategies.com
Tuesday, May 10, 16

2. Outline of Slides
・Extending NIST Big Data Reference Architecture to CPS SoS
・Extending NIST Cloud Reference Architecture to CPS SoS
・Extending NIST CPS Framework to CPS SoS
・Mapping Smart Grid and Smart City to NIST Reference Architectures
Tuesday, May 10, 16

3. Key Points
・Many CPS applications (e.g. Smart X) will involve systems of systems (SoS)
・NIST Reference Architectures needed to be extended for SoS
・The current Big Data Reference Architecture assumes closely coupled processing
・Terminology: The Big Data Reference Architecture models a “Data Hub”
・Loosely coupled data processing SoS can be modeled by connecting multiple Data Hubs
・These Data Hubs can range in size from device gateways through Fog to Clouds
・The current Cloud Reference Model assumes elastic data center resources
・Cloud delivery and deployment models should be extended to SoS (e.g. Smart X)
・The current CPS Reference Framework assumes a single CPS system
・ This Framework must be extended to CPS SoS for Smart X applications
・Terminology: A building block for CPS SoS is a “CPS Hub
・A CPS Hub has Data Hub capabilities + the ability to generate and process commands
・Loosely coupled CPS SoS can be modeled by connecting multiple CPS Hubs
Tuesday, May 10, 16

4. Extending Big Data Reference Architecture
to Cyber-Physical System of Systems
External Slides used as Examples of Architecture Models
Tuesday, May 10, 16

5. NIST Big Data Reference Architecture
From http://bigdatawg.nist.gov/_uploadfiles/M0397_v1_2395481670.pdf
Tuesday, May 10, 16

6. Key Points for Big Data Reference Architecture for CPS
• Large scale CPS System of Systems (SoS) will produce immense amounts of data generating
requirements beyond the current scale of “Big Data” volume, velocity, and/or variety.
• Examples of CPS SoS include most Smart X applications. See http://www.slideshare.net/
bobmarcus/iot-use-cases
• The requirements for these SoS (e.g. performance, resource constraints, modularity, privacy) will
necessitate hierarchical processing
• This hierarchy can include extremely large number of device layer nodes, large numbers of Fog
layer intermediate nodes, and multiple Cloud capabilities
• See the interface, middle layer, and data processing slides accessible from http://
www.slideshare.net/bobmarcus/inventory-of-my-cps-slide-sets
• It will be necessary to extend the current NIST Big Data Reference Architecture (BDRA) to
support the many emerging hierarchical CPS SoS applications
• Consider the BDRA as a model for a “Data Hub”
• Data processing SoS can be modeled by connecting multilayer Data Hubs
• A key question is: What are the similarities and differences of the RA roles in different layers?
• What interfaces are needed between roles (e.g. System Orchestrators) in different layers?
Tuesday, May 10, 16

7. Enterprise Data Hub from Cloudera
From http://www.cloudera.com/resources/recordedwebinar/enterprise-data-hub--the-next-big-thing-in-big-data.html
Tuesday, May 10, 16

8. Rotated Modified NIST Big Data Reference Architecture
Data Output Interface
for Data Consumers
Data Input Interface
for Data Providers
Data Hub
Framework Provider
Data Hub Application ProviderData Hub
Orchestrator
Output Data
Input Data
Service
Use
Service
Use
Note that Data Consumer and Data Provider are replaced by interfaces that can be used by multiple providers or consumers
Data Hub
Tuesday, May 10, 16

9. Hierarchical Smart City Architecture from U of Rhode Island
From http://dl.acm.org/citation.cfm?id=2818898&CFID=733103700&CFTOKEN=57270032 (AMS Digital Library)
Tuesday, May 10, 16

10. Data
Data Data
Data Data
Data
Gateway Layer e.g.
Thousands of Nodes
Fog Layer e.g.
Hundreds of Nodes
Cloud Layer e.g.
Multiple Clouds
Recursive Rotated NIST Big Data Reference Architecture for IoT
Cloud Data Hub
Fog Data Hub Fog Data Hub
Device Data Hub Device Data Hub Device Data Hub
Data
Devices Layer e.g.
Millions of Nodes Sensors Sensors Sensors SensorsSensors Sensors
Tuesday, May 10, 16

11. Extending NIST Cloud Reference Architecture
to Cyber-Physical System of Systems
External Slides used as Examples of Architecture Models
Tuesday, May 10, 16

12. Cloud Reference Architecture
From http://www.nist.gov/customcf/get_pdf.cfm?pub_id=909505
Tuesday, May 10, 16

13. Questions for Cloud Reference Architecture for Layered CPS
• Should there be Data Provider Roles to input data into the Cloud?
• Can Delivery Models (IaaS, PaaS, SaaS) be extended to lower layers (e.g. Fog)
• Can Deployment Models(Private, Public, Hybrid, Community) be extended to lower layers?
• Are Deployment Models captured in the Cloud Reference Architecture?
Tuesday, May 10, 16

14. ITU Big Data in a Cloud Context (IoT is a possible CSN)
From https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-Y.3600-201511-I!!PDF-E&type=items
Tuesday, May 10, 16

15. IoT PaaS Stack from Mulesoft
From http://www.slideshare.net/sumitcan/iot-architecture
iPaaS = Integration Platform as a Service (https://www.mulesoft.com/resources/cloudhub/what-is-ipaas-gartner-provides-reference-model
Tuesday, May 10, 16

16. Extending NIST CPS Framework
to Cyber-Physical System of Systems
External Slides used as Examples of Architecture Models
Tuesday, May 10, 16

17. Domain Framework for CPS Systems
From http://www.cpspwg.org/Portals/3/docs/CPS%20PWG%20Draft%20Framework%20for%20Cyber-Physical%20Systems%20Release%200.8%20September%202015.pdf
Tuesday, May 10, 16

18. Suggestion for NIST’s CPS Framework Extension
• NIST’s CPS Framework models a single CPS System
• It is necessary to extend the Framework to Systems of Systems for Smart X applications
• This would be a first step towards a Smart City Framework for NIST
www.nist.gov/public_affairs/releases/upload/smartcities_cps_budgetsheet.pdf
• This extension should probably precede the building of realistic testbeds
• Terminology: A CPS Hub is a node that processes data, commands, messages, and queries
• A loosely coupled CPS SoS can be modeled by connected multiple CPS Hubs
• These Hubs can range in scale from device interface level to Cloud
Tuesday, May 10, 16

19. CPS and Smart Grid as System of Systems from NIST
From www.nist.gov/smartgrid/upload/SGAC-Meeting-Presentations.pdf
Tuesday, May 10, 16

20. Node in a Hierarchical Control System from James Albus of NIST
From https://en.wikipedia.org/wiki/Hierarchical_control_system
Tuesday, May 10, 16

21. Data Hub Extended to CPS Hub
External Output Interface
External Input interface
Subsystem Framework
Provider (e.g. Data
Stores, Software and
Hardware Resources)
Subsystem Application
Provider (e.g. Analytics,
Services, Data Processing)
Subsystem Orchestrator
(e.g. Management of
Resources and
Processing)
Input Data
Service
Interfaces
Note that Data Consumer and Data Provider are replaced by interfaces that can be used multiple providers or consumers
CPS Hub
Output Data
Commands,
Messages,
Requests
Service
Interfaces
Commands,
Messages,
Requests
Data
Tuesday, May 10, 16

22. IoT Cloud Service Centric Landscape for Service Providers
From http://dw.connect.sys-con.com/session/2999/Dennis_Ward.pdf
Tuesday, May 10, 16

23. An Example of a CPS Hub-based Architecture for IoT
Data
Data Data
Data
Data Data
CPS Hub
CPS Hub CPS Hub
CPS Hub
CPS Hub CPS Hub
Data
Commands
Commands CommandsCommands
Commands
Commands
SensorsActuators ActuatorsActuators Sensors
Sensors
Commands Commands
CommandsDataData
Data
Tuesday, May 10, 16

24. Mapping Smart Grid and Smart City
to NIST Reference Architectures
Tuesday, May 10, 16

25. NIST Conceptual Architecture for the Smart Grid
From http://www.multispeak.org/ABOUT/SMARTGRID/Pages/MultiSpeakandtheNISTSmartGrid.aspx
and www.nist.gov/smartgrid/upload/NIST_Framework_Release_2-0_corr.pdf
Tuesday, May 10, 16

26. Smart Grid Questions
• How does NIST’s Smart Grid Conceptual Model map to the Big Data Reference Architecture?
• How does NIST’s Smart Grid Conceptual Model map to the Cloud Reference Architecture?
• How does NIST’s Smart Grid Conceptual Model map to future CPS Frameworks ?
Tuesday, May 10, 16

27. Suggestions for Extending Current NIST Reference Architectures
• The NIST Public Working Groups should try to map CPS System of Systems Use Cases (e.g.
Smart Grid, Smart City) to their Reference Architectures and Frameworks
• This will require extending these Reference Architectures and Frameworks to handle
hierarchical resource allocation, data processing, security, privacy, and control from device
layers through heterogeneous subsystems to Cloud-based analytics
• One possible approach is to use the current Reference Architectures as models for “Data
Hubs’ and “CPS Hubs” (Data Hub + Command Processing)
• Loosely coupled CPS SoS can then be modeled as connected multilayer Hubs
• It will be useful to have a planning discussion among the NIST Cloud, CPS, and Big Data
Public Working Groups to scope out and harmonize future extensions to support Smart X
applications
• The extended Reference Architectures and Frameworks could contribute to NIST’s Smart
Cities Program www.nist.gov/public_affairs/releases/upload/smartcities_cps_budgetsheet.pdf
Tuesday, May 10, 16

28. Mapping Large-Scale Scientific Applications
to the NIST Big Data Architecture
External Slide used as Example of Architecture Models
Tuesday, May 10, 16

29. Top Down Large Scale Scientific Applications
• Some large-scale scientific applications can be considered CPS applications
• Typically large data sets can be gathered from one or more sources into centralized locations
• Some data processing (e.g. filtering, analytics) can be performed in the centralized locations
• Data can be distributed from these centralized locations to many other processing locations
• This data distribution can continue recursively to even more processing locations
• The data analytics can be increasingly less resource intensive at the distributed locations
• These architectures can also be modeled by connecting Data Hubs
• The data flows are top-down rather than bottom-up as in IoT applications
Tuesday, May 10, 16

30. World-Wide Large Hadron Collider(LHC) Computing Grid
From https://sciencenode.org/feature/how-grid-computing-helped-cern-hunt-higgs.php
Tuesday, May 10, 16

31. Data
Data
Data
Data
Data
Data
Tier 2
Tier 1
Tier 0
Recursive Rotated NIST Big Data Reference Architecture for LHC
Data Hub
Data Hub Data Hub
Data Hub Data Hub Data Hub
Tuesday, May 10, 16