This document discusses semantic interoperability and reasoning techniques for heterogeneous IoT devices and data in smart buildings. It describes using ontologies and semantic annotations to model building components, properties, and their relationships. Semantic matching of component inputs and outputs can then enable automatic configuration of monitoring and control systems based on the available devices. Reasoning over the semantic knowledge graph allows reconfiguration when devices are added, removed or properties change over time.

1. ECVET Training for Operatorsof IoT-enabledSmart Buildings (VET4SBO)
2018-1-RS01-KA202-000411
Level 3
Module 3: State-of-the-art operation and maintenance
practices for sustainable buildings
Unit 3.2: Semantic interoperability and semantic
reasoning techniques to address heterogeneity of
devices and data

2. Outline
1. Logic Theories, Deductive Inference and Declarative
Languages
2. The vision of semantics in smart buildings
3. Revision of monitoring and control components
4. Building “Things” knowledge modelling
5. Semantic annotation of building “Things”
6. Semantic matching of “Things” in buildings
7. Semantic reasoning for automatic configuration

3. Logic Theories, Deductive Inference and Declarative
Languages
Background knowledge
Read the short document with name:
GMilis-LogicTheory&Inference-v1.0

4. The vision of semantics in smart buildings
The configuration or reconfiguration needs of feedback control
systems in large-scale systems, including buildings, where the
availability of sensors, actuators, controllers and other data
processing and analytics functions changes dynamically over time,
can be effectively automated through the use of ontology-based
knowledge models and deductive inference techniques (see later).
These techniques facilitate the automatic management of
information about the IoT components, the storage of knowledge
about feedback control engineering, as well as the implementation of
necessary reasoning algorithms.

5. The vision of semantics in smart buildings
This functionality can be offered through appropriate software solutions that act as
supervisory systems and undertakes to communicate with installed components, as
well as with human operators and cloud services, “understand” what sensing,
actuation, processing and control capability is available in the building and “think”
on behalf of the operators/engineers to appropriately re-configure all feedback
control loops.
The content of this Unit is largely based on recent work in IoT, about “SEMIoTICS:
Semantically-Enhanced IoT-enabled Intelligence Control Systems” [1, 2]. The
implementation of the solution adds a middle layer between the human operators,
e.g. control engineers and the IoT components installed in the building for the
monitoring and control of certain properties.

6. The vision of semantics in smart buildings

7. Revision of monitoring and control components
• xp
(𝑘), xp
(𝑘 − 1) ∈ 𝑅 𝑛 𝑥 : vector of state-variablesand memory, of dimension nx,
• 𝑓 𝑝
(. ): the dynamicsof the system to be monitored/controlled
• vp 𝑘 , wp 𝑘 ∈ 𝑅 𝑛 𝑣: controlled/uncontrolledinputvector
• φ 𝑘 : faults in the buildingsystems
• h(𝑘) : input signal produced by third interdependentsystems
• ζp
𝑘 : vector of other parameters related to the buildingsystem dynamics

8. Revision of monitoring and control components
• 𝑣 𝑎
(𝑘); 𝑣 𝑎
(𝑘 − 1) : actuator output signal and memory
• fa
(. ) : actuator output dynamics
• ua
(𝑘) : signal that drives the action
• ζa
(𝑘) : parameters vector

9. Revision of monitoring and control components
• 𝑦s
(𝑘): sensor output signal
• 𝑓 𝑠
(. ): sensor output dynamics
• 𝑥 𝑠
(𝑘) : sensor input vector. States of building properties.
• 𝜁 𝑠
(k): parameters vector

10. Revision of monitoring and control components
• uc(𝑘): control decision signal
• fc(. ): controller output dynamics
• yc
(𝑘): controller input vector (representing the plant's feedback as given to the
controller)
• 𝜁 𝑐
(𝑘) : parameters vector (including the reference trajectory r(k))

11. Revision of monitoring and control components
• 𝒚′ 𝒌 , 𝒖 𝒌 : processed measurementsignal and processed control decision respectively
• 𝒇 𝒚(.), 𝒇 𝒖(. ): pre- and post-control functions’ implementations
• 𝒚 𝒚
𝒌 : pre-control function input signal (statemeasurements vector)
• 𝜻 𝒚
𝒌 , 𝜻 𝒖
(𝒌): parameters vectors of pre- and post- control processing functions respectively
• 𝒖 𝒖 𝒌 : post-control function input signal (control decision signal)

12. Building “Things” knowledge modelling
Set of Things / knowledge objects
Example system

13. Building “Things” knowledge modelling
Classes / Types of Things

14. Building “Things” knowledge modelling
Relation (Bipartite) Graphs

15. Semantic annotation of building “Things”
As has been seen, all building and IoT component knowledge is modelled in a
big graph. The graph described all considered types of control system
components: Building dynamics, Sensors, Actuators, Controllers, Processing
Functions (Pre-Control, Post-Control and Parameter Functions).
The design of the graph can be based on the OWL-S “Service Profile” model
[3], which facilitates the modelling of the service offered by each type of
component.
i.e. each component has inputs, outputs, parameters, as well as some
additional information for its categorisation.

16. Semantic annotation of building “Things”
The semantic characterisation of the control system components is mainly based on the SSN ontology
[4]. The SSN ontology defines sensors and actuators as “Systems”that “observe”/“act-on” a certain
“property” of a “feature-of-interest” of the environment/building in which they are installed. For
instance, a sensor maymeasure the property “temperature” of the feature-of-interest “room 1” in a
given building.
The same ontology defines that such a “System”, in order to provide its intended service, implements a
“Procedure” that has certain “Inputs” and “Outputs”.
The terms “feature of interest” refers to specific “locations” in the building. “Locations” here do not
refer to a representation of coordinates in a geographic map; they refer to parts of the building and
objects in the building that correspond to certain relative positions; e.g. “heater 1”, “room 1” “window
1” are locations and subsequently “features-of-interest” in the building. In order to model relations
between locations, concepts of the GeoSPARQL model [5], e.g. “touches”, “inside”, “contains”, etc. can
be used.

17. Semantic annotation of building “Things”
Therefore, the services offered by control system components can be modelled
explicitly in a way to facilitate their online invocation, by combining the
“Procedure” concept of SSN with the “Service” concept of OWL-S.
For convenience, we may refer to the inputs, outputs and parameters associated
with a component/service, collectively as “end-points” of that component/service.
The adopted way of annotating/describing the components, allows us to model the
knowledge about all produced/consumed signals using the “Five Ws and one H”
method [6], which has been proposed for capturing and communicating the correct
information about an entity in a reporting or decision making context.

18. Semantic annotation of building “Things”
It can be seen that the semantic annotation space is defined by four dimensions:
Λ ≡ 𝐿 × 𝑄 × 𝑃 × 𝑀,
That is, an element of the space is represented by the specific values in a quadruple of
respective variables:
• Variable l represents the plant’s “feature-of-interest” and answers to the question
“WHERE”, e.g. taking values from a set L = {office; zone 1; zone 2; door;, window; ambient;
wall 1; ceiling 1; heater 1}. The set can be the output of the building design using a CAD
software. e.g. an extract of a BIM [7].
• Variable q represents the studied property of the feature-of-interest and answers to the
question “WHAT”, e.g. taking values from a set Q = { temperature; energy; opening; flow
rate; filtration rate; fan speed; time|. The values of this set, as well as of the measurement
unit below, can be retrieved from existing models (e.g. the current version or future
extensions of the Building Information Model [7]).

19. Semantic annotation of building “Things”
• Variable p represents the role of the signal/variable in the control system configuration and answers
to the question “WHY”, e.g. taking values from a set P = {state; stateMeasurement;controlDecision;
disturb; referenceValue; plantTopology; regulate; increase; decrease}. These values are given at the
time of annotating the component, either manually selected by the engineer/technician or
automatically by downloading the information from the Internet.
• Variable m represents the measurementunit of the property, where applicable, and answers to the
question “HOW”, e.g. taking values from a set M= {Celsius; Fahrenheit; kWatt; kilogramsPerSecond;
percentage}.
The question “WHO” is explicitly answered through the link of endpoints to specific components,
whereas the question “WHEN” is out of the scope of the decision making discussed here.

20. Semantic annotation of building “Things”
The Semantic Annotation operation is defined as:
λ . : A ↦ Λ
where:
• A is the set of all end-points of control system components
• Λ is the annotation space (quadruple) defined earlier

21. Semantic annotation of building “Things”
The input in the figure may be the “office”
in degrees Celsius and denotes a point in
the space , as:
y1
c
= {𝑙: 𝑜𝑓𝑓𝑖𝑐𝑒, 𝑞: 𝑡𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒,
𝑝: 𝑠𝑡𝑎𝑡𝑒𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡, 𝑚: 𝐶𝑒𝑙𝑠𝑖𝑢𝑠}
In the same way, the semantic annotations
of the example output and parameter are:
u1
c = {𝑙:ℎ𝑒𝑎𝑡𝑒𝑟, 𝑞: 𝑓𝑙𝑜𝑤 𝑟𝑎𝑡𝑒,
𝑝: 𝑐𝑜𝑛𝑡𝑟𝑜𝑙𝐷𝑒𝑐𝑖𝑠𝑖𝑜𝑛, 𝑚: 𝐾𝑔/𝑠}
ζ1
c = {𝑙: 𝑑𝑜𝑜𝑟, 𝑞: 𝑜𝑝𝑒𝑛𝑖𝑛𝑔,
𝑝: 𝑏𝑢𝑖𝑙𝑑𝑖𝑛𝑔 𝑡𝑜𝑝𝑜𝑙𝑜𝑔𝑦, 𝑚:%} A control system component with an example semantic model
of an input, an output and a parameter

22. Semantic annotation of building “Things”

23. Semantic annotation of building “Things”

24. Semantic matching of “Things” in buildings
The Semantic Matching operator is defined as:
ρ: Λ × Λ ↦ {⊤, ⊥}
Input: a pair of semantic annotations (Output-Input)
For instance:
ρ( office; temperature; stateMeasurement; Celsius , {{office; temperature;
stateMeasurement; any{office; temperature; stateMeasurement; any}) = ⊤
Transformations can also happen through “semantic rules” for deductive inference
(e.g. relations between locations)

25. Semantic matching of “Things” in buildings

26. Semantic reasoning for automatic configuration
IoT components in a building

27. Semantic reasoning for automatic configuration
Domain knowledge:
𝐿 = {𝑙1: 𝑟𝑜𝑜𝑚 1; 𝑙2: 𝑎𝑚𝑏𝑖𝑒𝑛𝑡, 𝑙3: 𝑑𝑜𝑜𝑟, 𝑙4: 𝑤𝑒𝑠𝑡 𝑤𝑎𝑙𝑙 1}
𝑄 = {𝑞1: 𝑡𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒; 𝑞2:ℎ𝑒𝑎𝑡 𝑒𝑛𝑒𝑟𝑔𝑦, 𝑞3: 𝑈 − 𝑣𝑎𝑙𝑢𝑒, 𝑞4: 𝑓𝑙𝑜𝑤 𝑟𝑎𝑡𝑒}
𝑃 = 𝑝1: 𝑖𝑛𝑐𝑟𝑒𝑎𝑠𝑒; 𝑝2: 𝑠𝑡𝑎𝑡𝑒, 𝑝3: 𝑏𝑢𝑖𝑙𝑑𝑖𝑛𝑔 𝑡𝑜𝑝𝑜𝑙𝑜𝑔𝑦, 𝑝4: 𝑠𝑡𝑎𝑡𝑒𝑀𝑒𝑎𝑠𝑢𝑟𝑒𝑚𝑒𝑛𝑡, 𝑝5: 𝑐𝑜𝑛𝑡𝑟𝑜𝑙𝐷𝑒𝑐𝑖𝑠𝑖𝑜𝑛
𝑀 = {𝑚1: 𝐶𝑒𝑙𝑠𝑖𝑢𝑠; 𝑚2: 𝐹𝑎ℎ𝑟𝑒𝑛ℎ𝑒𝑖𝑡, 𝑚3: 𝐽𝑜𝑢𝑙𝑒, 𝑚4:
𝑊
𝑚2 𝐾
, 𝑚5:… }
Relations:
𝑐𝑜𝑛𝑡𝑎𝑖𝑛𝑠 𝑟𝑜𝑜𝑚 1, 𝑤𝑒𝑠𝑡 𝑤𝑎𝑙𝑙 1

28. Semantic reasoning for automatic configuration

29. Semantic reasoning for automatic configuration

30. Semantic reasoning for automatic configuration

31. Semantic reasoning for automatic configuration

32. Semantic reasoning for automatic configuration
Examples of semantic matchings
between components and the
subsequent configurations of the
temperature control system.
There are three different ways for the
IoT components to be used in order to
achieve the control objectives.
Human operators would have not been
able to figure out all possible solutions
without the support of the semantic
supervisor.

33. Resources
[1] Milis, George, Panayiotou, Christos, & Polycarpou, Marios. (2017). Semantically-Enhanced Online Configuration of Feedback
Control Schemes. IEEE Transactions on Cybernetics. http://doi.org/10.1109/TCYB.2017.2680740
[2] Milis, George, Panayiotou, Christos, & Polycarpou, Marios. (2017). SEMIoTICS: Semantically-enhanced IoT-enabled Intelligent
Control Systems. IEEE Internet of Things Journal, (Special Issue IoT Feedback Control). http://doi.org/10.5281/zenodo.1053854
[3] D. Martin, M. Burstein, J. Hobbs, O. Lassila, D. McDermott, S. McIlraith, S. Narayanan, M. Paolucci, B. Parsia, T. Payne, E. Sirin, N.
Srinivasan, and K. Sycara. (2004) OWL-S: Semantic Markup for Web Services. Accessed: 2017-07-24. [Online]. Available:
https://www.w3.org/Submission/OWL-S/
[4] A. Haller, K. Janowicz, S. Cox, D. L. Phuoc, K. Taylor, M. Lefrançois, R. Atkinson, R. García-Castro, J. Lieberman, and C. Stadler.
Semantic Sensor Network Ontology. Accessed: 2017-07-24. [Online]. Available: https://www.w3.org/TR/vocab-ssn/
[5] GeoSPARQL - A Geographic Query Language for RDF Data. Accessed: 2017-07-24. [Online]. Available:
http://www.opengeospatial.org/standards/geosparql
[6] C. Griths and M. Costi, GRASP : the solution. Cardi, UK: Proactive Press, 2011.
[7] D. Conover, D. Crawley, S. Hagan, D. Knight, C. Barnaby, C. Gulledge, R. Hitchcock, S. Rosen, B. Emtman, G. Holness, D. Iverson,
M. Palmer, and C.Wilkins, An Introduction to Building Information Modeling (BIM) - A Guide for ASHRAE Members. Amer. Soc. of
Heating, Refrig. and Air-Cond. Eng., 2009.
…and references therein

34. Disclaimer
For further information, relatedto the VET4SBO project, please visit the project’swebsite at https://smart-building-
operator.euor visit us at https://www.facebook.com/Vet4sbo.
Downloadour mobile app at https://play.google.com/store/apps/details?id=com.vet4sbo.mobile.
This project (2018-1-RS01-KA202-000411) has been funded with support from the European Commission (Erasmus+
Programme). Thispublicationreflects the views only of the author, and the Commission cannot be held responsible
for any use which may be made of the informationcontainedtherein.