The document discusses techniques for feature selection in industrial automation systems using domain knowledge from an industrial feature ontology. It presents an approach for semantic-guided feature selection that reduces the feature space using dependencies defined in the ontology without accessing the actual data. An embedded model feature selection approach is also described that incorporates these semantic dependencies into linear models through a graph regularization term. The techniques are evaluated on simulation and real-world manufacturing data and shown to improve performance over traditional feature selection methods.

1. MES
Data Acquisition, Analysis
Why
Tracking & Tracing
When
Execution
What
Resource
Who
Specification
How
Unrestricted © Siemens AG 2015. All rights reserved
Semantic-guided Feature Selection
for Industrial Automation Systems
M. Ringsquandl, S. Lamparter, S. Brandt, T. Hubauer, R. Lepratti

2. Page 2 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

3. Page 3 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

4. Page 4 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Industrial Automation Systems
Layered Architecture
Layer 1 – Field Device Layer
Production
Instruments
Identification
Systems
Drive
Systems
Power
Supplies
Field Devices
Electr. & Mech. Engineering
Knowledge

5. Page 5 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Industrial Automation Systems
Layered Architecture
Layer 2 – Control Layer
Real-time Control
Industrial
Communication
Human-Machine
Interfaces
Switching
Technology
Control Layer
Field Devices
Electr. & Mech. Engineering
Knowledge
Control & Automation
Engineering Knowledge

6. Page 6 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Industrial Automation Systems
Layered Architecture
Layer 3 – Supervisory Layer
Control Layer
Field Devices
Engineering
Stations
Energy
Management
Asset
Management
Data Acquisition
Systems
Supervisory Layer
Electr. & Mech. Engineering
Knowledge
Control & Automation
Engineering Knowledge
IT-System Knowledge

7. Page 7 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Industrial Automation Systems
Layered Architecture
Layer 4 – Management Layer
Control Layer
Field Devices
Supervisory Layer
Management Layer
Operations
Management
Plant Engineering
Production
Execution
Manufacturing
Intelligence
Electr. & Mech. Engineering
Knowledge
Control & Automation
Engineering Knowledge
IT-System Knowledge
Manufacturing Operations
Knowledge

8. Page 8 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
MES
Introduction
Industrial Automation Systems
Data Collection on Manufacturing Operations Layer
Manufacturing Operations Management
Quality Inventory
Maintenance Production
ERP
Observation
Motor Torque
Conveyor Motor
2015-03-01T12:31:00
Door Assembly
Torquemeter
1200
featureOfInterestobservedProperty
Contextualize as
Unified Semantic Data Model
Thousands of Tags
and Events
Control Layer
Field Devices
Supervisory Layer

9. Page 9 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

10. Page 10 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Data Access and Analytics
Using Domain Knowledge
Going beyond Ontology-based Data Access (see [1])
Historic and
Real-time data
Data Access
Control Layer
Field Devices
Supervisory Layer
Management Layer
ETL
Analytics
OBDA
DomainKnowledge
Domain Knowledge
?

11. Page 11 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Data Access and Analytics
Application of Machine Learning Models
High-dimensional and Linked Data – Select optimal subset of features, cf. [2]
Manufacturing Operations Management
Quality Inventory
Maintenance Production
FS
Fi
Model
Feature
Selection
Model Fitting
Do we need
to check all
of them?

12. Page 12 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

13. Page 13 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

14. Page 14 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Approach
Industrial Feature Ontology
Extension of Semantic Sensor Network Ontology (see [3])
DomainKnowledge
Legacy
Model
Legacy
Model
Legacy
Model
Motor Temperature
dependsOn
Motor Speed
Model dependencies
between data

15. Page 15 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

16. Page 16 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Approach
Semantic-guided Feature Selection
Feature Ontology reduces Feature Space without looking at actual data
FeatureOntology
Legacy
Model
Legacy
Model
Legacy
Model
Response – Define
dependencies on the
variable we want to
predict

17. Page 17 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Approach
Semantic-guided Feature Selection
Role chain axioms propagating feature dependencies
FeatureOntology
Legacy
Model
Legacy
Model
Legacy
Model

18. Page 18 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Approach
Semantic-guided Feature Selection
Conceptual definition of relevant features
Invoke reasoner before Data Access

19. Page 19 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

20. Page 20 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Approach
Embedded Model Feature Selection
Ontology as
RDF-Graph
• Lasso Regularization:
• Graph Kernel Lasso:
• Linear Model:
Calculate Graph Kernel Matrix
Based on sub-graphs
Augment Linear Model with a semantic regularization term (see [4])
„Semantic“
Bias
What if we still want a
sparse solution?
• Use Laplacian of Graph Kernel Matrix:
“Degree – Adjacency Matrix”

21. Page 21 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

22. Page 22 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

23. Page 23 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Cycle Time Forecasting
Outgoing
Goods
PackagingAssemblyConveyingLoading Quality Test
predict
Cycle Time
Use Linear Model to estimate time until product is finished
Feature
Selection
Model
Fitting
• Collect data from different
layers and processes
• Contextualize w.r.t.
product (cycle time)
FeatureOntology

24. Page 24 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

25. Page 25 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Evaluation Results
Semantic Feature Selection
Feature selection performance
• Technomatix Plant Simulation Data Set:
47
29
18
0
10
20
30
40
50
No Feature Selection Semantic Feature
Selection
P-value based
Selection
Features
0.08 0.06
0.00
0.10
0.20
No Feature Selection Semantic Feature
Selection
P-value based
Selection
1.36E+11
1000
1E+11
2E+11
Normalized Model Error
Product Type Conveyor Speed Control Alarms … Cycle Time
47 Features
2000Instances

26. Page 26 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Evaluation Results
Embedded Feature Selection
22.9
42.9 42.8 43.8
0
10
20
30
40
50
Lasso ElsaticNet Graph Lasso Graph Kernel
Lasso
Features
0.48 0.46
0.54
0.43
0.00
0.10
0.20
0.30
0.40
0.50
0.60
Lasso ElasticNet Graph Lasso Graph Kernel
Lasso
Normalized Model Error
Embedded Feature Selection and Model Performance
• Small sample size n=40
• Results based on 10-Fold Cross-Validation
Product Type Conveyor Speed Control Alarms … Cycle Time
47 Features
40Instances

27. Page 27 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
• Evaluation Results
• Production Cycle Time Forecasting
Use Case
• Linear Model-Embedded Feature Selection
• Semantic-guided Feature Selection
• Industrial Feature Ontology
Our Approach
• Data Access and Analytics
• Industrial Automation Systems
Introduction
Outline

28. Page 28 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Summary
Major Takeaways
Semantic Feature Selection
• Domain knowledge from legacy models allows us to capture known dependencies between variables
• We can perform feature selection via semantic reasoning – without looking at the data
 It gives competitive results
 It reduces number crunching efforts
Embedded Model Feature Selection
• Extended graph regularization leverages from known dependencies
 They introduce a “semantic bias” to learning of hypothesis
 Can help to boost performance for small data sets

29. Page 29 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Outlook
Possibilities for Future Work
Ontology-Based Data Access
• Integrate Feature Selection directly into Query Answering
Additional Sources of Domain Knowledge
• Extract further dependencies from Product-Lifecycle and Engineering Systems
Evaluate on real-life plant data
• Apply techniques on real-life large-scale automation systems

30. Page 30 October 2015 Corporate Technology Unrestricted © Siemens AG 2015. All rights reserved
Literature
[1] M. Rodríguez-Muro, R. Kontchakov, and M. Zakharyaschev, “Ontology-based data access: Ontop of
databases,” in Proc. of the 12th Int. Sem. Web Conf., 2013.
[2] J. Tang, S. Alelyani, and H. Liu, “Feature Selection for Classification: A Review. 2013”
[3] M. Compton, P. Barnaghi, L. Bermudez, R. García-Castro, O. Corcho, S. Cox, J. Graybeal, M. Hauswirth, C.
Henson, A. Herzog, V. Huang, K. Janowicz, W. D. Kelsey, D. Le Phuoc, L. Lefort, M. Leggieri, H. Neuhaus, A.
Nikolov, K. Page, A. Passant, A. Sheth, and K. Taylor, “The SSN ontology of the W3C semantic sensor network
incubator group,” Web Semant. Sci. Serv. Agents World Wide Web, vol. 17, pp. 25–32, Dec. 2012.
[4] C. Li and H. Li, “Network-constrained regularization and variable selection for analysis of genomic data,”
Bioinformatics, vol. 24, no. 9, pp. 1175–1182, 2008.