The 5th Aslla Symposium.

1. INDUSTRY 4.0 @ GHENT UNIVERSITY
(GLOBAL CAMPUS) - IMEC:
FROM PREDICTIVE MAINTENANCE TO
PERFORMANCE MONITORING
October 23-26, 2018 @ Gangneung, Korea
The 5th Aslla Symposium
Wesley De Neve, Sofie Van Hoecke, Erik Mannens

2. OUTLINE
• Background
• Use cases
‒ use case 1: wind turbine condition monitoring
‒ use case 2: incipient bearing fault detection
‒ use case 3: anomaly detection in steel coil production
‒ use case 4: greenhouse condition monitoring
2

3. BACKGROUND
3

4. WESLEY DE NEVE
• Academic credentials
̶ Master’s degree in computer science (2002)
 from Ghent University, Belgium
̶ Ph.D. degree in computer science engineering (2007)
 from Ghent University, Belgium
̶ post-doctoral experience (2007-2011)
 at ICU / KAIST, Korea
• Employment: Professor
̶ IDLab, Ghent University, Belgium (2011 - current)
̶ Ghent University Global Campus, Korea (2014 - current)
4

5. SOFIE VAN HOECKE
• Academic credentials
̶ Master’s degree in computer science engineering (2003)
 from Ghent University, Belgium
̶ Ph.D. degree in computer science engineering (2009)
 from Ghent University, Belgium
• Employment: Professor
̶ University College West-Flanders (2011 - 2013)
̶ IDLab, Ghent University-imec, Belgium (2013 - current)
5

6. ERIK MANNENS
• Academic credentials
̶ Master’s degree in electro-mechanical engineering (1992)
 from KAHO Ghent, Belgium
̶ Master’s degree in computer science (1995)
 from KU Leuven, Belgium
̶ Ph.D. degree in computer science engineering (2011)
 from Ghent University, Belgium
• Employment: Professor & Research Valorization Director
̶ IDLab, Ghent University-imec, Belgium (2005 - current)
6

7. ORGANIZATIONAL OVERVIEW
7

8. GHENT UNIVERSITY
• Public university
‒ located in Ghent, Belgium
‒ established in 1817
8

9. GHENT UNIVERSITY
• Counts 42,000 students and 9,000 staff members
‒ about 4,000 foreign students and 800 foreign staff members
• Consists of 11 faculties, counting 117 departments
‒ campus buildings are distributed all over the city
9
Congress Center
‘Het Pand’
Faculty of Engineering
and Architecture
Aula Academia

10. GHENT UNIVERSITY GLOBAL CAMPUS
10
Biotech campus located in Songdo, Incheon, Korea (+250 students)
(part of the Incheon Global Campus, together with SUNY Stony Brook, GMU, & the University of Utah)

11. • R&D hub in nanoelectronics and digital technologies
‒ brings together 3,500 scientists and engineers from over 70 countries
• R&D groups in Belgium (headquarters: Leuven), the Netherlands,
Taiwan, USA, China, and offices in India and Japan
11
SOTA cleanroom
of 1 billion EURO
(in collaboration with
Samsung, a/o)
IMEC

12. IDLAB
• Internet Technologies and Data Science Lab
‒ founded on October 1, 2016
 merger of four already existing labs
‒ headcount
 40 professors
 50 post-doctoral researchers
 200 researchers
 15 support staff members
12
iGent Tower & AA Tower, Ghent, Belgium

13. IDLAB
• Education
‒ responsible for teaching more than 100 courses
• Fundamental and applied research
‒ machine learning and data mining
‒ semantic intelligence
‒ multimedia processing
‒ distributed intelligence for IoT
‒ cloud and Big Data infrastructures
‒ wireless and wired networking
13

14. Google DeepMind
(+ Ghent University)
four of our graduates are now working at
Google DeepMind and one at Google Brain
Sedol Lee
Google DeepMind Challenge Match
in March 2016 @ Seoul, Korea
verdict: 4–1 for AlphaGo
IDLAB
14

15. INDUSTRY 4.0 PLATFORMS
• Machines & Factories (M&F)
‒ an industrial research-oriented platform from
Ghent University linked to the manufacturing industry
‒ groups the activities of about 125 researchers from
different research groups, including 25 professors
• Flanders Make
‒ strategic research center for the manufacturing industry
in Flanders, set up by the Flemish government
‒ counts 400 researchers
15

16. PREDICTIVE MAINTENANCE &
PERFORMANCE MONITORING
16

17. MAINTENANCE STRATEGIES
• Preventive maintenance
‒ periodically inspect and replace components
• Reactive maintenance
‒ if something breaks, replace it
• Predictive maintenance
1) predict if, and possibly when, something is going to break
2) predict degradation of components
3) predict incipient faults
17

18. MAINTENANCE STRATEGIES
• Preventive maintenance
‒ periodically inspect and replace components
• Reactive maintenance
‒ if something breaks, replace it
• Predictive maintenance
1) predict if, and possibly when, something is going to break
2) predict degradation of components
3) predict incipient faults
18

19. USE CASE 1
19

20. WIND TURBINE CONDITION MONITORING
20
• Offshore maintenance is expensive
‒ specialized equipment is typically needed for landing and working at
offshore infrastructure
• Offshore maintenance is difficult to schedule
‒ depends on weather and wave conditions

21. WIND TURBINE CONDITION MONITORING
21
• Operation & maintenance actions may account for ± 30% of
the total production cost of offshore electricity in wind farms
• Early fault detection highly needed
‒ mitigates fault propagation and higher costs when detected (too) late

22. PERFORMANCE MONITORING
22
• Create a model that can predict the power output of a wind
turbine given several inputs
Predictive model
Predicted power
output
• Wind speed
• Wind direction
• Rotation speed
• Yaw
• Pitch
data collected from 3 wind turbines
over a time span of 3 months
(10 weeks of training data and 2 weeks of validation data)
random forests

23. ANOMALY DETECTION IN PRODUCTION
23
• Abnormal operation
‒ difference in predicted output
and actual output
‒ monitored through dashboard
Predictive model
Predicted power
output
• Wind speed
• Wind direction
• Rotation speed
• Yaw
• Pitch
Detection module
Actual power
output
Power production monitoring (Turbine 13)

24. USE CASE 2
24

25. INCIPIENT BEARING FAULT DETECTION
25
• Bearings are in all kinds of machines
‒ conveyer belts, wind turbines, cars, …
• Goal
‒ to detect incipient faults so to prevent faults
from escalating

26. INCIPIENT BEARING FAULT DETECTION
26
Time
Inadequate
lubrication
Pitting
Hard
particles Bearing
blockage
Component
failure
Machine
failure
• Goal: to detect condition of a bearing at this point in time
‒ maintenance can be scheduled accordingly and
reactive maintenance can be prevented

27. INCIPIENT BEARING FAULT DETECTION
27
Captured (1) vibration data
and (2) thermal imaging
data from bearings in
different conditions
Data set
Training set
Feature
extraction
Learning
algorithm
Model Predictions

28. INCIPIENT BEARING FAULT DETECTION
28
By using different machine learning algorithms (SVM, RDF, CNN)
for analyzing vibration data and infrared video,
we can automatically detect incipient faults with a high accuracy
Model
Inadequate
lubrication
Raceway
fault
Imbalance
Hard particles

29. INCIPIENT BEARING FAULT DETECTION
• Infrared imaging analysis
̶ handcrafted features + SVM: accuracy of 88.25%
• Vibration data analysis
̶ handcrafted features + RDF: accuracy of 87.25%
̶ CNN (end-to-end learning): accuracy of 91.77%
• Use of a multimodal approach (fusion)
̶ accuracy of over 99%
29

30. USE CASE 3
30

31. ANOMALY DETECTION IN STEEL MILL
31
• Any broken weld leads to
interrupts of several minutes
‒ comes with a high cost
• Fault prediction and cause
detection based on static and
dynamic parameters
‒ temperature
‒ pressure
‒ thickness
‒ speed
Metric GBRT RCN
True pos. 40 46
False neg. 10 4
False pos. 184 379
True neg. 558 363
Accuracy 77,60 % 70,46 %

32. USE CASE 4
32

33. SMART FARMING
33
? Where will we find enough food for 9 billion people by 2050?
Farming 4.0: use sensors to make farms more “intelligent” and
connected, so to be able to collect data about crop yields, soil-
mapping, fertilizer applications, weather, and plant growth
Collected data allow for improved decision-making
→
→

34. GREENHOUSE CONDITION MONITORING
34
condition monitoring in automated / programmable greenhouses (food computers)
plants on a conveyer belt
RGB imaging
hyperspectral imaging
adjustment of conditions

35. GREENHOUSE CONDITION MONITORING
35
• Drone-based phenomics
̶ measurement of physical and bio-chemical
traits of organisms as a function of genetic
and environmental changes
̶ hypothesis: gives less stress to plants
• Research questions
̶ what is (not) possible?
̶ what is the role of (deep) data analytics?
̶ how to scale to a real-world farm?
Indoor farming expert says Korea, Japan need to up investment
The Korea Herald 11/2017

36. 36
4. Point cloud
matching
6. Determine
skeleton
5. Segmentation
organ
recognition
Structure-from-
Motion
Growing
Neural Gas
Convolutional Neural Networks
Point Feature
Histograms (PFH)
descriptors
7. Estimate
phenotypic
parameters
Conditional
Random Fields
3. Filtering
denoising
2. MVS-SFM
point cloud
1. Image
sequence
GREENHOUSE CONDITION MONITORING

37. TAKE-AWAY MESSAGES
• IDLab has 10-15 professors working on machine learning,
together with 50-100 pre- and post-doctoral researchers
̶ on a wide range of use cases
̶ on different types of (sensor) data
• Open to different types of R&D collaboration
̶ imec
̶ Ghent University
̶ Ghent University Global Campus
37

38. Thank you for your attention!
{wesley.deneve, sofie.vanhoecke, erik.mannens}@ugent.be