AI Reliability use case result of anomaly detection using autoencoders based on sensor data from production equipment

1. AI Reliability
September 24th
, 2021

2. Goal And Objectives
2
▪ Сonfigure and demonstrate the capabilities to identify process and equipment anomalies to predict unplanned
shutdowns for two combined-cycle power plants. Analyze the following 6 subsystems: GT, ST, 2 Generators,
Boiler, Gas compressor
Goal
Objectives
Data sources P&ID
Visually detected
defects
Planned and
unplanned
shutdowns history
Sensors
▪ Creating and Replicating Models Between Similar Objects. Because of the strong differences in power generation
infrastructure objects, the ability to scale created models to objects of the same type has not been shown to be
effective.
▪ Data Integration. Services for handling downloaded engineering data from different types of sources, reducing the time
spent on cleanup, preparation, and other data management tasks.
▪ Implement and manage a large number of models in production. Use of multiple infrastructure components and
multi-model orchestration.
▪ Development of advanced user interfaces. Display both anomaly data and the potential economic impact of preventing
identified anomalies, operator actions, and output for key users on achieving planned targets.
▪ Target percentage of predicted complete unscheduled shutdowns in the test sample > 66%.

3. Anomaly Detection Models

4. • By default: 1 year train, 4 years test with retrain every 2 months
• Approach was adopted to every particular subsystem
• Test period complete unplanned shutdowns: 23 (out of 27)
• 14 ML models with rolling window approach - 336 ML models versions for dedicated periods required:
(6 subsystems + 1 power unit as a whole) * 2 power units * 4 test years * 6 models per year
Iteration 1
Test
Train Test
Iteration 2
Train Buffer zone between train/test
1Y 2M
1M Unplanned
shutdowns
Rolling window train/test approach was used
1Y 1M 2M

5. 5
Several Models trained to recognize expected tags values
Expected by ML model value
Actual value
Subsystem power
Anomaly level detected
ML model recognizes how this tag behave according to the past and to other tags behavior

6. 6
Using Expected Value We Can Identify Possible Anomalies
Expected by ML model value
Actual value
Subsystem power
Anomaly level detected
Vibration should go down according to
all the other tags behavior, but it goes
up. Anomaly level increases

7. 7
336 Models tested about anomalous equipment behavior
generated
• Risk score (anomaly level) calculated on basis of all the subsystem tags anomaly levels - for every hour in test period
for every subsystem and for power unit as a whole
• If Risk score is higher than threshold – Alert was generated
Boiler PU-2 example:
Train period Test period
Alert generation threshold
Alerts before shutdowns –
shutdown predicted
Alerts not before shutdowns: could be
valuable (real anomaly) or wrong
No alert before shutdowns –
shutdown not predicted
Subsystem anomaly level

8. 8
Production team reviewed alerts in detail
Application
detected anomaly:
High discharge
behind the air
house
Alert review example: PU-1 GT, shutdown 03.03.21, alert 23.01.2021 (36 days before shutdown), another alert 12.02.2021 (17 days before)
Investigation report conclusion: high discharge behind the air house: dP of the fine filters is high, the CSF is damaged
Drift
Application detected anomaly: dP of the fine filters is high at both points Application detected anomaly: dP CSF is low at both points (damaged?)
Expected by ML
model value
Tag actual value
Subsystem power
Anomaly level
• 2600 tags mapped to subsystems and key failure mode tags dedicated with alarm/ protection setpoints understanding
• >19 alerts reviewed:
• 77 iterations totally for 15 AI models
• >5 new diagnostic metrics developed
• >770 plots with anomalous tags behavior compared with investigation acts to check if alert relates to a shutdown
• 5 out of 19 alerts were not approved (2 of them after detailed internal discussions. Trial had preliminary recall 69.6% before)
• 5 expert rules developed jointly on basis of AI anomalies
• Production team demonstrated deep expertise and high readiness to interpret AI alerts and develop expert rules

9. Machine Learning model test results
9
Number of
shutdowns in test
period
Alerts before
shutdowns generated
(before review)
Statistical
Recall
Correctly detected
alerts
(approved by SME)
Reviewed Recall
(Anomaly relates to
a shutdown)
Average alerts
frequency
Alerts in a week:
Average lead
time, days
23 19 82,6% 14 60,9% 0,8 9,8
• Statistical recall 82.6% – alerts generated by the solution in advance of unplanned shutdowns
• Approved Recall 60.9% – alerts verified and confirmed by experts, proving that early anomaly detection can be
operationally used for unplanned shutdown prediction, i.e., anomalies detected are related to shutdowns and provide
potential reasons or early warning indications. All confirmed alerts either were generated earlier than staff detected a
defect (11 alerts), or provided new information about defect evolution (3 alerts)
• It is valuable to consider an accuracy rate of 82.6% when comparing with world practices, since there is usually no
investigation for the alert-shutdown relationship
• Of all 23 failure events, only 15 were technically predictable based on the supplied data, as agreed. The Reliability
solution picked 14 of these (or 93%)
* A methodology has been developed that will allow significantly reduce the number of faulse positives
during production implementation

10. Recall references science papers
1) ”Neural network-based data-driven modelling of anomaly detection in thermal power plant” from
Automatika Journal for Control, Measurement, Electronics, Computing and Communications
Published online: 07 Jul 2017 https://core.ac.uk/download/pdf/212472032.pdf
2) “Fault Detection in a Compined Cycle Power Based on Neural Networks …”
Master’s Thesis from Montan Universitat, Philipp Rohrweck, 2021
https://pure.unileoben.ac.at/portal/files/6491873/AC16191180.pdf
• Statistical recall 50%-60% is good result if you model subsystem as a whole. All the results with better accuracy were
delivered for detailed equipment or particular failure mode modelling
• It is worth using an accuracy rate of 82.6% when comparing with world practices, since there is usually no check for the
alert-shutdown relationship

11. Results overview

12. Operational staff efficiency increase
12
Number of alarms raised
per month per power unit
Few
times
per
day
<2
With app
Lead time, in days
3
10
With app
Reviewed Recall in %
(share of shutdowns correctly predicted)
35%
61%
With app
Performance can be further improved during the production deployment:
• More than 20 ML models shall be configured per power unit rather than 6 models per the trial project: equipment/ defect level
• Full set of PU tags should be analyzed. E.g., Steam turbine-2 generator currents are in protection metrics only. More sensor data can be ingested for the
project.
• It is valuable to add metrics for cumulative effect on equipment reliability estimation
• False Positives drastic decrease due to special treatment of G CV, subsystem power, alarm or expert set points usage
Baseline
(Alarm setpoints)
Baseline
Baseline