Building reliable data-driven predictive systems requires a considerable amount of human effort, especially in the data preparation and cleaning phase. In many application domains, multiple preprocessing steps need to be applied in sequence, constituting a `workflow' and facilitating reproducibility. The concatenation of such workflow with a predictive model forms a Multi-Component Predictive System (MCPS). Automatic MCPS composition can speed up this process by taking the human out of the loop, at the cost of model transparency (i.e. not being comprehensible by human experts). In this paper, we adopt and suitably re-define the Well-handled with Regular Iterations Work Flow (WRI-WF) Petri nets to represent MCPSs. The use of such WRI-WF nets helps to increase the transparency of MCPSs required in industrial applications and make it possible to automatically verify the composed workflows. We also present our experience and results of applying this representation to model soft sensors in chemical production plants.

1. Modelling Multi-Component
Predictive Systems as Petri Nets
Manuel Martín Salvador, Marcin Budka, Bogdan Gabrys
Bournemouth University, UK
{msalvador, mbudka, bgabrys}@bournemouth.ac.uk
ISC’2017
Warsaw, Poland
May 31st, 2017

3. Outline
1. Motivation
2. Proposal
3. Automation
4. Experiments
5. Conclusion and future work

4. MOTIVATION

5. Predictive modelling
Labelled
Data
Supervised
Learning
Algorithm
Predictive
Model

6. Predictive systems in the industry
Fault detection
Online prediction of hard-to-measure variables
Process monitoring
Demand forecasting
Classification based on computer vision
Picture is Creative Commons by Jm3

7. Need of preprocessing
Garbage in, garbage out
Missing data
Outliers
High dimensionality
Normalisation
Lack of preprocessing can lead to
inconsistent models

8. Multi-Component Predictive Systems
Data Postprocessing PredictionsPreprocessing Predictive
Model

9. Multi-Component Predictive Systems
Data
Predictive
Model
Postprocessing PredictionsPreprocessing Predictive
Model
Predictive
Model

10. Multi-Component Predictive Systems
Preprocessing
Data
Predictive
Model
Postprocessing Predictions
Preprocessing
Preprocessing Predictive
Model
Predictive
Model

11. Requirements in the industry
Reliability - to provide truthful results
Robustness - to work under any circumstances or inconvenience
Transparency - to be comprehensible by human experts
Reproducibility - to replicate the results of an study
Low maintenance cost - to keep the system up-to-date at low cost

12. PROPOSAL

13. ● Function composition: Difficult to model parallel paths. Can’t
representate states of a system.
● Directed Acyclic Graph: Not enough to model process state or
temporal behaviour..
● Petri net: Very flexible and robust mathematical background.
Expressivepower
Y = h(g(f(X)))
f g hX Y
f g hX Y
How to model MCPS?

14. Mathematical modelling language invented in 1939 by Carl Adam Petri
token
place
transition
arc
N = (P,T,F)
Petri net

15. Example of Petri net
Reception Waiting
Room
Check in
Consulting
Room
Exit
Call in
Examination
and diagnosis
Patient

16. Example of Petri net
Reception Waiting
Room
Check in
Consulting
Room
Exit
Call in
Examination
and diagnosis

17. Example of Petri net
Reception Waiting
Room
Check in
Consulting
Room
Exit
Call in
Examination
and diagnosis

18. Example of Petri net
Reception Waiting
Room
Check in
Consulting
Room
Exit
Call in
Examination
and diagnosis

19. Example of Petri net
Reception Waiting
Room
Check in
Consulting
Room
Exit
Call in
Examination
and diagnosis

20. Example of Petri net
Reception Waiting
Room
Check in
Consulting
Room
Exit
Call in
Examination
and diagnosis

21. Example of Petri net
Reception Waiting
Room
Check in
Consulting
Room
Exit
Call in
Examination
and diagnosis

22. Example of Petri net
Reception Waiting
Room
Check in
Consulting
Room
Exit
Call in
Examination
and diagnosis

23. Petri nets can be more complex
Source: http://bit.ly/1XZQhYZ

24. A Petri net is an MCPS iff all the following conditions apply:
● The Petri net is a WRI-WF-net
● The places P{i,o} have only a single input and a single output.
● The Petri net is 1-bounded.
● The Petri net is 1-sound.
● The Petri net is ordinary.
● All the transitions with multiple inputs or outputs are AND-join or AND-split,
respectively.
● Any token is a tensor (i.e. multidimensional array)
Modelling MCPS as Petri net

25. Example of MCPS
Classifier
o
Replace
missing
values
Dimensionality
reduction
Outlier
handling
token
(data) i
place
transition
MCPS = (P, Tλ
, F)

26. AUTOMATION

27. Manual
● WEKA
● RapidMiner
● Knime
● IBM SPSS
Automatic
● Auto-WEKA (Bayesian optimisation)
● Auto-sklearn (Bayesian optimisation + Meta-learning)
● TPOT (Genetic programming)
● e-Lico IDA (Ontologies + Planning)
Example of WEKA workflow
MCPS composition

28. What are the best algorithms to process my data?
Algorithm Selection

29. How to tune the hyperparameters to get the best performance?
Hyperparameter Optimisation

30. Combined Algorithm Selection and Hyperparameter configuration problem
k-fold cross validation
Objective function
(e.g. classification error)
HyperparametersMCPSs
Training dataset
Validation dataset
Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms.
In: Proc. of the 19th ACM SIGKDD. (2013) 847–855
Martin Salvador M., Budka M., Gabrys B.: Automatic composition and optimisation of multicomponent predictive systems. IEEE Transactions on Automation
Science and Engineering. under review - preprint available at https://arxiv.org/abs/1612.08789
CASH problem for MCPS

31. WEKA methods as search space
One-click black box
Data + Time Budget → MCPS
Our contribution
● Recursive extension of complex
hyperparameters in the search space.
● Composition and optimisation of
MCPSs (including WEKA filters,
predictors and meta-predictors)
● Petri net output as PNML format
Open-source. Download at:
https://github.com/dsibournemouth/autoweka
Auto-WEKA for MCPS

32. WoPeD: Workflow Petri Net Designer
Open-source. Download:
http://woped.dhbw-karlsruhe.de
Edit, analyze and
simulate PNs
Load and save PNML

33. EXPERIMENTS

34. Building soft sensors for process industry
Task: build a soft sensor to predict continuous
values (regression)
7 datasets from real chemical production
processes
70% training and optimisation, 30% testing
Auto-WEKA: 25 runs for 30 hours with different
seeds, keep the best.
Optimisation measure: RMSE
Baseline: 4 most popular methods for soft
sensors (PCR, PLS, MLP and RBF)
dataset RMSE of
best (test)
Difference
with baseline
absorber 0.8989 ↑ 0.0844
catalyst 0.0736 ↑ 0.1144
debutanizer 0.1745 ↓ 0.0035
drier 1.3744 ↑ 0.0573
oxeno 0.0226 ↑ 0.0042
sulfur 0.0366 ↑ 0.0030
thermalox 0.6904 ↑ 0.6170

36. ● Data distribution can change over time and affect predictions
○ External factors (e.g. weather conditions, new regulations)
○ Internal factors (e.g. quality of materials, equipment deterioration)
Source: INFER project
Maintaining an MCPS

37. GFMMZ-Score PCA Min-Max
Time
i p1
p2
p3
o
data
meta-data
prediction
[-3.1, 2.7]
x1
= 3.6
[-3.1, 3.6]
Reactive adaptation of MCPS

38. Conclusion and future work
Automatic composition of MCPS can speed up the process of building predictive
systems though can end up being a black-box process
Representing MCPSs as Petri nets has a number of benefits:
● Increase transparency
● Verification
● Vendor-independent
Future work:
● Workflow algebra to model MCPSs adaptation
● Timed Petri nets to model task duration and delays

39. THANKS!
Paper: http://bit.ly/mcps-petri-nets
Slides: http://www.slideshare.net/draxus
Manuel <msalvador@bournemouth.ac.uk>
@draxus