Open Lecture Wil van der Aalst

Process Mining On the Interplay Between Data Science and Behavioral Science
Open Lecture High Tech Campus Eindhoven, 11-12-2014
Wil van der Aalst
Scientific director of the DSC/e

https://www.coursera.org/course/procmin
A new profession is emerging, just like computer science in the early 1980-ties!

Requires a combination of competences!

©Wil van der Aalst & TU/e (use only with permission & acknowledgements)
data
mining
process
mining visualization
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
statistics industrial
engineering
databases
stochastics
privacy
algorithms
visual
analytics
DSC/e Data Science Competences

data
mining
process
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
industrial
engineering
databases
privacy
algorithms
visual
analytics
data
science
statistics
stochastics
Statistics and Stochastics

data
mining
process
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics
Data Mining and Machine Learning

data
mining
process
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics
Process Mining

data
mining
process
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics
Large Scale Distributed Computing

data
mining
process
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics
Visualization and Visual Analytics

data
mining
process
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics
Domain Knowledge

data
mining
process
data
science domain
knowledge
machine
learning
large scale
distributed
computing
engineering
behavioral/
social
sciences
databases
stochastics
privacy
algorithms
visual
analytics
Behavioral/Social Sciences & Privacy

data
mining
process
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics
Industrial Engineering

data
mining
process
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics
Databases & Algorithms

Internet of Events

Internet of Events: 4 sources of event data
Internet of Events

Internet of ContentInternet of Events“Big Data”

Internet of ContentInternet of People“social” Internet of Events“Big Data”

Internet of ContentInternet of People“social” Internet of Things“cloud” Internet of Events“Big Data”

Internet of ContentInternet of People“social” Internet of ThingsInternet of Places“cloud”“mobility” Internet of Events“Big Data”

"Der Datenflüsterer"
Building a relationship with data

DSC/e: Competences and Research Programs
28 groups involved
Context: Why are we using data science,
does it have the intended effect, and will
people accept it?
Analysis: How to turn data into real value
(models, answers/decisions, and
visualizations/insights)?
Enabling technologies: How to get the
data and deal with computational/
infrastructural challenges (big data and
hard questions)?
Probability and Statistics
Stochastic Networks
Data Mining
Process Mining
Visualization
Large-Scale Distributed
Systems
Data-Intensive Algorithms
Data-Driven Operations
Management
Data-Driven Innovation and
Business
Human and Social Analytics
Privacy, Security, Ethics, and
Governance
Internet of Things
systems
infrastructures
cities
organizations
people
[RP1] Process Analytics:
Improving Service While Cutting Costs
[RP2] Customer Journey:
Correlating Events to Learn and Influence
Customer Behavior
[RP3] Smart Maintenance & Diagnostics:
Safeguarding Availability
[RP4] Quantified Self:
Improving Performance and Well-Being
[RP5] Data Value and Privacy:
Economic and Legal Aspects of Data Science
[RP6] Smart Cities:
Ensuring Safety and Convenience for Citizens
[RP7] Smart Grids:
Data Intensive Infrastructures
[RP1] Process Analytics:
Improving Service While Cutting Costs
[RP2] Customer Journey:
Correlating Events to Learn and Influence Customer Behavior
[RP3] Smart Maintenance & Diagnostics:
Safeguarding Availability
[RP4] Quantified Self:
Improving Performance and Well-Being
[RP5] Data Value and Privacy:
Economic and Legal Aspects of Data Science
[RP6] Smart Cities:
Ensuring Safety and Convenience for Citizens
[RP7] Smart Grids:
Data Intensive Infrastructures

Data Science Flagship (Philips & DSC/e)
•4 Strategic topics
•4 TU/e departments
•16 PhD students
•30 Data science specialists
1.Data Driven Value Propositions
2.Healthcare Smart Maintenance
3.Optimizing Healthcare Workflows
4.Continuous Personal Health

data
mining
process
mining
visualization
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics

data
mining
process
mining
visualization
data
science
behavioral/
social
sciences
domain
knowledge
machine
learning
large scale
distributed
computing
engineering
databases
stochastics
privacy
algorithms
visual
analytics
formal
methods
business
process
management
concurrency
business
process re-engineering
process
science
model
checking Petri nets
BPMN

register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
Case Activity Timestamp Resource
432 register travel request (a) 18-3-2014:9.15 John
432 get support from local manager (b) 18-3-2014:9.25 Mary
432 check budget by finance (d) 19-3-2014:8.55 John
432 decide (e) 19-3-2014:9.36 Sue
432 accept request (g) 19-3-2014:9.48 Mary
Play-In
Play-Out
Replay
Let's play

Play-Out
register travel request (a) get detailed motivation letter (c) get support from local manager (b) check budget by finance (d) decide (e) accept request (g) reject request (h) reinitiate request (f) startend Case Activity TimestampResource432 register travel request (a)18-3-2014:9.15John432 get support from local manager (b) 18-3-2014:9.25Mary432 check budget by finance (d)19-3-2014:8.55John432 decide (e)19-3-2014:9.36Sue432 accept request (g)19-3-2014:9.48Mary

Play Out: A possible scenario
register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
AND-split
XOR-split
XOR-join
XOR-join
AND-join
XOR-split
XOR-join
a b d e g
Case Activity Timestamp Resource
432 register travel request (a) 18-3-2014:9.15 John
432 get support from local manager (b) 18-3-2014:9.25 Mary
432 check budget by finance (d) 19-3-2014:8.55 John
432 decide (e) 19-3-2014:9.36 Sue
432 accept request (g) 19-3-2014:9.48 Mary

Play Out: Another scenario
register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
a d c e f b d e h

Play Out: Process model allows for many
more scenarios
register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
abdeg
abcefbdeh
adceg adbeh
acbefbdeh
acdefcdefbdeh
adcefcdefbdefbdeg
adceahd beh abdeg
acdefcdefabcdbeehfb deg
adcefcdefbdefbdeg
abdeg
adceh
adbeh
acdefcdefabcdbeehf bdeg
adcefcdefbdefbdeg
abdeg

Play-In
register travel request (a) get detailed motivation letter (c) get support from local manager (b) check budget by finance (d) decide (e) accept request (g) reject request (h) reinitiate request (f) startend Case Activity TimestampResource432 register travel request (a)18-3-2014:9.15John432 get support from local manager (b) 18-3-2014:9.25Mary432 check budget by finance (d)19-3-2014:8.55John432 decide (e)19-3-2014:9.36Sue432 accept request (g)19-3-2014:9.48Mary

Loesje van der Aalst
desire line

Play In: Simple process allowing for 4 traces
abdeg
adbeg
abdeh
adbeh
abdeg
adbeg
abdeh
adbeh
abdeh
abdeh
abdeh
adbeh
adbeh
register travel request (a) get support from local manager (b) check budget by finance (d) decide (e) accept request (g) reject request (h) startend

Play In: Process allowing for more traces
abdeg
abcefbdeh
adceg
adbeh
acbefbdeh
acdefcdefbdeh
adcefcdefbdefbdeg
abdeg
adceh
adbeh
acbefbdeg
acdefcdefbdeh
adcefcdefbdefbdeg
abdeg
adceh
adbeh
acbefbdeg
acdefcdefbdeh
adcefcdefbdefbdeg
abdeg
register travel request (a) get detailed motivation letter (c) get support from local manager (b) check budget by finance (d) decide (e) accept request (g) reject request (h) reinitiate request (f) startend

No modeling needed!

Example Process Discovery (Dutch housing agency, 208 cases, 5987 events)

Replay
register travel request (a) get detailed motivation letter (c) get support from local manager (b) check budget by finance (d) decide (e) accept request (g) reject request (h) reinitiate request (f) startendCase Activity TimestampResource432 register travel request (a)18-3-2014:9.15John432 get support from local manager (b) 18-3-2014:9.25Mary432 check budget by finance (d)19-3-2014:8.55John432 decide (e)19-3-2014:9.36Sue432 accept request (g)19-3-2014:9.48Mary

event data
process model

desire line
very safe system

Replay
register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
a c d e g

Replay
register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
a c
check budget (d)
is missing!
e g
?

Replay
register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
a c h d e g
reject request (h) is
impossible ?

Conformance Checking (WOZ objections Dutch municipality, 745 objections, 9583 event, f= 0.988)

Replay with timestamps
register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
a9.15 c9.20 d9.35 e10.15 g11.30
9.15
9.20
9.35
10.15
11.30
5 55
20 40
75

Replay with timestamps
register travel
request (a)
get detailed
motivation
letter (c)
get support
from local
manager (b)
check budget
by finance (d)
decide (e)
accept
request (g)
reject
request (h)
reinitiate
request (f)
start end
5 55
20
40
75
15
20
60
45
65
20
25
45
55
50
5 55
20 40
75
15
20
60
45
65
20
25
45
55
50
5
20 55
40
75
15
20
60
45
65
20
25
45
55
50

Performance Analysis Using Replay (WOZ objections Dutch municipality, 745 objections, 9583 event, f= 0.988)

software
system
(process)
model
event
logs
models
analyzes
discovery
records
events, e.g.,
messages,
transactions,
etc.
specifies
configures
implements
analyzes
supports/
controls
enhancement
conformance
“world”
people machines
organizations
components
business
processes
Overview
Play-In
Play-Out
Replay

Process Mining Software

600+ plug-ins available covering the whole process mining spectrum

Starting point for process mining: Event data
student name
course name
exam date
mark
Peter Jones
Business Information systems
16-1-2014
8
Sandy Scott
16-1-2014
5
Bridget White
16-1-2014
9
John Anderson
16-1-2014
8
Sandy Scott
BPM Systems
17-1-2014
7
Bridget White
BPM Systems
17-1-2014
8
Sandy Scott
Process Mining
20-1-2014
5
Bridget White
Process Mining
20-1-2014
9
John Anderson
Process Mining
20-1-2014
8
…
…
…
…
case id
activity name
timestamp
other data
every row is an event
(here: an exam attempt)

Another event log: patient treatment
patient
activity
timestamp
doctor
age
cost
5781
make X-ray
23-1-2014@10.30
Dr. Jones
45
70.00
5541
blood test
23-1-2014@10.18
Dr. Scott
61
40.00
5833
blood test
23-1-2014@10.27
Dr. Scott
24
40.00
5781
blood test
23-1-2014@10.49
Dr. Scott
45
40.00
5781
CT scan
23-1-2014@11.10
Dr. Fox
45
1200.00
5833
surgery
23-1-2014@12.34
Dr. Scott
24
2300.00
5781
handle payment
23-1-2014@12.41
Carol Hope
45
0.00
5541
radiation therapy
23-1-2014@13.57
Dr. Jones
61
140.00
5541
radiation therapy
23-1-2014@13.08
Dr. Jones
61
140.00
…
…
…
…
…
…
case id
activity name
timestamp
other data
resource

Another event log: order handling
order number
activity
timestamp
user
product
quantity
9901
register order
22-1-2014@09.15
Sara Jones
iPhone5S
1
9902
register order
22-1-2014@09.18
Sara Jones
iPhone5S
2
9903
register order
22-1-2014@09.27
Sara Jones
iPhone4S
1
9901
check stock
22-1-2014@09.49
Pete Scott
iPhone5S
1
9901
ship order
22-1-2014@10.11
Sue Fox
iPhone5S
1
9903
check stock
22-1-2014@10.34
Pete Scott
iPhone4S
1
9901
handle payment
22-1-2014@10.41
Carol Hope
iPhone5S
1
9902
check stock
22-1-2014@10.57
Pete Scott
iPhone5S
2
9902
cancel order
22-1-2014@11.08
Carol Hope
iPhone5S
2
…
…
…
…
…
…
case id
activity name
timestamp
other data
resource

Simplifying event logs when focusing on control-flow
order number
activity
timestamp
user
product
quantity
9901
register order
22-1-2014@09.15
Sara Jones
iPhone5S
1
9902
register order
22-1-2014@09.18
Sara Jones
iPhone5S
2
9903
register order
22-1-2014@09.27
Sara Jones
iPhone4S
1
9901
check stock
22-1-2014@09.49
Pete Scott
iPhone5S
1
9901
ship order
22-1-2014@10.11
Sue Fox
iPhone5S
1
9903
check stock
22-1-2014@10.34
Pete Scott
iPhone4S
1
9901
handle payment
22-1-2014@10.41
Carol Hope
iPhone5S
1
9902
check stock
22-1-2014@10.57
Pete Scott
iPhone5S
2
9902
cancel order
22-1-2014@11.08
Carol Hope
iPhone5S
2
…
…
…
…
…
…
[ register_order, check_stock,ship_order,handle_payment,
register_order, check_stock,cancel_order,
register_order, check_stock , …]

Simple event log
•An event log is a multiset of traces (same trace may appear multiple times).
•A trace is a sequence of activity names (we abstract from all other attributes, but events are ordered).

Goal of Alpha algorithm
a
b
c
e d
p2
end
p4
p1 p3
start
Event log contains all possible
traces of model and vice versa.

Another example
a
b
c
f d
p2
end
p4
p1 p3
start
e
p5
Generalization: event
log contains only
subset of all possible
traces of model.

Notation is less relevant (e.g. BPMN)
a
b
c
e d
p2
end
p4
p1 p3
start
a
start end
b
c
e
d

Another BPMN example
a
b
c
f d
p2
end
p4
p1 p3
start
e
p5
a
start end
b
c
e
d
f

>,,||,# relations
•Direct succession: x>y iff for some case x is directly followed by y.
•Causality: xy iff x>y and not y>x.
•Parallel: x||y iff x>y and y>x
•Choice: x#y iff not x>y and not y>x.
a>b
a>c
a>e
b>c
b>d
c>b
c>d
e>d
ab
ac
ae
bd
cd
ed
b||c
c||b
abcd
acbd
aed
b#e
e#b
c#e
a#d
…

Basic Idea Used by Alpha Algorithm (1)
ab(a) sequence pattern: a→b

a
b
c
(b) XOR-split pattern:
a→b, a→c, and b#c
a
b
c
(b) XOR-split pattern:
a→b, a→c, and b#c
b
c
d
(c) XOR-join pattern:
b→d, c→d, and b#c

a
b
c
(d) AND-split pattern:
a→b, a→c, and b||c
b
c
d
(e) AND-join pattern:
b→d, c→d, and b||c
a
b
c
(d) AND-split pattern:
a→b, a→c, and b||c

Example Revisited
Result produced by
the Alpha algorithm
a>b
a>c
a>e
b>c
b>d
c>b
c>d
e>d
ab
ac
ae
bd
cd
ed
b||c
c||b
b#e
e#b
c#e
a#d
…
a
b
c
e d
p2
end
p4
p1 p3
start

Split event logs based on activity labels
abdef
acdef
adbef
adcef
abdeg
acdeg
adbeg
adceg

Split {a,b,c,d,e,f,g,h} into {a,b,c,d} and {e,f,g} using sequence decomposition
abdef
acdef
adbef
adcef
abdeg
acdeg
adbeg
adceg

Result
abd
acd
adb
adc
abd
acd
adb
adc
ef
ef
ef
ef
eg
eg
eg
eg
seq
abdef
acdef
adbef
adcef
abdeg
acdeg
adbeg
adceg

Split {a,b,c,d} into {a} and {b,c,d} using sequence decomposition
abd
acd
adb
adc
abd
acd
adb
adc
ef
ef
ef
ef
eg
eg
eg
eg
seq

Result
bd
cd
db
dc
bd
cd
db
dc
ef
ef
ef
ef
eg
eg
eg
eg
seq
a
a
a
a
a
a
a
a
seq

Split {e,f,g} into {e} and {f,g} using sequence decomposition
bd
cd
db
dc
bd
cd
db
dc
ef
ef
ef
ef
eg
eg
eg
eg
seq
a
a
a
a
a
a
a
a
seq

Result
bd
cd
db
dc
bd
cd
db
dc
e
e
e
e
e
e
e
e
seq
a
a
a
a
a
a
a
a
seq
seq
f
f
f
f
g
g
g
g

Split {f,g} into {f} and {g} using XOR decomposition
bd
cd
db
dc
bd
cd
db
dc
e
e
e
e
e
e
e
e
seq
a
a
a
a
a
a
a
a
seq
seq
f
f
f
f
g
g
g
g

Result
bd
cd
db
dc
bd
cd
db
dc
e
e
e
e
e
e
e
e
seq
a
a
a
a
a
a
a
a
seq
seq
f
f
f
f
g
g
g
g
xor

Split {b,c,d} into {b,c} and {d} using AND decomposition
bd
cd
db
dc
bd
cd
db
dc
e
e
e
e
e
e
e
e
seq
a
a
a
a
a
a
a
a
seq
seq
f
f
f
f
g
g
g
g
xor

Result
d
d
d
d
d
d
d
d
e
e
e
e
e
e
e
e
seq
a
a
a
a
a
a
a
a
seq
seq
f
f
f
f
g
g
g
g
xor
b
c
b
c
b
c
b
c
par

Split {b,c} into {b} and {c} using XOR decomposition
d
d
d
d
d
d
d
d
e
e
e
e
e
e
e
e
seq
a
a
a
a
a
a
a
a
seq
seq
f
f
f
f
g
g
g
g
xor
b
c
b
c
b
c
b
c
par

Result
d
d
d
d
d
d
d
d
e
e
e
e
e
e
e
e
seq
a
a
a
a
a
a
a
a
seq
seq
par
b
b
b
b
c
c
c
c
xor
…
no further decomposition is possible

Process tree
abdef
acdef
adbef
adcef
abdeg
acdeg
adbeg
adceg
seq
abd
acd
adb
adc
ef
eg
a
bd
cd
db
dc
seq
seq
e
f
g xor
f
g
par
d
b
c xor
b
c
a
start
b
c
d
e
f
g
end
a
c
b
d
e
f
g start end

process miningdata-oriented analysis process model analysis (simulation, verification, optimization, gaming, etc.) (data mining, machine learning, business intelligence)

process miningdata-oriented analysis process model analysis performance- oriented questions, problems and solutionscompliance- oriented questions, problems and solutions(simulation, verification, optimization, gaming, etc.) (data mining, machine learning, business intelligence)

Process Mining
Data Science in Action
38.000+ people joined!

Open Lecture Wil van der Aalst

Recommended

Recommended

More Related Content

Similar to Open Lecture Wil van der Aalst

Similar to Open Lecture Wil van der Aalst (20)

More from High Tech Campus Eindhoven

More from High Tech Campus Eindhoven (8)

Recently uploaded

Recently uploaded (20)

Open Lecture Wil van der Aalst