The document summarizes three studies conducted as part of a PhD investigating the process of process modeling and its relation to modeling quality. The first study visualized how people create process models through an analysis of modeling actions. The second study explored relationships between modeling behaviors and quality, finding that structured modeling related to better understandability. The third study developed the Structured Process Modeling Theory to explain observations of modeling styles and their cognitive impacts.

1. 1/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Jan Claes
Supervisors UGent : Geert Poels & Frederik Gailly
Supervisors TU/e : Paul Grefen & Irene Vanderfeesten
Investigating the process of process modeling
and its relation to modeling quality
The Role of Structure Serialization

2. 2/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Structure of the PhD
CHAPTER 5
CONCLUSION
CHAPTER 4
THEORIZATION
CHAPTER 3
EXPLORATION
CHAPTER 2
VISUALIZATION
CHAPTER 1
INTRODUCTION

3. 3/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
CHAPTER 1 – INTRODUCTION
Research objectives

4. 4/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Context
Business Process
Management
Conceptual
ModelingPhD

5. 5/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Context
Article availableOrder received
Check
availability
No
Yes
late deliveryundeliverable
Payment received
Procurement
Remove article
From catalogue
Inform
customer
Financial
settlement
Ship article
Customer informed
Article removed
Inform
customer
+
+
(From BPMN Quick Guide, OMG, 2015)

6. 6/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Definitions
 Definition 1: Business process
“A business process consists of a set of activities that are performed in
coordination in an organizational and technical environment. These activities
jointly realize a business goal.” (Weske, 2007, p. 5)
 Definition 2: Business process model
“A business process model is a mostly graphical representation that documents
the different steps that are or that have to be performed in the execution of a
particular business process under study, together with their execution
constraints such as the allowed sequence or the potential responsible actors for
these steps.”
 Definition 3: Process of process modeling
“the sequence of steps a modeler performs in order to translate his mental
image of the process into a formal, explicit and mostly graphical process
specification: the process model.”

7. 7/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 1: visualization
Research Objective 1
Build knowledge about
how people create models
Overall objective
Curiosity-driven
Build knowledge about PPM
Research Objective 2
Build knowledge about
relation with quality
Research Objective 3
Build knowledge about
structured modeling
Study 3: theorizationStudy 2: exploration
Research Objectives

8. 8/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
CHAPTER 2 – VISUALIZATION
PPMChart

9. 9/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Data collection
Pnina Soffer
Matthias Weidlich
Barbara Weber
Jakob Pinggera
Stefan Zugal
Jan Mendling
Hajo Reijers
Irene Vanderfeesten
Dirk Fahland
Observational
data
Cheetah
Experimental
Platform
‘Experiment’
design

10. 10/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
CREATE_ACTIVITY
CREATE_START_EVENT
CREATE_END_EVENT
CREATE_AND
CREATE_XOR
CREATE_EDGE
Data collection
MOVE_ACTIVITY
MOVE_START_EVENT
MOVE_END_EVENT
MOVE_AND
MOVE_XOR
DELETE_ACTIVITY
DELETE_START_EVENT
DELETE-END_EVENT
DELETE_AND
DELETE_XOR
DELETE_EDGE
NAME_ACTIVITY
RENAME_ACTIVITY
NAME_EDGE
RENAME_EDGE

11. 11/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 1 – Visualization
(From Pinggera et al., 2014)

12. 12/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 1 – Visualization

13. 13/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 1 – Visualization
PPMChart
 CREATE_ACTIVITY
 CREATE_START_EVENT
 CREATE_END_EVENT
 CREATE_AND
 CREATE_XOR
 CREATE_EDGE
 MOVE_ACTIVITY
 MOVE_START_EVENT
 MOVE_END_EVENT
 MOVE_AND
 MOVE_XOR
 DELETE_ACTIVITY
DELETE_START_EVENT
 DELETE-END_EVENT
 DELETE_AND
 DELETE_XOR
 DELETE_EDGE
 NAME_ACTIVITY
 RENAME_ACTIVITY
 NAME_EDGE
 RENAME_EDGE
 Start event
 Edge
 Activity
 Gateway
 Edge
 Activity
 Edge
 Edge
 Activity
 Edge
 Gateway
 Edge
7
29
8
9
32
14
30
31
10
33
56
34
time
modelelements

14. 14/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Fast
modelingSlow
modelingInitial
delayMany
pauzesFew
elementsMany
elements No
(separate)
lay-outing
Quick
lay-outingDedicated
lay-outing
phase
Continuous
lay-outingUnpaired
event
creation
Paired
event
creation
No pauzes
Serialization
Paired
gateway
creation
Delayed
edge
creation
Chunked
modeling
Study 1 – Visualization
Based on dataset of 357 unique modeling executions

15. 15/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 1 – Visualization
Evaluation
 Sample of intended users (6 academic researchers)
 Five extreme examples in PPMChart or Dotted Chart
 Observe and measure amount, quality, and timing
of insights gained through the visualization
 Observe and ask about perceived usefulness
Results
 Perceived as useful
 More cognitive effective than Dotted Chart

16. 16/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
CHAPTER 3 – EXPLORATION
Relation with quality

17. 17/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
Based on dataset of 40 unique modeling executions
Fast
modeling
Slow
modeling
Initial
delay
Many
pauzes
Few
elements
Many
elements
No
(separate)
lay-outing
Quick
lay-outing
Dedicated
lay-outing
phase
Continuous
lay-outing
Unpaired
event
creation
Paired
event
creation
No pauzesSerializationPaired
gateway
creation
Delayed
edge
creation
Chunked
modeling

18. 18/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
Fast
modeling
Slow
modeling
Initial
delay
Many
pauzes
Few
elements
Many
elements
No
(separate)
lay-outing
Quick
lay-outing
Dedicated
lay-outing
phase
Continuous
lay-outing
Unpaired
event
creation
Paired
event
creation
No pauzes Paired
gateway
creation
Delayed
edge
creation
Chunked
modeling
Based on dataset of 40 unique modeling executions
Serialization

19. 19/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
Fast
modeling
Slow
modeling
Initial
delay
Many
pauzes
Few
elements
Many
elements
No
(separate)
lay-outing
Quick
lay-outing
Continuous
lay-outing
Unpaired
event
creation
Paired
event
creation
No pauzesSerializationPaired
gateway
creation
Delayed
edge
creation
Chunked
modeling
Based on dataset of 40 unique modeling executions
Dedicated
lay-outing
phase

20. 20/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
Fast
modeling
Slow
modeling
Initial
delay
Many
pauzes
Few
elements
Many
elements
No
(separate)
lay-outing
Quick
lay-outing
Dedicated
lay-outing
phase
Unpaired
event
creation
Paired
event
creation
No pauzesSerializationPaired
gateway
creation
Delayed
edge
creation
Chunked
modeling
Based on dataset of 40 unique modeling executions
Continuous
lay-outing

21. 21/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
Slow
modeling
Initial
delay
Many
pauzes
Few
elements
Many
elements
No
(separate)
lay-outing
Quick
lay-outing
Dedicated
lay-outing
phase
Continuous
lay-outing
Unpaired
event
creation
Paired
event
creation
No pauzesSerializationPaired
gateway
creation
Delayed
edge
creation
Chunked
modeling
Based on dataset of 40 unique modeling executions
Fast
modeling

22. 22/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
Fast
modeling
Initial
delay
Many
pauzes
Few
elements
Many
elements
No
(separate)
lay-outing
Quick
lay-outing
Dedicated
lay-outing
phase
Continuous
lay-outing
Unpaired
event
creation
Paired
event
creation
No pauzesSerializationPaired
gateway
creation
Delayed
edge
creation
Chunked
modeling
Slow
modeling
Based on dataset of 40 unique modeling executions

23. 23/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
Structuredness Movement Speed
Based on dataset of 40 unique modeling executions
Fast
modeling
Slow
modeling
Quick
lay-outing
Dedicated
lay-outing
phase
Continuous
lay-outing
Serialization
Chunked
modeling
CONJECTURE 1
CONJECTURE 2
CONJECTURE 3

24. 24/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
Conjecture 1: Structured modeling
results in
understandable models
Conjecture 2: A high number of move operations
results in
less understandable models
Conjecture 3: Slow modeling
results in
less understandable models

25. 25/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Structuredness
• MaxSimulBlock
• PercNumBlockAsAWhole
Speed
• TotTime
• TotCreateTime
Movement
• AvgMoveOnMovedElements
• PercNumElementsWithMoves
Study 2 – Exploration
Model quality
• Perspicuity
a model that is unambiguously interpretable and can be made sound with only small
adaptations based on minimal assumptions on the modeler’s intentions with the model
CONJECTURE1

26. 26/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
T-test
t=-2,231 (p=0,028)
T-test
t=2,199 (p=0,030)
Based on dataset of 103 unique modeling executions
CONJECTURE1

27. 27/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
T-test
t=-1,984 (p=0,049)
T-test
t=0,457 (p=0,648)
Based on dataset of 103 unique modeling executions
CONJECTURE2

28. 28/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 2 – Exploration
T-test
t=-2,183 (p=0,031)
T-test
t=2,505 (p=0,014)
Based on dataset of 103 unique modeling executions
CONJECTURE3

29. 29/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
CHAPTER 4 – THEORISATION
Structured Process Modeling Theory (SPMT)

30. 30/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 3 – Theorization
Based on dataset of 103 unique modeling executions
T-test
t=-2,231
(p=0,028)
T-test
t=2,199
(p=0,030)
CONJECTURE 1
structuredness
T-test
t=-1,984
(p=0,049)
T-test
t=0,457
(p=0,648)
T-test
t=-2,183
(p=0,031)
T-test
t=2,505
(p=0,014)
CONJECTURE 2
movement
CONJECTURE 3
speed

31. 31/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 3 – Theorization
Combined
Flow-oriented Aspect-oriented
Undirected
Based on dataset of 118 unique modeling executions

32. 32/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 3 – Theorization
 Observation 1. Almost all modelers paused frequently
during the modeling process
 Observation 2. A large group can be categorized as
“flow-oriented process modeling”
 Observation 3. A smaller group can be categorized as
“aspect-oriented process modeling”
 Observation 4. Another large group used a combination
of both former styles
 Observation 5. Another small group can be categorized
as “undirected process modeling”
 Observation 6. The “undirected” sessions lasted longer
than the other approaches
Based on dataset of 118 unique modeling executions

33. 33/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 3 – Theorization
 Impression 1. Modelers need serialization of the modeling
process to deal with its complexity
 Impression 2. Structured serializing of the modeling process
helps avoiding ‘mistakes’
 Impression 3. Structured serializing does not support every
modeler to avoid ‘mistakes’ to the same extent
Based on dataset of 118 unique modeling executions

34. 34/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 3 – Theorization
Cognitive Load Theory
 Working memory capacity is limited
 Working memory overload causes decrease in
• Effectiveness (i.e., more mistakes)
• Efficiency (i.e., more time and effort)
• Learning
Cognitive Fit Theory
 Load is lower when there is a fit
• Between representation, tool or strategy on the one hand
• And task or modeler on the other hand

35. 35/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 3 – Theorization
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
field-dependency need for structure
– +
course of intrinsic cognitive load
for process modeling phases
course of intrinsic cognitive load
for aggregation phases
course of cognitive overload
course of intrinsic cognitive load
for strategy building phases
+ + +
serialization style fitstructuredness of serialization
– –– –
1 2 3

36. 36/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Study 3 – Theorization
Evaluation of utility
Novelty (uses existing theories in fundamental new way)
Parsimony (11 constructs, 15 associations)
Consistency (can explain additional observations)
Plausibility (accurate and profound explanation)
Credibility (building blocks are established theories)
Transferability (problem solving in general)
Consistency based on dataset of 143 unique modeling executions

37. 37/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
CHAPTER 5 – CONCLUSION
Summary & Future work

38. 38/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
FUTURE
WORK
Knowledge gaps
Knowledge contribution Research instrument
Knowledge gap 4
How to change one’s
modeling strategy?
Knowledge gap 3
How should one model
in a specific context?
Study 1: visualization
Contributions
A – PPMChart
B – 22 patterns
C – 13 observations
Study 2: exploration
Contributions
D – 8 patterns
E – 3 conjectures
F – 1 metric
Knowledge gap 1
How do people
currently model?
Knowledge gap 2
Which strategy is
intrinsically better?
A B
C
D E F
H
HG
I
Study 3: theorization
Contributions
G – 5 styles
H1 – 6 observations
H2 – 3 impressions
I – SPMT
I

39. 39/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Future Work
Use developed knowledge to
Develop prescriptive theory
 Cognitive profile  best modeling approach
Develop method (SPMM)
 (1) Determine (2) learn (3) apply best approach
Develop tool support
 Cognitive tests
 Interactive digital approach tutorial

40. 40/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Reflection – Deduction, induction, abduction
Rule Case
Result
Deduction
Reveals effect
Results in certainty
Induction
Case Result
Rule
Reveals mechanism
Results in probability
Abduction
Result Case
Rule
Reveals cause
Results in possibility
Process modeling as application domain
> ordering of data facilitates pattern recognition <

41. 41/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Reflection – Student observation
Assumption: overload has same causes
for students and practitioners
Assumption: overload has same consequences
for students and practitioners
STUDENTS…
… are representative participants
… don’t suffer from Expert-Reversal Effect
… form a homogeneous group
… provide heterogeneous set of observations
… reach point of overload faster than practitioners

42. 42/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Reflection – Empirical behavioral research
Process modeling = complex and dynamic task
Identify/measure/control confounding variables
Ignore/assume constant/assume minimal effect
Include multiple variables in model
Mix techniques (de/in/abduction, quanti/qualitative)
Open-world assumption
> conclusions are incomplete unless proven otherwise <
> no conclusions from insignificant results <
> more accurate, but slower progress <

43. 43/43
www.janclaes.info
INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION
Thanks for your attention!
Do you have any questions?
Jan Claes
jan.claes@ugent.be
http://www.janclaes.info
Twitter: @janclaesbelgium