Slides of my presentation at the UGent MIS research seminar of June, 2 June 2015, Gent, B

1. 1/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
Jan Claes
Supervisors UGent : Geert Poels & Frederik Gailly
Supervisors TU/e : Paul Grefen & Irene Vanderfeesten
Investigating the process of process modeling
and its relation to modeling quality
The Role of Structure Serialization

2. 2/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 1 – INTRODUCTION
Research gaps

3. 3/33
www.janclaes.info
Context
Increasing complexity of organizations
(globalization, customization, cost-effectiveness, …)
Process orientation
(efficiency, responsiveness, differentiation)
Process models
(representing process steps and execution constraints)
Process of Process Modeling
(translate mental image of process into formal model)

4. 4/33
www.janclaes.info
Research gaps
 GAP 1. Need for accurate measurements
 GAP 2. Need for detailed, yet cognitive effective
visualizations
 GAP 3. Knowledge about how people construct
process models (=PPM)
 GAP 4. Knowledge about relation between PPM
and model quality
 GAP 5. Need for practical process modeling methods
 GAP 6. Knowledge about process modeling
challenges

5. 5/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 2 – VISUALIZATION
PPMChart

6. 6/33
www.janclaes.info
Study 1 – Visualization
Current techniques
 Too high-level (Modeling Phase Diagrams)
 Not cognitive effective (Dotted Chart)
Design method
 9 principles of cognitive effective visualization
Evaluation method
 Qualitative evaluation with 6 academic researchers
 Modeling pattern discovery

7. 7/33
www.janclaes.info
Study 1 – Visualization
Cognitive effective visualization design principles
 Visual expressiveness (maximal use of graphical variables)
 Perceptual discriminability (visual matches conceptual distance)
 Graphic economy (maximal six values per variable)
 Dual coding (combine graphics with text)
 Semiotic clarity (exactly one symbol per exactly one concept)
 Semantic transparency (intuitiveness through natural mapping)
 Complexity management (modularization and hierarchical structuring)
 Cognitive integration (easy integration with other charts/models)
 Cognitive fit (fit with task and user)

8. 8/33
www.janclaes.info
Study 1 – Visualization
PPMChart
 CREATE_ACTIVITY
 CREATE_START_EVENT
 CREATE_END_EVENT
 CREATE_AND
 CREATE_XOR
 CREATE_EDGE
 MOVE_ACTIVITY
 MOVE_START_EVENT
 MOVE_END_EVENT
 MOVE_AND
 MOVE_XOR
 DELETE_ACTIVITY
DELETE_START_EVENT
 DELETE-END_EVENT
 DELETE_AND
 DELETE_XOR
 DELETE_EDGE
 NAME_ACTIVITY
 RENAME_ACTIVITY
 NAME_EDGE
 RENAME_EDGE
 Start event
 Edge
 Activity
 Gateway
 Edge
 Activity
 Edge
 Edge
 Activity
 Edge
 Gateway
 Edge
7
29
8
9
32
14
30
31
10
33
56
34
time
modelelements

9. 9/33
www.janclaes.info
Fast
modelingSlow
modelingInitial
delayMany
pauzesFew
elementsMany
elements No
(separate)
lay-outing
Quick
lay-outingDedicated
lay-outing
phase
Continuous
lay-outingUnpaired
event
creation
Paired
event
creation
No pauzes
Serialization
Paired
gateway
creation
Delayed
edge
creation
Chunked
modeling
Study 1 – Visualization
“Modeling
patterns”
Based on dataset of 357 unique modeling executions

10. 10/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 3 – EXPLORATION
Relation with quality

11. 11/33
www.janclaes.info
Study 2 – Exploration
Relation between modeling patterns and
process model quality
Exploration method
 Compare PPMCharts with process models
 Discover links
Evaluation method
 Measure definition
 Quantitative data collection
 T-tests

12. 12/33
www.janclaes.info
Study 2 – Exploration
Fast
modeling
Slow
modeling
Quick
lay-outing
Dedicated
lay-outing
phase
Continuous
lay-outing
Serialization
Chunked
modeling
Structuredness Movement Speed
Based on dataset of 40 unique modeling executions

13. 13/33
www.janclaes.info
Study 2 – Exploration
Conjecture 1: Structured modeling
results in
understandable models
Conjecture 2: A high number of move operations
results in
less understandable models
Conjecture 3: Slow modeling
results in
less understandable models

14. 14/33
www.janclaes.info
 Structuredness
• MaxSimulBlock
• PercNumBlockAsAWhole
 Speed
• TotTime
• TotCreateTime
 Movement
• AvgMoveOnMovedElements
• PercNumElementsWithMoves
Study 2 – Exploration
Measurement
 Model quality
• Perspicuity
a model that is unambiguously interpretable and can be made sound
with only small adaptations based on minimal assumptions on the
modeler’s intentions with the model

15. 15/33
www.janclaes.info
Study 2 – Exploration
T-test
t=-2,231 (p=0,028)
T-test
t=2,199 (p=0,030)
Based on dataset of 103 unique modeling executions

16. 16/33
www.janclaes.info
Study 2 – Exploration
T-test
t=-1,984 (p=0,049)
T-test
t=0,457 (p=0,648)
Based on dataset of 103 unique modeling executions

17. 17/33
www.janclaes.info
Study 2 – Exploration
T-test
t=-2,183 (p=0,031)
T-test
t=2,505 (p=0,014)
Based on dataset of 103 unique modeling executions

18. 18/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 4 – THEORISATION
Structured Process Modeling Theory (SPMT)

19. 19/33
www.janclaes.info
Study 3 – Theorization
Explanatory theory
Theory building method
 6 observations, 3 impressions (induction)
 Explanation via existing theories (deduction)
Evaluation method
 Assessment of novelty, parsimony, consistency,
plausibility, credibility, and transferability
 Inconclusive empirical results, but open-world
assumption

20. 20/33
www.janclaes.info
Study 3 – Theorization
Combined
Flow-oriented Aspect-oriented
Undirected
“Modeling
styles”
Based on dataset of 118 unique modeling executions

21. 21/33
www.janclaes.info
Study 3 – Theorization
 Observation 1. Almost all modelers paused frequently
during the modeling process
 Observation 2. A large group can be categorized as
“flow-oriented process modeling”
 Observation 3. A smaller group can be categorized as
“aspect-oriented process modeling”
 Observation 4. Another large group used a combination
of both former styles
 Observation 5. Another small group can be categorized
as “undirected process modeling”
 Observation 6. The “undirected” sessions lasted longer
than the other approaches
Based on dataset of 118 unique modeling executions

22. 22/33
www.janclaes.info
Study 3 – Theorization
 Impression 1. Modelers need serialization of the modeling
process to deal with its complexity
 Impression 2. Structured serializing of the modeling process
helps avoiding ‘mistakes’
 Impression 3. Structured serializing does not support every
modeler to avoid ‘mistakes’ to the same extent
Based on dataset of 118 unique modeling executions

23. 23/33
www.janclaes.info
Study 3 – Theorization
Cognitive Load Theory
 Working memory capacity is limited
 Working memory overload causes decrease in
• Effectiveness (i.e., more mistakes)
• Efficiency (i.e., more time and effort)
• Learning
Cognitive Fit Theory
 Load is lower when there is a fit
• Between representation, tool or strategy on the one hand
• And task or modeler on the other hand

24. 24/33
www.janclaes.info
Study 3 – Theorization
A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term+ +–
input material representation fit
working memory capacity
extraneous cognitive load germane cognitive load
cognitive schema construction
process model quality overall construction time
cognitive overload
intrinsic cognitive load
++
+
+++
–
task complexity
+
prior knowledge
––
– –

25. 25/33
www.janclaes.info
Study 3 – Theorization
A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B
learning style
degree of serialization
adopted serialization style
field-dependency need for structure
– +
course of intrinsic cognitive load
for process modeling phases
course of intrinsic cognitive load
for aggregation phases
course of cognitive overload
course of intrinsic cognitive load
for strategy building phases
+ + +
serialization style fitstructuredness of serialization
– –– –
1 2 3

26. 26/33
www.janclaes.info
Study 3 – Theorization
Novelty (uses existing theories in fundamental new way)
Parsimony (11 constructs, 15 associations)
Consistency (can explain additional observations)
Plausibility (accurate and profound explanation)
Credibility (building blocks are established theories)
Transferability (problem solving in general)
Falsifiability (inconclusive, but open-world assumed)
Utility (only on longer term)
Consistency based on dataset of 143 unique modeling executions

27. 27/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
CHAPTER 5 – CONCLUSION
Summary & Future work

28. 28/33
www.janclaes.info
Studies
Research
Cycle 3
Exploration
Research Cycle 1
Structured Process Modeling Theory
(SPMT)
RC6
SPMT
measures RC5
Cognitive
measures
RC4
Perspicuity
Engineering
Cycle 1
PPMChart
RC2
Modeling
styles
RC 7. Design validation
RC 7. Research design
EC 2. Problem investigation
RC 3. Problem investigation
RC 3. Research design
RC 3. Design validation
RC 3. Evaluation
RC 4. Evaluation
RC 4. Problem investigation
RC 4. Research design
RC 4. Design validation
RC 4. Research
RC 3. Research
EC 1. Problem investigation
EC 1. Solution design
EC 1. Design validation
EC 1. Implementation
EC 1. Evaluation
RC 1. Problem investigation
RC 2. Problem investigation
RC 2. Research design
RC 2. Design validation
RC 2. Research
RC 2. Evaluation
RC 1. Research design
RC 1. Design validation
EC 2. Evaluation
RC 8. Evaluation
RC 8. Research
RC 8. Design validation
RC 8. Research design
RC 8. Problem investigation
RC 7. Evaluation
RC 7. Research
RC 7. Problem investigation
EC 2. Design validation
EC 2. Solution design
RC 1. Evaluation
RC 1. Research
RC 6. Evaluation
RC 6. Research
RC 6. Design validation
RC 6. Research design
RC 6. Problem investigation
RC 5. Evaluation
RC 5. Research
RC 5. Design validation RC 5. Research design
RC 5. Problem investigation
EC 2. Implementation
EC2
Structured Process
Modeling Method
(SPMM)
RC8
Training
RC7
Influenceability
of method

29. 29/33
www.janclaes.info
Studies
Study 1. Visualization
• EC1. How can the operations of the process of process modeling
be presented in a cognitive effective and efficient way?
• RC2. How do people construct process models in terms of
modeling styles?
PPMChart
Research instrument
(visualization)
Study 3. Theorization
• RC1. Why do people struggle with the complexity of process
modeling?
• RC2. How do people construct process models in terms of
modeling patterns?
SPMT
Theory – type II
(explanation)
Study 2. Exploration
• RC3. How are process and product of modeling related?
• RC4. How to measure (syntax) errors with cognitive origin?
• RC2. How do people construct process models in terms of
modeling patterns?
Process vs. product
Conjectures
(exploration)
GAP 2
GAP 3
GAP 6
GAP 3
GAP 4
GAP 1
GAP 3

30. 30/33
www.janclaes.info
Future work
Study 5. Tool support
• EC3. How to support measurement of cognitive profile?
• EC4. How to support measurement of modeling effectiveness
and efficiency?
• EC5. How to support the SPMM
SPMTool
Tool support
(implementation)
GAP 1
GAP 1
GAP 5
Study 4. Method
• EC2. How to create process models in an effective and efficient
way?
• RC7. Is it possible to change a modeler’s approach towards
process modeling?
• RC8. How to transform the SPMT into a prescriptive theory?
SPMM
Practical method
(prescription)
GAP 5
GAP 6
GAP 5

31. 31/33
www.janclaes.info
Key publications
Publications in international journals
 Indexed by Web Of Science
• J. Claes, I. Vanderfeesten, J. Pinggera, H.A. Reijers, B. Weber, G. Poels, A visual
analysis of the process of process modeling, Information Systems and e-Business
Management, Vol 13(1), p. 147-190, 2015.
 Under review
• J. Claes, I. Vanderfeesten, F. Gailly, P. Grefen, G. Poels, The Structured Process
Modeling Theory (SPMT) A cognitive view on why and how modelers benefit from
structuring the process of process modeling, resubmitted after revision to Information
Systems Frontiers.

32. 32/33
www.janclaes.info
Key publications
Publications in international conference
proceedings
 Indexed by Web Of Science
• J. Claes, I. Vanderfeesten, H.A. Reijers, J. Pinggera, M. Weidlich, S. Zugal, D. Fahland,
B. Weber, J. Mendling, G. Poels, Tying Process Model Quality to the Modeling Process:
The Impact of Structuring, Movement, and Speed, Proc. BPM '12, LNCS 7481,
Springer, 2012, p. 33-48.
• J. Claes, I. Vanderfeesten, J. Pinggera, H.A. Reijers, B. Weber, G. Poels, Visualizing the
Process of Process Modeling with PPMCharts, Proc. BPM '12 Workshops, LNBIP 132,
Springer, 2012, p. 744-755.
• J. Claes, F. Gailly, G. Poels, Cognitive Aspects of Structured Process Modeling, Proc.
CAiSE '13 Workshops, LNBIP 148, Springer, p. 168-173, 2013.

33. 33/33
FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION
www.janclaes.info
Thanks for your attention!
Do you have any questions?
Jan Claes
jan.claes@ugent.be
http://www.janclaes.info
Twitter: @janclaesbelgium