This document summarizes a study that examined how the process of constructing process models impacts the quality and understandability of the resulting models. The researchers analyzed data from modelers constructing process models using an experimental platform that recorded their actions. They found that structured modeling, with fewer object moves and slower modeling, resulted in more understandable models, while an increased number of moves negatively impacted understandability. The goal was to understand process modeling to improve training and tools.

1. Tying Process Model Quality to the Modeling Process:
The Impact of Structuring, Movement, and Speed
Jan Claes, Irene Vanderfeesten, Hajo A. Reijers, Jakob Pinggera, Matthias
Weidlich, Stefan Zugal, Dirk Fahland, Barbara Weber, Jan Mendling, Geert Poels

2. The Process of Process Modeling
RESEARCH
Properties of Properties of Properties of Properties of
textual description modeler modeling process resulting model
Modelers were asked to construct a process model of a
case based on a given informal textual description
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 2/16

3. Cheetah Experiment Platform
http://cheetahplatform.org
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 3/16

4. Replay model construction process
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 4/16

5. Event Log
CREATE_XOR_GATEWAY
CREATE_ACTIVITY CREATE_EDGE
CREATE_START_EVENT CREATE_AND_GATEWAY
MOVE_ACTIVITY
DELETE_ACTIVITY
NAME_EDGE CREATE_EDGE_BENDPOINT
RENAME_ACTIVITY
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 5/16

6. PPMChart: examples
Create activity
Create AND gateway
Create edge
Move activity
Rename activity
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 6/16

7. PPMChart: examples
Create activity ACT
Create
XOR gateway X X
Create edge
ACT
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 7/16

8. PPMChart: examples
Move objects
short after creation
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 8/16

9. PPMChart: examples
Move objects
at the end
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 9/16

10. PPMChart: examples
Move objects
regularly
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 10/16

11. Conjectures
C1: Structured modeling is positively related to
the understandability of the resulting model.
C2: A high number of move operations is
negatively related to the understandability of
the resulting model.
C3: Slow modeling is negatively related to the
understandability of the resulting model.
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 11/16

12. Metrics
Understandability
Measured by perspicuity:
“a model that is unambiguously interpretable and can be made
sound with only small adaptations based on minimal
assumptions on the modeler’s intentions with the model ”
Calculation
 Correct syntax errors/ambiguities if possible
 Transform BPMN model to WF-net
 Check soundness of WF-net
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 12/16

13. Metrics
Structured modeling
Creating blocks ‘as a whole’ (before moving on
to the creation of the rest of the model)
ACT
ACT X X
ACT
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 13/16

14. Results
C1: structured modeling – understandability
maxSimulBlock perspicuity
perspicuity   0,028*
percNumBlockAsAWhole perspicuity  0,030*
C2: number of moves – understandability
avgMoveOnMovedElements
avgMoveOnMovedElements perspicuity
perspicuity   0,049*
0,049*
percNumElementsWithMoves
percNumElementsWithMoves perspicuity
perspicuity   0,648
0,648
C3: speed – understandability
totTime
totTime perspicuity
perspicuity   0,031*
0,031*
totCreateTime
totCreateTime perspicuity
perspicuity   0,014*
0,014*
see statistics in the paper (*) statistically significant values at the 95% confidence level
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 14/16

15. Conclusion
Goal
 Understand process of process modeling to improve
training/tools
Conjectures
 C1: Structured modeling is pos. related to understandability
 C2: A lot of moves might be neg. related to understandability
 C3: Slow modeling is neg. related to understandability
Further work
 Zoom on structured modeling
 Investigate other conjectures
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 15/16

16. Closing
Thank you for your attention!
Do you have any questions?
Jan Claes
jan.claes@ugent.be Jan Claes Stefan Zugal
http://www.janclaes.info Irene Vanderfeesten Dirk Fahland
Twitter: @janclaesbelgium
Hajo A. Reijers Barbara Weber
Jakob Pinggera Jan Mendling
Matthias Weidlich Geert Poels
Tying Process Model Quality to the Modeling Process: Jan Claes - BPM‘12 - 4 September 2012
The Impact of Structuring, Movement, and Speed 16/16