PhD pre-defense September 2015

317 views

Published on

Slides of my presentation at my private defense (pre-defense) 11 September 2015, Gent, B

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
317
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • OMG. (2015). BPMN Quick Guide. Retrieved June 8, 2015, from http://www.bpmn.org/
  • Pinggera, J., Zugal, S., & Furtner, M. (2014). The modeling mind: Behavior patterns in process modeling. In I. Bider, K. Gaaloul, J. Krogstie, S. Nurcan, H. A. Proper, R. Schmidt, & P. Soffer (Eds.), Proceedings of the 15th International Conference on Business Process Modeling, Development, and Support (BPMDS ’14) and the 19th International Conference on Exploring Modelling Methods for Systems Analysis and Design (EMMSAD  '14), Thessaloniki, Greece, Ju (Vol. LNBIP 175, pp. 1–16). Springer.
    Poor visual expressiveness
    Poor perceptual discriminability
    Poor semantic transparency
    Poor cognitive integration
  • Song, M. S., & Van der Aalst, W. M. P. (2007). Supporting process mining by showing events at a glance. In K. Chari & A. Kumar (Eds.), Proceeding of the Seventeenth Annual Workshop on Information Technologies and Systems (WITS’07) (pp. 139–145).
  • WMC capacity: Miller, G. A. (1956). The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, 63(2), 81–97.
    CLT: Sweller, J., Van Merriënboer, J. J. G., & Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10(3), 251–296.
    CFT: Vessey, I., & Galletta, D. (1991). Cognitive Fit: An Empirical Study of Information Acquisition. Information Systems Research, 2(1), 63–84.
  • Reiter, R. (1978). On closed world data bases. In H. Gallaire & J. Minker (Eds.), Logic and Data Bases (pp. 55–76). Springer-Verlag Berlin Heidelberg.
    Motro, A., & Smets, P. (1996). Uncertainty management in information systems: From needs to solutions. US: Springer.
    Moore, P., & Pham, H. Van. (2015). On context and the open world assumption. In Proceedings of the 29th International Conference on Advanced Information Networking and Applications - Workshops (WAINA ’15), Gwangju, Korea, March 24-27, 2015 (pp. 387–392). IEEE Computer Society Press.
  • PhD pre-defense September 2015

    1. 1. 1/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Jan Claes Supervisors UGent : Geert Poels & Frederik Gailly Supervisors TU/e : Paul Grefen & Irene Vanderfeesten Investigating the process of process modeling and its relation to modeling quality The Role of Structure Serialization
    2. 2. 2/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Structure of the PhD CHAPTER 5 CONCLUSION CHAPTER 4 THEORIZATION CHAPTER 3 EXPLORATION CHAPTER 2 VISUALIZATION CHAPTER 1 INTRODUCTION
    3. 3. 3/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION CHAPTER 1 – INTRODUCTION Research objectives
    4. 4. 4/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Context Business Process Management Conceptual ModelingPhD
    5. 5. 5/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Context Article availableOrder received Check availability No Yes late deliveryundeliverable Payment received Procurement Remove article From catalogue Inform customer Financial settlement Ship article Customer informed Article removed Inform customer + + (From BPMN Quick Guide, OMG, 2015)
    6. 6. 6/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Definitions  Definition 1: Business process “A business process consists of a set of activities that are performed in coordination in an organizational and technical environment. These activities jointly realize a business goal.” (Weske, 2007, p. 5)  Definition 2: Business process model “A business process model is a mostly graphical representation that documents the different steps that are or that have to be performed in the execution of a particular business process under study, together with their execution constraints such as the allowed sequence or the potential responsible actors for these steps.”  Definition 3: Process of process modeling “the sequence of steps a modeler performs in order to translate his mental image of the process into a formal, explicit and mostly graphical process specification: the process model.”
    7. 7. 7/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 1: visualization Research Objective 1 Build knowledge about how people create models Overall objective Curiosity-driven Build knowledge about PPM Research Objective 2 Build knowledge about relation with quality Research Objective 3 Build knowledge about structured modeling Study 3: theorizationStudy 2: exploration Research Objectives
    8. 8. 8/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION CHAPTER 2 – VISUALIZATION PPMChart
    9. 9. 9/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Data collection Pnina Soffer Matthias Weidlich Barbara Weber Jakob Pinggera Stefan Zugal Jan Mendling Hajo Reijers Irene Vanderfeesten Dirk Fahland Observational data Cheetah Experimental Platform ‘Experiment’ design
    10. 10. 10/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION CREATE_ACTIVITY CREATE_START_EVENT CREATE_END_EVENT CREATE_AND CREATE_XOR CREATE_EDGE Data collection MOVE_ACTIVITY MOVE_START_EVENT MOVE_END_EVENT MOVE_AND MOVE_XOR DELETE_ACTIVITY DELETE_START_EVENT DELETE-END_EVENT DELETE_AND DELETE_XOR DELETE_EDGE NAME_ACTIVITY RENAME_ACTIVITY NAME_EDGE RENAME_EDGE
    11. 11. 11/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 1 – Visualization (From Pinggera et al., 2014)
    12. 12. 12/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 1 – Visualization
    13. 13. 13/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 1 – Visualization PPMChart  CREATE_ACTIVITY  CREATE_START_EVENT  CREATE_END_EVENT  CREATE_AND  CREATE_XOR  CREATE_EDGE  MOVE_ACTIVITY  MOVE_START_EVENT  MOVE_END_EVENT  MOVE_AND  MOVE_XOR  DELETE_ACTIVITY DELETE_START_EVENT  DELETE-END_EVENT  DELETE_AND  DELETE_XOR  DELETE_EDGE  NAME_ACTIVITY  RENAME_ACTIVITY  NAME_EDGE  RENAME_EDGE  Start event  Edge  Activity  Gateway  Edge  Activity  Edge  Edge  Activity  Edge  Gateway  Edge 7 29 8 9 32 14 30 31 10 33 56 34 time modelelements
    14. 14. 14/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Fast modelingSlow modelingInitial delayMany pauzesFew elementsMany elements No (separate) lay-outing Quick lay-outingDedicated lay-outing phase Continuous lay-outingUnpaired event creation Paired event creation No pauzes Serialization Paired gateway creation Delayed edge creation Chunked modeling Study 1 – Visualization Based on dataset of 357 unique modeling executions
    15. 15. 15/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 1 – Visualization Evaluation  Sample of intended users (6 academic researchers)  Five extreme examples in PPMChart or Dotted Chart  Observe and measure amount, quality, and timing of insights gained through the visualization  Observe and ask about perceived usefulness Results  Perceived as useful  More cognitive effective than Dotted Chart
    16. 16. 16/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION CHAPTER 3 – EXPLORATION Relation with quality
    17. 17. 17/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration Based on dataset of 40 unique modeling executions Fast modeling Slow modeling Initial delay Many pauzes Few elements Many elements No (separate) lay-outing Quick lay-outing Dedicated lay-outing phase Continuous lay-outing Unpaired event creation Paired event creation No pauzesSerializationPaired gateway creation Delayed edge creation Chunked modeling
    18. 18. 18/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration Fast modeling Slow modeling Initial delay Many pauzes Few elements Many elements No (separate) lay-outing Quick lay-outing Dedicated lay-outing phase Continuous lay-outing Unpaired event creation Paired event creation No pauzes Paired gateway creation Delayed edge creation Chunked modeling Based on dataset of 40 unique modeling executions Serialization
    19. 19. 19/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration Fast modeling Slow modeling Initial delay Many pauzes Few elements Many elements No (separate) lay-outing Quick lay-outing Continuous lay-outing Unpaired event creation Paired event creation No pauzesSerializationPaired gateway creation Delayed edge creation Chunked modeling Based on dataset of 40 unique modeling executions Dedicated lay-outing phase
    20. 20. 20/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration Fast modeling Slow modeling Initial delay Many pauzes Few elements Many elements No (separate) lay-outing Quick lay-outing Dedicated lay-outing phase Unpaired event creation Paired event creation No pauzesSerializationPaired gateway creation Delayed edge creation Chunked modeling Based on dataset of 40 unique modeling executions Continuous lay-outing
    21. 21. 21/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration Slow modeling Initial delay Many pauzes Few elements Many elements No (separate) lay-outing Quick lay-outing Dedicated lay-outing phase Continuous lay-outing Unpaired event creation Paired event creation No pauzesSerializationPaired gateway creation Delayed edge creation Chunked modeling Based on dataset of 40 unique modeling executions Fast modeling
    22. 22. 22/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration Fast modeling Initial delay Many pauzes Few elements Many elements No (separate) lay-outing Quick lay-outing Dedicated lay-outing phase Continuous lay-outing Unpaired event creation Paired event creation No pauzesSerializationPaired gateway creation Delayed edge creation Chunked modeling Slow modeling Based on dataset of 40 unique modeling executions
    23. 23. 23/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration Structuredness Movement Speed Based on dataset of 40 unique modeling executions Fast modeling Slow modeling Quick lay-outing Dedicated lay-outing phase Continuous lay-outing Serialization Chunked modeling CONJECTURE 1 CONJECTURE 2 CONJECTURE 3
    24. 24. 24/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration Conjecture 1: Structured modeling results in understandable models Conjecture 2: A high number of move operations results in less understandable models Conjecture 3: Slow modeling results in less understandable models
    25. 25. 25/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Structuredness • MaxSimulBlock • PercNumBlockAsAWhole Speed • TotTime • TotCreateTime Movement • AvgMoveOnMovedElements • PercNumElementsWithMoves Study 2 – Exploration Model quality • Perspicuity a model that is unambiguously interpretable and can be made sound with only small adaptations based on minimal assumptions on the modeler’s intentions with the model CONJECTURE1
    26. 26. 26/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration T-test t=-2,231 (p=0,028) T-test t=2,199 (p=0,030) Based on dataset of 103 unique modeling executions CONJECTURE1
    27. 27. 27/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration T-test t=-1,984 (p=0,049) T-test t=0,457 (p=0,648) Based on dataset of 103 unique modeling executions CONJECTURE2
    28. 28. 28/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 2 – Exploration T-test t=-2,183 (p=0,031) T-test t=2,505 (p=0,014) Based on dataset of 103 unique modeling executions CONJECTURE3
    29. 29. 29/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION CHAPTER 4 – THEORISATION Structured Process Modeling Theory (SPMT)
    30. 30. 30/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 3 – Theorization Based on dataset of 103 unique modeling executions T-test t=-2,231 (p=0,028) T-test t=2,199 (p=0,030) CONJECTURE 1 structuredness T-test t=-1,984 (p=0,049) T-test t=0,457 (p=0,648) T-test t=-2,183 (p=0,031) T-test t=2,505 (p=0,014) CONJECTURE 2 movement CONJECTURE 3 speed
    31. 31. 31/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 3 – Theorization Combined Flow-oriented Aspect-oriented Undirected Based on dataset of 118 unique modeling executions
    32. 32. 32/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 3 – Theorization  Observation 1. Almost all modelers paused frequently during the modeling process  Observation 2. A large group can be categorized as “flow-oriented process modeling”  Observation 3. A smaller group can be categorized as “aspect-oriented process modeling”  Observation 4. Another large group used a combination of both former styles  Observation 5. Another small group can be categorized as “undirected process modeling”  Observation 6. The “undirected” sessions lasted longer than the other approaches Based on dataset of 118 unique modeling executions
    33. 33. 33/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 3 – Theorization  Impression 1. Modelers need serialization of the modeling process to deal with its complexity  Impression 2. Structured serializing of the modeling process helps avoiding ‘mistakes’  Impression 3. Structured serializing does not support every modeler to avoid ‘mistakes’ to the same extent Based on dataset of 118 unique modeling executions
    34. 34. 34/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 3 – Theorization Cognitive Load Theory  Working memory capacity is limited  Working memory overload causes decrease in • Effectiveness (i.e., more mistakes) • Efficiency (i.e., more time and effort) • Learning Cognitive Fit Theory  Load is lower when there is a fit • Between representation, tool or strategy on the one hand • And task or modeler on the other hand
    35. 35. 35/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 3 – Theorization A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B learning style degree of serialization adopted serialization style field-dependency need for structure – + course of intrinsic cognitive load for process modeling phases course of intrinsic cognitive load for aggregation phases course of cognitive overload course of intrinsic cognitive load for strategy building phases + + + serialization style fitstructuredness of serialization – –– – 1 2 3
    36. 36. 36/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Study 3 – Theorization Evaluation of utility Novelty (uses existing theories in fundamental new way) Parsimony (11 constructs, 15 associations) Consistency (can explain additional observations) Plausibility (accurate and profound explanation) Credibility (building blocks are established theories) Transferability (problem solving in general) Consistency based on dataset of 143 unique modeling executions
    37. 37. 37/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION CHAPTER 5 – CONCLUSION Summary & Future work
    38. 38. 38/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION FUTURE WORK Knowledge gaps Knowledge contribution Research instrument Knowledge gap 4 How to change one’s modeling strategy? Knowledge gap 3 How should one model in a specific context? Study 1: visualization Contributions A – PPMChart B – 22 patterns C – 13 observations Study 2: exploration Contributions D – 8 patterns E – 3 conjectures F – 1 metric Knowledge gap 1 How do people currently model? Knowledge gap 2 Which strategy is intrinsically better? A B C D E F H HG I Study 3: theorization Contributions G – 5 styles H1 – 6 observations H2 – 3 impressions I – SPMT I
    39. 39. 39/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Future Work Use developed knowledge to Develop prescriptive theory  Cognitive profile  best modeling approach Develop method (SPMM)  (1) Determine (2) learn (3) apply best approach Develop tool support  Cognitive tests  Interactive digital approach tutorial
    40. 40. 40/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Reflection – Deduction, induction, abduction Rule Case Result Deduction Reveals effect Results in certainty Induction Case Result Rule Reveals mechanism Results in probability Abduction Result Case Rule Reveals cause Results in possibility Process modeling as application domain > ordering of data facilitates pattern recognition <
    41. 41. 41/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Reflection – Student observation Assumption: overload has same causes for students and practitioners Assumption: overload has same consequences for students and practitioners STUDENTS… … are representative participants … don’t suffer from Expert-Reversal Effect … form a homogeneous group … provide heterogeneous set of observations … reach point of overload faster than practitioners
    42. 42. 42/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Reflection – Empirical behavioral research Process modeling = complex and dynamic task Identify/measure/control confounding variables Ignore/assume constant/assume minimal effect Include multiple variables in model Mix techniques (de/in/abduction, quanti/qualitative) Open-world assumption > conclusions are incomplete unless proven otherwise < > no conclusions from insignificant results < > more accurate, but slower progress <
    43. 43. 43/43 www.janclaes.info INTRODUCTION VISUALIZATION EXPLORATION THEORIZATION CONCLUSION Thanks for your attention! Do you have any questions? Jan Claes jan.claes@ugent.be http://www.janclaes.info Twitter: @janclaesbelgium

    ×