Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

UGent MIS research seminar June 2015

267 views

Published on

Slides of my presentation at the UGent MIS research seminar of June, 2 June 2015, Gent, B

Published in: Business
  • Be the first to comment

  • Be the first to like this

UGent MIS research seminar June 2015

  1. 1. 1/33 FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION www.janclaes.info Jan Claes Supervisors UGent : Geert Poels & Frederik Gailly Supervisors TU/e : Paul Grefen & Irene Vanderfeesten Investigating the process of process modeling and its relation to modeling quality The Role of Structure Serialization
  2. 2. 2/33 FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION www.janclaes.info CHAPTER 1 – INTRODUCTION Research gaps
  3. 3. 3/33 www.janclaes.info Context Increasing complexity of organizations (globalization, customization, cost-effectiveness, …) Process orientation (efficiency, responsiveness, differentiation) Process models (representing process steps and execution constraints) Process of Process Modeling (translate mental image of process into formal model)
  4. 4. 4/33 www.janclaes.info Research gaps  GAP 1. Need for accurate measurements  GAP 2. Need for detailed, yet cognitive effective visualizations  GAP 3. Knowledge about how people construct process models (=PPM)  GAP 4. Knowledge about relation between PPM and model quality  GAP 5. Need for practical process modeling methods  GAP 6. Knowledge about process modeling challenges
  5. 5. 5/33 FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION www.janclaes.info CHAPTER 2 – VISUALIZATION PPMChart
  6. 6. 6/33 www.janclaes.info Study 1 – Visualization Current techniques  Too high-level (Modeling Phase Diagrams)  Not cognitive effective (Dotted Chart) Design method  9 principles of cognitive effective visualization Evaluation method  Qualitative evaluation with 6 academic researchers  Modeling pattern discovery
  7. 7. 7/33 www.janclaes.info Study 1 – Visualization Cognitive effective visualization design principles  Visual expressiveness (maximal use of graphical variables)  Perceptual discriminability (visual matches conceptual distance)  Graphic economy (maximal six values per variable)  Dual coding (combine graphics with text)  Semiotic clarity (exactly one symbol per exactly one concept)  Semantic transparency (intuitiveness through natural mapping)  Complexity management (modularization and hierarchical structuring)  Cognitive integration (easy integration with other charts/models)  Cognitive fit (fit with task and user)
  8. 8. 8/33 www.janclaes.info Study 1 – Visualization PPMChart  CREATE_ACTIVITY  CREATE_START_EVENT  CREATE_END_EVENT  CREATE_AND  CREATE_XOR  CREATE_EDGE  MOVE_ACTIVITY  MOVE_START_EVENT  MOVE_END_EVENT  MOVE_AND  MOVE_XOR  DELETE_ACTIVITY DELETE_START_EVENT  DELETE-END_EVENT  DELETE_AND  DELETE_XOR  DELETE_EDGE  NAME_ACTIVITY  RENAME_ACTIVITY  NAME_EDGE  RENAME_EDGE  Start event  Edge  Activity  Gateway  Edge  Activity  Edge  Edge  Activity  Edge  Gateway  Edge 7 29 8 9 32 14 30 31 10 33 56 34 time modelelements
  9. 9. 9/33 www.janclaes.info Fast modelingSlow modelingInitial delayMany pauzesFew elementsMany elements No (separate) lay-outing Quick lay-outingDedicated lay-outing phase Continuous lay-outingUnpaired event creation Paired event creation No pauzes Serialization Paired gateway creation Delayed edge creation Chunked modeling Study 1 – Visualization “Modeling patterns” Based on dataset of 357 unique modeling executions
  10. 10. 10/33 FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION www.janclaes.info CHAPTER 3 – EXPLORATION Relation with quality
  11. 11. 11/33 www.janclaes.info Study 2 – Exploration Relation between modeling patterns and process model quality Exploration method  Compare PPMCharts with process models  Discover links Evaluation method  Measure definition  Quantitative data collection  T-tests
  12. 12. 12/33 www.janclaes.info Study 2 – Exploration Fast modeling Slow modeling Quick lay-outing Dedicated lay-outing phase Continuous lay-outing Serialization Chunked modeling Structuredness Movement Speed Based on dataset of 40 unique modeling executions
  13. 13. 13/33 www.janclaes.info Study 2 – Exploration Conjecture 1: Structured modeling results in understandable models Conjecture 2: A high number of move operations results in less understandable models Conjecture 3: Slow modeling results in less understandable models
  14. 14. 14/33 www.janclaes.info  Structuredness • MaxSimulBlock • PercNumBlockAsAWhole  Speed • TotTime • TotCreateTime  Movement • AvgMoveOnMovedElements • PercNumElementsWithMoves Study 2 – Exploration Measurement  Model quality • Perspicuity a model that is unambiguously interpretable and can be made sound with only small adaptations based on minimal assumptions on the modeler’s intentions with the model
  15. 15. 15/33 www.janclaes.info Study 2 – Exploration T-test t=-2,231 (p=0,028) T-test t=2,199 (p=0,030) Based on dataset of 103 unique modeling executions
  16. 16. 16/33 www.janclaes.info Study 2 – Exploration T-test t=-1,984 (p=0,049) T-test t=0,457 (p=0,648) Based on dataset of 103 unique modeling executions
  17. 17. 17/33 www.janclaes.info Study 2 – Exploration T-test t=-2,183 (p=0,031) T-test t=2,505 (p=0,014) Based on dataset of 103 unique modeling executions
  18. 18. 18/33 FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION www.janclaes.info CHAPTER 4 – THEORISATION Structured Process Modeling Theory (SPMT)
  19. 19. 19/33 www.janclaes.info Study 3 – Theorization Explanatory theory Theory building method  6 observations, 3 impressions (induction)  Explanation via existing theories (deduction) Evaluation method  Assessment of novelty, parsimony, consistency, plausibility, credibility, and transferability  Inconclusive empirical results, but open-world assumption
  20. 20. 20/33 www.janclaes.info Study 3 – Theorization Combined Flow-oriented Aspect-oriented Undirected “Modeling styles” Based on dataset of 118 unique modeling executions
  21. 21. 21/33 www.janclaes.info Study 3 – Theorization  Observation 1. Almost all modelers paused frequently during the modeling process  Observation 2. A large group can be categorized as “flow-oriented process modeling”  Observation 3. A smaller group can be categorized as “aspect-oriented process modeling”  Observation 4. Another large group used a combination of both former styles  Observation 5. Another small group can be categorized as “undirected process modeling”  Observation 6. The “undirected” sessions lasted longer than the other approaches Based on dataset of 118 unique modeling executions
  22. 22. 22/33 www.janclaes.info Study 3 – Theorization  Impression 1. Modelers need serialization of the modeling process to deal with its complexity  Impression 2. Structured serializing of the modeling process helps avoiding ‘mistakes’  Impression 3. Structured serializing does not support every modeler to avoid ‘mistakes’ to the same extent Based on dataset of 118 unique modeling executions
  23. 23. 23/33 www.janclaes.info Study 3 – Theorization Cognitive Load Theory  Working memory capacity is limited  Working memory overload causes decrease in • Effectiveness (i.e., more mistakes) • Efficiency (i.e., more time and effort) • Learning Cognitive Fit Theory  Load is lower when there is a fit • Between representation, tool or strategy on the one hand • And task or modeler on the other hand
  24. 24. 24/33 www.janclaes.info Study 3 – Theorization A B The more A, the more B A B The more A, the less B A B The more A, the more B on the long term+ +– input material representation fit working memory capacity extraneous cognitive load germane cognitive load cognitive schema construction process model quality overall construction time cognitive overload intrinsic cognitive load ++ + +++ – task complexity + prior knowledge –– – –
  25. 25. 25/33 www.janclaes.info Study 3 – Theorization A B A determines BA B The more A, the more B+ A B The more A, the less B– A B A translates into B learning style degree of serialization adopted serialization style field-dependency need for structure – + course of intrinsic cognitive load for process modeling phases course of intrinsic cognitive load for aggregation phases course of cognitive overload course of intrinsic cognitive load for strategy building phases + + + serialization style fitstructuredness of serialization – –– – 1 2 3
  26. 26. 26/33 www.janclaes.info Study 3 – Theorization Novelty (uses existing theories in fundamental new way) Parsimony (11 constructs, 15 associations) Consistency (can explain additional observations) Plausibility (accurate and profound explanation) Credibility (building blocks are established theories) Transferability (problem solving in general) Falsifiability (inconclusive, but open-world assumed) Utility (only on longer term) Consistency based on dataset of 143 unique modeling executions
  27. 27. 27/33 FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION www.janclaes.info CHAPTER 5 – CONCLUSION Summary & Future work
  28. 28. 28/33 www.janclaes.info Studies Research Cycle 3 Exploration Research Cycle 1 Structured Process Modeling Theory (SPMT) RC6 SPMT measures RC5 Cognitive measures RC4 Perspicuity Engineering Cycle 1 PPMChart RC2 Modeling styles RC 7. Design validation RC 7. Research design EC 2. Problem investigation RC 3. Problem investigation RC 3. Research design RC 3. Design validation RC 3. Evaluation RC 4. Evaluation RC 4. Problem investigation RC 4. Research design RC 4. Design validation RC 4. Research RC 3. Research EC 1. Problem investigation EC 1. Solution design EC 1. Design validation EC 1. Implementation EC 1. Evaluation RC 1. Problem investigation RC 2. Problem investigation RC 2. Research design RC 2. Design validation RC 2. Research RC 2. Evaluation RC 1. Research design RC 1. Design validation EC 2. Evaluation RC 8. Evaluation RC 8. Research RC 8. Design validation RC 8. Research design RC 8. Problem investigation RC 7. Evaluation RC 7. Research RC 7. Problem investigation EC 2. Design validation EC 2. Solution design RC 1. Evaluation RC 1. Research RC 6. Evaluation RC 6. Research RC 6. Design validation RC 6. Research design RC 6. Problem investigation RC 5. Evaluation RC 5. Research RC 5. Design validation RC 5. Research design RC 5. Problem investigation EC 2. Implementation EC2 Structured Process Modeling Method (SPMM) RC8 Training RC7 Influenceability of method
  29. 29. 29/33 www.janclaes.info Studies Study 1. Visualization • EC1. How can the operations of the process of process modeling be presented in a cognitive effective and efficient way? • RC2. How do people construct process models in terms of modeling styles? PPMChart Research instrument (visualization) Study 3. Theorization • RC1. Why do people struggle with the complexity of process modeling? • RC2. How do people construct process models in terms of modeling patterns? SPMT Theory – type II (explanation) Study 2. Exploration • RC3. How are process and product of modeling related? • RC4. How to measure (syntax) errors with cognitive origin? • RC2. How do people construct process models in terms of modeling patterns? Process vs. product Conjectures (exploration) GAP 2 GAP 3 GAP 6 GAP 3 GAP 4 GAP 1 GAP 3
  30. 30. 30/33 www.janclaes.info Future work Study 5. Tool support • EC3. How to support measurement of cognitive profile? • EC4. How to support measurement of modeling effectiveness and efficiency? • EC5. How to support the SPMM SPMTool Tool support (implementation) GAP 1 GAP 1 GAP 5 Study 4. Method • EC2. How to create process models in an effective and efficient way? • RC7. Is it possible to change a modeler’s approach towards process modeling? • RC8. How to transform the SPMT into a prescriptive theory? SPMM Practical method (prescription) GAP 5 GAP 6 GAP 5
  31. 31. 31/33 www.janclaes.info Key publications Publications in international journals  Indexed by Web Of Science • J. Claes, I. Vanderfeesten, J. Pinggera, H.A. Reijers, B. Weber, G. Poels, A visual analysis of the process of process modeling, Information Systems and e-Business Management, Vol 13(1), p. 147-190, 2015.  Under review • J. Claes, I. Vanderfeesten, F. Gailly, P. Grefen, G. Poels, The Structured Process Modeling Theory (SPMT) A cognitive view on why and how modelers benefit from structuring the process of process modeling, resubmitted after revision to Information Systems Frontiers.
  32. 32. 32/33 www.janclaes.info Key publications Publications in international conference proceedings  Indexed by Web Of Science • J. Claes, I. Vanderfeesten, H.A. Reijers, J. Pinggera, M. Weidlich, S. Zugal, D. Fahland, B. Weber, J. Mendling, G. Poels, Tying Process Model Quality to the Modeling Process: The Impact of Structuring, Movement, and Speed, Proc. BPM '12, LNCS 7481, Springer, 2012, p. 33-48. • J. Claes, I. Vanderfeesten, J. Pinggera, H.A. Reijers, B. Weber, G. Poels, Visualizing the Process of Process Modeling with PPMCharts, Proc. BPM '12 Workshops, LNBIP 132, Springer, 2012, p. 744-755. • J. Claes, F. Gailly, G. Poels, Cognitive Aspects of Structured Process Modeling, Proc. CAiSE '13 Workshops, LNBIP 148, Springer, p. 168-173, 2013.
  33. 33. 33/33 FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION www.janclaes.info Thanks for your attention! Do you have any questions? Jan Claes jan.claes@ugent.be http://www.janclaes.info Twitter: @janclaesbelgium

×