BPM Cluster Meeting 2014

274 views

Published on

Slides of my presentation at BPM Cluster meeting, 21 August 2014, Eindhoven, NL

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
274
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

BPM Cluster Meeting 2014

  1. 1. FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION Why do people struggle with the complexity 1/31 of constructing a process model? Jan Claes Teaching assistant : PhD 2009 – 2015 Supervisors UGent : Geert Poels & Frederik Gailly Supervisors TU/e : Paul Grefen & Irene Vanderfeesten www.janclaes.info
  2. 2. 2/31 Business Process Modeling (BPM) www.janclaes.info Assumptions  Constructing process models is useful  Constructing the models is cognitively challenging Business Process Modeling (BPM)  Process model quality “what is a good process model?”  Process of process modeling (PPM) “how to make a good process model (effectiveness) using a good approach (efficiency)?”
  3. 3. FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION 3/31 How do people construct process models? www.janclaes.info
  4. 4. 4/31 PPMChart visualization www.janclaes.info  CREATE_ACTIVITY  CREATE_START_EVENT  CREATE_END_EVENT  CREATE_AND  CREATE_XOR  CREATE_EDGE  MOVE_ACTIVITY  MOVE_START_EVENT  MOVE_END_EVENT  MOVE_AND  MOVE_XOR  DELETE_ACTIVITY  DELETE_START_EVENT  DELETE-END_EVENT  DELETE_AND  DELETE_XOR  DELETE_EDGE  NAME_ACTIVITY  RENAME_ACTIVITY  NAME_EDGE  RENAME_EDGE  Start event Edge  Activity Edge Gateway Edge Edge  Activity  Activity Edge Edge Gateway 1 2 3 4 5 6 7 8 9 10 11 12 time model elements
  5. 5. 5/31 Flow-oriented modeling www.janclaes.info
  6. 6. 6/31 Aspect-oriented modeling www.janclaes.info
  7. 7. 7/31 Undirected modeling www.janclaes.info
  8. 8. Obs. 1. Majority used serialized approach Obs. 2. Large group flow-oriented modeling Obs. 3. Small group aspect-oriented modeling Obs. 4. Large group combine FO and AO Obs. 5. Small group undirected modeling Obs. 6. Remainder is uncategorized Obs. 7. Undirected sessions lasted longer 8/31 Observations (objective) www.janclaes.info
  9. 9. FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION 9/31 What about the quality of the corresponding models? www.janclaes.info
  10. 10. 10/31 Impressions (subjective) Impr. 1. Modelers need serialization Impr. 2. Structured serialization helps avoiding ‘mistakes’ Impr. 3. Structured serialization does not support every modeler to the same extent www.janclaes.info
  11. 11. FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION 11/31 Is there a rational for these observations and impressions? www.janclaes.info
  12. 12. 12/31 Kinds of human memory www.janclaes.info input Sensory memory Working memory Long term memory performance OBSERVATION SELECTION ORGANIZATION TRANSFER RETRIEVAL
  13. 13. 13/31 Types of cognitive load www.janclaes.info free capacity germane load extraneous load Time Instantaneous load overall load intrinsic load assumed capacity instantaneous load overload
  14. 14. 14/31 Cognitive Load Theory (CLT) case complexity prior knowledge input material representation fit – – + – intrinsic cognitive load + + + cognitive overload www.janclaes.info working memory capacity extraneous cognitive load germane cognitive load schema construction process model quality + + overall construction effort + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term –
  15. 15. I ♥︎ chaos I #$@&! chaos 15/31 Cognitive Fit Theory (CFT) Need for structure Field dependency Learning style www.janclaes.info Sensory Visual Inductive Active Sequential Intuitive Auditory Deductive Reflective Global DON’T SEE THE FOREST THROUGH THE TREES
  16. 16. FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION 16/31 What can we do with this knowledge? www.janclaes.info
  17. 17. 17/31 Theory building CoOIgmbnspietrirevvesas tiohionenossry Structured Process Modeling Theory www.janclaes.info 32
  18. 18. learning style adopted serialization serialization style fit 1 2 3 structuredness of serialization – – – – A + B The more A, the more B A – B The more A, the less B A B A determines B A B A translates into B 18/31 Structured Process Modeling Theory (SPMT) www.janclaes.info degree of serialization style need for structure field-dependency – + intrinsic cognitive load for modeling phases intrinsic cognitive load for aggregation phases cognitive overload intrinsic cognitive load for strategy building phases + + +
  19. 19. FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION 19/31 How will the developed theory be tested? www.janclaes.info
  20. 20. with selection of participants (e.g., outliers) 20/31 Methodology 1. Cognitive tests prior to the test session  Collect modeler data 2. Observational modeling session  At TU/e in BPM lecture of September 26  Collect modeler, modeling and model data 3. Post-experiment interviews  Collect additional, qualitative data www.janclaes.info
  21. 21. No treatment Measure every involved construct of the theory 21/31 Experiment set-up  Modeler, modeling process, process model Calculate (cor)relations of the theory www.janclaes.info
  22. 22. 22/31 PRIOR TO THE EXPERIMENT SESSION www.janclaes.info
  23. 23. 23/31 Experiment set-up 1. Prior to the experiment session  Software to be installed on personal laptop ( = download from website and unzip)  5 cognitive tests (time range 5-30 min) • Reading span test (RSPAN) • Operation span test (OSPAN) • Counting span test (CSPAN) • Hidden figures test (HFT) • Need for structure (NFS) www.janclaes.info
  24. 24. 24/31 Experiment set-up 1. Prior to the experiment session  To be completed at home  Before deadline  Software collects data in one-zip file per test which participants have to submit www.janclaes.info
  25. 25.  Students will receive their individual results as incentive to not cheat on these tests  Late responses or too much errors reveals cheating  Student is not allowed to start actual experiment if the test results are not submitted in time 25/31 Experiment set-up 1. Prior to the experiment session www.janclaes.info
  26. 26. 26/31 AT THE EXPERIMENT SESSION www.janclaes.info
  27. 27. 27/31 Experiment set-up 2. At the experiment session  Modeling task and post-task survey  In the same, already installed program (using a different start-up code)  Software automatically collects data • Each operation in the tool ( = each activity in the modeling process) • Constructed process model, number of operations, modeling time in one-zip file which they have to submit www.janclaes.info
  28. 28.  Student does not get credit if not all data (6 zip files) are submitted  Technical problems > replacement assignment (e.g., help me with coding the collected data) 28/31 Experiment set-up 2. At the experiment session www.janclaes.info
  29. 29. 29/31 AFTER THE EXPERIMENT SESSION www.janclaes.info
  30. 30.  Data will be analyzed and outliners will be defined  Selection of students will be invited for post-experiment • Explain how they have modeled • Explain their cognitive profile • Explain which results we expected and why • Ask about their opinion: why did they (not) achieve the predicted results? 30/31 Experiment set-up 3. After the experiment session interview (voluntarily)  Collect qualitative data www.janclaes.info
  31. 31. > Do you have feedback on my research plans? < 31/31 Thanks for you attention! > set-up is useful for theory testing? < > set-up is feasible (enough)? < > other suggestions? < Does someone has some time to test the software? www.janclaes.info Jan Claes jan.claes@ugent.be http://www.janclaes.info Twitter: @janclaesbelgium
  32. 32. FACULTY OF ECONOMICS AND BUSINESS ADMINISTRATION How can the concepts of the theory be measured? 32/31 www.janclaes.info
  33. 33. 33/31 Cognitive Load Theory (CLT) case complexity prior knowledge input material representation fit – – + – intrinsic cognitive load + + + cognitive overload www.janclaes.info working memory capacity extraneous cognitive load germane cognitive load schema construction process model quality + + overall construction effort + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term –
  34. 34. For many years, my family and friends have been working on the farm. SPOT Because the stuffy was room, Bob went air for some fresh outside. TRAIL 34/31 Cognitive Load Theory (CLT) Reading span case complexity prior knowledge – – + – We were fifty miles out at sea before we lost sight of the land. BAND intrinsic cognitive load + + + Operation span cognitive IS (8 / 4) overload – 1 = 1 ? BEAR IS (6 X 2) – 2 = 10 ? BEANS IS (10 X 2) – 6 = 12 ? DAD Counting span Count the dark blue circles www.janclaes.info + + overall construction effort input material representation fit extraneous cognitive load germane cognitive load schema construction process model quality + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term – working memory capacity Reading span - Operation span - Counting span
  35. 35. 35/31 Cognitive Load Theory (CLT) case complexity prior knowledge – – + – intrinsic cognitive load + + + cognitive overload Example questions I did experience problems using the modeling tool (5-point Likert) Using the modeling language was not difficult (5-point Likert) I found it easy to understand the case description (5-point Likert) www.janclaes.info + + overall construction effort working memory capacity extraneous cognitive load germane cognitive load schema construction process model quality + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term – input material representation fit Fit with modeling task = constant Fit with modeler through questions
  36. 36. 36/31 Cognitive Load Theory (CLT) – – + – intrinsic cognitive load + + + cognitive overload www.janclaes.info prior knowledge input material representation fit working memory capacity extraneous cognitive load germane cognitive load schema construction process model quality + + overall construction effort + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term – case complexity Case complexity = constant (number of activities and arrows in solution model can be used as proxy for number of concepts and relations)
  37. 37. 37/31 Cognitive Load Theory (CLT) case complexity – – + – intrinsic cognitive load + + + cognitive overload Example questions I am familiar with the mortgage handling process (5 point-Likert) How many process models did you ever create? I am experienced in process modeling (5-point-Likert) www.janclaes.info + + overall construction effort input material representation fit working memory capacity extraneous cognitive load germane cognitive load schema construction process model quality + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term – prior knowledge Prior domain, modeling language and modeling method knowledge through questions
  38. 38. 38/31 Cognitive Load Theory (CLT) case complexity prior knowledge – – – intrinsic cognitive load + + + Example questions I found it hard to complete the task (5-point-Likert) www.janclaes.info + + overall construction effort input material representation fit working memory capacity schema construction process model quality + + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term – extraneous cognitive load germane cognitive load cognitive overload Through questions
  39. 39. 39/31 Cognitive Load Theory (CLT) case complexity prior knowledge input material representation fit – – + – intrinsic cognitive load + + + cognitive overload www.janclaes.info working memory capacity extraneous cognitive load germane cognitive load process model quality + + overall construction effort + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term – schema construction
  40. 40. 40/31 Cognitive Load Theory (CLT) case complexity prior knowledge input material representation fit – – + – intrinsic cognitive load + + + cognitive overload www.janclaes.info working memory capacity extraneous cognitive load germane cognitive load schema construction + + overall construction effort + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term – process model quality Syntactical quality: number of cognitive mistakes Semantic quality: assessment by experts using checklist (penalty for over- and underfit)
  41. 41. 41/31 Cognitive Load Theory (CLT) case complexity prior knowledge input material representation fit – – + – intrinsic cognitive load + + + cognitive overload www.janclaes.info working memory capacity extraneous cognitive load germane cognitive load schema construction process model quality + + + – A + B The more A, the more B A – B The more A, the less B A + B The more A, the more B on the long term – overall construction effort Time between start of reading case description and last operation in the tool Number of operations
  42. 42. learning style adopted serialization serialization style fit structuredness of serialization – – – – A + B The more A, the more B A – B The more A, the less B A B A determines B A B A translates into B 42/31 Structured Process Modeling Theory (SPMT) www.janclaes.info degree of serialization style need for structure field-dependency – + intrinsic cognitive load for modeling phases intrinsic cognitive load for aggregation phases cognitive overload intrinsic cognitive load for strategy building phases + + +
  43. 43. learning style adopted serialization serialization style fit structuredness of serialization – – – – Example questions When I write a text: • I usually work on content, structure, wording and format right from the start • I usually first decide on the content, then the structure, next I write sentences A + B The more A, the more B A – B The more A, the less B A B A determines B A B A translates into B 43/31 Structured Process Modeling Theory (SPMT) and finally I work on the formatting www.janclaes.info degree of serialization style need for structure field-dependency – + intrinsic cognitive load for modeling phases intrinsic cognitive load for aggregation phases cognitive overload intrinsic cognitive load for strategy building phases + + + Through questions
  44. 44. learning style adopted serialization serialization style fit structuredness of serialization – – – – Need for Structure scale 12 questions with 6-point Likert scale A + B The more A, the more B A – B The more A, the less B A B A determines B A B A translates into B 44/31 Structured Process Modeling Theory (SPMT) www.janclaes.info degree of serialization style need for structure field-dependency – + intrinsic cognitive load for modeling phases intrinsic cognitive load for aggregation phases cognitive overload intrinsic cognitive load for strategy building phases + + + Through questions
  45. 45. learning style adopted serialization serialization style fit structuredness of serialization – – – – A + B The more A, the more B A – B The more A, the less B A B A determines B A B A translates into B 45/31 Structured Process Modeling Theory (SPMT) www.janclaes.info degree of serialization style need for structure field-dependency – + intrinsic cognitive load for modeling phases intrinsic cognitive load for aggregation phases cognitive overload intrinsic cognitive load for strategy building phases + + + Hidden Figures Test
  46. 46. learning style adopted serialization serialization style fit structuredness of serialization – – – – degree of serialization Flow-oriented – Aspect-oriented – Combination Happy Path First – Undirected Uncategorized A + B The more A, the more B A – B The more A, the less B A B A determines B A B A translates into B 46/31 Structured Process Modeling Theory (SPMT) www.janclaes.info style need for structure field-dependency – + intrinsic cognitive load for modeling phases intrinsic cognitive load for aggregation phases cognitive overload intrinsic cognitive load for strategy building phases + + + Assessment by experts
  47. 47. learning style adopted serialization structuredness of serialization serialization style fit – – – – Global learner = aspect-oriented, sequential = flow-oriented High need for structure = no undirected Field-dependent = flow-oriented A + B The more A, the more B A – B The more A, the less B A B A determines B A B A translates into B 47/31 Structured Process Modeling Theory (SPMT) www.janclaes.info degree of serialization style need for structure field-dependency – + intrinsic cognitive load for modeling phases intrinsic cognitive load for aggregation phases cognitive overload intrinsic cognitive load for strategy building phases + + + Theoretical match Post-experiment interview
  48. 48. learning style adopted serialization serialization style fit structuredness of serialization – – – – intrinsic cognitive load for modeling phases intrinsic cognitive load for aggregation phases A + B The more A, the more B A – B The more A, the less B A B A determines B A B A translates into B 48/31 Structured Process Modeling Theory (SPMT) www.janclaes.info degree of serialization style need for structure field-dependency – + + + + cognitive overload intrinsic cognitive load for strategy building phases See earlier: through questions

×