Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

An Empirical Investigation of the Intuitiveness of Process Landscape Designs


Published on

Process landscapes define the scope and relationships between an organization’s business processes and are therefore essential for their management. However, in contrast to business process diagrams, where nowadays BPMN prevails, process landscape diagrams lack standardization, which results in numerous process landscape designs. Accordingly, our goal was to investigate how intuitive are current landscape designs to users with low expertise, as well as users having expertise in BPMN and landscape modeling. A total of 302 subjects participated in the research showing that previous expertise impacts the interpretation of land-scape elements and designs whereas, in the case of having contextual infor-mation, subjects responded more consistently. The results also show that the basic relationships between processes are intuitive to users, also in the case when only proximity between shapes is facilitated. Our findings may imply future de-signs of languages for process landscapes. They also may be useful for those who actually model process landscape diagrams and search for suitable notations.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

An Empirical Investigation of the Intuitiveness of Process Landscape Designs

  1. 1. An Empirical Investigation of the Intuitiveness of Process Landscape Designs Gregor Polančič, Pavlo Brin, Lucineia Heloisa Thom, Encarna Sosa, Mateja Kocbek Bule Business Process Modeling, Development, and Support the 21th edition of the BPMDS series, in conjunction with CAISE'20, 8-9 June, 2020
  2. 2. Background • Large organizations may manage thousands of business process diagrams in their process repositories, as they form a knowledge base, which enables a competitive advantage (Smirnov et al., 2012) (Kunze et al., 2011). • To maintain an overview, these systems of business processes have to be somehow organized and presented.
  3. 3. What is a process landscape? • A diagram aimed for representing organizational processes on a bird’s-eye view. Innovation process Product planning process Product development process Marketing process Order management process Prototype Product innovation Product innovation Product specification Customer order Stakeholders Stakeholders Market challenge Customer expectations Market requirements Innovative products Shipping
  4. 4. Positioning process landscapes in a process architecture Process landscape Strategic process models Operational process models Process architecture New order received Store Prepare invoice Maintenance 1.00 am Perform online store maintenance Customer Check products availability 1 minute Restart products availability service Send products status Product status Order confirmation received Order confirmation New product received Distribution center Product delivery notification Stock update Send invoice Invoice Software update New update received Perform online store update Zoom-in Concretization Zoom-out Abstraction
  5. 5. Process landscapes VS. process diagrams Process landscapes • „Black-box“ process • Synthesis of processes • Depict relationships between processes • Processes clustering Process diagrams • „White-box“ process • Analysis of processes • Depict „control-flow“ and other within-process relationships • Process decomposition New order received Process order Check availability No Yes Ship order Order shipped Ordermanagement process Items available? Order delayed
  6. 6. Process landscape designs • No standardized languages for creating process landscapes exist (Malinova et al., 2014). • Consequently, organizations, as well as process modeling tool vendors (e.g., ARIS Express, Visual Paradigm, Vizi Modeler, and Signavio), define their own ‘overviews of processes’ most commonly by imitating ‘value chain’ diagrams. • What about BPMN 2.0?
  7. 7. Applying BPMN 2.0 • By default BPMN 2.0 does not support the wide landscapes and complexities that exist in the process-modeling domain (Van Nuffel and De Backer, 2012)(von Rosing et al., 2014, p. 447). • However, the evidence from academia and practice show that BPMN is used for modeling of process landscapes in an informal way (Muehlen and Ho, 2008; Polančič et al., 2017). • Lack of landscape concepts. • Invalid BPMN 2.0 diagrams and/or • Syntactically incompatiple with current landscape diagrams Abstract collaboration diagrams Conversation diagrams Enterprise-wide process diagrams
  8. 8. Variety of process landscape designs Which design is „bettter“?
  9. 9. Enabling effective diagrammatic communication User A User B Interpreted process landscape Interpreted Process landscape Level of matching (Effectiveness) Effectiveness is positively impacted if signs (i.e. elements) are clear from its appearance alone!
  10. 10. Empirical research • The main goal of our work was to investigate the intuitiveness of the representations of process landscape designs as found in academia and industry. WHY? • In case of intuitive representations, a user may infer the correct meaning from the appearance of a notational element. • In case of non-intuitive representations a user would be likely to infer a different meaning from the appearance of a notational element  this could negatively impact the comprehension of a diagram and corresponding decisions made. • In this light, we defined the following research questions which could be tested empirically: • RQ1: Are „common“ landscape designs semantically transparent to ‘novice users’? • RQ2: How does the previous knowledge impacts the comprehension of process landscape designs?
  11. 11. Dependent variable: semantic transparency • “The key to designing visual notations that are understandable to naïve users is a property called semantic transparency” (Caire et al.,2013), which means that the meaning (semantics) of a sign is clear (i.e. intuitive, transparent) from its appearance alone. • In our research, semantic transparency was measured with the percentage of correctly identified meanings of the investigated elements. Caire, P., Genon, N., Heymans, P., & Moody, D. L. (2013). Visual notation design 2.0: Towards user comprehensible requirements engineering notations. In 2013 21st IEEE International Requirements Engineering Conference, RE 2013 - Proceedings (pp. 115-124). [6636711]
  12. 12. Research approach • Pre-experimental design, more specifically ‘one group posttest only design’ (Christensen et al., 2011). • Subjects were introduced with individual or groups (i.e. contaxtual information) of landscape elements. • Subjects‘ task was to associate the stated representations with the provided definitions (i.e. denotation mapping, decoding). • Alternative depictions for an element were provided (i.e. symbol redundancy). Landscape element Depiction Definition (The meaning of an element) (e.g. a circle, a dotted line, and a rectangle) (e.g. a process, a collection of processes, and a trigger relationship) Denotation mapping
  13. 13. Online questionnaire • Research was performed in January 2019. • 588 subjects were invited to participate. • 347 subjects partially completed the questionnaire. • 302 subjects successfully completed the questionnaire. • 65% of the subjects came from Slovenia, • 35% of the subjects came from Ukraine.
  14. 14. Results Expertise Male Female Slovenia Ukraine Total T. [%] Inexperienced 80 68 102 46 148 49% BPMN Expertise 34 18 44 8 52 17% Landscape modeling expertise 7 9 10 6 16 5% All 161 141 197 105 302 100%
  15. 15. Comprehensionof landscapeelements Most consistently recognized as a „Process“ element BPMN experts recognize this element as a „Process Collection“ Data-related elements were consistently recognized. Processes collections were NOT recognized sucesfully. Recognized either as a Process or a Patipant (depending of expertise) „Support“ and „Management“ processes were consistently recognized.
  16. 16. The impact of contextual information Individual elements Diagram 12% 29% 12% 58% 72% 67% Level of expertise: inexperienced users
  17. 17. Comprehension of between-processes relationships (1/2) Consistently recognized as a conditional flow Less consistent responses 50%+ of BPMN and Landscape experts associate this with an information flow. 29% Of all subjects associate this with a „generic-specific“ relationship 38%+ of BPMN and Landscape experts associate this with a „generic- specific“ relationship. 38% of inexperienced users associate this with a conditional flow. of all subject associate this with a trigger relationship. of inexperienced users associate this with an information flow. 36%
  18. 18. Comprehension of between-processes relationships (2/2) Of all subjects recognized these with sequentially 72%+ Of all subjects associated these with independent processes 72%+ 63% Of all subjects associated this with parallelly performed processes 73% Of all subjects associated this with parallelly performed processes 66% Of all subjects associated this with sequentially performed processes Consistent responses in all cases
  19. 19. RQ1: Are „common“ landscape designs semantically transparent to ‘novice users’? Semantic transpatent Semantic opaque + Semantic perverse Generic-specific Support process Sequentially Management process Support process Participant Process Process Datastore Document Conditional flow Conditional flow Sequential flow Information flow Generic-speciffic Sequentially Sequentially
  20. 20. RQ2: How does the previous knowledge impacts the comprehension of process landscape designs? Process(chevron) Process(Archimate) Supportprocess Participant Managementprocess Processcollection Database Document Collection(chevron) Collection(Archimate) Average Inexperienced 29% 18% 12% 27% 12% 5% 59% 85% 1% 4% 25% Landscapes 39% 25% 29% 20% 25% 22% 88% 92% 2% 2% 34% BPMN 31% 25% 19% 19% 25% 13% 69% 88% 6% 0% 30% Inexperienced in landscapes and BPMN Expertise in landscapes modeling Expertise in BPMN modeling Comprehension of landscape elements
  21. 21. Validity threats and Future work • Misconceptions related to the specifications of concepts (naming, translations, …). • Wrong focus of the subjects when inferring the meaning of the depictions (e.g. process composition). • Risks of generalization beyond the non-random sample. • Limitations of self-reported expertise. • In 2002 Hitchman stated: “Very little is documented about why particular graphical conventions are used. Texts generally state what a particular symbol means without giving any rationale for the choice of symbols or saying why the symbol chosen is to be preferred to those already available. The reasons for choosing graphical conventions are generally shrouded in mystery” (Hitchman,2002). • On the path to standardization of landscape designs (e.g. via a common language) our research may provide answers on how to effectively represents the language‘s concepts.
  22. 22. Thank you for your attetion! Full paper is available as a Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 387).