Awareness Checklist: Reviewing the Quality of Awareness Support in Collaborative Applications


Published on

Presentation from CRIWG 2010, session on awareness.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Studies have found awareness to be a very important component of a collaborative system Questionnaires and interviews, logging, etc may be used to assess awareness support, however, they require users. Even though users are probably the best evaluators of a system..
  • Quality assurance: extent to which quality is controlled in an organization. Early on, QA was based on inspections, but it evolved through methods such as statistical quality control, six-sigma and total quality management. QA extends not only to traditional industry but also to software, through e.g. cleanroom software engineering and SQFD. Formal technical reviews involve several people in a formal meeting during which a software artifact is presented, discussed, approved. Collaborative systems=technology+humans, concerned with group characteristics, communication, etc. In this setting WA must deal with multiple stakeholders, domains of concern and technology components. Difficult to assess due to complexity, time, cost. 3 existing groupware evaluation methods adapt FTR to collaborative systems: GHE (based on 8 heuristics that codify best practices in collab systems development), GWA (stepping through task sequences to explore possible failures) and KMA (checklist to assess knowledge circulation). However, in these methods, concern with awareness is diluted among many other concerns. Beyond these methods, the other technique found in the literature concerned with awareness, requires significant time and effort to accomplish.
  • Map of awareness elements and their relationships Awareness elements, main aspects and types of awareness they support
  • Time vs place is a classic categorization? in collaborative systems. This highlights spatial issues and the extent users have to go to to access the group. Studies of media richness and media naturalness show that communication mediated by technology loses several important features such as nonverbal cues, rapid feedback and arousal. In this line of reasoning, the notion of place is fundamental to adapt the medium to the group and task. Spaces provide additional context to places such as physical location, topology and mobility. We may identify five types of space
  • Space introduces geographical relationships: physical location, topology, mobility
  • Mainly concerns mobility
  • For example: virtual meeting rooms e.g. discussion forums, navigation is logical, not spatial
  • Final type of space
  • Workspace awareness is specialization of broader concept: situation awareness
  • Design elements mentioned in previous section that influence or contribute to awareness support, organized into 14 categories
  • Relationships between design categories and awareness elements Negatively influences workspace awareness because communication channels tend to be a limiting factor
  • Five experts, defined relationships between 77 design elements and awareness categories. They were supplied with strong relationships (=4), and had to add moderate (=2) and weak (=1) relationships. Accumulated correlations with value less than 2 were zeroed. Then normalized, to avoid design categories with more elements to have a higher importance in the overall score, and so the sum is 100%.
  • Correlations for each design element: strong positive, positive, uncorrelated, negative, strong negative Positive means implementation contributes to realize the design element, negative means implementation is detrimental to design requirement Each score (2,1,-1,-2) is multiplied by correlation in correlations matrix for the awareness category Then adding results, normalizing to a 0-100 scale, separately for positive and negative factors.
  • In each category, best outcome is 100 positive and 0 negative scores
  • Raise developers attention to understand if that type of awareness is required and how to better support it. Physical awareness: mobility and location play an important role but are not adequately supported
  • Developer is not forced to require all the design elements
  • Awareness Checklist: Reviewing the Quality of Awareness Support in Collaborative Applications

    1. 1. Awareness Checklist: Reviewing the Quality of Awareness Support in Collaborative Applications   Pedro Antunes, Claudio Sapateiro, Jos é A. Pino, Valeria Herskovic and Sergio F. Ochoa Department of Informatics of the Faculty of Sciences, University of Lisbon Computer Science Department, Universidad de Chile
    2. 2. Contents <ul><li>Introduction </li></ul><ul><li>Awareness Elements </li></ul><ul><li>Awareness Checklist Proposal </li></ul><ul><li>Case Study </li></ul><ul><li>Conclusions </li></ul>
    3. 3. Awareness <ul><li>Distinctive feature of collaborative systems </li></ul><ul><li>How to assess awareness support? </li></ul><ul><ul><li>Questionnaires, interviews, logging </li></ul></ul><ul><ul><li>Without user participation ? </li></ul></ul><ul><li>Awareness checklist </li></ul><ul><ul><li>Useful to system developers </li></ul></ul><ul><ul><li>Complement or substitute users’ evaluation </li></ul></ul>
    4. 4. Related Work
    5. 5. Methodology <ul><li>Definition of </li></ul><ul><ul><li>Awareness types </li></ul></ul><ul><ul><li>Design elements contributing to awareness </li></ul></ul><ul><ul><li>Correlations between design elements and awareness types with help from experts </li></ul></ul><ul><li>Construction of awareness checklist </li></ul><ul><li>Validation of checklist with case studies </li></ul>
    6. 6. Awareness Elements
    7. 7. Time x Place <ul><li>Same place, different place, any place </li></ul><ul><li>Co-located, virtual co-located, remote places </li></ul><ul><li>Drawbacks of technology-mediated communication </li></ul><ul><li>Collaboration awareness: perception of temporal and spatial structures in group of peers </li></ul>[Rodden&Blair, 1991] [Johansen et al. 1991]
    8. 8. Geographical space <ul><li>Location </li></ul><ul><ul><li>Cartesian </li></ul></ul><ul><ul><li>Topological </li></ul></ul><ul><li>Distance </li></ul><ul><li>Orientation </li></ul>[Dix et al. 2000]
    9. 9. Physical space <ul><li>Mobility </li></ul><ul><ul><li>Wandering, </li></ul></ul><ul><ul><li>visiting, </li></ul></ul><ul><ul><li>traveling </li></ul></ul><ul><ul><li>Fixed, </li></ul></ul><ul><ul><li>mobile, </li></ul></ul><ul><ul><li>autonomous, </li></ul></ul><ul><ul><li>free, </li></ul></ul><ul><ul><li>embedded, </li></ul></ul><ul><ul><li>pervasive </li></ul></ul>[Kristoffersen&Ljunberg 1998] [Dix et al. 2000]
    10. 10. Virtual space <ul><li>Collection of computer-supported interaction spaces </li></ul><ul><ul><li>Conceptual topology </li></ul></ul><ul><ul><li>Interactive </li></ul></ul><ul><ul><li>Shared </li></ul></ul><ul><ul><li>Malleable </li></ul></ul><ul><ul><li>Populated </li></ul></ul><ul><li>Navigation </li></ul><ul><ul><li>Spatial </li></ul></ul><ul><ul><li>Logical </li></ul></ul>[Rodden, 1996]
    11. 11. Social space <ul><li>Broader issues related to social practice and context </li></ul><ul><li>Geographical, physical and virtual affordances + social interaction, cultural meaning, experience, knowledge </li></ul>[Brewer&Dourish, 2008]
    12. 12. Workspace <ul><li>Container of places with ongoing activities </li></ul><ul><li>Peripheral awareness: understand activities of others in nearby places </li></ul><ul><li>Workspace awareness: another person’s interactions </li></ul><ul><ul><li>Who </li></ul></ul><ul><ul><li>What </li></ul></ul><ul><ul><li>Where </li></ul></ul><ul><ul><li>When </li></ul></ul><ul><ul><li>How </li></ul></ul>[Snowdon et al., 2000] [Liechti, 1998] [Gutwin&Greenberg, 1999]
    13. 13. Situation awareness <ul><li>Perception of elements in current situation </li></ul><ul><li>Comprehension of current situation </li></ul><ul><li>Projection of future status </li></ul>[Endsley et al., 1988]
    14. 14. Design elements Individual cognition, distributed cognition, team cognition Externalization 14 Events, actions, resources, critical elements, meaning, future scenarios Internalization 13 Parallel activities, coordinated activities, mutually adjusted activities, loosely coupled, tightly coupled Interdependence 12 Feedback, feedthrough, backchannel feedback Interaction 11 Who, what, where, when, how, task history Task 10 Eye-gaze orientation, body orientation, voice filtering, portholes/peepholes Attention 9 Participants, roles, activities, privileges, group history Membership 8 Private, group, public, data access privileges, concurrency control, floor control, version control, virtual constraints, virtual places, virtual topology, virtual attributes Virtuality 7 Viewports, links, radar views, teleports Navigation 6 Physical constraints, physical places, physical topology, physical attributes Physicality 5 Wandering, visiting, traveling, fixed, mobile, autonomous, independent, embedded, pervasive Mobility 4 Cartesian locations, topological locations, distances, orientation, focus/nimbus Spatiality 3 Synchronous, asynchronous, network connectivity, message delivery, network management Communication 2 Same place, different place, any place, co-located, virtually co-located, remote Accessibility 1 Design elements Design category
    15. 15. Main relationships <ul><li>Example: </li></ul><ul><ul><li>Design element: Different place </li></ul></ul><ul><ul><li>Design category: Accesibility </li></ul></ul><ul><ul><li>Main relationship: Time x place </li></ul></ul><ul><ul><li>Also negatively influences workspace awareness </li></ul></ul>Internalization, externalization Situation 7 Task, interaction, interdependence Workspace 6 Membership, attention Social space 5 Navigation, virtuality Virtual space 4 Mobility, physicality Physical space 3 Spatiality Geographical space 2 Accessibility, communication Time x place 1 Design categories Type of awareness
    16. 16. Correlation Matrix
    17. 17. Example of use: COIN
    18. 18. Awareness Checklist + implementation contributes to realize design element - implementation is detrimental to design requirement
    19. 19. Awareness Report
    20. 20. Awareness Report <ul><li>Low positive scores: </li></ul><ul><ul><li>Physical awareness (3) </li></ul></ul><ul><ul><li>Virtual space awareness (4) </li></ul></ul><ul><li>High negative scores: </li></ul><ul><ul><li>Situation awareness (7) </li></ul></ul>
    21. 21. Conclusions <ul><li>Awareness checklist based on QA principles, 77 design elements, 7 awareness types </li></ul><ul><li>Correlations matrix contains knowledge extracted from experts </li></ul><ul><li>Developers must balance benefit of awareness item with information overload and implementation cost </li></ul><ul><li>Allows formal review of awareness support, may help examine and improve awareness support </li></ul>
    22. 22. Thank you for your attention