Incos2010 irene


Published on

"study of the effect of awareness on synchronous collaborative problem solving", INCOS 2010

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • My name is Irene Angelica Chounta and I come from the human computer interaction group of the university of patras. I will present our study of awareness effects on synchronous collaborative problem solving activities
  • In this presentation we will overview the objectives of the study, describe the experimental set up and the collaborative sessions that are analyzed and focus on phenomena of attention shifting and awareness that emerged during the collaborative sessions. Furthermore we present our future work on the field
  • Ithe objective of this research is to study the attention shifting and awareness phenomena that can emerge during collaboration, and how these phenomena can affect the collaboration process and quality. we also study how attention is affected by the collaborative nature of a task, the task’s content and address the relation between attention and awareness mechanisms in collaborative systems.
    At last we study how eye movements and eye gaze patterns can be used in order to evaluate groupware applications as well as to judge the quality of collaboration
  • We present a qualitative study where six participants were grouped in three dyads. These dyads represented distinctive collaborative cases. Each dyad had to collaborate in order to perform a problem solving task.

    From each dyad a user was monitored by an eye tracker.
  • The three dyads were formed in such way so as to stand for distinctive cases of collaborative activity. Therefore we have Case A where users were of similar knowledge background, Case B where an evaluation expert and a novice had to collaborate and Case C where there was a user with experience in collaborative applications and web technologies and an inexperienced student.
  • These three dyads had to collaborate synchronously for about 20 minutes in order to perform the given task. Collaboration was mediated by Synergo, a groupware application originally used for designing state diagrams
  • Synergo provides a common workspace that can be used for drawing diagrams and a chat tool for exchanging written messages. Each dyad had to use the chat tool in order to communicate and the common workspace to jointly perform the task
    The task was a cognitive walkthrough evaluation of a website and the dyads had to create a state diagram depicting the results of the evaluation
  • Since our objective was to study attention and awareness phenomena in regard to collaboration practice we focused on actions that produce a visual effect or visual change on the shared resources, the common workspace and the chat area. Such actions are the chat messages for the chat tool and basic objects editing such as insertion, movement, paste, resizing, modification of content and deletion.

    In order to provide a metric for the response to visual changes we defined and estimated the response time that can be perceived as the time difference from when an action with visual effect of a user took place since the collaborating party focused on this visual change
  • In case A the users had similar knowldege background ,they collaborated smoothly and the result of their collaboration was of good quality

    Dyad A spent 10 minutes out of 20 for browsing the under evaluation website and the rest of the time they collaborated through the groupware application.

    their activity was similar in the workspace and the chat

    The users chose to divide the task and each one undertook a part. In the end of the activity they linked their work.

    In the beginning the activity unfolded smoothly and the collaborators were able to follow each others work. When their workload started growing, delays and awareness failures occurred.

    These delays in eye-gaze response are shown in the figure for the workspace in the upper graph and the chat tool in the lower, where the time periods of high workload are marked in red color.
  • An example from case A is demonstrated here.
    The horizontal axis is the timeline.

    User A’ faces a problem in connecting two objects and asksfor help (19:32:21). User A, is working for the completion of a goal. In the same time he checksthe chat area and composes a reply to his collaborator(19:33:34). However his goal is not fulfilled so he returns tothe task and forgets to send the reply. Meanwhile, User A’sends another message. That event reminds User A of thetext message that he has composed but never sent. The lackof awareness, not only costs in time but also interfered inboth users’ progress more than once.
  • “Fig. 6” presents the series of actions of User B and UserB’ for a time period of almost two minutes. During thisperiod User B’ creates the required state diagram and User Btakes no significant part in the task activity exceptcomposing chat messages. He is confused about theobjective of the task and asks for help. User B edits theworkspace after receiving a chat message with instructions.He inserts an object (19:01:19), revises his action and deletesit. He is confused about the activity and also deletes theobjects of his partner for no obvious reason. This pattern isrepeated. When he seesnew objects emerging on the workspace he perceives them as “junk” and deletes them without realizing that they arecreated by his collaborator. An awareness mechanism whichwould notify users about the author of workspace objectscould have been useful in this case.
  • From the analysis of eye gaze from the collaborators it was shown that users who are focused on a task do not shift attention to the work of their partner. In many cases they tend to ignore their partner’s actions, even when they are aware of them, if are not directly related to their personal goal.
    Users who choose to divide a task and work seperately are difficult to recoordinate when needed, for example when one of them needs help
    Those with low activity are more responsive to events and changes as expected.
  • Collaboration activities are difficult to model andanalyze, The outcome ofa collaborative activity depends on the quality ofcollaboration itself, as well as on the tools thatmediate the activity.

    Our intention has been to use the information providedby eye-tracking devices for the evaluation of awareness mechanisms as well as assesment of collaboration’s quality

    Statistics produced by collaborative application to describe users activity can lead us to misleading assumptions.
    For example if statistics show that a user has a small number of actions this can be perceived either as if the user is not motivated towards collaboration or as the user is facing problem regarding the task or even the use of the application
    It holds the same for a user who does not reply promptly to messages. A teacher or evaluator can either suggest that the user is unwilling to collaborate or that he was unaware of incoming questions.
    We believe that the use of eye gaze can be used and provide information for such ambiguous cases.
  • Eye gaze can be used for the design process of awareness mechanisms. Attention and awareness s
  • Incos2010 irene

    1. 1. Study of the effect of awareness on synchronous collaborative problem-solving Chounta Irene-Angelica, Avouris Nicolaos HCI Group Dept. of Electrical and Computer Engineering University of Patras
    2. 2. Presentation’s Structure • Objective of the study • Description of the study setup and the cases examined • Awareness and attention shifting phenomena in sessions – Examples from collaborative sessions • Discussion on findings • Further Research
    3. 3. Objectives • Study of awareness and attention shifting phenomena during collaboration • Study of the interplay between task, attention and dynamics of collaboration • Use of eye-movements for groupware evaluation and assessment of the quality of collaboration
    4. 4. Study Setup • Qualitative Study • Six participants forming 3 dyads to represent distinctive cases • One student of each dyad monitored by eye-tracker • Participants in dyads worked together in order to perform a collaborative problem-solving task
    5. 5. Formation of Dyads 3 dyads that represent distinctive cases • Case A: Users of similar knowledge background • Case B: An inexperienced user collaborating with an expert in the field • Case C: Collaboration of disparate skills users Dyads User monitored by eye- tracker Collaborative Partner Case A User A User A’ Case B User B, inexperienced User B’, expert Case C User C, expert User C’, inexperienced
    6. 6. Collaboration Scheme • Synchronous Collaboration via groupware application Synergo • Duration of collaborative sessions: 20’ minutes • Collaborative Task: Website Evaluation using the Cognitive Walkthrough method
    7. 7. Problem-Solving Task in Synergo • Joint development of state diagram depicting the task execution Common workspace Drawing tools Chat tool Requested State Diagram
    8. 8. Actions under study We study actions that provoke visual changes on the shared resources: •Resize Object •Modify Text •Delete Object •Chat message •Insert object •Move object •Paste Object Estimate the response time to visual changes: response time=|(time an action took place- time the collaborating partner realized the action)|
    9. 9. Analysis of Activity-Case A •Users of similar knowledge background •Smooth collaboration and results of good quality •Equally active on the workspace and chat •Time was spent equally on browsing the website and building the state diagram •Users divided the task and worked separately most of the time. User A avg_res_workspace 00:01:00 avg_res_chat 00:01:01 avg_res_total 00:01:00 Actor User A User A' Actions 28 (54%) 23 (46%) Messages 19 (41%) 27 (59%) •Attention shiftings occurred when workload was low •Delays and Awareness failures occurred as activity unfolded
    10. 10. Analysis of Activity-Case A - Example User A’ asks for help User A’ asks for help again User A’ asks for help for the third time User A composes a reply but forgets to send it User A remembers to send message
    11. 11. •Users of not similar experience and knowledge •Poor session in terms of collaboration quality •Both used the chat tool equally while the experienced user (User B’) took over the workspace activity •Intense chat activity Analysis of Activity-Case B•The inexperienced user (User B) monitored all activity •Finally User B lost interest and focused on the chat Actor User B User B' Actions 17 (26%) 46 (74%) Messages 35 (53%) 30 (47%) User B avg_res_workspace 00:00:53 avg_res_chat 00:00:10 avg_res_total 00:00:36
    12. 12. Analysis of Activity-Case B - Example User B’ inserts object User B’ inserts object again User B inserts object User B deletes his object User B deletes partner’s object User B deletes partner’s object for the second time
    13. 13. Analysis of Activity-Case C •Users of disparate skills •Poor session in terms of collaboration quality but provided satisfactory solution •The experienced user (User C) took over chat and workspace activity •Help was not provided to inexperienced user (User C’) when needed •User C’ had no feedback from User C User C avg_res_workspace 00:00:20 avg_res_chat 00:00:39 avg_res_total 00:00:27 Actor User C User C' Actions 31 (71%) 13 (29%) Messages 22 (62%) 14 (38%) •Though User C was able to see in short time the messages of his partner, he did not reply •Gradually communication was diminished and collaboration failed
    14. 14. Discussion(1) We study three distinctive cases of collaboration activity: • Dyad of same background and motivation • Dyad of different expertise but motivated towards collaboration • Dyad with disparate skills and not motivated The eye gaze patterns of users during collaboration have been recorded and analyzed
    15. 15. Discussion(2) • Users immersed into an activity (high cognitive workload) remain unaware of their collaborators’ actions • Users with high cognitive workload tend to ignore partner’s actions that are not directly related to their goal • Users with low activity tend to be more responsive to incoming events/visual changes
    16. 16. Use of eye gaze to assess quality of collaboration Statistics alone can mislead us to false conclusions concerning quality of collaboration, i.e.: The use of eye gaze patterns can help identification of such ambiguous cases User with small number of actions User who does not quickly reply to messages Unwilling to collaborate ? Unaware of incoming message Not motivated ? Facing trouble
    17. 17. Eye gaze for awareness mechanisms design Eye gaze patterns and eye behavior can be proven useful for awareness mechanisms design and evaluation •Awareness mechanisms should attract the attention but also vary according to task’s nature •Awareness support should adapt to the workload of users. •Awareness Regarding Content •Additional mechanisms that will provide help or guidance can be combined with awareness support
    18. 18. Future Research • design and implement adaptive awareness mechanisms • integrate them in collaborative applications • and conduct studies to determine how the characteristics of the awareness mechanisms affect the collaborative activity. • Use the results and methods in the implementation of a groupware evaluation tool
    19. 19. Questions are welcome Thank you