Mental Workload at
Transitions between Multiple
           Devices
   in Personal Information
         Management
       M...
Highlights from
     Mental Workload at
Transitions between Multiple
           Devices
   in Personal Information
       ...
Personal information,
  Multiple devices
Problem Domain

•   Is there a difference in mental workload
    when users perform tasks using multiple
    devices?
•   I...
Mental workload
•   [...] “that portion of operator information
    processing capacity or resources that is
    actually ...
Experiment design
               Files                                                                             Calenda...
Files task, L0 — TEPR
                                                                B
                                  ...
Calendar task — TEPR
                                                                                                 Not
...
Contacts task — TEPR
                                              0.00623                       0.01158   0.0224
        ...
Files L0: Migration Step
                                                             Files Task, Participant P8, Level L1...
Files L0: Migration Step
                                                         Files Task, Participant P18, Level L0
  ...
Files — Time
                 600
                                                                Without Sync Support
   ...
Calendar task — Time
                                          F(1,34)=4.1842
                                            ...
Contacts task — Time
                                           No Sync Support
                                          ...
Calendar task — TEPR, Time
                                                                                               ...
Workload — Time


•   TLX correlation signi cant only for Files.
•   No signi cant correlation between TEPR
    and time.
...
TEPR: Files
                                                                  Files Level L0                              ...
TEPR: Calendar
                                                                                     Calendar Level L0     ...
TEPR: Contacts
                                                        Contacts Level L0                                  ...
Mental Workload in PIM
•   Workload is a useful metric to understand
    PIM at a sub-task level.
•   Provides alternative...
Holistic Usability

•   Computing is situated, cognition is
    distributed
•   ∴ Must evaluate tools & tasks in situ.
•  ...
Future work: Hot Cognition

 •   Maslow’s hierarchy applied to interaction:
     •   1: Function, 2: Usability, 3: Pleasur...
Questions & comments

  ?
               !
Questions & comments

  ?
               !
      Thank you!
Upcoming SlideShare
Loading in …5
×

Mental Workload at Transitions between Multiple Devices in Personal Information Management

1,300 views

Published on

Knowledge workers increasingly use multiple devices such as desktop computers, laptops, cell phones, and PDAs for personal information management (PIM) tasks. This paper presents the results of a study that examined users' mental workload in this context, especially when transitioning tasks from one device to another. In a preliminary survey of 220 knowledge workers, users reported high frustration with current devices' support for task migration, e.g. accessing files from multiple machines. To investigate further, we conducted a controlled experiment with 18 participants. While they performed PIM tasks, we measured their mental workload using subjective measures and physiological measures. Some systems provide support for transitioning users' work between devices, or for using multiple devices together; we explored the impact of such support on mental workload and task performance. Participants performed three tasks (Files, Calendar, Contacts) with two treatment conditions each (lower and higher support for migrating tasks between devices.)

Workload measures obtained using the subjective NASA TLX scale were able to discriminate between tasks, but not between the two conditions in each task. Task-Evoked Pupillary Response, a continuous measure, was sensitive to changes within each task. For the Files task, a significant increase in workload was noted in the steps before and after task migration. Participants entered events faster into paper calendars than into an electronic calendar, though there was no observable difference in workload. For the Contacts task, time-on-task was equal, but mental workload was higher when no synchronization support was available between their cell phone and their laptop. Little to no correlation was observed between task performance and both workload measures, except in isolated instances. This suggests that neither task performance metrics nor workload assessments alone offer a complete picture of device usability in multi-device personal information ecosystems. Traditional usability metrics that focus on efficiency and effectiveness are necessary, but not sufficient, to evaluate such designs. Given participants' varying subjective perceptions of these systems and differences in task-evoked pupillary response, aspects of hot cognition such as emotion, pleasure, and likability show promise as important parameters in the evaluation of PIM systems.

Published in: Technology, Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,300
On SlideShare
0
From Embeds
0
Number of Embeds
165
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Mental Workload at Transitions between Multiple Devices in Personal Information Management

    1. 1. Mental Workload at Transitions between Multiple Devices in Personal Information Management Manas Tungare & Manuel Pérez-Quiñones
    2. 2. Highlights from Mental Workload at Transitions between Multiple Devices in Personal Information Management Manas Tungare & Manuel Pérez-Quiñones
    3. 3. Personal information, Multiple devices
    4. 4. Problem Domain • Is there a difference in mental workload when users perform tasks using multiple devices? • Increased perception of task difficulty? • Does switching to alternate strategies result in lower workload?
    5. 5. Mental workload • [...] “that portion of operator information processing capacity or resources that is actually required to meet system demands.” [O’Donnell and Eggemeier, 1986] • Subjective measure: NASA Task Load Index (TLX) • Physiological measure: Task-Evoked Pupillary Response (TEPR)
    6. 6. Experiment design Files Calendar Contacts Participant Code: Date: Treatment: Session: W T 2009S January 5 to January 11, 2009 January 2009 February M T W T F S S M T F S Home Calendar 1 2 3 4 1 Week 1 5 6 7 8 9 10 11 2 3 4 5 6 7 8 12 13 14 15 16 17 18 9 10 11 12 13 14 15 January 2009 19 20 21 22 23 24 25 16 17 18 19 20 21 22 26 27 28 29 30 31 23 24 25 26 27 28 PIM Study - Home Monday 5 Tuesday 6 Wednesday 7 Thursday 8 Friday 9 Saturday 10 Sunday 11 8 AM 9 AM 10 AM Participant Code: Date: Treatment: Session: W T 2009S January 5 to January 11, 2009 January 2009 February 11 AM M T W T F S S M T F S Home Calendar 1 2 3 4 1 Week 1 5 6 7 8 9 10 11 2 3 4 5 6 7 8 NOON 12 13 14 15 16 17 18 9 10 11 12 13 14 15 Level 0 January 2009 19 20 21 22 23 24 25 16 17 18 19 20 21 22 1 PM 26 27 28 29 30 31 23 24 25 26 27 28 PIM Study - Home 2 PM Team Outing Monday 5 Tuesday 6 Wednesday 7 Thursday 8 Friday 9 Saturday 10 Sunday 11 3 PM 8 AM 4 PM Dentist's appoint! ment 9 AM 5 PM 10 AM 6 PM Michael's Little League game (tenta! 11 AM 7 PM tive; confirm with Alex) NOON 8 PM 1 PM 9 PM 2 PM Team Outing Page 1/1 3 PM 4 PM Dentist's appoint! ment 5 PM 6 PM Michael's Little League game (tenta! 7 PM tive; confirm with Alex) 8 PM 9 PM Page 1/1 No support for Multiple paper No support for le migration calendars synchronization Level 1 System supports Devices support Online calendars le migration synchronization
    7. 7. Files task, L0 — TEPR B B B 20 L0 L1 AB AB AB ! ! 10 ! A Adjusted Pupil Radius A ! A ! 0 ! ! ! ! Transition Step −0 1 1 2 3 4 5 6 7 8 9
    8. 8. Calendar task — TEPR Not signi cant L0 L1 20 Adjusted Pupil Radius 10 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 0 ! ! ! ! ! ! ! ! ! ! ! −0 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Step within Task
    9. 9. Contacts task — TEPR 0.00623 0.01158 0.0224 0.03476 0.007127 L0 L1 0.04138 25 20 Adjusted Pupil Radius 15 ! ! ! ! ! 10 ! 5 ! ! ! ! ! ! 0 − 5 1 2 3 4 5 6 Step within Task
    10. 10. Files L0: Migration Step Files Task, Participant P8, Level L1 30 Migration Step 20 Migration Step Pupil Radius (eye image pixels) 10 0 S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 −0 1 −0 2 −0 3 200 400 600 800
    11. 11. Files L0: Migration Step Files Task, Participant P18, Level L0 30 20 Pupil Radius (eye image pixels) 10 0 S0 S1 S2 S3 S4 S5 S6 S7 S8 S9S10 −0 1 Migration Step −0 2 −0 3 200 400 600 800 1000
    12. 12. Files — Time 600 Without Sync Support With Sync Support 500 F(1,34)=8.8318 p=0.005404 400 Time Taken (s) 312 (276) 300 200 109 (83) ! 100 ! ! ! ! ! ! ! ! ! 0 1 2 3 4 5 6 7 8 9 10
    13. 13. Calendar task — Time F(1,34)=4.1842 p=0.0486 Both 2 & 6 required Paper Calendar Online Calendar scheduling an event 80 ! F(1,34)=4.3345 p=0.04495 ! 60 ! ! Time Taken (s) ! 40 ! ! ! 20 ! ! ! ! ! ! ! ! 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
    14. 14. Contacts task — Time No Sync Support Sync Support 150 ! Time Taken (s) ! 100 ! 50 ! ! ! ! 1 2 3 4 5 6 7
    15. 15. Calendar task — TEPR, Time No Sync Support Sync Support 150 ! L0 L1 25 20 Time Taken (s) ! 100 Adjusted Pupil Radius 15 ! ! ! ! ! 10 ! 5 ! 50 ! ! ! ! ! ! ! ! ! ! 0 − 5 1 2 3 4 5 6 1 2 3 4 5 6 7 Step within Task Step # Same time, but higher workload in L0: More mental workload for same performance
    16. 16. Workload — Time • TLX correlation signi cant only for Files. • No signi cant correlation between TEPR and time. • Need to evaluate both separately.
    17. 17. TEPR: Files Files Level L0 Files Level L1 ! ! ! !! ! ! ! !! 20 ! ! !! ! ! ! ! ! !! 60 !! ! !! ! ! !! ! ! ! ! !! ! ! !! !! ! ! ! !! 10 !! !! ! ! ! ! !! ! ! ! ! ! !! ! ! ! 40 !! ! ! ! ! ! ! ! ! Adjusted Pupil Radius Adjusted Pupil Radius !! ! ! !! ! ! ! ! ! ! ! ! ! ! !! ! 0 ! ! !! ! ! ! !! ! ! ! !! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! 20 !! ! ! ! ! !! ! ! ! !! ! −0 ! ! ! !! ! ! ! 1 ! ! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! !! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! !! ! ! !! ! !! ! ! ! ! ! 0 ! ! ! ! ! −0 ! ! !! ! ! ! ! 2 ! ! ! ! ! ! !!! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! !! !! ! ! ! ! ! ! ! !! ! ! ! !! !! ! ! ! !! ! −0 ! ! −0 3 ! ! 2 ! ! ! 0 200 400 600 800 1000 1200 0 50 100 150 200 250 300 Time on Task (seconds) Time on Task (seconds) r=-0.062, p=0.46 r=0.15, p=0.06
    18. 18. TEPR: Calendar Calendar Level L0 Calendar Level L1 60 ! ! ! ! ! ! ! ! 40 ! ! ! ! ! !! ! ! ! ! ! ! 40 ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! Adjusted Pupil Radius Adjusted Pupil Radius ! ! ! !! ! ! ! ! ! 20 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 20 ! ! !! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! !! ! ! ! ! !!! ! ! ! ! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !!! ! ! ! !! !!! !!! ! ! ! ! ! ! ! ! ! ! !! ! !! ! ! !!! ! ! ! ! ! 0 ! ! ! ! ! !! ! ! ! ! ! !! !! !! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! !! ! ! ! ! ! !!!!! ! !! ! ! ! ! ! ! ! ! ! ! ! ! !!!! ! ! !!! ! !!! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! !! ! !! !! !! ! !!!! !! !! ! ! ! !! ! !! ! ! ! ! !! ! ! ! ! !!!!! ! ! !!!!! ! ! ! ! 0 ! ! ! !! !! ! !! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! !!! !! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! !! !! ! ! ! ! ! ! ! ! ! ! !! ! ! !!! ! ! ! !!! !! ! ! ! !! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! −0 ! ! ! ! ! ! 2 ! !! ! ! !! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! !! ! ! −0 ! ! 2 0 20 40 60 80 100 120 0 20 40 60 80 100 120 Time on Task (seconds) Time on Task (seconds) r=–-0.11, p=0.08 r=–0.067, p=0.28
    19. 19. TEPR: Contacts Contacts Level L0 Contacts Level L1 ! ! ! ! ! ! ! ! 60 ! ! ! ! ! ! ! ! ! ! ! ! 10 ! ! ! ! ! !! ! ! ! ! ! ! !! ! ! !! ! ! ! !! ! 40 ! ! ! ! Adjusted Pupil Radius Adjusted Pupil Radius ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! !! ! ! ! ! ! 0 !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! !! 20 ! ! !! ! ! ! ! ! !! ! ! !! ! ! ! ! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! !! ! ! ! ! ! ! ! ! ! ! ! ! −0 ! ! ! ! ! ! !! ! 1 ! ! ! ! ! ! ! ! ! ! !! ! !! ! ! !! ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! 0 !! ! ! ! ! ! ! ! ! ! ! !! ! ! ! −0 2 50 100 150 200 50 100 150 200 250 300 350 Time on Task (seconds) Time on Task (seconds) r=–0.013, p=0.18 r=0.415, p=0.68
    20. 20. Mental Workload in PIM • Workload is a useful metric to understand PIM at a sub-task level. • Provides alternative to performance measurement (e.g. same performance but high workload.) • Continuous measure of workload provides important information about sub-tasks that subjective measures could not capture
    21. 21. Holistic Usability • Computing is situated, cognition is distributed • ∴ Must evaluate tools & tasks in situ. • Device transitions are not designed well. • Traditional usability: effectiveness, efficiency and satisfaction (ISO de nition)
    22. 22. Future work: Hot Cognition • Maslow’s hierarchy applied to interaction: • 1: Function, 2: Usability, 3: Pleasure. • Usability is necessary, but not sufficient. • Emotion, pleasure, frustration, likability • Quality of life, Perceived ease of use/ usability metrics, happiness.
    23. 23. Questions & comments ? !
    24. 24. Questions & comments ? ! Thank you!

    ×