• Like
Preliminary Examination Proposal Slides
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Preliminary Examination Proposal Slides

  • 1,839 views
Published

Slides from my Proposal Defense held on May 5, 2008.

Slides from my Proposal Defense held on May 5, 2008.

Published in Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,839
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
52
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Preliminary Examination Manas Tungare Advisory Committee: Dr. Manuel Pérez-Quiñones Dr. Stephen H. Edwards Dr. Edward A. Fox Prof. Steve Harrison Dr. Tonya Smith-Jackson
  • 2. Talk outline 0 ~45 min Presentation & questions Additional comments, suggestions OK to record audio? Your questions/comments are welcome at any time. Slides contain only major citations. The document contains full citations.
  • 3. Talk outline • Introduction • Problem statement • A review of my work so far • Research questions • How my research plan will address these • Planned schedule
  • 4. Introduction
  • 5. Human Computer Interaction Personal Multi-Platform Information User Interfaces Management
  • 6. Personal Information
  • 7. Multiple devices
  • 8. Problems and workarounds • Constant need for manual synchronization • Give up using multiple computers • Copy addresses and phone numbers on sticky notes • Use USB flash drives to cart data around • Email files to themselves
  • 9. Evaluation issues in PIM Paraphrased from discussions at the CHI 2008 Workshop on Personal Information Management, April 2008. • Evaluating new PIM tools • Comparing PIM tools developed by diverse research groups • Choosing suitable reference tasks for PIM • Measures that are valid across tasks
  • 10. Problem Statement
  • 11. Understanding PIM • Understanding users and how they use multiple devices to accomplish PIM • Identify common device configurations in information ecosystems • Identify tasks performed on each device • Identify problems, frustrations
  • 12. Mental workload in Information Ecosystems • What is the mental workload incurred by users when they are trying to use multiple devices for personal information management? • For those tasks that users have indicated are frustrating for them, do the alternate strategies result in lower mental workload? • Are multi-dimensional subjective workload assessment techniques (such as NASA TLX) an accurate indicator of operator performance in information ecosystems?
  • 13. Research: Phase I Understanding users’ PIM practices across devices
  • 14. Research Questions • Devices and activities • What is the distribution of users who use multiple devices? Most common devices? Common PIM tasks? Tasks bound to a device? • The use of multiple devices together • Factors in choice of new devices • Device failures
  • 15. Research Questions • Devices and activities • The use of multiple devices together • Which devices were commonly used in groups? Methods employed to share data among these devices? Problems and frustrations? • Factors in choice of new devices • Device failures
  • 16. Research Questions • Devices and activities • The use of multiple devices together • Factors in choice of new devices • What are some of the factors that influence users’ buying decisions for new devices? Integrating a device into current set of devices? • Device failures
  • 17. Research Questions • Devices and activities • The use of multiple devices together • Factors in choice of new devices • Device failures • How often do users encounter failures in their information ecosystems? Common types of failures? Coping with failure?
  • 18. Survey: August 2007 • Knowledge workers (N=220) • Highlights from preliminary results: • 96% use at least one laptop • 71% use at least one desktop • Lots of frustrated users (as expected) • Longer discussion in[Tungare and Pérez-Quiñones 2008]
  • 19. Survey analysis • Content analysis to uncover common tasks • Quantitative analysis to determine typical set of devices for experiment • Recruit two students to code random subset of survey; ensure high inter-rater reliability • Design Phase II experiment based on these findings
  • 20. Content analysis: example “The last device I acquired was a cell phone from Verizon. I would have liked to synchronize data from my laptop or my PDA with it but there seems to be no reasonable way to do so. I found a program that claimed to be able to break in over bluetooth but it required a fair amount of guess work as to data rates etc and I was never able to actually get it to do anything. In the end I gave up. Fortunately I dont know that many people and I usually have my PDA with me so it isnt a big deal but frankly I dont know how Verizon continues to survive with the business set...”
  • 21. Content analysis: example Device 1 “The last device I acquired was a cell phone from Device 2 Device 2 Verizon. I would have liked to synchronize data from my laptop or my PDA with it but there seems to be no Task reasonable way to do so. I found a program that Problem 1 claimed to be able to break in over bluetooth but it required a fair amount of guess work as to data rates etc and I was never able to actually get it toProblem 2 do anything. In the end I gave up. Fortunately I dont know that many people and I usually have my PDA with me Conclusion so it isnt a big deal but frankly I dont know how Verizon continues to survive with the business set...”
  • 22. Research: Phase II Measurement of mental workload and task performance of users while they perform representative PIM tasks
  • 23. Mental workload • [...] “That portion of an operator’s limited capacity actually required to perform a particular task.” [O’Donnell and Eggemeier, 1986] • Low to moderate levels of workload are associated with acceptable levels of operator performance [Wilson and Eggemeier, 2006] • Often used as a measure of operator performance
  • 24. Mental workload as a measure of operator performance • Alternative: direct measurement of task performance: • Time taken to perform task, • Number of errors, etc. • Task metrics are more difficult to measure • Need instrumentation of equipment • Scores cannot be compared across tasks
  • 25. Measuring mental workload Figure 8.6 NASA Task Load Index H art and Staveland’s N ASA Task Loa d Ind ex (TLX) method assesses work loa d on five 7-point sc ales. Increments of high, me dium and low estimates for e a ch point result in 21 gra d ations on the sc ales. NASAame N Task D ate TLX Mental Demand How mentally d emanding was the task? Very Low Very High • NASA TLX: Physic al D emand How physic ally d emanding was the task? Task Load Index Very Low Very High Temporal Demand How hurrie d or rushe d was the p a c e of the task? • SWAT: Very Low Performanc e Very High How suc c essful were you in a c complishing what Subjective workload assessment technique you were aske d to do? Perfe ct F ailure Effort How hard did you have to work to a c complish your level of p erformanc e? • WP: Very Low Very High Workload Profile Frustration How inse cure, discoura g e d, irritate d, stresse d, and annoye d wereyou? Very Low Very High
  • 26. Validity of workload measures • Mental workload consistently shown to be negatively correlated with performance metrics [Bertram et al. 1992] • Airline cockpits [Ballas et al. 1992] • Navigation [Schryver 1994] • Multi-device computing environments: information ecosystems [None yet!]
  • 27. Research Question 1 • RQ: What is the mental workload incurred by users in certain common tasks that were considered difficult in Phase I? • Hypothesis: Subjective assessment of mental workload will be high in these tasks • Experiment: Measure mental workload for several representative tasks performed in information ecosystems
  • 28. Research Question 2 • RQ: Is a decrease in mental workload a factor that motivates changes in users’ information management strategies? • Hypothesis: Users adopt strategies that will eventually lead to lowered mental workload • Experiment: Compare mental workload for tasks identified as difficult, and for their respective workarounds
  • 29. Research Question 3 • RQ: Are subjective assessments of mental workload an accurate indicator of operator performance in this domain? • Hypothesis: Mental workload measured by NASA TLX (including existing dimensions, and possibly new dimensions) can be used to predict operator performance • Experiment: (Attempt to) correlate workload assessments with operator performance
  • 30. Experiment design • Representative tasks from the content analysis of Phase I • Identify devices, tasks, strategies, etc. and use these to give users benchmark tasks • Measure mental workload • Other benchmark tasks too • To have a baseline
  • 31. Expected contributions • Understanding users and how they use multiple devices to accomplish PIM • Comparing workloads in different information ecosystems • Formative feedback for designers • Validating NASA TLX as an accurate predictor of task performance in information ecosystems
  • 32. Schedule May 08 June 08 July 08 Aug 08 Sep 08 Perform content analysis for Phase I Determine tasks Recruitment, IRB, Scheduling Study Conduct Experiments Perform analysis Write dissertation Prepare publications
  • 33. Questions & comments ? ! Thank you! Note to self: Turn off audio recording before committee deliberation.
  • 34. Supporting Slides
  • 35. Mental workload and task performance [O’Donnell, Eggemeier 1986] Performance Mental workload
  • 36. Why NASA TLX • Higher correlation with performance as compared to SWAT and WP [Rubio & Díaz, 2004] • Validated in several environments since1988 [several, 1988-present]
  • 37. NASA Task Load Index NASA TLX procedure H art and Staveland’s N ASA Task Loa d Ind ex (TLX) method assesses work loa d on five 7-point sc ales. Increments of high, me dium and low estimates for e a ch point result in 21 gra d ations on the sc ales. N ame Task D ate Mental Demand How mentally d emanding was the task? Very Low Very High Physic al D emand How physic ally d emanding was the task? 20 steps Very Low Very High Temporal Demand How hurrie d or rushe d was the p a c e of the task?
  • 38. NASA TLX procedure ntal D emand Frustra Me tion Leve l Pairwise Comparisons
  • 39. Quantitative analysis Work Desktop Home Desktop Laptop Media player Cell phone PDA cell phone 52 32 29 25 24 22 20 19 18 Number of participants using these devices as a group
  • 40. Content analysis • Techniques from [Neuendorf 2004, Krippendorf 2004] • Inter-rater reliability with 2 additional coders (expected Cohen’s ! quot; 0.6~0.7) • Purpose of content analysis is to design the experiment, not to draw conclusions • Coding: a priori versus emergent • Challenge: converging on representative tasks
  • 41. Experimental setup • Explain features • Training period with example tasks • Account for experience • Stratified samples? • Participant recruitment • CHCI, CS@VT, CRC, Google (?)