User studies and usability testing with remote participants

  • 707 views
Uploaded on

Session at InfoCamp Seattle 2011, including presentation and discussion on user studies and usability testing with participants in remote locations

Session at InfoCamp Seattle 2011, including presentation and discussion on user studies and usability testing with participants in remote locations

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
707
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Our study spanned over three weeks.At the beginning of the study, for each participant, we provided an overview of study, got their consent, setup SuperIDR on their personal computers <CLICK> and trained them to use it, including having them complete a set of training tasks.Then, we conducted a pre-study interview, asking participants about their fish ID practices and their fish ID background and experience.

Transcript

  • 1. Conducting User/Usability Studies with Remote Participants
    Uma Murthy
    http://umurthy.com
    @umurthy
    InfoCamp Seattle
    9 October, 2011
  • 2. We’ll talk about …
    My remote user study experience
    Discussion
  • 3. A qualitative study of subimage and SuperIDR use
    How do people use subimages in fish identification?
    Characteristics of subimages and related information
    Contexts and strategies of working with subimages in fish identification
    How does SuperIDR support that use?
    3
  • 4. Study procedures
    - Study overview
    - Informed consent
    - SuperIDR setup
    - SuperIDR training
    - Pre-study interview
    SuperIDR use in work/study (diary)
    Unknown specimen identification tasks
    SuperIDR use in work/study (diary)
    Unknown specimen identification tasks
    SuperIDR use in work/study (diary)
    Post-study interview
    4
  • 5. Study procedures
    - Study overview
    - Informed consent
    - SuperIDR setup
    - SuperIDR training
    - Pre-study interview
    SuperIDR use in work/study (diary)
    Unknown specimen identification tasks
    SuperIDR use in work/study (diary)
    Unknown specimen identification tasks
    SuperIDR use in work/study (diary)
    Post-study interview
    5
  • 6. Qualitative study procedures
    - Study overview
    - Informed consent
    - SuperIDR setup
    - SuperIDR training
    - Pre-study interview
    SuperIDR use in work/study (diary)
    Unknown specimen identification tasks
    SuperIDR use in work/study (diary)
    Unknown specimen identification tasks
    SuperIDR use in work/study (diary)
    Post-study interview
    6
  • 7. Qualitative study procedures
    - Study overview
    - Informed consent
    - SuperIDR setup
    - SuperIDR training
    - Pre-study interview
    SuperIDR use in work/study (diary)
    Unknown specimen identification tasks
    SuperIDR use in work/study (diary)
    Unknown specimen identification tasks
    SuperIDR use in work/study (diary)
    Post-study interview
    7
  • 8. Study setup
    8
  • 9. Data analysis was informed by Grounded Theory
    9
    Interview responses
    Screen captures & spoken thoughts
    Diary entries
    SuperIDR logs
    Species ID materials
    Species ID responses
    Database image
  • 10. Murthy, U., Tzy Li, Lin, Hallerman, E., Fox, E. A., Pérez-Quiñones, M. A., Delcambre, L. M., and Torres, R. da S. Use of Subimages in Fish Species Identification: A Qualitative Study. In Proceedings of the 11th ACM/IEEE Joint Conference on Digital Libraries, Ottawa, 2011.
  • 11. Discussion notes
    The ever-evolving notes document is on Googls docs. Email me if you would like to contribute to it.
  • 12. Challenges
    Finding participants, who can give time. Eg. It is harder to recruit faculty than grad students than undergrads
    Hard to get the attention of transient users, who visit your site briefly<
    Hard to work with participants who may not be tech savvy
    Real-time recruitment of users (while they use technology) and having an interaction at the same time can be challenging
  • 13. Ideas to enable studies
    Grab a user's attention to recruit them
    Recruit people who use that or similar technology as part of their job
    Have a help/FAQ page for users to refer to
    Have Live meeting/Skype link on the side (not as a pop-up)
    Make connecting with them personal - use your blog, twitter, linkedIn
    Ask users what features to build
  • 14. Remote usability/user studies tool recommendations
    Morae – capture user data remotely
    Adobe Connect, GoToMeeting: screen share/capture
    Skype: screen sharing
    QuickTime: screen capture
    QuickTime: Audio capture as a separate file
    Camtasia facial/screen capture
    meet.webex.com share screen, video camera
    WebEx save files to computer or server
    Phone to PC audio adapter
    Google Docs - share documentation, scripts, and other information
    Mechanical Turk - do repeatable, simple, tasks
    Twitter - connect with participants
    TeamViewer - remote screen sharing and presentation tool
  • 15. Resources
    http://www.slideshare.net/EdChi/tutorial-on-using-amazon-mechanical-turk-mturk-for-hci-research
  • 16. Upcoming panel at Grace Hopper Conference
    http://umurthy.com/research/ghc2011-panel/