User studies and usability testing with remote participants

1,048 views

Published on

Session at InfoCamp Seattle 2011, including presentation and discussion on user studies and usability testing with participants in remote locations

Published in: Technology, Art & Photos
  • Be the first to comment

User studies and usability testing with remote participants

  1. 1. Conducting User/Usability Studies with Remote Participants<br />Uma Murthy<br />http://umurthy.com<br />@umurthy<br />InfoCamp Seattle<br />9 October, 2011<br />
  2. 2. We’ll talk about …<br />My remote user study experience<br />Discussion<br />
  3. 3. A qualitative study of subimage and SuperIDR use<br />How do people use subimages in fish identification?<br />Characteristics of subimages and related information<br />Contexts and strategies of working with subimages in fish identification<br />How does SuperIDR support that use?<br />3<br />
  4. 4. Study procedures<br />- Study overview<br />- Informed consent<br />- SuperIDR setup<br />- SuperIDR training<br />- Pre-study interview<br />SuperIDR use in work/study (diary)<br />Unknown specimen identification tasks<br />SuperIDR use in work/study (diary)<br />Unknown specimen identification tasks<br />SuperIDR use in work/study (diary)<br />Post-study interview<br />4<br />
  5. 5. Study procedures<br />- Study overview<br />- Informed consent<br />- SuperIDR setup<br />- SuperIDR training<br />- Pre-study interview<br />SuperIDR use in work/study (diary)<br />Unknown specimen identification tasks<br />SuperIDR use in work/study (diary)<br />Unknown specimen identification tasks<br />SuperIDR use in work/study (diary)<br />Post-study interview<br />5<br />
  6. 6. Qualitative study procedures<br />- Study overview<br />- Informed consent<br />- SuperIDR setup<br />- SuperIDR training<br />- Pre-study interview<br />SuperIDR use in work/study (diary)<br />Unknown specimen identification tasks<br />SuperIDR use in work/study (diary)<br />Unknown specimen identification tasks<br />SuperIDR use in work/study (diary)<br />Post-study interview<br />6<br />
  7. 7. Qualitative study procedures<br />- Study overview<br />- Informed consent<br />- SuperIDR setup<br />- SuperIDR training<br />- Pre-study interview<br />SuperIDR use in work/study (diary)<br />Unknown specimen identification tasks<br />SuperIDR use in work/study (diary)<br />Unknown specimen identification tasks<br />SuperIDR use in work/study (diary)<br />Post-study interview<br />7<br />
  8. 8. Study setup<br />8<br />
  9. 9. Data analysis was informed by Grounded Theory <br />9<br />Interview responses<br />Screen captures & spoken thoughts<br />Diary entries<br />SuperIDR logs<br />Species ID materials<br />Species ID responses<br />Database image<br />
  10. 10. Murthy, U., Tzy Li, Lin, Hallerman, E., Fox, E. A., Pérez-Quiñones, M. A., Delcambre, L. M., and Torres, R. da S. Use of Subimages in Fish Species Identification: A Qualitative Study. In Proceedings of the 11th ACM/IEEE Joint Conference on Digital Libraries, Ottawa, 2011.<br />
  11. 11. Discussion notes<br />The ever-evolving notes document is on Googls docs. Email me if you would like to contribute to it.<br />
  12. 12. Challenges<br />Finding participants, who can give time. Eg. It is harder to recruit faculty than grad students than undergrads<br />Hard to get the attention of transient users, who visit your site briefly<<br />Hard to work with participants who may not be tech savvy<br />Real-time recruitment of users (while they use technology) and having an interaction at the same time can be challenging<br />
  13. 13. Ideas to enable studies<br />Grab a user's attention to recruit them<br />Recruit people who use that or similar technology as part of their job<br />Have a help/FAQ page for users to refer to<br />Have Live meeting/Skype link on the side (not as a pop-up)<br />Make connecting with them personal - use your blog, twitter, linkedIn<br />Ask users what features to build<br />
  14. 14. Remote usability/user studies tool recommendations<br />Morae – capture user data remotely<br />Adobe Connect, GoToMeeting: screen share/capture<br />Skype: screen sharing<br />QuickTime: screen capture<br />QuickTime: Audio capture as a separate file<br />Camtasia facial/screen capture<br />meet.webex.com share screen, video camera<br />WebEx save files to computer or server<br />Phone to PC audio adapter<br />Google Docs - share documentation, scripts, and other information<br />Mechanical Turk - do repeatable, simple, tasks<br />Twitter - connect with participants<br />TeamViewer - remote screen sharing and presentation tool<br />
  15. 15. Resources<br />http://www.slideshare.net/EdChi/tutorial-on-using-amazon-mechanical-turk-mturk-for-hci-research<br />
  16. 16. Upcoming panel at Grace Hopper Conference<br />http://umurthy.com/research/ghc2011-panel/<br />

×