Hvad Er Remote Usability Testing af Jan Stage, AAU

344 views

Published on

Oplægget blev holdt ved InfinIT-arrangementet "Temadag om Remote Usability Testing" afholdt den 17. januar 2012. Læs mere på http://www.infinit.dk/dk/hvad_kan_vi_goere_for_dig/viden/reportager/mere_tid_prioritering_og_penge_til_brugbarhedstests.htm

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
344
On SlideShare
0
From Embeds
0
Number of Embeds
61
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hvad Er Remote Usability Testing af Jan Stage, AAU

  1. 1. Hvad er Remote Usability Testing?Jan StageProfessor, PhDForskningsleder i Informationssystemer (IS)/Human-Computer Interaction (HCI)Aalborg Universitet, Institut for Datalogi, HCI-Labjans@cs.aau.dk
  2. 2. Oversigt • Forhindringer i praksis • Ideen om remote usability testing • Realisering af ideenInstitut for Datalogi 2
  3. 3. Oversigt • Forhindringer i praksis • Ideen om remote usability testing • Realisering af ideenInstitut for Datalogi 3
  4. 4. To undersøgelser af forhindringer i praksis 50 Primært Nordjylland 45 43 1. Software-virksom- 40 heder, typisk svar fra 35 mellemledere 30 2. Deltagere i en-dages 25 26 25 23 konference om 20 usability 15 15 13 11 10 8 8 10 6 • Tallene angiver procent 5 5 5 af svarene i hver 1 0 0 undersøgelse 0 Resource Developer Customer Test Conducting No problems Low priority No answ er demands mindset participation participants tests Første undersøgelse Anden undersøgelseInstitut for Datalogi 4
  5. 5. Oversigt • Forhindringer i praksis • Ideen om remote usability testing • Definition og baggrund • Tidligere arbejde • Synkrone og asynkrone metoder • Realisering af ideenInstitut for Datalogi 5
  6. 6. Definition and Motivation • Remote usability testing is a situation where “the evaluators are separated in space and/or time from users” (Castillo, Hartson and Hix, 1998) • Well-known challenges • Building and setting up usability labs • Getting users to the lab and making them feel comfortable • New challenges • Users, developers and evaluators are distributed geographically, sometimes even in different countries and time zones • Outsourcing • Open Source Software development • Global software development • Remote usability evaluation is becoming increasingly relevantInstitut for Datalogi 6
  7. 7. Earlier Work (1) • The idea of remote evaluation emerged between 1994 and 1998 • Literature survey on remote usability testing • 21 references • Early: presents a variety of methods • Recently: practical advice and experiences combined with pros/consInstitut for Datalogi 7
  8. 8. Earlier Work (2)Institut for Datalogi 8
  9. 9. Earlier Work (3) • Two types of methods: synchronous / asynchronous (various forms) • The majority of references focus on synchronous usability evaluation: Simulating a conventional laboratory-based think-aloud test • Overall: • Few systematic comparisons of remote methods • Mostly comparisons of a single remote method and the conventional lab-based methodInstitut for Datalogi 9
  10. 10. Synchronous Methods Laboratory testing (LAB) • Conventional method • Benchmark for comparison Remote synchronous testing (RS) • Spatial separation: User and test monitor in separate rooms • Remote sharing of desktop • Webcam connection with picture in the bottom corner of the screen • Audio connection Data: Video recording of the test monitor’s screen and the audioInstitut for Datalogi 10
  11. 11. Asynchronous Methods Remote asynchronous user testing (AU) • Based on the idea of users reporting the critical incidents they experience (Castillo, Hartson and Hix, 1998; Hartson and Castillo, 1998; Hartson, Castillo, Kelso and Neale, 1996) • The users solve the tasks with the system • Simultaneously fill in a web-based form each time they encounter a usability problem • The form briefly explains what a usability problem is and how it can be categorized • In the form, they describe each problem and categorize it’s severity • Spatial and temporal separation Remote asynchronous expert testing (AE) • Same method as AU but with users that have expertise in usability engineering (graduate students in HCI) Data: A set of described usability problems and a start and end time for each userInstitut for Datalogi 11
  12. 12. Survey of Remote Asynchronous Methods Method Paper # Auto logging 20, 28, 29, 30, 34, 35, 36, 37 Interview 10, 26, 29, 30, 35 Questionnaires 10, 19, 25, 28, 29, 30, 32, 34, 35, 36, 37 User-reported critical incident 1, 5, 6, 7, 13, 14, 31 Unstructured problem reporting 10, 19, 38 Forum 21 Diary 31Institut for Datalogi 12
  13. 13. Oversigt • Forhindringer i praksis • Ideen om remote usability testing • Realisering af ideenInstitut for Datalogi 13
  14. 14. Realisering af ideen • Forhindringer og udfordringer • Tekniske: Kan man få det til at fungere • Metodiske: giver det brugbare resultater • Praktiske: kan man gøre det i praksis • Det skal vi arbejde med resten af dagen …Institut for Datalogi 14
  15. 15. aaInstitut for Datalogi 15

×