Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Temadag	
  om	
  evaluering	
  af	
  fysiske	
  prototyper	
  og	
  
produkter	
  	
  
	
  
Infinit	
  Interessegruppen	
  ...
Program	
  
9.30:	
  	
  	
  Welcome	
  /	
  Infint	
  and	
  Delta	
  
9.40:	
  	
  	
  Brugerinvolvering	
  og	
  elektro...
Challenges	
  when	
  doing	
  usability	
  
tests	
  on	
  physical	
  devices	
  
	
  
Outline:	
  
•  Are	
  there	
  s...
Methods?	
  
Looking	
  at	
  user	
  tes8ng	
  methods	
  in	
  textbooks,	
  ar8cles,	
  etc.	
  
reveals	
  that:	
  
W...
Usability	
  -­‐	
  ISO	
  9241	
  defini8on	
  
The	
  effec,veness,	
  efficiency	
  and	
  sa,sfac,on	
  with	
  
which	
  ...
JUS	
  Survey	
  

uxpa.org

A	
  survey	
  of	
  UXPA’s	
  Journal	
  of	
  Usability	
  Studies	
  for	
  the	
  past	
 ...
Challenges	
  
Physical	
  products	
  –	
  contrary	
  to	
  soXware	
  products,	
  web,	
  
etc	
  –	
  typically	
  re...
Challenges	
  -­‐	
  prototyping	
  
Building	
  Mockups	
  for	
  tes8ng:	
  
•  Lo-­‐Fi	
  -­‐	
  Very	
  oXen	
  done,	...
Mockups	
  
Solu8ons?	
  
•  Combining	
  real	
  device	
  with	
  simulated	
  interac8on	
  

Good solution, but only
g...
Challenges:	
  	
  Test	
  design	
  
Experimental	
  designs	
  become	
  more	
  complicated	
  when	
  both	
  the	
  
...
cent grip study, CAD models of the estimated based on grip style, fingers’ contact points and
modelization still encounter...
Summing	
  up	
  
Usability	
  tes8ng	
  of	
  physical	
  products	
  does	
  not	
  really	
  call	
  for	
  
specific	
 ...
Plenary	
  Discussion	
  
•  Are	
  you	
  going	
  to	
  do	
  a	
  usability	
  test	
  on	
  a	
  physical	
  
product	...
Upcoming SlideShare
Loading in …5
×

Challenges when doing usability tests on physical devices af Lars Bo Larsen, Aalborg Universitet

729 views

Published on

Oplægget blev holdt ved InfinIT-arrangementet "Temadag om evaluering af fysiske prototyper og produkter", der blev afholdt den 16. januar 2014.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Challenges when doing usability tests on physical devices af Lars Bo Larsen, Aalborg Universitet

  1. 1. Temadag  om  evaluering  af  fysiske  prototyper  og   produkter       Infinit  Interessegruppen  for  usability  og   Interak8onsdesign   Delta,  onsdag  16/1  2014     Infinit Workshop - Lars Bo Larsen
  2. 2. Program   9.30:      Welcome  /  Infint  and  Delta   9.40:      Brugerinvolvering  og  elektronisk  skitsering  af  nye  produkter            Morten  Wagner,  Delta   10.10:  Challenges  when  doing  usability  tests  on  physical  devices            Lars  Bo  Larsen,  AAU   10.35    Pause   10.50    Usability  of  Medical  Devices  –  Possibili8es  and  Challenges  when                  designing  and  tes8ng  physical  products              Morten  Purup,  Radiometer  Medical  Aps   11.20    Exploratory  user  research  with  people  that  use  drug  delivery  devices            Chris  Monnier  og  Peter  Urban  Novo  Nordisk   12.00    Plenary  Discussion   12.30  –  13.    Lunch   Infinit Workshop - Lars Bo Larsen 16/01/14 p. 2
  3. 3. Challenges  when  doing  usability   tests  on  physical  devices     Outline:   •  Are  there  specific  methods  for  usability  tes8ng  for  physical   devices?   •  What  are  the  challenges  when  tes8ng  on  physical   products?   •  Examples  from  mobile  devices     Infinit Workshop - Lars Bo Larsen 16/01/14 p. 3
  4. 4. Methods?   Looking  at  user  tes8ng  methods  in  textbooks,  ar8cles,  etc.   reveals  that:   While  physical  devices/products  are  oXen  men8oned:   •  No  usability  theories  or  methods  are  specifically  targeted  physical   devices   •  Ie.  “the  usual”  methods  applies  to  physical  products  (which  also   makes  sense)     The  conclusion  must  therefore  be  that  the  challenges  are   opera8onal,  rather  than  methodical:   •  Logis8cs,  costs,  lab  vs.  field,  mockups,  test  plan  and  tasks,  etc.   Infinit Workshop - Lars Bo Larsen 16/01/14 p. 4
  5. 5. Usability  -­‐  ISO  9241  defini8on   The  effec,veness,  efficiency  and  sa,sfac,on  with   which  specified  users  achieve  specified  goals  in   par8cular  environments.     •  effec,veness:  the  accuracy  and  completeness  with  which  specified   users  can  achieve  specified  goals  in  par8cular  environments   •  efficiency:  the  resources  expended  in  rela8on  to  the  accuracy  and   completeness  of  goals  achieved   •  sa,sfac,on:  the  comfort  and  acceptability  of  the  work  system  to  its   users  and  other  people  affected  by  its  use     -­‐  universal   Infinit Workshop - Lars Bo Larsen 16/01/14 p. 5
  6. 6. JUS  Survey   uxpa.org A  survey  of  UXPA’s  Journal  of  Usability  Studies  for  the  past  8   years  (130  ar8cles)  years  showed:   While  about  10-­‐15%  of  the  papers  present  methodical  approaches:   •  Not  one  single  paper  addressed  tes8ng  of  physical  products  as  the   main  methodical  focus     •  •    Very  few  papers  (<5)  present  and  discuss  tests  of  physical  products,  (e.g.   “vo8ng  machines”)  -­‐  however  not  from  a  methodological  point  of  view   A  higher  propor8on  discussed  mobile  or  medical  devices,  but  mostly  as   concrete  studies,  not  methods   Infinit Workshop - Lars Bo Larsen 16/01/14 p. 6
  7. 7. Challenges   Physical  products  –  contrary  to  soXware  products,  web,   etc  –  typically  require  much  more  handling:   •  •  •  •  •  Difficult  and  expensive  to  distribute  for  field  tes8ng  and  may   en8rely  prohibit  remote  tes8ng   Can  pose  problems  for  recording  usability  performance   parameters   Prototypes  can  be  quite  expensive  and  8me  consuming  to   produce  and  might  only  exist  in  very  few  (one)  instances  –   contrary  to  e.g.  wireframing   Test  users  oXen  need  the  whole  device  -­‐  oXen  not  possible  to   separate  the  physical  appearance  from  e.g.  soXware  driven   interac8on  (displays,  etc)   SW  and  HW  might  not  become  available  for  tes8ng  at  the  same   8me   Infinit Workshop - Lars Bo Larsen 16/01/14 p. 7
  8. 8. Challenges  -­‐  prototyping   Building  Mockups  for  tes8ng:   •  Lo-­‐Fi  -­‐  Very  oXen  done,  not  really  a  big  problem:   Infinit Workshop - Lars Bo Larsen 16/01/14 p.8
  9. 9. Mockups   Solu8ons?   •  Combining  real  device  with  simulated  interac8on   Good solution, but only goes so far – and for some types of products Infinit Workshop - Lars Bo Larsen 16/01/14 p. 9
  10. 10. Challenges:    Test  design   Experimental  designs  become  more  complicated  when  both  the   hw  and  sw  designs  are  subjected  to  usability  tests   simultaneously:   •  More  variables  to  keep  track  of  and  decorrelate   •  Solu8on  would  typically  be  to  test  separately  as  far  as   possible,  but  at  some  point  everything  must  come   together   Infinit Workshop - Lars Bo Larsen 16/01/14
  11. 11. cent grip study, CAD models of the estimated based on grip style, fingers’ contact points and modelization still encounters some practical difficulties [3], erated, investigating user’s proximity relative anthropometric properties. A proper categorization [4]. have The simulation results show that the Though some standardization bodies of two already ways of procedure led to the identification main proposed some preliminary hand phantoms, they utilize a ribution in determining the total loss holding mobiles while talking, naming them “firm” and “soft” hand ntom torso alone. The palm-handset grip that is not supported by grip studies [3]. Thanks to a grip styles [5] it was possible to generate more recent contributionrespectively. In the “firm” grip style the fingers are th absorption and mismatch loss. detailed placed models. the handset so of this work is intermediate hand around The objective that while the to phalanges FDTD side, the distal ones reach its front investigate throughtouch its (Finite Difference Time Domain) region, proximity factors, body loss, efficiency, with a user’s proximity effects for talk does in simulations thecorresponding palm-handset gap thatmode not exceed the length of the on both absorption and In the “soft” grip mobile phones, focusing longest proximal phalanx.mismatch NTRODUCTION loss andstyle the hand holds the handset onlyuser’sthe distal phalanges, isolating the contribution of both with torso and creating an air gap between the palm andof the palm- that does the handset hand e used in close proximity with the to the total loss. Moreover the influence Fig. 1. Test room [5]. handset not exceed the length of the thumb. s in a detrimental effect in its gap will be investigated too. This paper is structured as a ces [1]. While it was shown thatfollows: Section II describes the grip study and its main results. morphic Mannequin) phantom can Section III illustrates the used procedure to generate orso in average sense [2], the proper hand phantoms. In Section IV all the FDTD hand simulations are presented. Section V finally summarizes our ters some practical difficulties [3], conclusions. dardization bodies have already ry hand phantoms, they utilize a II. GRIP STUDY DESCRIPTION rted by grip studies [3]. Thanks to a A t was possible to generate morerecent contribution within the COST Action 2100 [5] reports a first grip study for talk and data modes in mobile The objective of this work is to phones, D (Finite Difference Time Domain) where a rigorous investigation methodology was used over roximity effects for talk mode in a sample population of 100 subjects; thanks to an unobtrusive data acquisition system (figures 1, 2) and a proper on both absorption and mismatch ntribution of both user’s torsoinvestigation protocol most of the experimental biases were and minimized, allowing the collection of stable and oreover the influence of the palmcomprehensive statistics. The Fig. 1. Test room [5]. location was index finger’s tigated too. This paper is structured confirmed to be in the back region of the handset in most scribes the grip study and its main Fig. 2: Example of videotape screenshot for talk mode (only 12/21 webcams cases ates the used procedure to generate (figure 3). The palm-handset distance was indirectly are displayed here) [5]. In Section IV all the FDTD Section V finally summarizes our STUDY DESCRIPTION Challenges  –  data  recording   Grip study of mobile phones at AAU labs •  Data capture using 21 webcams •  100 test persons. Tasks were texting, making and answering calls From: “User's proximity effects in mobile phones” within the COST Action by: Mauro Pelosi, Gert F. Pedersen et al. 3rd European Conference on Antennas and Propagation, 2009 2100 [5] for talk and data modes in mobile nvestigation methodology was used 16/01/14 p. 11 n of 100 subjects; thanks to an Infinit Workshop - Lars Bo Larsen 1022 on system (figures 1, 2) and a proper st of the experimental biases were he collection of stable and
  12. 12. Summing  up   Usability  tes8ng  of  physical  products  does  not  really  call  for   specific  theore8c  or  methodical  approaches  and  has  not  drawn   much  special  alen8on  in  the  community   However,  there  are  many  prac8cal  problems  not  present  with   pure  sw  products:   •  •  •  •  •  •  Logis8cs   Mockups   Data  recording   Costs   Complexity   …..   Infinit Workshop - Lars Bo Larsen 16/01/14 p. 12
  13. 13. Plenary  Discussion   •  Are  you  going  to  do  a  usability  test  on  a  physical   product  –  what  problems  are  you  facing?   •  From  your  experiences,  can  you  recommend   solu8ons,  tools  etc  for  some  of  the  challenges   you’ve  met?     Infinit Workshop - Lars Bo Larsen 16/01/14 13

×