Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Related Books

Free with a 30 day trial from Scribd

See all

DIY mobile usability testing - IA Summit 2011

  1. 1. we are ...
  2. 2. Bernard, packet core engineer at NSN
  3. 3. Belén, interaction designer at Intel
  4. 4. usability testing a process that employs people as testing participants who are representative of the target audience to evaluate the degree to which a product meets specific usability criteria. Handbook of usability testing 2nd Ed., J. Rubin and D. Chisnell
  5. 5. please, stand up everyone
  6. 6. sit down if you don’t have a US cellphone with a data plan
  7. 7. sit down if you don’t like beer
  8. 8. sit down if you are absolutely terrified by the idea of being our test subject
  9. 9. why recording? memory aid powerful communication tool
  10. 10. actions reactions
  11. 11. dut = mut
  12. 12. dut = mut + afec where: dut = desktop usability testing mut = mobile usability testing afec = a few extra challenges
  13. 13. which phone? which context? which connection?
  14. 14. which phone? which context? which connection?
  15. 15. web task success rates feature phones 38% smartphones 55% touch phones 75% Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009 http://www.useit.com/alertbox/mobile-usability.html
  16. 16. web task success rates feature phones 38% smartphones 55% touch phones 75% Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009 http://www.useit.com/alertbox/mobile-usability.html
  17. 17. web task success rates feature phones 38% smartphones 55% touch phones 75% Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009 http://www.useit.com/alertbox/mobile-usability.html
  18. 18. web task success rates feature phones 38% smartphones 55% touch phones 75% Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009 http://www.useit.com/alertbox/mobile-usability.html
  19. 19. handset usability affects test results
  20. 20. remember ... test with participants’ own phones if not possible, include training and warm-up tasks
  21. 21. which phone? which context? which connection?
  22. 22. field vs. lab It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
  23. 23. experts disagree according to our study there was no difference in the number of problems that occurred in the two test settings. Our hypothesis that more problems would be found in the field was not supported Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
  24. 24. experts disagree evaluations conducted in field settings can reveal problems not otherwise identified in laboratory evaluations It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
  25. 25. but they all agree evaluations in the field (are) more complex and time-consuming It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006 testing in the field requires double the time in comparison to the laboratory Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
  26. 26. testing in the lab is better than no testing
  27. 27. remember ... for most software, lab testing is fine if you must do field testing do it late plan and run pilot tests be prepared (like the Scouts)
  28. 28. which phone? which context? which connection?
  29. 29. !"#$%& '&()"*+,-*.//$0,1*")(2 US Mobile Data Market Update Q4 2010 and 2010, Chetan Sharma Technology & Strategy Consulting http://www.chetansharma.com/usmarketupdate2010.htm "$$89::;;;<="#$%&,"%)*%<=+* 67 !"!"#$%&'("%)*%'!+&,-.$/&01' 2..'3/0"$,'3#,#)4#5 #$%&"'())
  30. 30. remember ... do not test over wi-fi cover participants’ data costs
  31. 31. tsdohoeaygtrtwt dut = (mut + afec) where: dut = desktop usability testing mut = mobile usability testing afec = a few extra challenges tsdohoeaygtrtwt = the small detail of how on earth are you going to record the whole thing
  32. 32. why recording? memory aid powerful communication tool
  33. 33. 4 approaches to the small detail of how on earth are you going to record the whole thing
  34. 34. 1. wearable equipment . Methods and techniques for field-based usability testing of mobile geo-applications, I. Delikostidis (2007) International Institute for Geo-Information Science and Earth Observation (Enschede, The Netherlands)
  35. 35. allows testing in the field
  36. 36. but ... difficult and time-consuming to set up intrusive, uncomfortable and heavy
  37. 37. 2. screen-capture applications UsabilityExpo by Digia for S60 3rd and 5th ed, WinMo 6.1+ and Maemo 5 http://www.digia.com/C2256FEF0043E9C1/0/405001348
  38. 38. provides high quality screen recording
  39. 39. but no application will support all platforms http://www.digia.com/C2256FEF0043E9C1/0/405001348
  40. 40. it’s not cheap https://shop.digia.com
  41. 41. and participants won’t appreciate you installing stuff on their phones
  42. 42. 3. document cameras Handheld Usability, S. Weiss (page 174)
  43. 43. Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)
  44. 44. Nielsen Norman Elmo TT-02RX Teachers Tool document camera with autofocus controlled remotely from a laptop records screen and fingers webcam records both video streams participant’s face fed onto a laptop and recorded with Morae Photo from Nielsen Norman Mobile Usability workshop handout (London, 22 May 2009)
  45. 45. popular and well documented
  46. 46. it’s not cheap http://www.bhphotovideo.com/c/product/644591-REG/Elmo_1304.html
  47. 47. participants must keep within the camera range
  48. 48. phone must lay on a desk or be hold at a flat angle http://www.flickr.com/photos/zabriskiepoint/2806511301/sizes/m/in/photostream/
  49. 49. 4. mounted devices
  50. 50. 4a. ready-made mounted devices Mobile Device Camera by Tracksys http://www.tracksys.co.uk/product- details.php?id=9 Mobile Device Camera by Noldus http://www.noldus.com/human-behavior- research/accessories/mobile-device-camera- mdc#
  51. 51. 4b. DIY mounted devices Little Springs Design http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/
  52. 52. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/
  53. 53. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/ Usability Sciences http://www.usabilitysciences.com/services/lab- based-usability-testing/mobile-usability-testing
  54. 54. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/ Google Towards the Perfect Infrastructure for Usability Testing on Mobile Usability Sciences Devices, R. Schusteritsch, C.Y. Wei, http://www.usabilitysciences.com/services/lab- M. LaRosa - Google (CHI 2007) based-usability-testing/mobile-usability-testing
  55. 55. natural interaction with the phone
  56. 56. but it’s not cheap http://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
  57. 57. messy to build http://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/
  58. 58. and ... if bulky they can prevent single-hand use if heavy they can become tiring during long tests
  59. 59. easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face and fingers gives enough video quality
  60. 60. easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face & fingers gives enough video quality
  61. 61. easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face & fingers gives enough video quality
  62. 62. the spirit http://www.wired.com/gadgetlab/2010/07/the-pencil-ipad-stand-smart-enough-to-impress-a-new-yorker/
  63. 63. the ingredients
  64. 64. two meccano trunions (part no. A126)
  65. 65. two 5 and 6-hole meccano strips (part nos. 5 & 4)
  66. 66. one 11-hole meccano strip (part no. 2)
  67. 67. six meccano screws & nuts (part nos. 69 & 37h)
  68. 68. one 13-20mm jubilee clip
  69. 69. one HUE HD webcam
  70. 70. a second USB webcam
  71. 71. a USB male to female extension cable
  72. 72. blu tack
  73. 73. an allen key
  74. 74. a meccano wrench
  75. 75. a screwdriver
  76. 76. a Windows computer
  77. 77. screen recording software
  78. 78. a task You are planning a visit to the Denver Zoo after this (great) session. Go to www.denverzoo.org and find out which bus number will bring you to the zoo from the Colorado Convention Centre.
  79. 79. how was that?
  80. 80. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face and fingers gives enough video quality
  81. 81. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face and fingers gives enough video quality
  82. 82. 15-model meccano set 18.33 Hue HD webcam 28.94 Philips webcam 52.08 additional meccano parts 4.90 blu tack 0.98 jubilee clips (x2) 1.55 USB cable 8.99 screwdriver 5.99 CamStudio 0.00 total (in GBP) 121.76
  83. 83. in USD: 196.40
  84. 84. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face and fingers gives enough video quality
  85. 85. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face and fingers gives enough video quality
  86. 86. weight: 125 grams an iPhone weighs 137 grams an iPad weighs 680 grams I weigh 55,000 grams a blue whale weighs 136,400,000 grams
  87. 87. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face and fingers gives enough video quality
  88. 88. Nokia E66 slider with 12-key numeric keypad
  89. 89. Nokia E71 BB Bold candybar with qwerty keyboard
  90. 90. iPhone 3G touchscreen candybar
  91. 91. Nokia N900 slider touchscreen with qwerty keyboard
  92. 92. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face and fingers gives enough video quality
  93. 93. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed use supports all form factors runs tests with participants’ phones captures screen, face and fingers gives enough video quality
  94. 94. we expect much of our buildings: they need to have firm foundations, solid structures, pleasing aesthetics. We should expect the same of emerging mobile systems. Mobile Interaction Design, M. Jones and G. Marsden (2005)
  95. 95. thanks! belenbarrospena@gmail.com b@runningwithbulls.com @belenpena @bernardtyers http://belenpena.posterous.com http://www.runningwithbulls.com

×