Your SlideShare is downloading. ×
0
we are ...
Bernard, packet core engineer at NSN
Belén, interaction designer at Intel
usability testinga process that employs people as testingparticipants who are representative of thetarget audience to eval...
please, stand up   everyone
sit down if you don’thave a US cellphone   with a data plan
sit down if you don’t      like beer
sit down if you areabsolutely terrified bythe idea of being our     test subject
why recording? memory aid powerful communication tool
actions          reactions
dut = mut
dut = mut + afecwhere:dut = desktop usability testingmut = mobile usability testingafec = a few extra challenges
which phone?which context?which connection?
which phone?which context?which connection?
web task success rates         feature phones                               38%           smartphones                     ...
web task success rates         feature phones                               38%           smartphones                     ...
web task success rates         feature phones                               38%           smartphones                     ...
web task success rates         feature phones                               38%           smartphones                     ...
handset usabilityaffects test results
remember ... test with participants’ own phones if not possible, include training and warm-up tasks
which phone?which context?which connection?
field vs. labIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen,...
experts disagreeaccording to our study there was nodifference in the number of problems thatoccurred in the two test setti...
experts disagreeevaluations conducted in field settings canreveal problems not otherwise identifiedin laboratory evaluations...
but they all agreeevaluations in the field (are) morecomplex and time-consumingIt’s Worth the Hassle! The Added Value of Ev...
testing in the lab isbetter than no testing
remember ... for most software, lab testing is fine if you must do field testing   do it late   plan and run pilot tests   b...
which phone?which context?which connection?
!"#$%& &()"*+,-*.//$0,1*")(2US Mobile Data Market Update Q4 2010 and 2010, Chetan Sharma Technology & Strategy Consultingh...
remember ... do not test over wi-fi cover participants’ data costs
tsdohoeaygtrtwtdut = (mut + afec)where:dut = desktop usability testingmut = mobile usability testingafec = a few extra cha...
why recording? memory aid powerful communication tool
4 approaches to thesmall detail of how on earth   are you   going to   record the   whole   thing
1. wearable equipment     .     Methods and techniques for field-based usability testing of mobile geo-applications, I. Del...
allows testing in    the field
but ...  difficult and time-consuming to set up  intrusive, uncomfortable and heavy
2. screen-capture applications   UsabilityExpo by Digia for S60 3rd and 5th ed, WinMo 6.1+ and Maemo 5   http://www.digia....
provides high quality  screen recording
but no application willsupport all platformshttp://www.digia.com/C2256FEF0043E9C1/0/405001348
it’s not cheaphttps://shop.digia.com
and participants won’t appreciateyou installing stuff on their phones
3. document camerasHandheld Usability, S. Weiss (page 174)
Google         Towards the Perfect Infrastructure for Usability Testing on Mobile Devices         R. Schusteritsch, C.Y. W...
Nielsen Norman                         Elmo TT-02RX Teachers Tool                                   document camera with a...
popular and well  documented
it’s not cheaphttp://www.bhphotovideo.com/c/product/644591-REG/Elmo_1304.html
participants mustkeep within thecamera range
phone must lay on         a desk or be hold         at a flat anglehttp://www.flickr.com/photos/zabriskiepoint/2806511301/si...
4. mounted devices
4a. ready-made mounted devices                                                   Mobile Device Camera by Tracksys         ...
4b. DIY mounted devices Little Springs Design http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobi...
4b. DIY mounted devices Little Springs Design                                       Nick Bowmast http://www.bowmast.com/mo...
4b. DIY mounted devices Little Springs Design                                       Nick Bowmast http://www.bowmast.com/mo...
4b. DIY mounted devices Little Springs Design                                       Nick Bowmast http://www.bowmast.com/mo...
natural interaction with       the phone
but it’s not cheaphttp://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-fo...
messy to buildhttp://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/
and ...  if bulky they can prevent single-hand  use  if heavy they can become tiring during  long tests
easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with ...
easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with ...
easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with ...
the spirithttp://www.wired.com/gadgetlab/2010/07/the-pencil-ipad-stand-smart-enough-to-impress-a-new-yorker/
theingredients
two meccano trunions (part no. A126)
two 5 and 6-hole meccano strips (part nos. 5 & 4)
one 11-hole meccano strip (part no. 2)
six meccano screws & nuts (part nos. 69 & 37h)
one 13-20mm jubilee clip
one HUE HD webcam
a second USB webcam
a USB male to female extension cable
blu tack
an allen key
a meccano wrench
a screwdriver
a Windows computer
screen recording software
a taskYou are planning a visit to the DenverZoo after this (great) session.Go to www.denverzoo.org and find outwhich bus nu...
how was that?
b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests ...
b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests ...
15-model meccano set                 18.33Hue HD webcam                        28.94Philips webcam                       5...
in USD: 196.40
b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests ...
b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests ...
weight: 125 grams an iPhone weighs 137 grams an iPad weighs 680 grams I weigh 55,000 grams a blue whale weighs 136,400,000...
b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests ...
Nokia E66slider with 12-keynumeric keypad
Nokia E71      BB Bold      candybar with      qwerty keyboard
iPhone 3Gtouchscreencandybar
Nokia N900             slider             touchscreen             with qwerty             keyboard
b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests ...
b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests ...
we expect much of our buildings: theyneed to have firm foundations, solidstructures, pleasing aesthetics. We shouldexpect t...
thanks!belenbarrospena@gmail.comb@runningwithbulls.com@belenpena @bernardtyershttp://belenpena.posterous.comhttp://www.run...
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
DIY mobile usability testing - IA Summit 2011
Upcoming SlideShare
Loading in...5
×

DIY mobile usability testing - IA Summit 2011

12,235

Published on

Our presentation for the IA Summit 2011 in Denver.

Published in: Technology, Business
2 Comments
24 Likes
Statistics
Notes
  • useful links to european research
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • http://XSEO.CO - You have a business and you are not familiar? Why not develop on the Internet? Want a website for your business? Want an online store? Want a presentation website for your company? WE HAVE THE SAME PRICE FOR ALL!
    id messenger: xseoco
    icq:423966704
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
12,235
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
173
Comments
2
Likes
24
Embeds 0
No embeds

No notes for slide

Transcript of "DIY mobile usability testing - IA Summit 2011"

  1. 1. we are ...
  2. 2. Bernard, packet core engineer at NSN
  3. 3. Belén, interaction designer at Intel
  4. 4. usability testinga process that employs people as testingparticipants who are representative of thetarget audience to evaluate the degree towhich a product meets specific usabilitycriteria.Handbook of usability testing 2nd Ed., J. Rubin and D. Chisnell
  5. 5. please, stand up everyone
  6. 6. sit down if you don’thave a US cellphone with a data plan
  7. 7. sit down if you don’t like beer
  8. 8. sit down if you areabsolutely terrified bythe idea of being our test subject
  9. 9. why recording? memory aid powerful communication tool
  10. 10. actions reactions
  11. 11. dut = mut
  12. 12. dut = mut + afecwhere:dut = desktop usability testingmut = mobile usability testingafec = a few extra challenges
  13. 13. which phone?which context?which connection?
  14. 14. which phone?which context?which connection?
  15. 15. web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html
  16. 16. web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html
  17. 17. web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html
  18. 18. web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html
  19. 19. handset usabilityaffects test results
  20. 20. remember ... test with participants’ own phones if not possible, include training and warm-up tasks
  21. 21. which phone?which context?which connection?
  22. 22. field vs. labIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
  23. 23. experts disagreeaccording to our study there was nodifference in the number of problems thatoccurred in the two test settings. Ourhypothesis that more problems would befound in the field was not supportedUsability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
  24. 24. experts disagreeevaluations conducted in field settings canreveal problems not otherwise identifiedin laboratory evaluationsIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
  25. 25. but they all agreeevaluations in the field (are) morecomplex and time-consumingIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006 testing in the field requires double the time in comparison to the laboratory Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
  26. 26. testing in the lab isbetter than no testing
  27. 27. remember ... for most software, lab testing is fine if you must do field testing do it late plan and run pilot tests be prepared (like the Scouts)
  28. 28. which phone?which context?which connection?
  29. 29. !"#$%& &()"*+,-*.//$0,1*")(2US Mobile Data Market Update Q4 2010 and 2010, Chetan Sharma Technology & Strategy Consultinghttp://www.chetansharma.com/usmarketupdate2010.htm "$$89::;;;<="#$%&,"%)*%<=+* 67 !"!"#$%&("%)*%!+&,-.$/&01 2..3/0"$,3#,#)4#5 #$%&"())
  30. 30. remember ... do not test over wi-fi cover participants’ data costs
  31. 31. tsdohoeaygtrtwtdut = (mut + afec)where:dut = desktop usability testingmut = mobile usability testingafec = a few extra challengestsdohoeaygtrtwt = the smalldetail of how on earth are yougoing to record the whole thing
  32. 32. why recording? memory aid powerful communication tool
  33. 33. 4 approaches to thesmall detail of how on earth are you going to record the whole thing
  34. 34. 1. wearable equipment . Methods and techniques for field-based usability testing of mobile geo-applications, I. Delikostidis (2007) International Institute for Geo-Information Science and Earth Observation (Enschede, The Netherlands)
  35. 35. allows testing in the field
  36. 36. but ... difficult and time-consuming to set up intrusive, uncomfortable and heavy
  37. 37. 2. screen-capture applications UsabilityExpo by Digia for S60 3rd and 5th ed, WinMo 6.1+ and Maemo 5 http://www.digia.com/C2256FEF0043E9C1/0/405001348
  38. 38. provides high quality screen recording
  39. 39. but no application willsupport all platformshttp://www.digia.com/C2256FEF0043E9C1/0/405001348
  40. 40. it’s not cheaphttps://shop.digia.com
  41. 41. and participants won’t appreciateyou installing stuff on their phones
  42. 42. 3. document camerasHandheld Usability, S. Weiss (page 174)
  43. 43. Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)
  44. 44. Nielsen Norman Elmo TT-02RX Teachers Tool document camera with autofocus controlled remotely from a laptop records screen and fingers webcam records both video streams participant’s face fed onto a laptop and recorded with Morae Photo from Nielsen Norman Mobile Usability workshop handout (London, 22 May 2009)
  45. 45. popular and well documented
  46. 46. it’s not cheaphttp://www.bhphotovideo.com/c/product/644591-REG/Elmo_1304.html
  47. 47. participants mustkeep within thecamera range
  48. 48. phone must lay on a desk or be hold at a flat anglehttp://www.flickr.com/photos/zabriskiepoint/2806511301/sizes/m/in/photostream/
  49. 49. 4. mounted devices
  50. 50. 4a. ready-made mounted devices Mobile Device Camera by Tracksys http://www.tracksys.co.uk/product- details.php?id=9 Mobile Device Camera by Noldus http://www.noldus.com/human-behavior- research/accessories/mobile-device-camera- mdc#
  51. 51. 4b. DIY mounted devices Little Springs Design http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/
  52. 52. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/
  53. 53. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/ Usability Sciences http://www.usabilitysciences.com/services/lab- based-usability-testing/mobile-usability-testing
  54. 54. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/ Google Towards the Perfect Infrastructure for Usability Testing on Mobile Usability Sciences Devices, R. Schusteritsch, C.Y. Wei, http://www.usabilitysciences.com/services/lab- M. LaRosa - Google (CHI 2007) based-usability-testing/mobile-usability-testing
  55. 55. natural interaction with the phone
  56. 56. but it’s not cheaphttp://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
  57. 57. messy to buildhttp://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/
  58. 58. and ... if bulky they can prevent single-hand use if heavy they can become tiring during long tests
  59. 59. easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  60. 60. easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face & fingersgives enough video quality
  61. 61. easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face & fingersgives enough video quality
  62. 62. the spirithttp://www.wired.com/gadgetlab/2010/07/the-pencil-ipad-stand-smart-enough-to-impress-a-new-yorker/
  63. 63. theingredients
  64. 64. two meccano trunions (part no. A126)
  65. 65. two 5 and 6-hole meccano strips (part nos. 5 & 4)
  66. 66. one 11-hole meccano strip (part no. 2)
  67. 67. six meccano screws & nuts (part nos. 69 & 37h)
  68. 68. one 13-20mm jubilee clip
  69. 69. one HUE HD webcam
  70. 70. a second USB webcam
  71. 71. a USB male to female extension cable
  72. 72. blu tack
  73. 73. an allen key
  74. 74. a meccano wrench
  75. 75. a screwdriver
  76. 76. a Windows computer
  77. 77. screen recording software
  78. 78. a taskYou are planning a visit to the DenverZoo after this (great) session.Go to www.denverzoo.org and find outwhich bus number will bring you to thezoo from the Colorado ConventionCentre.
  79. 79. how was that?
  80. 80. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  81. 81. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  82. 82. 15-model meccano set 18.33Hue HD webcam 28.94Philips webcam 52.08additional meccano parts 4.90blu tack 0.98jubilee clips (x2) 1.55USB cable 8.99screwdriver 5.99CamStudio 0.00 total (in GBP) 121.76
  83. 83. in USD: 196.40
  84. 84. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  85. 85. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  86. 86. weight: 125 grams an iPhone weighs 137 grams an iPad weighs 680 grams I weigh 55,000 grams a blue whale weighs 136,400,000 grams
  87. 87. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  88. 88. Nokia E66slider with 12-keynumeric keypad
  89. 89. Nokia E71 BB Bold candybar with qwerty keyboard
  90. 90. iPhone 3Gtouchscreencandybar
  91. 91. Nokia N900 slider touchscreen with qwerty keyboard
  92. 92. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  93. 93. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  94. 94. we expect much of our buildings: theyneed to have firm foundations, solidstructures, pleasing aesthetics. We shouldexpect the same of emerging mobilesystems.Mobile Interaction Design, M. Jones and G. Marsden (2005)
  95. 95. thanks!belenbarrospena@gmail.comb@runningwithbulls.com@belenpena @bernardtyershttp://belenpena.posterous.comhttp://www.runningwithbulls.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×