DIY mobile usability testing - IA Summit 2011
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

DIY mobile usability testing - IA Summit 2011

  • 12,857 views
Uploaded on

Our presentation for the IA Summit 2011 in Denver.

Our presentation for the IA Summit 2011 in Denver.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • useful links to european research
    Are you sure you want to
    Your message goes here
  • http://XSEO.CO - You have a business and you are not familiar? Why not develop on the Internet? Want a website for your business? Want an online store? Want a presentation website for your company? WE HAVE THE SAME PRICE FOR ALL!
    id messenger: xseoco
    icq:423966704
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
12,857
On Slideshare
9,593
From Embeds
3,264
Number of Embeds
22

Actions

Shares
Downloads
171
Comments
2
Likes
25

Embeds 3,264

http://uxformobile.com 1,845
http://www.diymobileusabilitytesting.net 302
http://blog.mobilefoo.com 273
http://www.scoop.it 205
http://diymut.wordpress.com 171
http://www.temafrank.com 154
http://www.sitemotif.com 134
http://cloudworks.ac.uk 66
http://library.iasummit.org 40
http://lanyrd.com 20
http://abtasty.com 17
http://kecke.wordpress.com 7
http://paper.li 5
http://www.slideshare.net 5
http://speakerrate.com 5
http://twitter.com 4
url_unknown 4
http://a0.twimg.com 2
http://translate.googleusercontent.com 2
http://frankonlinemarketing.com 1
http://webcache.googleusercontent.com 1
https://twitter.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. we are ...
  • 2. Bernard, packet core engineer at NSN
  • 3. Belén, interaction designer at Intel
  • 4. usability testinga process that employs people as testingparticipants who are representative of thetarget audience to evaluate the degree towhich a product meets specific usabilitycriteria.Handbook of usability testing 2nd Ed., J. Rubin and D. Chisnell
  • 5. please, stand up everyone
  • 6. sit down if you don’thave a US cellphone with a data plan
  • 7. sit down if you don’t like beer
  • 8. sit down if you areabsolutely terrified bythe idea of being our test subject
  • 9. why recording? memory aid powerful communication tool
  • 10. actions reactions
  • 11. dut = mut
  • 12. dut = mut + afecwhere:dut = desktop usability testingmut = mobile usability testingafec = a few extra challenges
  • 13. which phone?which context?which connection?
  • 14. which phone?which context?which connection?
  • 15. web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html
  • 16. web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html
  • 17. web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html
  • 18. web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html
  • 19. handset usabilityaffects test results
  • 20. remember ... test with participants’ own phones if not possible, include training and warm-up tasks
  • 21. which phone?which context?which connection?
  • 22. field vs. labIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
  • 23. experts disagreeaccording to our study there was nodifference in the number of problems thatoccurred in the two test settings. Ourhypothesis that more problems would befound in the field was not supportedUsability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
  • 24. experts disagreeevaluations conducted in field settings canreveal problems not otherwise identifiedin laboratory evaluationsIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
  • 25. but they all agreeevaluations in the field (are) morecomplex and time-consumingIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006 testing in the field requires double the time in comparison to the laboratory Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
  • 26. testing in the lab isbetter than no testing
  • 27. remember ... for most software, lab testing is fine if you must do field testing do it late plan and run pilot tests be prepared (like the Scouts)
  • 28. which phone?which context?which connection?
  • 29. !"#$%& &()"*+,-*.//$0,1*")(2US Mobile Data Market Update Q4 2010 and 2010, Chetan Sharma Technology & Strategy Consultinghttp://www.chetansharma.com/usmarketupdate2010.htm "$$89::;;;<="#$%&,"%)*%<=+* 67 !"!"#$%&("%)*%!+&,-.$/&01 2..3/0"$,3#,#)4#5 #$%&"())
  • 30. remember ... do not test over wi-fi cover participants’ data costs
  • 31. tsdohoeaygtrtwtdut = (mut + afec)where:dut = desktop usability testingmut = mobile usability testingafec = a few extra challengestsdohoeaygtrtwt = the smalldetail of how on earth are yougoing to record the whole thing
  • 32. why recording? memory aid powerful communication tool
  • 33. 4 approaches to thesmall detail of how on earth are you going to record the whole thing
  • 34. 1. wearable equipment . Methods and techniques for field-based usability testing of mobile geo-applications, I. Delikostidis (2007) International Institute for Geo-Information Science and Earth Observation (Enschede, The Netherlands)
  • 35. allows testing in the field
  • 36. but ... difficult and time-consuming to set up intrusive, uncomfortable and heavy
  • 37. 2. screen-capture applications UsabilityExpo by Digia for S60 3rd and 5th ed, WinMo 6.1+ and Maemo 5 http://www.digia.com/C2256FEF0043E9C1/0/405001348
  • 38. provides high quality screen recording
  • 39. but no application willsupport all platformshttp://www.digia.com/C2256FEF0043E9C1/0/405001348
  • 40. it’s not cheaphttps://shop.digia.com
  • 41. and participants won’t appreciateyou installing stuff on their phones
  • 42. 3. document camerasHandheld Usability, S. Weiss (page 174)
  • 43. Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)
  • 44. Nielsen Norman Elmo TT-02RX Teachers Tool document camera with autofocus controlled remotely from a laptop records screen and fingers webcam records both video streams participant’s face fed onto a laptop and recorded with Morae Photo from Nielsen Norman Mobile Usability workshop handout (London, 22 May 2009)
  • 45. popular and well documented
  • 46. it’s not cheaphttp://www.bhphotovideo.com/c/product/644591-REG/Elmo_1304.html
  • 47. participants mustkeep within thecamera range
  • 48. phone must lay on a desk or be hold at a flat anglehttp://www.flickr.com/photos/zabriskiepoint/2806511301/sizes/m/in/photostream/
  • 49. 4. mounted devices
  • 50. 4a. ready-made mounted devices Mobile Device Camera by Tracksys http://www.tracksys.co.uk/product- details.php?id=9 Mobile Device Camera by Noldus http://www.noldus.com/human-behavior- research/accessories/mobile-device-camera- mdc#
  • 51. 4b. DIY mounted devices Little Springs Design http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/
  • 52. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/
  • 53. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/ Usability Sciences http://www.usabilitysciences.com/services/lab- based-usability-testing/mobile-usability-testing
  • 54. 4b. DIY mounted devices Little Springs Design Nick Bowmast http://www.bowmast.com/mob-device-cam/ http://www.littlespringsdesign.com/blog/2008/Jun/usability- testing-for-mobile-devices-2/ Google Towards the Perfect Infrastructure for Usability Testing on Mobile Usability Sciences Devices, R. Schusteritsch, C.Y. Wei, http://www.usabilitysciences.com/services/lab- M. LaRosa - Google (CHI 2007) based-usability-testing/mobile-usability-testing
  • 55. natural interaction with the phone
  • 56. but it’s not cheaphttp://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
  • 57. messy to buildhttp://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/
  • 58. and ... if bulky they can prevent single-hand use if heavy they can become tiring during long tests
  • 59. easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  • 60. easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face & fingersgives enough video quality
  • 61. easy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face & fingersgives enough video quality
  • 62. the spirithttp://www.wired.com/gadgetlab/2010/07/the-pencil-ipad-stand-smart-enough-to-impress-a-new-yorker/
  • 63. theingredients
  • 64. two meccano trunions (part no. A126)
  • 65. two 5 and 6-hole meccano strips (part nos. 5 & 4)
  • 66. one 11-hole meccano strip (part no. 2)
  • 67. six meccano screws & nuts (part nos. 69 & 37h)
  • 68. one 13-20mm jubilee clip
  • 69. one HUE HD webcam
  • 70. a second USB webcam
  • 71. a USB male to female extension cable
  • 72. blu tack
  • 73. an allen key
  • 74. a meccano wrench
  • 75. a screwdriver
  • 76. a Windows computer
  • 77. screen recording software
  • 78. a taskYou are planning a visit to the DenverZoo after this (great) session.Go to www.denverzoo.org and find outwhich bus number will bring you to thezoo from the Colorado ConventionCentre.
  • 79. how was that?
  • 80. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  • 81. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  • 82. 15-model meccano set 18.33Hue HD webcam 28.94Philips webcam 52.08additional meccano parts 4.90blu tack 0.98jubilee clips (x2) 1.55USB cable 8.99screwdriver 5.99CamStudio 0.00 total (in GBP) 121.76
  • 83. in USD: 196.40
  • 84. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  • 85. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  • 86. weight: 125 grams an iPhone weighs 137 grams an iPad weighs 680 grams I weigh 55,000 grams a blue whale weighs 136,400,000 grams
  • 87. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  • 88. Nokia E66slider with 12-keynumeric keypad
  • 89. Nokia E71 BB Bold candybar with qwerty keyboard
  • 90. iPhone 3Gtouchscreencandybar
  • 91. Nokia N900 slider touchscreen with qwerty keyboard
  • 92. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  • 93. b&b’seasy to put togethercheaprepeatableallows holding the deviceallows one-handed usesupports all form factorsruns tests with participants’ phonescaptures screen, face and fingersgives enough video quality
  • 94. we expect much of our buildings: theyneed to have firm foundations, solidstructures, pleasing aesthetics. We shouldexpect the same of emerging mobilesystems.Mobile Interaction Design, M. Jones and G. Marsden (2005)
  • 95. thanks!belenbarrospena@gmail.comb@runningwithbulls.com@belenpena @bernardtyershttp://belenpena.posterous.comhttp://www.runningwithbulls.com