DIY mobile usability testing - IA Summit 2011

we are ...

Bernard, packet core engineer at NSN

Belén, interaction designer at Intel

usability testinga process that employs people as testingparticipants who are representative of thetarget audience to evaluate the degree towhich a product meets speciﬁc usabilitycriteria.Handbook of usability testing 2nd Ed., J. Rubin and D. Chisnell

please, stand up everyone

sit down if you don’thave a US cellphone with a data plan

sit down if you don’t like beer

sit down if you areabsolutely terriﬁed bythe idea of being our test subject

why recording? memory aid powerful communication tool

actions reactions

dut = mut

dut = mut + afecwhere:dut = desktop usability testingmut = mobile usability testingafec = a few extra challenges

which phone?which context?which connection?

web task success rates feature phones 38% smartphones 55% touch phones 75%Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009http://www.useit.com/alertbox/mobile-usability.html

handset usabilityaffects test results

remember ... test with participants’ own phones if not possible, include training and warm-up tasks

ﬁeld vs. labIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006

experts disagreeaccording to our study there was nodifference in the number of problems thatoccurred in the two test settings. Ourhypothesis that more problems would befound in the ﬁeld was not supportedUsability Testing of Mobile Applications: A Comparison between Laboratory and Field TestingA. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005

experts disagreeevaluations conducted in ﬁeld settings canreveal problems not otherwise identiﬁedin laboratory evaluationsIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006

but they all agreeevaluations in the ﬁeld (are) morecomplex and time-consumingIt’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the FieldC.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006 testing in the ﬁeld requires double the time in comparison to the laboratory Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005

testing in the lab isbetter than no testing

remember ... for most software, lab testing is ﬁne if you must do ﬁeld testing do it late plan and run pilot tests be prepared (like the Scouts)

!"#$%& &()"*+,-*.//$0,1*")(2US Mobile Data Market Update Q4 2010 and 2010, Chetan Sharma Technology & Strategy Consultinghttp://www.chetansharma.com/usmarketupdate2010.htm "$$89::;;;<="#$%&,"%)*%<=+* 67 !"!"#$%&("%)*%!+&,-.$/&01 2..3/0"$,3#,#)4#5 #$%&"())

remember ... do not test over wi-ﬁ cover participants’ data costs

tsdohoeaygtrtwtdut = (mut + afec)where:dut = desktop usability testingmut = mobile usability testingafec = a few extra challengestsdohoeaygtrtwt = the smalldetail of how on earth are yougoing to record the whole thing

why recording? memory aid powerful communication tool

4 approaches to thesmall detail of how on earth are you going to record the whole thing

1. wearable equipment . Methods and techniques for ﬁeld-based usability testing of mobile geo-applications, I. Delikostidis (2007) International Institute for Geo-Information Science and Earth Observation (Enschede, The Netherlands)

allows testing in the ﬁeld

but ... difﬁcult and time-consuming to set up intrusive, uncomfortable and heavy

2. screen-capture applications UsabilityExpo by Digia for S60 3rd and 5th ed, WinMo 6.1+ and Maemo 5 http://www.digia.com/C2256FEF0043E9C1/0/405001348

provides high quality screen recording

but no application willsupport all platformshttp://www.digia.com/C2256FEF0043E9C1/0/405001348

it’s not cheaphttps://shop.digia.com

and participants won’t appreciateyou installing stuff on their phones

3. document camerasHandheld Usability, S. Weiss (page 174)

Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)

Nielsen Norman Elmo TT-02RX Teachers Tool document camera with autofocus controlled remotely from a laptop records screen and ﬁngers webcam records both video streams participant’s face fed onto a laptop and recorded with Morae Photo from Nielsen Norman Mobile Usability workshop handout (London, 22 May 2009)

http://uxformobile.com	1845
http://blog.mobilefoo.com	273
http://www.diymobileusabilitytesting.net	273
http://www.temafrank.com	154
http://www.scoop.it	147
http://www.sitemotif.com	116
http://diymut.wordpress.com	70
http://cloudworks.ac.uk	57
http://library.iasummit.org	37
http://abtasty.com	17
http://lanyrd.com	17
http://paper.li	5
http://www.slideshare.net	5
http://speakerrate.com	5
http://twitter.com	4
url_unknown	4
http://kecke.wordpress.com	3
http://a0.twimg.com	2
http://translate.googleusercontent.com	2
http://frankonlinemarketing.com	1
http://webcache.googleusercontent.com	1

DIY mobile usability testing - IA Summit 2011

by Belen Barros Pena

Accessibility

Categories

Upload Details

Usage Rights

21 Embeds 3,038

Statistics

DIY mobile usability testing - IA Summit 2011 Presentation Transcript