SlideShare a Scribd company logo
1 of 115
Do it yourself mobile usability testing at EuroIA 2010
we are ...
Bernard, packet core engineer at NSN
Belén, interaction designer at LBi UK
the proverbial poll
how many have
   observed a mobile
usability testing session?
how many have
planned or moderated
mobile usability testing?
usability testing
a process that employs people as testing
participants who are representative of the
target audience to evaluate the degree to
which a product meets specific usability
criteria.
Handbook of usability testing 2nd Ed., J. Rubin and D. Chisnell
Do it yourself mobile usability testing at EuroIA 2010
why recording?

no need to rely on notes
and memory

a powerful communication
means
actions

          reactions
dut = mut
dut = mut + afec

where:

dut = desktop usability testing
mut = mobile usability testing
afec = a few extra challenges
which phone?

which context?

which connection?
which phone?

which context?

which connection?
web task success rates
         feature phones                               38%

           smartphones                                55%

          touch phones                                75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009
http://www.useit.com/alertbox/mobile-usability.html
web task success rates
         feature phones                               38%

           smartphones                                55%

          touch phones                                75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009
http://www.useit.com/alertbox/mobile-usability.html
web task success rates
         feature phones                               38%

           smartphones                                55%

          touch phones                                75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009
http://www.useit.com/alertbox/mobile-usability.html
web task success rates
         feature phones                               38%

           smartphones                                55%

          touch phones                                75%

Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009
http://www.useit.com/alertbox/mobile-usability.html
handset usability
affects test results
remember...

test with participants’ own
phones

if not possible, include
training and warm-up tasks
which phone?

which context?

which connection?
field vs. lab




It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field
C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
experts disagree
according to our study there was no
difference in the number of problems that
occurred in the two test settings. Our
hypothesis that more problems would be
found in the field was not supported
Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing
A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
experts disagree

evaluations conducted in field settings can
reveal problems not otherwise identified
in laboratory evaluations
It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field
C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
but they all agree
evaluations in the field (are) more
complex and time-consuming
It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field
C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006




                         testing in the field requires double the
                         time in comparison to the laboratory
                         Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing
                         A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
testing in the lab is
better than no testing
remember...

for most software, lab testing is fine
if you must do field testing...
     do it late
     plan and run pilot tests
     be prepared (like the Scouts)
which phone?

which context?

which connection?
http://www.coveragemaps.com/gsmposter_europe.htm
state of coverage: UK

mobile coverage for 2G is extensive at
98%. The picture for 3G coverage is
different. Figures are high overall (87%),
but vary markedly in the nations, with
only 43% availability in Northern Ireland
Mobile Evolution - Ofcom’s mobile sector assessment - 17 December 2009 (p35)
http://stakeholders.ofcom.org.uk/binaries/consultations/msa/statement/MSA_statement.pdf
Map of UK 3G geographic coverage by number of operators
Ofcom / GSM Association / Europa Technologies, Q1 2009
disable 3G and see
   what happens
remember...

do not test over wi-fi

cover participants’ data costs
tsdohoeaygtrtwt
dut = (mut + afec)

where:

dut = desktop usability testing
mut = mobile usability testing
afec = a few extra challenges
tsdohoeaygtrtwt = the small
detail of how on earth are you
going to record the whole thing
why recording?

no need to rely on notes
and memory

a powerful communication
means
4 approaches to the
small detail of how on earth
   are you   going to   record the   whole   thing
1. wearable equipment




Methods and techniques for field-based usability testing of mobile geo-applications, I. Delikostidis (2007)
International Institute for Geo-Information Science and Earth Observation (Enschede, The Netherlands)
allows testing in
    the field
but...


difficult and time-consuming set up
intrusive, uncomfortable and heavy
2. screen-capture applications




UsabilityExpo by Digia for S60 3rd and 5th ed, WinMo 6.1+ and Maemo OS2008
http://www.digia.com/C2256FEF0043E9C1/0/405001348
provides high quality
  screen recording
but no app will support
               all platforms




http://www.digia.com/C2256FEF0043E9C1/0/405001348
it’s not cheap




https://shop.digia.com
and participants
won’t appreciate
you installing
stuff on their
phones
3. document cameras




Handheld Usability, S. Weiss (page 174)
Google




Towards the Perfect Infrastructure for Usability Testing on Mobile Devices, R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)
Nielsen Norman



                                                                     Elmo TT-02RX Teachers Tool




       Photo from Nielsen Norman Mobile Usability workshop handout (London, 22 May 2009)
Nielsen Norman

                                   document camera with autofocus
                                   controlled remotely from a laptop
                                   records screen and fingers




  webcam records                                                                 both video streams
  participant’s face                                                             fed onto a laptop and
                                                                                 recorded with Morae




              Photo from Nielsen Norman Mobile Usability workshop handout (London, 22 May 2009)
Foolproof




            http://www.foolproof.co.uk/our-services/user-experience/facilities/
popular and well
  documented
but it’s not cheap, and...




http://www.bhphotovideo.com/c/product/644591-REG/Elmo_1304.html
participants must
keep within the
camera range
phone must lay
         on a desk or be
         hold at a flat angle




http://www.flickr.com/photos/zabriskiepoint/2806511301/sizes/m/in/photostream/
4. mounted devices (a) ready made




                                                  Mobile Device Camera by Tracksys
                                                  http://www.tracksys.co.uk/product-
                                                  details.php?id=9
     Mobile Device Camera by Noldus
     http://www.noldus.com/human-behavior-
     research/accessories/mobile-device-camera-
     mdc#
4. mounted devices (b) DIY




                                                             Nick Bowmast http://www.bowmast.com/mob-device-cam/

Little Springs Design
http://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-
for-mobile-devices-2/
4. mounted devices (b) DIY




     Google
     Towards the Perfect Infrastructure
                                            Usability Sciences
     for Usability Testing on Mobile
                                            http://www.usabilitysciences.com/services/lab-
     Devices, R. Schusteritsch, C.Y. Wei,
                                            based-usability-testing/mobile-usability-testing
     M. LaRosa - Google (CHI 2007)
natural interaction with
       the phone
but not cheap to buy




http://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
messy to build




http://www.littlespringsdesign.com/blog/2008/Jun/usability-testing-for-mobile-devices-2/
and...

if bulky they can prevent single-
hand use
if heavy they can become tiring
during long tests
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
1   2   3   4a   4b
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
1   2   3   4a   4b
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
1   2   3   4a   4b
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
the spirit




http://www.wired.com/gadgetlab/2010/07/the-pencil-ipad-stand-smart-enough-to-impress-a-new-yorker/
the
ingredients
two meccano trunions (part no. 126)
two 5 and 6-hole meccano strips (part nos. 5 & 4)
one 11-hole meccano strip (part no. 2)
six meccano screws & nuts (part nos. 69 & 37h)
a 13-20mm jubilee clip
one HUE HD webcam
a second USB webcam
a USB male to female extension cable
blu tack
an allen key
a meccano wrench
a screwdriver
a Windows computer (or virtual machine)
CamStudio (free and open source)
Do it yourself mobile usability testing at EuroIA 2010
Do it yourself mobile usability testing at EuroIA 2010
Do it yourself mobile usability testing at EuroIA 2010
a fake (but realistic)
impromptu usability test
task 1

Your friend from Dublin is flying to
London tonight. He is a cheap dude, so he
must be flying Ryanair. He hasn’t told you
which airport he is flying to, but you know
he should be landing at 12:45 in the
afternoon. Go to www.ryanair.com and
find out if his flight is on time.
task 2

Your friend from Dublin is flying to
London tonight. He is a posh dude, so he
must be flying BA. He hasn’t told you
which airport he is flying to, but you know
he should be landing at 11:10 in the
morning. Go to www.ba.com and find out
if his flight is on time.
how was that?
Do it yourself mobile usability testing at EuroIA 2010
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
15-model meccano set                 18.33
Hue HD webcam                        28.94
Philips webcam                       52.08
additional meccano parts              4.90
blu tack                              0.98
jubilee clips (x2)                    1.55
USB cable                             8.99
screwdriver                           5.99
CamStudio                             0.00
                     total (in GBP) 121.76
in Euro: 143.429
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
Do it yourself mobile usability testing at EuroIA 2010
Do it yourself mobile usability testing at EuroIA 2010
Do it yourself mobile usability testing at EuroIA 2010
Do it yourself mobile usability testing at EuroIA 2010
Do it yourself mobile usability testing at EuroIA 2010
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
weight: 125 grams
 an iPhone weighs 137 grams
 an iPad weighs 680 grams
 I weigh 55,000 grams
 a blue whale weighs 136,400,000 grams
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
Nokia E66
slider with 12-key
numeric keypad
Nokia E71

      BB Bold
      candybar with
      qwerty keyboard
iPhone 3G
touchscreen
candybar
Nokia N900

             slider
             touchscreen
             with qwerty
             keyboard
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
Do it yourself mobile usability testing at EuroIA 2010
Do it yourself mobile usability testing at EuroIA 2010
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
Do it yourself mobile usability testing at EuroIA 2010
b&b’s
easy to put together
cheap
repeatable
allows holding the device
allows one-handed hold
supports all form factors
captures screen, face & fingers
runs tests with participants’ phones
gives enough video quality
a call to arms




an Irish unit of the International Brigade   http://www.communistpartyofireland.ie/spainn.jpg
thanks!

   @belenpena @chistorra

 http://belenpena.posterous.com
http://www.runningwithbulls.com

More Related Content

Viewers also liked

Usability vs. User Experience: What's the difference?
Usability vs. User Experience: What's the difference?Usability vs. User Experience: What's the difference?
Usability vs. User Experience: What's the difference?Domain7
 
A simple mobile testing lab using our Meccano home-made rig
A simple mobile testing lab using our Meccano home-made rigA simple mobile testing lab using our Meccano home-made rig
A simple mobile testing lab using our Meccano home-made rigBelen Barros Pena
 
Cheap And Efficient Tools Euroia09
Cheap And Efficient Tools Euroia09Cheap And Efficient Tools Euroia09
Cheap And Efficient Tools Euroia09Belen Barros Pena
 
DIY Mobile Usability Testing - SXSW Interactive 2012
DIY Mobile Usability Testing - SXSW Interactive 2012DIY Mobile Usability Testing - SXSW Interactive 2012
DIY Mobile Usability Testing - SXSW Interactive 2012Belen Barros Pena
 
UX para desenvolvedores - QCon Rio 2015
UX para desenvolvedores - QCon Rio 2015UX para desenvolvedores - QCon Rio 2015
UX para desenvolvedores - QCon Rio 2015Samantha Rosa
 
Usability in the GeoWeb
Usability in the GeoWebUsability in the GeoWeb
Usability in the GeoWebDave Bouwman
 
Mobile Usability Evaluation
Mobile Usability EvaluationMobile Usability Evaluation
Mobile Usability EvaluationGarrett Stettler
 
Handbook Usability Testing - Capitulo 13
Handbook Usability Testing - Capitulo 13Handbook Usability Testing - Capitulo 13
Handbook Usability Testing - Capitulo 13Luiz Agner
 
Android App Development 02 : Activity & Layout
Android App Development 02 : Activity & LayoutAndroid App Development 02 : Activity & Layout
Android App Development 02 : Activity & LayoutAnuchit Chalothorn
 
WVRHC Desktop Usability Testing
WVRHC Desktop Usability TestingWVRHC Desktop Usability Testing
WVRHC Desktop Usability TestingTim Broadwater
 
Android2 composant-layout-menu (1)
Android2 composant-layout-menu (1)Android2 composant-layout-menu (1)
Android2 composant-layout-menu (1)Chaikhani Ibtissam
 
Usability & Usability Testing
Usability  & Usability TestingUsability  & Usability Testing
Usability & Usability Testingrebrennan
 
Introduction to Web Survey Usability Design and Testing
Introduction to Web Survey Usability Design and TestingIntroduction to Web Survey Usability Design and Testing
Introduction to Web Survey Usability Design and TestingJennifer Romano Bergstrom
 

Viewers also liked (19)

Usability vs. User Experience: What's the difference?
Usability vs. User Experience: What's the difference?Usability vs. User Experience: What's the difference?
Usability vs. User Experience: What's the difference?
 
UX Usability Heuristics
UX Usability HeuristicsUX Usability Heuristics
UX Usability Heuristics
 
A simple mobile testing lab using our Meccano home-made rig
A simple mobile testing lab using our Meccano home-made rigA simple mobile testing lab using our Meccano home-made rig
A simple mobile testing lab using our Meccano home-made rig
 
Cheap And Efficient Tools Euroia09
Cheap And Efficient Tools Euroia09Cheap And Efficient Tools Euroia09
Cheap And Efficient Tools Euroia09
 
DIY Mobile Usability Testing - SXSW Interactive 2012
DIY Mobile Usability Testing - SXSW Interactive 2012DIY Mobile Usability Testing - SXSW Interactive 2012
DIY Mobile Usability Testing - SXSW Interactive 2012
 
UX para desenvolvedores - QCon Rio 2015
UX para desenvolvedores - QCon Rio 2015UX para desenvolvedores - QCon Rio 2015
UX para desenvolvedores - QCon Rio 2015
 
Welding shop
Welding shopWelding shop
Welding shop
 
Usability in the GeoWeb
Usability in the GeoWebUsability in the GeoWeb
Usability in the GeoWeb
 
Mobile Usability Evaluation
Mobile Usability EvaluationMobile Usability Evaluation
Mobile Usability Evaluation
 
Handbook Usability Testing - Capitulo 13
Handbook Usability Testing - Capitulo 13Handbook Usability Testing - Capitulo 13
Handbook Usability Testing - Capitulo 13
 
Android App Development 02 : Activity & Layout
Android App Development 02 : Activity & LayoutAndroid App Development 02 : Activity & Layout
Android App Development 02 : Activity & Layout
 
WVRHC Desktop Usability Testing
WVRHC Desktop Usability TestingWVRHC Desktop Usability Testing
WVRHC Desktop Usability Testing
 
Android2 composant-layout-menu (1)
Android2 composant-layout-menu (1)Android2 composant-layout-menu (1)
Android2 composant-layout-menu (1)
 
Usability & Usability Testing
Usability  & Usability TestingUsability  & Usability Testing
Usability & Usability Testing
 
Usability Tests
Usability TestsUsability Tests
Usability Tests
 
Intranet Usability Testing
Intranet Usability TestingIntranet Usability Testing
Intranet Usability Testing
 
Introduction to Web Survey Usability Design and Testing
Introduction to Web Survey Usability Design and TestingIntroduction to Web Survey Usability Design and Testing
Introduction to Web Survey Usability Design and Testing
 
Carpentry shop
Carpentry shopCarpentry shop
Carpentry shop
 
Machine tools
Machine toolsMachine tools
Machine tools
 

Recently uploaded

COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 

Recently uploaded (20)

COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 

Do it yourself mobile usability testing at EuroIA 2010

  • 3. Bernard, packet core engineer at NSN
  • 6. how many have observed a mobile usability testing session?
  • 7. how many have planned or moderated mobile usability testing?
  • 8. usability testing a process that employs people as testing participants who are representative of the target audience to evaluate the degree to which a product meets specific usability criteria. Handbook of usability testing 2nd Ed., J. Rubin and D. Chisnell
  • 10. why recording? no need to rely on notes and memory a powerful communication means
  • 11. actions reactions
  • 13. dut = mut + afec where: dut = desktop usability testing mut = mobile usability testing afec = a few extra challenges
  • 16. web task success rates feature phones 38% smartphones 55% touch phones 75% Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009 http://www.useit.com/alertbox/mobile-usability.html
  • 17. web task success rates feature phones 38% smartphones 55% touch phones 75% Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009 http://www.useit.com/alertbox/mobile-usability.html
  • 18. web task success rates feature phones 38% smartphones 55% touch phones 75% Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009 http://www.useit.com/alertbox/mobile-usability.html
  • 19. web task success rates feature phones 38% smartphones 55% touch phones 75% Mobile usability, J. Nielsen’s Alertbox 20 Jul 2009 http://www.useit.com/alertbox/mobile-usability.html
  • 21. remember... test with participants’ own phones if not possible, include training and warm-up tasks
  • 23. field vs. lab It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
  • 24. experts disagree according to our study there was no difference in the number of problems that occurred in the two test settings. Our hypothesis that more problems would be found in the field was not supported Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
  • 25. experts disagree evaluations conducted in field settings can reveal problems not otherwise identified in laboratory evaluations It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006
  • 26. but they all agree evaluations in the field (are) more complex and time-consuming It’s Worth the Hassle! The Added Value of Evaluating the Usability of Mobile Systems in the Field C.M. Nielsen, M. Overgaard, M.B. Pedersen, J. Stage, S. Stenild - NordiCHI 2006 testing in the field requires double the time in comparison to the laboratory Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing A. Kaikkonen, T. Kallio, A. Kekäläinen, A. Kankainen, M. Cankar - Journal of Usability Studies, 2005
  • 27. testing in the lab is better than no testing
  • 28. remember... for most software, lab testing is fine if you must do field testing... do it late plan and run pilot tests be prepared (like the Scouts)
  • 31. state of coverage: UK mobile coverage for 2G is extensive at 98%. The picture for 3G coverage is different. Figures are high overall (87%), but vary markedly in the nations, with only 43% availability in Northern Ireland Mobile Evolution - Ofcom’s mobile sector assessment - 17 December 2009 (p35) http://stakeholders.ofcom.org.uk/binaries/consultations/msa/statement/MSA_statement.pdf
  • 32. Map of UK 3G geographic coverage by number of operators Ofcom / GSM Association / Europa Technologies, Q1 2009
  • 33. disable 3G and see what happens
  • 34. remember... do not test over wi-fi cover participants’ data costs
  • 35. tsdohoeaygtrtwt dut = (mut + afec) where: dut = desktop usability testing mut = mobile usability testing afec = a few extra challenges tsdohoeaygtrtwt = the small detail of how on earth are you going to record the whole thing
  • 36. why recording? no need to rely on notes and memory a powerful communication means
  • 37. 4 approaches to the small detail of how on earth are you going to record the whole thing
  • 38. 1. wearable equipment Methods and techniques for field-based usability testing of mobile geo-applications, I. Delikostidis (2007) International Institute for Geo-Information Science and Earth Observation (Enschede, The Netherlands)
  • 39. allows testing in the field
  • 40. but... difficult and time-consuming set up intrusive, uncomfortable and heavy
  • 41. 2. screen-capture applications UsabilityExpo by Digia for S60 3rd and 5th ed, WinMo 6.1+ and Maemo OS2008 http://www.digia.com/C2256FEF0043E9C1/0/405001348
  • 42. provides high quality screen recording
  • 43. but no app will support all platforms http://www.digia.com/C2256FEF0043E9C1/0/405001348
  • 45. and participants won’t appreciate you installing stuff on their phones
  • 46. 3. document cameras Handheld Usability, S. Weiss (page 174)
  • 47. Google Towards the Perfect Infrastructure for Usability Testing on Mobile Devices, R. Schusteritsch, C.Y. Wei, M. LaRosa - Google (CHI 2007)
  • 48. Nielsen Norman Elmo TT-02RX Teachers Tool Photo from Nielsen Norman Mobile Usability workshop handout (London, 22 May 2009)
  • 49. Nielsen Norman document camera with autofocus controlled remotely from a laptop records screen and fingers webcam records both video streams participant’s face fed onto a laptop and recorded with Morae Photo from Nielsen Norman Mobile Usability workshop handout (London, 22 May 2009)
  • 50. Foolproof http://www.foolproof.co.uk/our-services/user-experience/facilities/
  • 51. popular and well documented
  • 52. but it’s not cheap, and... http://www.bhphotovideo.com/c/product/644591-REG/Elmo_1304.html
  • 53. participants must keep within the camera range
  • 54. phone must lay on a desk or be hold at a flat angle http://www.flickr.com/photos/zabriskiepoint/2806511301/sizes/m/in/photostream/
  • 55. 4. mounted devices (a) ready made Mobile Device Camera by Tracksys http://www.tracksys.co.uk/product- details.php?id=9 Mobile Device Camera by Noldus http://www.noldus.com/human-behavior- research/accessories/mobile-device-camera- mdc#
  • 56. 4. mounted devices (b) DIY Nick Bowmast http://www.bowmast.com/mob-device-cam/ Little Springs Design http://www.littlespringsdesign.com/blog/2008/Jun/usability-testing- for-mobile-devices-2/
  • 57. 4. mounted devices (b) DIY Google Towards the Perfect Infrastructure Usability Sciences for Usability Testing on Mobile http://www.usabilitysciences.com/services/lab- Devices, R. Schusteritsch, C.Y. Wei, based-usability-testing/mobile-usability-testing M. LaRosa - Google (CHI 2007)
  • 59. but not cheap to buy http://www.90percentofeverything.com/2010/05/07/quick-tip-make-your-own-iphone-usability-testing-sled-for-5/
  • 61. and... if bulky they can prevent single- hand use if heavy they can become tiring during long tests
  • 62. easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 63. 1 2 3 4a 4b easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 64. 1 2 3 4a 4b easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 65. 1 2 3 4a 4b easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 68. two meccano trunions (part no. 126)
  • 69. two 5 and 6-hole meccano strips (part nos. 5 & 4)
  • 70. one 11-hole meccano strip (part no. 2)
  • 71. six meccano screws & nuts (part nos. 69 & 37h)
  • 73. one HUE HD webcam
  • 74. a second USB webcam
  • 75. a USB male to female extension cable
  • 80. a Windows computer (or virtual machine)
  • 81. CamStudio (free and open source)
  • 85. a fake (but realistic) impromptu usability test
  • 86. task 1 Your friend from Dublin is flying to London tonight. He is a cheap dude, so he must be flying Ryanair. He hasn’t told you which airport he is flying to, but you know he should be landing at 12:45 in the afternoon. Go to www.ryanair.com and find out if his flight is on time.
  • 87. task 2 Your friend from Dublin is flying to London tonight. He is a posh dude, so he must be flying BA. He hasn’t told you which airport he is flying to, but you know he should be landing at 11:10 in the morning. Go to www.ba.com and find out if his flight is on time.
  • 90. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 91. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 92. 15-model meccano set 18.33 Hue HD webcam 28.94 Philips webcam 52.08 additional meccano parts 4.90 blu tack 0.98 jubilee clips (x2) 1.55 USB cable 8.99 screwdriver 5.99 CamStudio 0.00 total (in GBP) 121.76
  • 94. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 100. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 101. weight: 125 grams an iPhone weighs 137 grams an iPad weighs 680 grams I weigh 55,000 grams a blue whale weighs 136,400,000 grams
  • 102. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 103. Nokia E66 slider with 12-key numeric keypad
  • 104. Nokia E71 BB Bold candybar with qwerty keyboard
  • 106. Nokia N900 slider touchscreen with qwerty keyboard
  • 107. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 110. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 111. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 113. b&b’s easy to put together cheap repeatable allows holding the device allows one-handed hold supports all form factors captures screen, face & fingers runs tests with participants’ phones gives enough video quality
  • 114. a call to arms an Irish unit of the International Brigade http://www.communistpartyofireland.ie/spainn.jpg
  • 115. thanks! @belenpena @chistorra http://belenpena.posterous.com http://www.runningwithbulls.com