• Save
Mobile Usability Testing
Upcoming SlideShare
Loading in...5
×
 

Mobile Usability Testing

on

  • 395 views

Overview of usability testing methods for mobile devices and apps. Includes information on usability, explanation of challenges introduced by the mobile context, and practical tools and techniques.

Overview of usability testing methods for mobile devices and apps. Includes information on usability, explanation of challenges introduced by the mobile context, and practical tools and techniques.

Statistics

Views

Total Views
395
Slideshare-icon Views on SlideShare
393
Embed Views
2

Actions

Likes
2
Downloads
0
Comments
0

1 Embed 2

https://twitter.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Mobile Usability Testing Mobile Usability Testing Presentation Transcript

    • Mobile Usability Testing Garrett Stettler AT&T Labs April 2, 2014 Presented to U.T. Austin School of Information, INF 385T
    • MSIS 2010 @gstett AT&T Labs Human Factors AT&T Foundry UX & Design Research Garrett Stettler
    • Preview ● Definition of “usability” ● Testing for usability ● Mobile challenges ● Tools and techniques ● Examples
    • “The user can do what he or she wants to do the way he or she expects to be able to do it, without hindrance, hesitation, or questions.”1 Defining “usability” 1. Rubin, J. & Chisnell, D. Handbook of Usability Testing, 2nd ed. Efficiency Effectiveness Satisfaction Discoverability Learnability Memorability ISO 9241 More: Dimensions of Usability
    • UCD and Experience Design Useful Usable Accessible Desirable Emotional Valuable Adoptable Social etc... Usability Testing User-Centered Design Experience Design
    • Usability Inspection Heuristic Evaluation Cognitive Walkthrough Pluralistic Walkthrough / Design Review
    • Usability Testing ● “Early and Often” ● Iterate ● Keep your goals in mind when choosing methods & metrics
    • Formative vs. Summative Formative Problem-finding during development Summative Measure against specific goals when development is complete In practice there isn’t always a clear distinction, and you should use quantitative metrics in both types.
    • Usability Metrics Objective and Subjective Task Completion Task Time Task Satisfaction Errors Perceived Difficulty Overall Satisfaction Expected Difficulty Clicks Conversion Rate UI Problems (Severity, Frequency) More: Making usability metrics count
    • Participants ● Many ways to find participants ● Magic Number 5 (J. Nielsen)... not magic ● Use more frequent tests (with smaller sample sizes) when possible. More: A Brief History of the Magic Number 5
    • Usability Testing Methods Lab-based Remote Moderated Remote Unmoderated Art: Todd Zazelenchuk, Elizabeth Boling
    • The Challenges of Mobile
    • Photo: Marcelo Druck (horizontal flip) Context
    • Photo: Kathy Cassidy
    • Photo: David van der Mark
    • Photo: Wesley Fryer
    • Photo: Ed Yourdon
    • Photo: Daniel Hoherd
    • Test in lab or in field? Disagreement among practitioners and in literature. Kjeldskov, Skov, et al (2004) Kakkonen, Kallio, et al (2005) Nielsen, Overgaard, et al (2006) Duh, Tan, et al (2006) Same UI problems identified in both Field study costly More/different UI issues found in field Field study costly “A lab experiment is recommended when the testing focus is on the user interface and application-oriented usability related issues. However, the results suggest that a field experiment is more suitable for investigating a wider range of factors affecting the overall acceptability of the designed mobile service.” Sun & May (2013)
    • Test in lab or in field? Field testing is getting easier and cheaper, but depending on your goals and circumstances a lab study may still be the best choice. Given the particulars of your situation, it’ s up to YOU to be pragmatic and determine the best test setting.
    • Photo: James Nash Devices
    • Use participant’s device? Using the participant’s device adds to realism, but introduces extra variables. Participants may be uncomfortable if software must be installed or modified. If not using the participant’s device, you may need to include a short training session or warm-up tasks. It depends...
    • Photo: Ikhlasul Amal Connectivity
    • Control connection? Regardless of whether or not it makes sense to reduce connectivity variables during your usability testing, your design MUST consider connection conditions in the field and you should test it under varying conditions. Not all cellular or WiFi connections are created equal, especially in congested settings. Location isn’t determined just by GPS, e.g., what happens if the WiFi radio is turned off? It depends...
    • Photo: City University Interaction Lab Recording
    • Why Record? Memory Aid - events happen quickly! Powerful way to present UI issues to stakeholders Ability for detailed analysis later
    • Recording Mobile (This photo is unfair, but recording mobile devices IS relatively tricky compared to PCs.)
    • Usability Testing Methods Lab-based Remote Moderated Remote Unmoderated Art: Todd Zazelenchuk, Elizabeth Boling
    • Recording - Screen Capture Easily view and record screen Some can capture gestures and participant’s face Pros Cons Most are not cross-platform or have other limitations Some have recurring costs Recording time can be limited by space Many don’t capture gestures or participant’s face
    • Recording - Screen Capture UX Recorder Records on-screen activity, gestures, front camera iOS only, websites only $60 or $2 per session AirPlay, Reflector app Mirror iOS to Mac or PC No gesture or front camera support Android apps (various) Mirror to Mac or PC No gesture or front camera support Magitest iOS only, can use native apps with some work $25
    • Recording - Cameras Flexible, device-agnostic Can be relatively inexpensive over time Better view of participant’s gestures and movement Pros Cons Need to focus and position camera User’s fingers can obscure view Need TWO cameras to record user’s face
    • Recording - Cameras Document Cameras and Desk Mounts
    • Recording - Cameras Problems with Document Cameras Fixed (participant cannot hold and move the device) Angle must usually remain constant Focus is usually fixed
    • Recording - Cameras Sleds - many inexpensive / DIY options
    • Recording - Cameras Problems with Sleds Can be heavy or otherwise unwieldy (e.g., unbalanced) Camera can get in participant’s way Can be difficult to rotate device (portrait <-> landscape) Do not always accommodate different device sizes
    • Recording - Cameras Free-standing / mounted cameras
    • Recording - Cameras Problems with Mounted Cameras User may block camera Lighting, glare, and focus can be a problem
    • Multi-Camera Human Factors Lab
    • DIY “Guerilla” Mobile Testing Lab ● Two laptops (1 with webcam) ● Device sled with camera ● Screen capture software (Camtasia, Silverback, Morae) ● WebEx
    • You don’t always need to record. You don’t always need to go high-tech.
    • Test Early! Test Often! Draw screens on Post-It notes Rubber-band screen shots to an actual phone Cardboard and adding-machine tape POP app (Prototyping on Paper) for iOS and Android
    • Usability Testing Methods Lab-based Remote Moderated Remote Unmoderated Art: Todd Zazelenchuk, Elizabeth Boling
    • Remote Moderated Testing UserTesting.com GoToAssist (Samsung, BlackBerry) GoToMeeting (iPad websites only) Still quite challenging for mobile MailChimp “hug” technique
    • Usability Testing Methods Lab-based Remote Moderated Remote Unmoderated Art: Todd Zazelenchuk, Elizabeth Boling
    • Remote UNmoderated Userzoom Usabilla Loop11 (to name a few) Lots of options Unmoderated remote testing can be a time- and cost-effective way to test a mobile app or website.
    • Recording - Wearable Someday we may be able to conduct true in-context testing without a rig like this: Delikostidis, 2007
    • Recording - Wearable GoPro: (Google Glass project, anyone?) Head-mounted camera
    • Recording - Eye Tracking
    • Examples (removed)
    • Now you try! Task-Based Usability Study Find some participants (food & beer are good incentives) Setup a “lab”. Whatever you can manage. Create a set of tasks. Intro to users. (Consent. Recording. “We’re not testing you.” Survey?) Run through tasks (randomize/counterbalance). Think-aloud protocol. Capture issues, errors, comments, metrics (e.g., success), path After all tasks, survey? Satisfaction (System Usability Scale) and/or other questions.
    • Summary ● Definition of “usability” ● Testing for usability ● Mobile challenges ● Tools and techniques ● Examples
    • Questions?