• Save
Making The Leap From Web To Mobile
Upcoming SlideShare
Loading in...5

Making The Leap From Web To Mobile



Best Practices in Mobile User Experience Research

Best Practices in Mobile User Experience Research

Amy Buckner, AnswerLab
Kris Mihalic, Yahoo!
UPA 2009, June 12, Portland



Total Views
Views on SlideShare
Embed Views



6 Embeds 22

http://paper.li 12
http://answerlab.com 4
http://www.slideshare.net 3
http://suikris.wordpress.com 1
http://www.lmodules.com 1
http://us-w1.rockmelt.com 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Making The Leap From Web To Mobile Making The Leap From Web To Mobile Presentation Transcript

    • Making the Leap from Web to Mobile
      Best Practices in Mobile User Experience Research
      Amy Buckner, AnswerLab
      Kris Mihalic, Yahoo!
      UPA 2009, June 12, Portland
    • Agenda
    • Introductions
    • State of Mobile
    • There are mobile phone subscribers worldwide
      4.1 Billion
      Only of US mobile phones are iPhones (~5 Million)
      Social networking on mobile phones is growing Y/Y
      There are more mobile IM users than mobile business email users in the US
    • Mobile Market Summary
      Globally, there are 800 million cars, 850 million PCs, 1.3 billion fixed landline phones, 1.4 billion credit cards, 1.5 billion TV sets – and 2.7 billion mobile phones – in use. (Experian 2008)
      eMarketer projects that worldwide spending on mobile advertising will reach a total of $19.1 billion in 2012 – up from 4.6 billion in 2008.
      Source: Experian 2008, eMarketer 2008
    • U.S. Mobile Web Users Growing Rapidly
      Source: m:metrics/comScore, 2009
    • Mobile Audience Size (# of users)
      In the U.S., the size of the Mobile Audience is 250 Million
      Source: m:metrics/comScore, 2009
    • U.S. Mobile Internet Usage
      Mobile Internet users are heavy users
      60% access the Internet from their phone at least once/day
      48% access the Internet more than once per day
      Source: TSM|TargetProfile 2007
    • Device Traffic
      Mobile phones dominate mobile Web traffic with 66%market share (US).
      BlackBerry and Smartphones (Windows Mobile, Palm, etc.) are heavy users of mobile Internet – combined 26% of traffic
      iPhone/iPod Touch has shown tremendous growth, capturing 8% of the mobile Internet market in the US
      Source: m:metrics/comScore, 2009
    • Widgets Improve Mobile Internet Experience
      Source: ‘Widgets Improve the Convenience of the Mobile Internet,’ Forrester, 2008
    • Group Exercise
    • Take out your cell phone and search for times that ‘Star Trek’ is playing in your neighborhood.
      Raise your hand when you have finished the task.
    • Discussion Points
      How long did it take?
      How did you do it? (web, app, SMS?)
      What problems, if any, did you have?
    • Mobile User Research Methods
    • Overview of Research Methods
      Quick and Dirty feedback
      Lab Studies
      Field Studies
      Online Surveys
      Basic Survey
      Survey with Behavioral Tracking (Keynote on iPhone only)
      Quantitative Behavioral Analysis
    • Quick and Dirty – What are obvious problems?
      Informal setting with Lo-Fi prototypes
      Fast iteration cycles
      Brain-damage check
      Finds critical usability issues in shortest time
      Insider/stakeholder view
      Anecdotal, unstructured results
      Small sample size
      Difficult to convey real usage scenarios
      Cross-functional team feedback
      Internal alpha/beta
    • Lab Studies – Can they use it?
      Allows for probing by moderator
      Can record interactions
      Best for prototype testing
      Allows for real-time viewing of interaction – team engagement
      Artificial environment
      Moderator bias; participant ‘pleasing’
      Small sample size
      Difficult to test all hardware / network scenarios
      Usability study
      Participatory design sessions
    • Field Studies – What is the context of usage?
      Assess usage under real conditions
      Candid user feedback
      Delivers unknown use-cases
      Small sample size
      Difficult to observe and probe
      Participant engagement difficult
      Requires robust product
      Diary study
      Community study
    • Online Surveys – What do they think of it?
      Can deliver statistically valid results
      Provides qualitative and/or quantitative data
      Geographical diversity (remote)
      Broad representation of devices
      Difficult to observe and probe
      Out-of-context interaction, esp. with desktop survey
      Technical constraints
      Desktop surveys
      On-device survey
      Survey with behavioral tracking
    • Behavioral Analysis – How do they use it?
      Reflects what people do, not say
      Statistically valid results
      Shows all behaviors, rather than those confined to a single task
      Probing / deep-dive difficult
      Context and intent unknown
      Attitudes and perceptions unknown
      Log data mining
    • Types of Prototypes
      Paper prototypes
      Paper and pencil
      Interactive prototypes
      Native, e.g. iPhone
      Simple HTML prototype
      High-fidelity prototype (with session variables)
    • Team Exercise & Case Study
    • Yahoo! Go
      Mobile Application
      Other Topics
    • Y! oneSearch integrated into Y! Go
    • Business Issue
      Drive mobile search uptake – Improved user experience can help accelerate growth of the service
      Improve relevancy – Understanding search intent can contribute to delivering more relevant results
      Product differentiation – New approach to search results page can deliver higher value to customers
    • Research Objectives
      Understand mobile search behavior in users’ daily lives
      Identify content users seekwhen conducting mobile searches
      Assess the context surrounding mobile searches
      Evaluate effectiveness of new product
      Identify opportunities to improve the user experience
    • Form into teams of 4.
      Develop a research plan for Yahoo! Go.
      You have 10 minutes.
    • Research Objectives
      Team Activity
      Understand mobile search behavior in users’ daily lives
      Identify content users seekwhen conducting mobile searches
      Assess the context surrounding mobile searches
      Evaluate effectiveness of new product
      Identify opportunities to improve the user experience
      Develop a research plan for Yahoo! Go
      • Recommended method(s)
      • Number of participants
      • Profile of participants
    • Discussion Points
      What methods do you recommend & why?
      What are your anticipated concerns?
      Any potential limitations?
    • Our Solution: One-Month Field Study (1)
      Daily Mobile Survey
      Digital Photos
      Voicemails & Pocket Card
      Daily SMS mini-survey
      Linked to phone number for voicemail
      Users sent photos of themselves or surroundings in the context of using oneSearch
      9 users
      Daily voicemails
      Laminated pocket card with key questions
    • ‘Pocket Card’ Questionnaire
    • Sample Voicemail Diary Messages -- Winchelle
      Friday 7:39pm
      I used oneSearch today, was looking for a hospital – directions and address to it. I was getting off work and sitting in my car, trying to find directions. It was about 4:30. I liked that it came up with the website and with information about that. And the directions on how to go.
      I didn’t like that it could not find my work address – which is saved on my Yahoo! directions (recently gone) saved places.
      I think using the actual internet through the PC will help me better because it knows where the address is. I’m not sure why the address is not coming up in my oneSearch and that sucked.
      Saturday 8:06pm
      I didn’t use oneSearch today. I looked for info, looked for a store and I was at home. I wanted to know if the store was open STORE HOURS (9:45am). I used the internet with my phone and used Google to search. It was not very helpful because I wanted to see what time they opened and I couldn’t find it. I’ve used oneSearch in the past and I guess the results were the same but I guess it’d be nice to go to the site for that particular store.
      Monday 10:48pm
      I used oneSearch to find Expedia, was at work on my lunch break. It was about 12:20. I liked that I can find it quickly. I guess the website didn’t support my mobile phone though, so I didn’t like that. Would rather book a flight/hotel at home because its supported.
    • Our Solution: One-Month Field Study (2)
      Mid-Check Interview
      Wrap-up Interview
      An initial 45-minute interview
      Verified technical capabilities of phone
      Explained research program
      20-minute check-in phone interview
      Initial feedback
      Final in-person interview of 75 minutes
      Clarified voicemail reports
      Overall impressions of the product from the month-long usage
    • Screening Criteria: 9 Experienced Mobile Users
      Have an unlimited data plan for WAP services through their mobile carrier
      Use their mobile for text messages (SMS) daily and use mobile more than three times per week for activities that require the transmission of online data
      Have Cingular, Sprint, T-Mobile, or Verizon as their mobile carrier
      Have a primary mobile device capable of downloading and using Y! Go
      Live or work within San Francisco
      Mix of gender, age (23-46+),and typical method of commuting to work (4 users primarily drive, 5 primarily take mass transit)
    • Sliding Scale for Participant Incentives
      $100 for the initial in-person interview
      $50 for the mid-way phone interview
      $150 for the final in-person interview
      $3 each day a report is given
    • Natural environment key to context
      All research activities on the mobile device
      Mix of methods
      Engage users in program
      Breadth of insights – context, usability, impressions
      Why did we choose this method?
    • What did we learn? (1)
      Perceived value increased with usage
      Usage typically driven by lack of computer availability
      However, certain scenarios drove phone choice over computer:
      Social setting
    • What did we learn? (2)
      Mental model based on computer usage
      Speed and relevance of search results highest area of frustration
      Lack of understanding that results were customized by widget
    • Daily SMS Survey Results
      *Website not adapted for mobile device
    • What did we learn? (3)
    • Summary of Use Case Frequency
      Note: The 166 searches were conducted by 9 users
    • Shopping
      Business’ number, location, directions, and hours of operation
      “While driving home from work, I did a search for Fry’s electronics on the local search and it came up with all the different Fry’s in the area, their numbers, as well as directions and all that. The results were all helpful and fast in helping me find the one closest.”
      “Since we were already out shopping, I wanted to find where the closest Citibank was to where we were.”
      Pricing/Product comparisons
      “I was at Costco and comparing TV prices. I wanted to know how much these same TV’s would be at the Circuit City down the street.”
    • Dining
      A restaurant’s number, location, directions, and hours of operation
      “I just woke up and didn’t want to boot my computer up but I wanted to find out what time this breakfast place opened so I could know how soon I could get there.”
      “I was in my car and wanted their number so I could call and place an order to pick it up on my way home from work.”
      Ideas of places to eat
      “I was with my wife in the Haight and we wanted to find nearby restaurants so we could decide what we wanted to eat that was within walking distance.”
      “I was in the South Bay with some friends and we wanted to find a good sushi place nearby.”
      Reservations & Reviews
      “I was hoping to make reservations straight from the phone, or at least call them to place them.”
      “I was out with some friends and curious about whether the place nearby was any good.”
    • Traveling
      Transit schedules and numbers
      “I wanted to find out when the next MUNI but the MUNI’s site wouldn’t work on my phone.”
      “I wanted the number to call a taxi. I was outside and tired of waiting for the bus.”
      Tourist activities
      “I wanted to see what kind of ferry tours we could do before looking for another museum that would be fun for my daughter.”
      “I was looking for fun things to do while at Hermosa beach.”
      Maps and directions
      “I was trying to get a map of the Mt. Shasta area and I wanted to plan which route to take. It would be nice if the driving directions had live updates on traffic and road conditions.”
      Flight status / check-in
      “I was in the taxi to the airport and wanted to check my flight’s status.”
    • Business Implications
      Drive mobile search uptake – Distribution through partnerships, marketing campaigns, opening to third party services and developers
      Improve relevancy – Focused on improving results for specific use cases (e.g. local businesses, flights, etc.)
      Product differentiation – Federated search results, integrated user experience across multiple services (e.g. search results + maps)
    • Mobile Usability Research Challenges & Tips
    • Variety of Mobile Devices
      Difficult to know and understand user experience across all devices
      Devices vary by model, browser type, carrier, and input type
      Moderator may be unable to help users through usability tasks – resetting prototypes, navigating – if interface is not well-known
      Design team should have a variety of devices simulators / devices on hand for testing
      Prototype designs should be tested across as many devices as possible
      Limit recruiting to only devices that have been tested
      Have a back-up plan with either a basic device or paper prototypes
    • Challenges
      Mobile Devices are Small and . . . Well, Very Mobile
      Create a hot zone on the table, encouraging users to keep the device within a narrow frame
      Consider light source and potential reflection on the phone; continually adjust mid testing; turn-off lights
      Use remote-controlled video camera (with technician in back room) or have a second technician available in interview room
      Project the mobile screen on a larger monitor in interview room and back room
      Creatively screen out participants whose finger size may distract from findings (if non-touch screen)
      Users move the device around while interacting and explaining, often moving video display out of focus
      Reflectors on screens and smudges can make video all reflection
      Small device screens make it difficult to see what users are doing
      Large fingers and long fingernails can cause unintended device responses
    • Challenges
      Varied Locations and Use Cases
      Mobile device usage occurs in a variety of places
      59% of Americans check email while in the bathroom*
      Difficult to recreate the true experience in a lab setting
      Create opportunities for feedback in context of usage
      Conduct field studies on beta products
      Source: Nielsen Mobile 2008
    • Challenges
      Varied Carrier Network Coverage
      Network coverage is inconsistent; difficult to predict accessibility during lab testing
      Difficult to validate that out-of-town lab truly has adequate coverage
      Visit facility with colleagues who have various carriers to confirm network availability in lab
      From out-of-town labs, request:
      Reference clients from last mobile studies
      List of carriers confirmed to have consistent coverage
      At a minimum, check carrier coverage in various cities online
    • Screen during recruiting for:
      Unlimited data plans
      Willingness to send / receive text messages during study
      Rescreen again upon arrival for study
      Calling / Data Plans Vary Widely
      Participants may be charged for mobile web access or sample texts sent during testing
      Participants may change data plan between recruiting interview and date of study
    • Challenges
      Varied User Experience
      Most mobile device owners only utilize a small percentage of total device capabilities
      43% of mobile subscribers do not use text messaging on a regular basis*
      Users may have no experience typing in a web address to access a site or with sending SMS messages (which may be critical to your study)
      If using a prototype phone for downloadable app, users may not be comfortable with it
      Pre-test participants to ensure appropriate device experience
      Mobile address typing
      Send SMS to recruiter
      Spend 2-5 minutes explaining how prototype phone works
      * Source: Nielsen 2008 ‘The Short Code Marketing Opportunity’
    • Other Helpful Tips
      Tell participants to bring recharger
      Simplify language (e.g., SMS vs. text message)
      Have a prototype strategy
      Back-end SMS text simulation
      Easy URL for access
      Back-up paper prototypes
      Index page for easy access to alternative flows
      Have plan for interruption from phone call or text (resetting session variables)
      Plan for getting screenshots for your report
      iPhone: Press hold button and home button at same time (saves into photo gallery)
      Use documented comps
    • Q&A
    • Thank You.
      Amy Buckner
      Managing Partner & Co-Founder, AnswerLab
      Kris Mihalic
      Head of Mobile User Experience and Design Research, Yahoo!
      Twitter: suikris