0
Making the Leap from Web to Mobile
Best Practices in Mobile User Experience Research

Amy Buckner, AnswerLab
Kris Mihalic,...
Agenda

Topic                                                 Minutes

1. Introduction                                    ...
Introductions

            Amy Buckner
            Managing Partner & Co-Founder of AnswerLab
            
    Over 10 yea...
State of Mobile




                        5



         CONFIDENTIAL
There are   4.1 Billion   mobile phone subscribers worldwide



  Only   2%   of US mobile phones are iPhones (~5 Million)...
Mobile Market Summary
 Globally, there are 800 million cars, 850 million PCs, 1.3 billion fixed landline
 phones, 1.4 bill...
U.S. Mobile Web Users Growing Rapidly




                                      Source: m:metrics/comScore, 2009   8



  ...
Mobile Audience Size (# of users)
In the U.S., the size of the Mobile Audience is 250 Million
  60.00%


             52%
...
U.S. Mobile Internet Usage


Mobile Internet
users are
heavy users
 
     60% access the
       Internet from their
      ...
Device Traffic



Mobile phones dominate mobile
Web traffic with 66%
market share (US).

BlackBerry and Smartphones
(Windo...
Widgets Improve Mobile Internet Experience




              Source: ‘Widgets Improve the Convenience of the Mobile Intern...
Group Exercise




                        13



         CONFIDENTIAL
Take out your cell phone and
search for times that ‘Star Trek’ is
 playing in your neighborhood.


Raise your hand when yo...
Discussion Points



   
     How long did it take?

     How did you do it? (web, app, SMS?)
   
  



     What problems...
Mobile User Research Methods




                               16



        CONFIDENTIAL
Overview of Research Methods


     Quick and Dirty feedback


     Lab Studies


     Field Studies


     Online Surveys...
Quick and Dirty – What are obvious problems?

                                                         Advantages

       ...
Lab Studies – Can they use it?

                                                       Advantages
                        ...
Field Studies – What is the context of usage?

                                                     Advantages

          ...
Online Surveys – What do they think of it?

                                                          Advantages

        ...
Behavioral Analysis – How do they use it?

                                                      Advantages

             ...
Types of Prototypes


       
     Paper prototypes
              
     Mocks
              
     Paper and pencil


     ...
Team Exercise & Case Study




                             24



        CONFIDENTIAL
Yahoo! Go


              Mobile Application
            
  

                   
   Search
                   
   News

 ...
Y! oneSearch integrated into Y! Go




                                     26



                  CONFIDENTIAL
Business Issue



     Drive mobile search uptake – Improved user experience can help
      accelerate growth of the servi...
Research Objectives



     Understand mobile search behavior in users’ daily lives


     Identify content users seek whe...
1.  Form into teams of 4.


2.  Develop a research plan for
    Yahoo! Go.


3.  You have 10 minutes.

                   ...
Team Activity                              Research Objectives


    Develop a research plan for Yahoo!              
    ...
Discussion Points



   
     What methods do you recommend & why?


   
     What are your anticipated concerns?


   
  ...
Our Solution: One-Month Field Study (1)



       Voicemails &
                         Daily Mobile Survey            Dig...
‘Pocket Card’ Questionnaire




                                      33



                       CONFIDENTIAL
Sample Voicemail Diary Messages -- Winchelle
Friday 7:39pm
I used oneSearch today, was looking for a hospital – directions...
Our Solution: One-Month Field Study (2)


       Pre-Interview        Mid-Check Interview       Wrap-up Interview




    ...
Screening Criteria: 9 Experienced Mobile Users




    Have an unlimited data plan for                       
    Have a p...
Sliding Scale for Participant Incentives




              
    $100 for the initial in-person interview


              
...
Why did we choose this method?


     Natural environment key to context


     All research activities on the mobile devi...
What did we learn? (1)


     Perceived value increased with usage


     Usage typically driven by lack of computer avail...
What did we learn? (2)


     Mental model based on
      computer usage


     Speed and relevance of search
      result...
Daily SMS Survey Results




                   *Website not adapted for mobile device   41



                           ...
What did we learn? (3)



Use Case                          Primary Content Wanted

Shopping          Business’ number, lo...
Summary of Use Case Frequency



            Note: The 166 searches were conducted by 9 users




                        ...
Shopping

     Business’ number, location, directions,
      and hours of operation
       
   “While driving home from wo...
Dining

     A restaurant’s number, location,
      directions, and hours of operation
       
   “I just woke up and didn...
Traveling

     Transit schedules and numbers
       
     “I wanted to find out when the next MUNI but
             the M...
Business Implications



    Drive mobile search uptake – Distribution through partnerships,
     marketing campaigns, ope...
Mobile Usability Research Challenges & Tips




                                        49



           CONFIDENTIAL
Variety of Mobile Devices

                  Challenges                                                       Tips 


    ...
Mobile Devices are Small and . . . Well, Very Mobile

                     Challenges                                     ...
Varied Locations and Use Cases

                         Challenges                                                  Tips ...
Varied Carrier Network Coverage

                   Challenges                                                            ...
Calling / Data Plans Vary Widely

                  Challenges                                                            ...
Varied User Experience

                         Challenges                                                               ...
Other Helpful Tips



     Tell participants to bring recharger


     Simplify language (e.g., SMS vs. text message)


  ...
Q&A




                     57



      CONFIDENTIAL
Thank You.

                       Amy Buckner
                       Managing Partner & Co-Founder, AnswerLab
           ...
Upcoming SlideShare
Loading in...5
×

Making The Leap From Web To Mobile

3,080

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,080
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Making The Leap From Web To Mobile"

  1. 1. Making the Leap from Web to Mobile Best Practices in Mobile User Experience Research Amy Buckner, AnswerLab Kris Mihalic, Yahoo! UPA 2009, June 12, Portland
  2. 2. Agenda Topic Minutes 1. Introduction 5 minutes 2. State of Mobile 10 minutes 3. Group Exercise: Mobile Device Task & Discussion 5 minutes 4. Mobile User Research Methods 20 minutes 5. Team Exercise: Develop Research Solution 15 minutes 6. Case Study 15 minutes 7. Mobile Usability Research Challenges & Tips 15 minutes 8. Q&A 5 minutes 3 CONFIDENTIAL
  3. 3. Introductions Amy Buckner Managing Partner & Co-Founder of AnswerLab   Over 10 years experience leading user research programs for Fortune 500 companies   Expertise in quantitative and qualitative methods, across web and mobile devices   Previous West Coast Director of Professional Services at Vividence (now Keynote) and Marketing Manager at SmartPlanet   Believes a glass of wine and a beef jerky stick count as dinner Kris Mihalic, PhD Head of Mobile User Experience and Design Research, Yahoo!   Seasoned user experience and design researcher for mobile and cross-channel products and services   Expertise in defining and executing UX and design research strategy in cross-functional teams   Experience teaching interface design methods, usability engineering, and telecommunication technologies   Lives on coffee 4 CONFIDENTIAL
  4. 4. State of Mobile 5 CONFIDENTIAL
  5. 5. There are 4.1 Billion mobile phone subscribers worldwide Only 2% of US mobile phones are iPhones (~5 Million) Social networking on mobile phones is growing 196% Y/Y There are 1.4x more mobile IM users than mobile business email users in the US 6 CONFIDENTIAL
  6. 6. Mobile Market Summary Globally, there are 800 million cars, 850 million PCs, 1.3 billion fixed landline phones, 1.4 billion credit cards, 1.5 billion TV sets – and 2.7 billion mobile phones – in use. (Experian 2008) eMarketer projects that worldwide spending on mobile advertising will reach a total of $19.1 billion in 2012 – up from 4.6 billion in 2008. Source: Experian 2008, eMarketer 2008 7 CONFIDENTIAL
  7. 7. U.S. Mobile Web Users Growing Rapidly Source: m:metrics/comScore, 2009 8 CONFIDENTIAL
  8. 8. Mobile Audience Size (# of users) In the U.S., the size of the Mobile Audience is 250 Million 60.00% 52% 50.00% 40.00% 31% 30% 30.00% 20.00% 10.00% 10% 0.00% SMS Users Mobile Media Users 3G Smartphone 9 Source: m:metrics/comScore, 2009 CONFIDENTIAL
  9. 9. U.S. Mobile Internet Usage Mobile Internet users are heavy users   60% access the Internet from their phone at least once/ day   48% access the Internet more than once per day Source: TSM|TargetProfile 2007 10 CONFIDENTIAL
  10. 10. Device Traffic Mobile phones dominate mobile Web traffic with 66% market share (US). BlackBerry and Smartphones (Windows Mobile, Palm, etc.) are heavy users of mobile Internet – combined 26% of traffic iPhone/iPod Touch has shown tremendous growth, capturing 8% of the mobile Internet market in the US Source: m:metrics/comScore, 2009 11 CONFIDENTIAL
  11. 11. Widgets Improve Mobile Internet Experience Source: ‘Widgets Improve the Convenience of the Mobile Internet,’ Forrester, 2008 12 CONFIDENTIAL
  12. 12. Group Exercise 13 CONFIDENTIAL
  13. 13. Take out your cell phone and search for times that ‘Star Trek’ is playing in your neighborhood. Raise your hand when you have finished the task. 14 CONFIDENTIAL
  14. 14. Discussion Points   How long did it take? How did you do it? (web, app, SMS?)   What problems, if any, did you have?   15 CONFIDENTIAL
  15. 15. Mobile User Research Methods 16 CONFIDENTIAL
  16. 16. Overview of Research Methods   Quick and Dirty feedback   Lab Studies   Field Studies   Online Surveys   Basic Survey   Survey with Behavioral Tracking (Keynote on iPhone only)   Quantitative Behavioral Analysis 17 CONFIDENTIAL
  17. 17. Quick and Dirty – What are obvious problems? Advantages   Informal setting with Lo-Fi prototypes   Fast iteration cycles   Brain-damage check   Finds critical usability issues in shortest time Challenges Examples   Insider/stakeholder view   Cross-functional team feedback   Anecdotal, unstructured results   Internal alpha/beta   Small sample size   Difficult to convey real usage scenarios 18 CONFIDENTIAL
  18. 18. Lab Studies – Can they use it? Advantages   Allows for probing by moderator   Can record interactions   Best for prototype testing   Allows for real-time viewing of interaction – team engagement Challenges Examples   Artificial environment   Usability study   Moderator bias; participant ‘pleasing’   Participatory   Small sample size design sessions   Difficult to test all hardware / network scenarios 19 CONFIDENTIAL
  19. 19. Field Studies – What is the context of usage? Advantages   Assess usage under real conditions   Candid user feedback   Delivers unknown use-cases Challenges Examples   Small sample size   Diary study   Difficult to observe and probe   Community study   Participant engagement difficult   Requires robust product 20 CONFIDENTIAL
  20. 20. Online Surveys – What do they think of it? Advantages   Can deliver statistically valid results   Provides qualitative and/or quantitative data   Geographical diversity (remote)   Broad representation of devices Challenges Examples   Difficult to observe and probe   Desktop surveys   Out-of-context interaction, esp. with desktop survey   On-device survey   Technical constraints   Survey with behavioral tracking 21 CONFIDENTIAL
  21. 21. Behavioral Analysis – How do they use it? Advantages   Reflects what people do, not say   Statistically valid results   Shows all behaviors, rather than those confined to a single task Challenges Examples   Probing / deep-dive difficult   Log data mining   Context and intent unknown   Attitudes and perceptions unknown 22 CONFIDENTIAL
  22. 22. Types of Prototypes   Paper prototypes   Mocks   Paper and pencil   Interactive prototypes   Apps   Flash   Native, e.g. iPhone   Browser-based   Simple HTML prototype   High-fidelity prototype (with session variables) 23 CONFIDENTIAL
  23. 23. Team Exercise & Case Study 24 CONFIDENTIAL
  24. 24. Yahoo! Go Mobile Application     Search   News   Sports   Weather   Other Topics 25 CONFIDENTIAL
  25. 25. Y! oneSearch integrated into Y! Go 26 CONFIDENTIAL
  26. 26. Business Issue   Drive mobile search uptake – Improved user experience can help accelerate growth of the service   Improve relevancy – Understanding search intent can contribute to delivering more relevant results   Product differentiation – New approach to search results page can deliver higher value to customers 27 CONFIDENTIAL
  27. 27. Research Objectives   Understand mobile search behavior in users’ daily lives   Identify content users seek when conducting mobile searches   Assess the context surrounding mobile searches   Evaluate effectiveness of new product   Identify opportunities to improve the user experience 28 CONFIDENTIAL
  28. 28. 1.  Form into teams of 4. 2.  Develop a research plan for Yahoo! Go. 3.  You have 10 minutes. 29 CONFIDENTIAL
  29. 29. Team Activity Research Objectives   Develop a research plan for Yahoo!   Understand mobile search behavior in users’ daily lives Go   Identify content users seek when   Include: conducting mobile searches •  Recommended method(s) •  Number of participants   Assess the context surrounding mobile searches •  Profile of participants   Evaluate effectiveness of new product   Identify opportunities to improve the user experience 30 CONFIDENTIAL
  30. 30. Discussion Points   What methods do you recommend & why?   What are your anticipated concerns?   Any potential limitations? 31 CONFIDENTIAL
  31. 31. Our Solution: One-Month Field Study (1) Voicemails & Daily Mobile Survey Digital Photos Pocket Card   9 users   Daily SMS mini-   Users sent survey photos of   Daily voicemails themselves or   Linked to phone   Laminated surroundings in number for pocket card with the context of voicemail key questions using oneSearch 32 CONFIDENTIAL
  32. 32. ‘Pocket Card’ Questionnaire 33 CONFIDENTIAL
  33. 33. Sample Voicemail Diary Messages -- Winchelle Friday 7:39pm I used oneSearch today, was looking for a hospital – directions and address to it. I was getting off work and sitting in my car, trying to find directions. It was about 4:30. I liked that it came up with the website and with information about that. And the directions on how to go. I didn’t like that it could not find my work address – which is saved on my Yahoo! directions (recently gone) saved places. I think using the actual internet through the PC will help me better because it knows where the address is. I’m not sure why the address is not coming up in my oneSearch and that sucked. Saturday 8:06pm I didn’t use oneSearch today. I looked for info, looked for a store and I was at home. I wanted to know if the store was open STORE HOURS (9:45am). I used the internet with my phone and used Google to search. It was not very helpful because I wanted to see what time they opened and I couldn’t find it. I’ve used oneSearch in the past and I guess the results were the same but I guess it’d be nice to go to the site for that particular store. Monday 10:48pm I used oneSearch to find Expedia, was at work on my lunch break. It was about 12:20. I liked that I can find it quickly. I guess the website didn’t support my mobile phone though, so I didn’t like that. Would rather book a flight/hotel at home because its supported. 34 CONFIDENTIAL
  34. 34. Our Solution: One-Month Field Study (2) Pre-Interview Mid-Check Interview Wrap-up Interview   An initial 45-   20-minute check-   Final in-person minute interview in phone interview of 75 interview minutes   Verified technical capabilities of   Initial feedback   Clarified phone voicemail reports   Q&A   Explained   Overall research impressions of program the product from the month-long usage 35 CONFIDENTIAL
  35. 35. Screening Criteria: 9 Experienced Mobile Users   Have an unlimited data plan for   Have a primary mobile device WAP services through their capable of downloading and mobile carrier using Y! Go   Use their mobile for text   Live or work within San Francisco messages (SMS) daily and use mobile more than three times per week for activities that require the   Mix of gender, age (23-46+), and transmission of online data typical method of commuting to work (4 users primarily drive, 5   Have Cingular, Sprint, T-Mobile, primarily take mass transit) or Verizon as their mobile carrier 36 CONFIDENTIAL
  36. 36. Sliding Scale for Participant Incentives   $100 for the initial in-person interview   $50 for the mid-way phone interview   $150 for the final in-person interview   $3 each day a report is given 37 CONFIDENTIAL
  37. 37. Why did we choose this method?   Natural environment key to context   All research activities on the mobile device   Mix of methods   Engage users in program   Breadth of insights – context, usability, impressions 38 CONFIDENTIAL
  38. 38. What did we learn? (1)   Perceived value increased with usage   Usage typically driven by lack of computer availability   However, certain scenarios drove phone choice over computer:   Social setting   Privacy   Convenience 39 CONFIDENTIAL
  39. 39. What did we learn? (2)   Mental model based on computer usage   Speed and relevance of search results highest area of frustration   Lack of understanding that results were customized by widget 40 CONFIDENTIAL
  40. 40. Daily SMS Survey Results *Website not adapted for mobile device 41 CONFIDENTIAL
  41. 41. What did we learn? (3) Use Case Primary Content Wanted Shopping Business’ number, location, directions, and hours of operation Entertaining Schedules/Venues of concerts, sport games, movies, and TV shows Researching Content from news/articles Dining Restaurants’ number, location, directions, and hours of operation Traveling Schedules and numbers of transportation (public transit, taxis, etc) Media Seeking Photos/Ring tones 42 CONFIDENTIAL
  42. 42. Summary of Use Case Frequency Note: The 166 searches were conducted by 9 users 43 CONFIDENTIAL
  43. 43. Shopping   Business’ number, location, directions, and hours of operation   “While driving home from work, I did a search for Fry’s electronics on the local search and it came up with all the different Fry’s in the area, their numbers, as well as directions and all that. The results were all helpful and fast in helping me find the one closest.”   “Since we were already out shopping, I wanted to find where the closest Citibank was to where we were.”   Pricing/Product comparisons   “I was at Costco and comparing TV prices. I wanted to know how much these same TV’s would be at the Circuit City down the street.” 44 CONFIDENTIAL
  44. 44. Dining   A restaurant’s number, location, directions, and hours of operation   “I just woke up and didn’t want to boot my computer up but I wanted to find out what time this breakfast place opened so I could know how soon I could get there.”   “I was in my car and wanted their number so I could call and place an order to pick it up on my way home from work.”   Ideas of places to eat   “I was with my wife in the Haight and we wanted to find nearby restaurants so we could decide what we wanted to eat that was within walking distance.”   Reservations & Reviews   “I was in the South Bay with some friends   “I was hoping to make reservations straight and we wanted to find a good sushi place from the phone, or at least call them to place nearby.” them.”   “I was out with some friends and curious about whether the place nearby was any good.” 45 CONFIDENTIAL
  45. 45. Traveling   Transit schedules and numbers   “I wanted to find out when the next MUNI but the MUNI’s site wouldn’t work on my phone.”   “I wanted the number to call a taxi. I was outside and tired of waiting for the bus.”   Tourist activities   “I wanted to see what kind of ferry tours we could do before looking for another museum that would be fun for my daughter.”   “I was looking for fun things to do while at Hermosa beach.”   Maps and directions   “I was trying to get a map of the Mt. Shasta area and I wanted to plan which route to take. It would be nice if the driving directions had live updates on traffic and road conditions.”   Flight status / check-in   “I was in the taxi to the airport and wanted to check my flight’s status.” 46 CONFIDENTIAL
  46. 46. Business Implications   Drive mobile search uptake – Distribution through partnerships, marketing campaigns, opening to third party services and developers   Improve relevancy – Focused on improving results for specific use cases (e.g. local businesses, flights, etc.)   Product differentiation – Federated search results, integrated user experience across multiple services (e.g. search results + maps) 48 CONFIDENTIAL
  47. 47. Mobile Usability Research Challenges & Tips 49 CONFIDENTIAL
  48. 48. Variety of Mobile Devices Challenges  Tips    Difficult to know and understand user   Design team should have a variety experience across all devices of devices simulators / devices on hand for testing   Devices vary by model, browser type, carrier, and input type   Prototype designs should be tested across as many devices as possible   Moderator may be unable to help users through usability tasks –   Limit recruiting to only devices that resetting prototypes, navigating – if have been tested interface is not well-known   Have a back-up plan with either a basic device or paper prototypes 50 CONFIDENTIAL
  49. 49. Mobile Devices are Small and . . . Well, Very Mobile Challenges  Tips    Create a hot zone on the table, encouraging   Users move the device around while users to keep the device within a narrow interacting and explaining, often moving frame video display out of focus   Consider light source and potential   Reflectors on screens and smudges can reflection on the phone; continually adjust make video all reflection mid testing; turn-off lights   Small device screens make it difficult to   Use remote-controlled video camera (with see what users are doing technician in back room) or have a second technician available in interview room   Large fingers and long fingernails can   Project the mobile screen on a larger cause unintended device responses monitor in interview room and back room   Creatively screen out participants whose finger size may distract from findings (if non- touch screen) 51 CONFIDENTIAL
  50. 50. Varied Locations and Use Cases Challenges  Tips    Mobile device usage occurs in a   Create opportunities for feedback in variety of places context of usage   59% of Americans check email while in the bathroom*   Conduct field studies on beta products   Difficult to recreate the true experience in a lab setting MOBILE DEVICE ACCESS 67.4% 55.2% 35.9% 15.8% 6.4% Source: Nielsen Mobile 2008 52 CONFIDENTIAL
  51. 51. Varied Carrier Network Coverage Challenges  Tips    Network coverage is inconsistent;   Visit facility with colleagues who difficult to predict accessibility during have various carriers to confirm lab testing network availability in lab   Difficult to validate that out-of-town   From out-of-town labs, request: lab truly has adequate coverage   Reference clients from last mobile studies   List of carriers confirmed to have consistent coverage   At a minimum, check carrier coverage in various cities online 53 CONFIDENTIAL
  52. 52. Calling / Data Plans Vary Widely Challenges  Tips    Participants may be charged for   Screen during recruiting for: mobile web access or sample texts   Unlimited data plans sent during testing   Willingness to send / receive text messages during study   Participants may change data plan between recruiting interview and date   Rescreen again upon arrival for of study study 54 CONFIDENTIAL
  53. 53. Varied User Experience Challenges  Tips    Most mobile device owners only   Pre-test participants to ensure utilize a small percentage of total appropriate device experience device capabilities   Mobile address typing   43% of mobile subscribers do not use   Send SMS to recruiter text messaging on a regular basis*   Spend 2-5 minutes explaining how   Users may have no experience typing prototype phone works in a web address to access a site or with sending SMS messages (which may be critical to your study)   If using a prototype phone for downloadable app, users may not be comfortable with it 55 * Source: Nielsen 2008 ‘The Short Code Marketing Opportunity’ CONFIDENTIAL
  54. 54. Other Helpful Tips   Tell participants to bring recharger   Simplify language (e.g., SMS vs. text message)   Have a prototype strategy   Back-end SMS text simulation   Easy URL for access   Back-up paper prototypes   Index page for easy access to alternative flows   Have plan for interruption from phone call or text (resetting session variables)   Plan for getting screenshots for your report   iPhone: Press hold button and home button at same time (saves into photo gallery)   Use documented comps 56 CONFIDENTIAL
  55. 55. Q&A 57 CONFIDENTIAL
  56. 56. Thank You. Amy Buckner Managing Partner & Co-Founder, AnswerLab amy@answerlab.com Kris Mihalic Head of Mobile User Experience and Design Research, Yahoo! kmihalic@yahoo-inc.com Twitter: suikris 58 CONFIDENTIAL
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×