Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to conduct field research for mobile apps

Mobile application use has grown exponentially. Testing mobile experiences brings additional challenges and opportunities because the context-of-use is not a static location like a workplace, but instead wherever a user happens to be with his or her phone. Field studies are useful because they help researchers collect more naturalistic data since they observe users in their context-of-us instead of a lab.

In this course you will learn:

The variety of methods and tools for field testing mobile experiences, and the pros / cons of each
How to develop a research plan for mobile field testing
Recruitment strategies for field testing
How to moderate a field test and record participants’ behaviors
You will learn by doing an actual field study of a mobile application near the conference location.

  • Be the first to comment

  • Be the first to like this

How to conduct field research for mobile apps

  1. 1. How to conduct field research for mobile apps MAY 2016
  2. 2. Workshop Goals Understand the pros/cons and different considerations for field research How to develop a research plan for mobile field research Practice conducting a field study Present findings and reflect on field studies for mobile apps
  3. 3. Schedule 5:30 – 6:15pm Designing, planning and conducting field research 6:45– 7:30pm Form groups and develop a field study plan 7:30 – 8:00pm Break 8:00 – 8:45pm Conduct field study 8:45 – 9:30pm Analyze and present findings and reflections
  4. 4. Suzanne Boyd FOUNDER & CEO Introductions Ian Wyosnick USER RESEARCHER
  5. 5. INTRODUCTIONS We’re a user-centered design consultancy focused on government agencies, nonprofits and enterprises with a social-impact mission.
  6. 6. Clients and impact
  7. 7. User-centered design process
  8. 8. Getting out of the lab and into the field
  9. 9. The “field” Coffee shops Libraries Conferences Hospitals Court house Ferry terminals Farmer’s markets Busses, trains and light rail Stations and bus stops
  10. 10. Behaviors and products we study Wayfinding Trip planning Purchasing Info seeking Gaming Web sites / prototypes Kiosks Apps
  11. 11. Focus on mobile apps
  12. 12. Some stats Mobile Marketing Statistics Compilation by Dave Chaffey - 2016 Internet usage mobile desktopmobile desktop 51% 42% 2.8 hrs / day 2.4 hrs / day Time on screens
  13. 13. Why do field studies?
  14. 14. What is a field study? A field study is a general method for collecting data about users, user needs, and product requirements that involves observation and interviewing. Data are collected about task flows, inefficiencies, and the organizational and physical environments of users. Investigators in field studies observe users as they work, taking notes on particular activities and often asking questions of the users. Observation may be either direct, where the investigator is actually present during the task, or indirect, where the task is viewed by some other means like a video recorder set up in an office. The method is useful early in product development to gather user requirements. It is also useful for studying currently executed tasks and processes. Usability Body of Knowledge
  15. 15. Research methods Analytics Eye tracking Observations Surveys Interviews Usability testing Components of field research
  16. 16. user + tool environment / context user + tool The field
  17. 17. Let’s brainstorm Public transit trip planner app Social media app Grocery list app Air travel app Game app Busses, trains, light rail, stations, bus stops If you are researching Then the field may be
  18. 18. Your thoughts What have you heard about field studies?
  19. 19. Benefits of field studies Study context of use Efficient Ancillary findings
  20. 20. Early morning at King Street Station | Seattle, WA Directly observe context of use
  21. 21. Context of use People Light Weather Other apps Artifacts Others?
  22. 22. Efficient Research with many people in a day Minimal recruitment cost Representative participants No lab costs
  23. 23. Ancillary findings Pain points Unmet needs Behaviors
  24. 24. Challenges of field studies Logistics and organization Weather Privacy and recording Off topic conversation ‘Difficult’ participants Rejection Observers Fatigue
  26. 26. Considerations for your study design generative evaluative Why are you conducting research? summative formative Who is conducting the research? solo team recruited intercept How will you find participants? How will you capture the data? video/audio no video/audio Where will you go? on the go set up shop Whose device will you use? participant researcher
  27. 27. + Go where the participants are + Find a captive audience - Less control over circumstances and permissions. + More control + Easier to record + Use your own devices - Less ancillary findings ON THE GO SET UP SHOP + Go where the participants are + Find a captive audience - Less control
  28. 28. Journal studies Recruited participants log their behaviors with the app over a period of time Capture experiences whenever and wherever they happen Many emerging tools available
  29. 29. Recording Equipment – Camera – iPad – Clip board – Paper and pencil – Sled
  30. 30. Recording Software & apps Input is the app Input is the video camera
  31. 31. Prepare LOGISTICS
  32. 32. Prepare study materials  Study plan and research questions  Recruitment criteria and screener  Consent form  Facilitator script  Scenario(s)  Note taking sheets  Post-test interview and metrics  Honoraria
  33. 33. Work the logistics  Obtaining permissions  Electricity availability and considerations  Internet availability and considerations  Official identification  Staff schedules and points of contact in the field  Scheduling and communication during the study Tip: Schedule sync-ups during the day to revise study plan accordingly
  34. 34. Don’t forget the details  Weather appropriate clothing  Extra pens and pencils  Clipboards  Camera/Recording devices  Backpacks or other bags to carry everything  Snacks to keep you going  Clear instructions for any recruited participants on where park and meet up
  35. 35. Conduct the study
  36. 36. Recruit and screen participants Find willing participants What are the behaviors or approximate personas you are looking for Hook ‘em Give them the pitch Screen for fit to user profile
  37. 37. Facilitating sessions Keep sessions short and within the promised amount of time (i.e. if you say it will take 15 minutes, make sure it does) Stay on script as it will ensure some rigor and consistency to the study
  38. 38. Take good notes Take a picture of the participant Agree on codes with other researchers Use forms Leverage checkboxes Anticipate usability issues and count instances If the study is free flowing, blank note sheets may be best
  39. 39. Example Usability Issues P1 P2 P3 Participant has Yelp downloaded on phone x x Participant uses multiple apps to complete scenario x Participant does not use filters x x x Scenario 1 P1 P2 P3 Completion S F S Post scenario metric score 4 2 5 # of requests for help 0 1 0 # of errors 1 3 1
  40. 40. Capture and synch up on themes as you go Log problems and positives after each participant Debrief with team along the way
  41. 41. Recruitment and screening Timeline Specific behavior Random Agrees to participate Declines to participate How do I select a participant? “Ok thank you, have a good day!” Hook ‘em! Initial Screener Qualified Disqualified Consent Agrees to participate Declines to participate Start Study! 15sec 15sec 30sec
  42. 42. Interview and study facilitation Timeline cont… Qualified Disqualified Taking all qualified participants Result of recruitment & screening “Ok thank you, have a good day!” Follow up demographics “Thank you, have a good day!” 2min 10min Optional: Do they fit in a needed user segment? No Yes Determine scenario Run usability test Post-study interview and metrics 2min 15min total
  43. 43. Analyze
  44. 44. Analyze data Compile quantitative data – Task success, Time on task, Satisfaction, Error rates, etc. Analyze qualitative data – Problems experiences, comments / recommendations, pathways, think a loud protocol Develop exact and concise problem statements – Good problem statement: 12/14 participants clicked on notifications instead of newsfeed – Poor problem statement: Clicked on wrong link. – Poor problem statement: Was confused about links.
  45. 45. Reminder Focus on what participants did and said.
  46. 46. Your field research mission
  47. 47. Mission You have just signed a contract to do usability work for Yelp. They have been experiencing a decrease in users and realize that they may need to improve their product. Yelp wants you to do field research to get qualitative and quantitative feedback on user behavior with mobile devices when deciding to find a restaurant while on the go. Design a field research study to understand how apps or websites are determining a user’s choice of where to eat, how they use those apps or websites, and how they may integrate with other apps or websites to do auxiliary tasks (i.e. getting directions and travelling to the restaurant).
  48. 48. Steps Create study plan – Research questions, roles and responsibilities, scenarios Conduct field research (at least 3 participants) Analyze data Report on findings – Study approach – Findings – Lessons learned and takeaways
  49. 49. Your thoughts What have you learned about field studies? Feedback survey link onid=292
  50. 50. Thank you.