Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
How to conduct field
research for mobile apps
MAY 2016
Workshop Goals
Understand the pros/cons and different considerations for
field research
How to develop a research plan for...
Schedule
5:30 – 6:15pm
Designing, planning and conducting field research
6:45– 7:30pm
Form groups and develop a field stud...
Suzanne Boyd
FOUNDER & CEO
Introductions
Ian Wyosnick
USER RESEARCHER
INTRODUCTIONS
We’re a user-centered design
consultancy focused on government
agencies, nonprofits and enterprises
with a s...
Clients and impact
User-centered design process
Getting out of the lab and into the field
The “field”
Coffee shops
Libraries
Conferences
Hospitals
Court house
Ferry terminals
Farmer’s markets
Busses, trains and l...
Behaviors and
products we study
Wayfinding
Trip planning
Purchasing
Info seeking
Gaming
Web sites / prototypes
Kiosks
Apps
Focus on mobile apps
Some stats
Mobile Marketing Statistics Compilation by Dave Chaffey - 2016
Internet usage
mobile desktopmobile desktop
51% ...
Why do field studies?
What is a field study?
A field study is a general method for collecting data about users, user
needs, and product requirem...
Research methods
Analytics
Eye tracking
Observations
Surveys
Interviews
Usability testing
Components of field research
user + tool
environment / context
user + tool
The field
Let’s brainstorm
Public transit trip planner app
Social media app
Grocery list app
Air travel app
Game app
Busses, trains,...
Your thoughts
What have you heard about field
studies?
Benefits of field
studies
Study context of use
Efficient
Ancillary findings
Early morning at King Street Station | Seattle, WA
Directly observe context of use
Context of use
People
Light
Weather
Other apps
Artifacts
Others?
Efficient
Research with many people in a day
Minimal recruitment cost
Representative participants
No lab costs
Ancillary findings
Pain points
Unmet needs
Behaviors
Challenges of field
studies
Logistics and organization
Weather
Privacy and recording
Off topic conversation
‘Difficult’ pa...
Design
OPTIONS AND CONSIDERATIONS
Considerations for your study design
generative evaluative
Why are you conducting research?
summative formative
Who is con...
+ Go where the participants are
+ Find a captive audience
- Less control over circumstances and permissions.
+ More contro...
Journal studies
Recruited participants log their
behaviors with the app over a
period of time
Capture experiences whenever...
Recording
Equipment
– Camera
– iPad
– Clip board
– Paper and pencil
– Sled
Recording
Software & apps
Input is the app
Input is the video camera
Prepare
LOGISTICS
Prepare study
materials
 Study plan and research
questions
 Recruitment criteria and screener
 Consent form
 Facilitat...
Work the logistics
 Obtaining permissions
 Electricity availability and
considerations
 Internet availability and
consi...
Don’t forget the details
 Weather appropriate clothing
 Extra pens and pencils
 Clipboards
 Camera/Recording devices
...
Conduct the study
Recruit and screen
participants
Find willing participants
What are the behaviors or
approximate personas you are
looking f...
Facilitating sessions
Keep sessions short and within
the promised amount of time
(i.e. if you say it will take 15
minutes,...
Take good notes
Take a picture of the participant
Agree on codes with other
researchers
Use forms
Leverage checkboxes
Anti...
Example
Usability Issues P1 P2 P3
Participant has Yelp downloaded on phone x x
Participant uses multiple apps to complete ...
Capture and synch up
on themes as you go
Log problems and positives
after each participant
Debrief with team along the
way
Recruitment and screening
Timeline
Specific
behavior
Random
Agrees to
participate
Declines to
participate
How do I
select ...
Interview and study facilitation
Timeline cont…
Qualified
Disqualified
Taking all
qualified
participants
Result of
recruit...
Analyze
Analyze data
Compile quantitative data
– Task success, Time on task, Satisfaction, Error rates, etc.
Analyze qualitative d...
Reminder
Focus on what participants did
and said.
Your field research mission
Mission
You have just signed a contract to do usability work for Yelp. They
have been experiencing a decrease in users and...
Steps
Create study plan
– Research questions, roles and responsibilities, scenarios
Conduct field research (at least 3 par...
Your thoughts
What have you learned about
field studies?
Feedback survey link
www.uxpa2016.org/sessionsurvey?sessi
onid=292
Thank you.
How to conduct field research for mobile apps
How to conduct field research for mobile apps
How to conduct field research for mobile apps
Upcoming SlideShare
Loading in …5
×

How to conduct field research for mobile apps

Mobile application use has grown exponentially. Testing mobile experiences brings additional challenges and opportunities because the context-of-use is not a static location like a workplace, but instead wherever a user happens to be with his or her phone. Field studies are useful because they help researchers collect more naturalistic data since they observe users in their context-of-us instead of a lab.

In this course you will learn:

The variety of methods and tools for field testing mobile experiences, and the pros / cons of each
How to develop a research plan for mobile field testing
Recruitment strategies for field testing
How to moderate a field test and record participants’ behaviors
You will learn by doing an actual field study of a mobile application near the conference location.

  • Be the first to comment

  • Be the first to like this

How to conduct field research for mobile apps

  1. 1. How to conduct field research for mobile apps MAY 2016
  2. 2. Workshop Goals Understand the pros/cons and different considerations for field research How to develop a research plan for mobile field research Practice conducting a field study Present findings and reflect on field studies for mobile apps
  3. 3. Schedule 5:30 – 6:15pm Designing, planning and conducting field research 6:45– 7:30pm Form groups and develop a field study plan 7:30 – 8:00pm Break 8:00 – 8:45pm Conduct field study 8:45 – 9:30pm Analyze and present findings and reflections
  4. 4. Suzanne Boyd FOUNDER & CEO Introductions Ian Wyosnick USER RESEARCHER
  5. 5. INTRODUCTIONS We’re a user-centered design consultancy focused on government agencies, nonprofits and enterprises with a social-impact mission.
  6. 6. Clients and impact
  7. 7. User-centered design process
  8. 8. Getting out of the lab and into the field
  9. 9. The “field” Coffee shops Libraries Conferences Hospitals Court house Ferry terminals Farmer’s markets Busses, trains and light rail Stations and bus stops
  10. 10. Behaviors and products we study Wayfinding Trip planning Purchasing Info seeking Gaming Web sites / prototypes Kiosks Apps
  11. 11. Focus on mobile apps
  12. 12. Some stats Mobile Marketing Statistics Compilation by Dave Chaffey - 2016 Internet usage mobile desktopmobile desktop 51% 42% 2.8 hrs / day 2.4 hrs / day Time on screens
  13. 13. Why do field studies?
  14. 14. What is a field study? A field study is a general method for collecting data about users, user needs, and product requirements that involves observation and interviewing. Data are collected about task flows, inefficiencies, and the organizational and physical environments of users. Investigators in field studies observe users as they work, taking notes on particular activities and often asking questions of the users. Observation may be either direct, where the investigator is actually present during the task, or indirect, where the task is viewed by some other means like a video recorder set up in an office. The method is useful early in product development to gather user requirements. It is also useful for studying currently executed tasks and processes. Usability Body of Knowledge
  15. 15. Research methods Analytics Eye tracking Observations Surveys Interviews Usability testing Components of field research
  16. 16. user + tool environment / context user + tool The field
  17. 17. Let’s brainstorm Public transit trip planner app Social media app Grocery list app Air travel app Game app Busses, trains, light rail, stations, bus stops If you are researching Then the field may be
  18. 18. Your thoughts What have you heard about field studies?
  19. 19. Benefits of field studies Study context of use Efficient Ancillary findings
  20. 20. Early morning at King Street Station | Seattle, WA Directly observe context of use
  21. 21. Context of use People Light Weather Other apps Artifacts Others?
  22. 22. Efficient Research with many people in a day Minimal recruitment cost Representative participants No lab costs
  23. 23. Ancillary findings Pain points Unmet needs Behaviors
  24. 24. Challenges of field studies Logistics and organization Weather Privacy and recording Off topic conversation ‘Difficult’ participants Rejection Observers Fatigue
  25. 25. Design OPTIONS AND CONSIDERATIONS
  26. 26. Considerations for your study design generative evaluative Why are you conducting research? summative formative Who is conducting the research? solo team recruited intercept How will you find participants? How will you capture the data? video/audio no video/audio Where will you go? on the go set up shop Whose device will you use? participant researcher
  27. 27. + Go where the participants are + Find a captive audience - Less control over circumstances and permissions. + More control + Easier to record + Use your own devices - Less ancillary findings ON THE GO SET UP SHOP + Go where the participants are + Find a captive audience - Less control
  28. 28. Journal studies Recruited participants log their behaviors with the app over a period of time Capture experiences whenever and wherever they happen Many emerging tools available
  29. 29. Recording Equipment – Camera – iPad – Clip board – Paper and pencil – Sled
  30. 30. Recording Software & apps Input is the app Input is the video camera
  31. 31. Prepare LOGISTICS
  32. 32. Prepare study materials  Study plan and research questions  Recruitment criteria and screener  Consent form  Facilitator script  Scenario(s)  Note taking sheets  Post-test interview and metrics  Honoraria
  33. 33. Work the logistics  Obtaining permissions  Electricity availability and considerations  Internet availability and considerations  Official identification  Staff schedules and points of contact in the field  Scheduling and communication during the study Tip: Schedule sync-ups during the day to revise study plan accordingly
  34. 34. Don’t forget the details  Weather appropriate clothing  Extra pens and pencils  Clipboards  Camera/Recording devices  Backpacks or other bags to carry everything  Snacks to keep you going  Clear instructions for any recruited participants on where park and meet up
  35. 35. Conduct the study
  36. 36. Recruit and screen participants Find willing participants What are the behaviors or approximate personas you are looking for Hook ‘em Give them the pitch Screen for fit to user profile
  37. 37. Facilitating sessions Keep sessions short and within the promised amount of time (i.e. if you say it will take 15 minutes, make sure it does) Stay on script as it will ensure some rigor and consistency to the study
  38. 38. Take good notes Take a picture of the participant Agree on codes with other researchers Use forms Leverage checkboxes Anticipate usability issues and count instances If the study is free flowing, blank note sheets may be best
  39. 39. Example Usability Issues P1 P2 P3 Participant has Yelp downloaded on phone x x Participant uses multiple apps to complete scenario x Participant does not use filters x x x Scenario 1 P1 P2 P3 Completion S F S Post scenario metric score 4 2 5 # of requests for help 0 1 0 # of errors 1 3 1
  40. 40. Capture and synch up on themes as you go Log problems and positives after each participant Debrief with team along the way
  41. 41. Recruitment and screening Timeline Specific behavior Random Agrees to participate Declines to participate How do I select a participant? “Ok thank you, have a good day!” Hook ‘em! Initial Screener Qualified Disqualified Consent Agrees to participate Declines to participate Start Study! 15sec 15sec 30sec
  42. 42. Interview and study facilitation Timeline cont… Qualified Disqualified Taking all qualified participants Result of recruitment & screening “Ok thank you, have a good day!” Follow up demographics “Thank you, have a good day!” 2min 10min Optional: Do they fit in a needed user segment? No Yes Determine scenario Run usability test Post-study interview and metrics 2min 15min total
  43. 43. Analyze
  44. 44. Analyze data Compile quantitative data – Task success, Time on task, Satisfaction, Error rates, etc. Analyze qualitative data – Problems experiences, comments / recommendations, pathways, think a loud protocol Develop exact and concise problem statements – Good problem statement: 12/14 participants clicked on notifications instead of newsfeed – Poor problem statement: Clicked on wrong link. – Poor problem statement: Was confused about links.
  45. 45. Reminder Focus on what participants did and said.
  46. 46. Your field research mission
  47. 47. Mission You have just signed a contract to do usability work for Yelp. They have been experiencing a decrease in users and realize that they may need to improve their product. Yelp wants you to do field research to get qualitative and quantitative feedback on user behavior with mobile devices when deciding to find a restaurant while on the go. Design a field research study to understand how apps or websites are determining a user’s choice of where to eat, how they use those apps or websites, and how they may integrate with other apps or websites to do auxiliary tasks (i.e. getting directions and travelling to the restaurant).
  48. 48. Steps Create study plan – Research questions, roles and responsibilities, scenarios Conduct field research (at least 3 participants) Analyze data Report on findings – Study approach – Findings – Lessons learned and takeaways
  49. 49. Your thoughts What have you learned about field studies? Feedback survey link www.uxpa2016.org/sessionsurvey?sessi onid=292
  50. 50. Thank you.

×