1.
How to conduct field
research for mobile apps
MAY 2016

2.
Workshop Goals
Understand the pros/cons and different considerations for
field research
How to develop a research plan for mobile field research
Practice conducting a field study
Present findings and reflect on field studies for mobile
apps

3.
Schedule
5:30 – 6:15pm
Designing, planning and conducting field research
6:45– 7:30pm
Form groups and develop a field study plan
7:30 – 8:00pm
Break
8:00 – 8:45pm
Conduct field study
8:45 – 9:30pm
Analyze and present findings and reflections

4.
Suzanne Boyd
FOUNDER & CEO
Introductions
Ian Wyosnick
USER RESEARCHER

5.
INTRODUCTIONS
We’re a user-centered design
consultancy focused on government
agencies, nonprofits and enterprises
with a social-impact mission.

6.
Clients and impact

7.
User-centered design process

8.
Getting out of the lab and into the field

9.
The “field”
Coffee shops
Libraries
Conferences
Hospitals
Court house
Ferry terminals
Farmer’s markets
Busses, trains and light rail
Stations and bus stops

10.
Behaviors and
products we study
Wayfinding
Trip planning
Purchasing
Info seeking
Gaming
Web sites / prototypes
Kiosks
Apps

11.
Focus on mobile apps

12.
Some stats
Mobile Marketing Statistics Compilation by Dave Chaffey - 2016
Internet usage
mobile desktopmobile desktop
51% 42% 2.8 hrs / day 2.4 hrs / day
Time on screens

13.
Why do field studies?

14.
What is a field study?
A field study is a general method for collecting data about users, user
needs, and product requirements that involves observation and
interviewing. Data are collected about task flows, inefficiencies, and
the organizational and physical environments of users.
Investigators in field studies observe users as they work, taking
notes on particular activities and often asking questions of the users.
Observation may be either direct, where the investigator is actually
present during the task, or indirect, where the task is viewed by some
other means like a video recorder set up in an office. The method is
useful early in product development to gather user requirements. It is
also useful for studying currently executed tasks and processes.
Usability Body of Knowledge

15.
Research methods
Analytics
Eye tracking
Observations
Surveys
Interviews
Usability testing
Components of field research

16.
user + tool
environment / context
user + tool
The field

17.
Let’s brainstorm
Public transit trip planner app
Social media app
Grocery list app
Air travel app
Game app
Busses, trains, light rail,
stations, bus stops
If you are researching Then the field may be

18.
Your thoughts
What have you heard about field
studies?

19.
Benefits of field
studies
Study context of use
Efficient
Ancillary findings

20.
Early morning at King Street Station | Seattle, WA
Directly observe context of use

21.
Context of use
People
Light
Weather
Other apps
Artifacts
Others?

22.
Efficient
Research with many people in a day
Minimal recruitment cost
Representative participants
No lab costs

23.
Ancillary findings
Pain points
Unmet needs
Behaviors

24.
Challenges of field
studies
Logistics and organization
Weather
Privacy and recording
Off topic conversation
‘Difficult’ participants
Rejection
Observers
Fatigue

25.
Design
OPTIONS AND CONSIDERATIONS

26.
Considerations for your study design
generative evaluative
Why are you conducting research?
summative formative
Who is conducting the research?
solo team
recruited intercept
How will you find participants?
How will you capture the data?
video/audio no video/audio
Where will you go?
on the go set up shop
Whose device will you use?
participant researcher

27.
+ Go where the participants are
+ Find a captive audience
- Less control over circumstances and permissions.
+ More control
+ Easier to record
+ Use your own devices
- Less ancillary findings
ON THE GO SET UP SHOP
+ Go where the participants are
+ Find a captive audience
- Less control

28.
Journal studies
Recruited participants log their
behaviors with the app over a
period of time
Capture experiences whenever
and wherever they happen
Many emerging tools available

29.
Recording
Equipment
– Camera
– iPad
– Clip board
– Paper and pencil
– Sled

30.
Recording
Software & apps
Input is the app
Input is the video camera

31.
Prepare
LOGISTICS

32.
Prepare study
materials
 Study plan and research
questions
 Recruitment criteria and screener
 Consent form
 Facilitator script
 Scenario(s)
 Note taking sheets
 Post-test interview and metrics
 Honoraria

33.
Work the logistics
 Obtaining permissions
 Electricity availability and
considerations
 Internet availability and
considerations
 Official identification
 Staff schedules and points of contact
in the field
 Scheduling and communication
during the study
Tip: Schedule sync-ups during the day to
revise study plan accordingly

34.
Don’t forget the details
 Weather appropriate clothing
 Extra pens and pencils
 Clipboards
 Camera/Recording devices
 Backpacks or other bags to carry
everything
 Snacks to keep you going
 Clear instructions for any
recruited participants on where
park and meet up

35.
Conduct the study

36.
Recruit and screen
participants
Find willing participants
What are the behaviors or
approximate personas you are
looking for
Hook ‘em
Give them the pitch
Screen for fit to user profile

37.
Facilitating sessions
Keep sessions short and within
the promised amount of time
(i.e. if you say it will take 15
minutes, make sure it does)
Stay on script as it will ensure
some rigor and consistency to
the study

38.
Take good notes
Take a picture of the participant
Agree on codes with other
researchers
Use forms
Leverage checkboxes
Anticipate usability issues and
count instances
If the study is free flowing, blank
note sheets may be best

39.
Example
Usability Issues P1 P2 P3
Participant has Yelp downloaded on phone x x
Participant uses multiple apps to complete scenario x
Participant does not use filters x x x
Scenario 1 P1 P2 P3
Completion S F S
Post scenario metric score 4 2 5
# of requests for help 0 1 0
# of errors 1 3 1

40.
Capture and synch up
on themes as you go
Log problems and positives
after each participant
Debrief with team along the
way

41.
Recruitment and screening
Timeline
Specific
behavior
Random
Agrees to
participate
Declines to
participate
How do I
select a
participant?
“Ok thank
you, have a
good day!”
Hook ‘em! Initial Screener
Qualified
Disqualified
Consent
Agrees to
participate
Declines to
participate
Start Study!
15sec 15sec 30sec

42.
Interview and study facilitation
Timeline cont…
Qualified
Disqualified
Taking all
qualified
participants
Result of
recruitment &
screening
“Ok thank
you, have a
good day!”
Follow up
demographics
“Thank
you, have a
good day!”
2min 10min
Optional:
Do they fit in a
needed user
segment?
No
Yes
Determine scenario
Run usability
test
Post-study
interview
and metrics
2min 15min total

43.
Analyze

44.
Analyze data
Compile quantitative data
– Task success, Time on task, Satisfaction, Error rates, etc.
Analyze qualitative data
– Problems experiences, comments / recommendations, pathways, think a loud
protocol
Develop exact and concise problem statements
– Good problem statement: 12/14 participants clicked on notifications instead of newsfeed
– Poor problem statement: Clicked on wrong link.
– Poor problem statement: Was confused about links.

45.
Reminder
Focus on what participants did
and said.

46.
Your field research mission

47.
Mission
You have just signed a contract to do usability work for Yelp. They
have been experiencing a decrease in users and realize that they
may need to improve their product. Yelp wants you to do field
research to get qualitative and quantitative feedback on user
behavior with mobile devices when deciding to find a restaurant
while on the go.
Design a field research study to understand how apps or websites
are determining a user’s choice of where to eat, how they use
those apps or websites, and how they may integrate with other
apps or websites to do auxiliary tasks (i.e. getting directions and
travelling to the restaurant).

48.
Steps
Create study plan
– Research questions, roles and responsibilities, scenarios
Conduct field research (at least 3 participants)
Analyze data
Report on findings
– Study approach
– Findings
– Lessons learned and takeaways

49.
Your thoughts
What have you learned about
field studies?
Feedback survey link
www.uxpa2016.org/sessionsurvey?sessi
onid=292

50.
Thank you.