Hearing that focus groups are a dead "yesterday" methodology? Wondering if & when they could be relevant to your company? This presentation outlines what they are, current trends, best practices for when to use and when to avoid focus groups, and the dreadfully-boring but oh-so-important topic of quality recruiting for qualitative research
Understanding the Pakistan Budgeting Process: Basics and Key Insights
Â
BellaVia Research: Mozilla: When to use focus groups
1. To Focus Group,
or Not To Focus Group:
This is the Question
Julie Francis / BellaVia Research
July 2012 / Presentation to Mozilla
1
2. Who Am I? Julie Francis
ď§ Founder and principal of BellaVia Research
ď§ 20 years experience
ď§ Market Research Background & Usability/UX since 2000
⢠Creative Good (NYC & SF): online UX strategy consultant
⢠Before that: Socratic Technologies, Audits & Surveys Worldwide
ď§ Range of work:
⢠Usability & User Experience
⢠Mobile / online / multi-channel
⢠Contextual inquiry / âethnography-liteâ
⢠Persona creation
ď§ âAgnosticâ when it comes to methodology
⢠What is the best way to answer the research question?
⢠What methodology best fits budget, need for observation, desired geographic reach, timing, etc.
⢠Huge range of methodologies / approaches
ď§ Arizona Public Service ď§ Diageo ď§ Sony
ď§ Arhaus.com ď§ Groupon.com ď§ Southern California Edison
ď§ Art Institute of Chicago ď§ Interbrand ď§ StubHub.com
ď§ ArtThatFits.com ď§ Motorola ď§ thebar.com
ď§ Bank of America ď§ OPOWER ď§ Walmart.com
ď§ Busch Gardens ď§ Planned Parenthood ď§ Wetpaint.com
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 2
3. How weâll spend our time todayâŚ
ď§ 60 minutes
ď§ Â˝ me talking
⢠Level setting â What the heck is a focus group?
⢠When to use focus groups
⢠When not to use focus groups
⢠About recruitingâŚ.
ď§ Â˝ us talking together
⢠Whatâs on your mind?
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 3
4. Research Methodologies: Decisions, Decisions
ď§ Weâre inundated with methodology optionsâŚ
ď§ Qualitative Research approaches:
ď§ In person
⢠Focus groups
⢠IDIs / one-on-one interviews
⢠Mini-groups / Dyads / Triads
⢠Contextual? In-lab?
ď§ Online/Remote
⢠Bulletin boards (open or closed)
⢠Diary studies, including mobile phone / text diary studies
⢠Remote moderated phone/online
⢠Social media âlisteningâ
ď§ And moreâŚ
ď§ Hybrid methodologies
⢠Infinite combinations of the above
⢠Hybrid is the current trend â
No methodology is perfect.
All have strengths & weaknessesâŚ
How to choose?
⢠a hybrid approach lets us
capitalize on upside of various approaches
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 4
5. Baseline: Whatâs a Focus Group?
ď§Focused Group Discussion
ď§ Moderated (trained human being)
ď§ 1.5 â 2 hours (usually)
ď§ 6-10 people (usually)
⢠Historically groups were larger (10 â 12)
⢠Current trend: Smaller groups (6-8 )
- Why? Deeper inquiry (the whole point of qual)
⢠Also Dyads & Triads (go really deep, but still get the benefit
of dialogue you miss in a one-on-one interviewâŚ)
(M&Ms are the official food of focus groups)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com Photo credit Willie Lunchmeat 5
6. Baseline: Whatâs a Focus Group?
ď§ Pre-recruited participants
⢠Professional recruiters (facility or independent)
⢠Client or facility-lists
⢠Strangers (usually)
⢠As with any kind of research, good recruiting is a must
(weed out âcheaters & repeatersâ)
(be careful with Craigâs List & Friends/Family recruiting)
⢠Recruited at least in part by phone (online surveys & email
blasts may also be involved) (definitely use the phone â
casual conversation helps ensure the person is a good fit
and helps identify cheaters)
⢠Over-recruit in anticipation of no-shows
ď§ Paid to participate (almost always)
⢠Lures âem in, & thanks people for their time
⢠Helps with âshow rateâ
⢠Addresses participation bias ď Tangent: for bulletin board and other online/written research,
(We want busy people not just broke people) use a written open-ended screening question for articulation.
⢠$75 & up Can the person express themselves well in writing with
minimal prodding?
⢠Better incentive ď¨ better show rate
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 6
7. Baseline: Logistics â Lab with Conference Table
ď§ Most groups are held in research facilities using a conference table setup & a one-way mirrorâŚ
Got lots of written exercises or
paper to juggle?
Use a conference table setupâŚ
watchLAB
One way mirror in facility ď¨
lots of real time observation
(get the most bang for your buck by inviting team
to watch real-time)
Budget conscious? Research doesnât need to
be âblindâ? Use your in-house lab.
In rural markets we sometimes set up in hotels. Amplify, Pleasanton
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 7
8. Baseline: Logistics â Lab with Living Room Setup
ď§ Thereâs a growing trend towards Living Room Setups at research facilities
watchLAB
A living room setup can feel more comfortable &
less foreign to people who arenât used to sitting
around conference tables.
GREAT for body language
(much of which is hidden by conference table)
watchLAB, San Francisco
Not good if there are lots of papers to juggle
(unless a clipboard will suffice)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 8
9. Baseline: Logistics â in Context
ď§ A lot of research (including focus groups) takes place in-
context, in the âreal worldâ
⢠Street intercept -- a great way to find âfreshâ participants
(beware of intercept bias)
⢠In-homes / Contextual â always good to see people in context â
Friend Groups, Momâs groups
⢠But⌠you lose the ability for your team to observe (one of the
greatest benefits of research)
TecEd, Inc. Ethnographic Field Study
Susan Hosking & Jon Littel
(photo from MomsSupport.org)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 9
10. Overheard RecentlyâŚ
âI donât believe in focus groupsâ
(followed by)
âIâm thinking about getting
together a group of people
to talk about this topic.â
ď (UmâŚcall it what you willâŚ
but thatâs what I call a focus group)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 10
11. Focus Groups are getting a bad rap
âFocus groups are
dead. The paradigm
has shifted.â âitâs observational
vs. directed market
researchâ
âawkward tacticâ
ď Do they deserve this bad rap? IMHO,
most âproblem groupsâ stem from
unrealistic expectations, improper use,
DIY moderators who lack training/
experience
ď Clients tell researchers âI want to do some
focus groupsâ when what they mean is âI
want to understand my targetâ (and focus
groups arenât always the right fit)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 11
12. Letâs not bury focus groups
All approaches can work well
when used appropriately
There is a role for focus groups
It can be incredibly insightful to hear
a group of people banter about your
product/service/category
See Francesca Johnson, Research Arts:
http://www.researcharts.com/2011/01/in-praise-of-focus-groups-and-moderators/
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 12
13. When to use focus groups?
ď§ IMHO, Focus Groups are best when you want/need to:
Explore needs / Explore
thoughts / feelings Explore brand consumer
perceptions language / issues
(fuel for innovation, not (yours, competitors)
iteration) (to inform/explain survey
research)
Understand Uncover what Explore use
decision process you donât cases
/ factors know⌠(why, where, when, &
howâŚ)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 13
14. Focus Groups are not my first choice forâŚ
ď§ IMHO, Focus Groups are not the best fit forâŚ
Usability
Advertising impact /
(product / website / Design iteration
communication
mobile â donât do it)
Understanding Situations where
what people really you worry about
do (instead of what people influencing
they say they do) each other
ď Sometimes we use groups anyway. Why?
ď Efficiency / Speed
(You can interview 24 people in groups much faster than you can in 1:1)
ď Budget (if itâs faster, itâs often cheaper too)
ď Client comfort (Clients ask for focus groups because itâs what they are used to)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 14
15. Focus Groups are Best WhenâŚ
They are open & exploratory
⢠You donât know what you donât know
⢠Use projective techniques to get below the surface (donât knock it until youâve tried it!)
Banter happens!
⢠You use the group dynamic to its full advantage! (and use a trained moderator, to help
manage the chaos)
⢠Let people interact, banter, build on each otherâs ideas, trigger each other with
experiences/questions, etc. Thatâs where some of the richest insights come fromâŚ
The guide isnât too packed
⢠Allow conversation to happen! Give people time to talk, banter, build, interact â let the
magic unfold
⢠Litmus test: Aim to wonder âUh oh, is this enough to fill 90 minutes?â
⢠(FYI, weâre seeing a trend towards clients packing guidesâŚwhich diminishes our ability to
get good quality insights.)
There is no counting
⢠Itâs not quant. Itâs qual. Itâs all about what drives opinion (the WHY), & the range of
opinion/experience. Itâs not definitive. Itâs often followed by quant research, for good
reason.
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 15
16. What do I mean by projective technique?
ď§ One example: picture selection
⢠i.e., âWhich of these images best describes how
you feel about _______________â? How and why
does that fit? (then probe, probe, and probe some
more)
⢠This seemingly bizarre approach is brilliant for
cutting through all the superficial, high level
words down to the core issue. Trust me, itâs
brilliant.
⢠Each pictures should have multiple interpretations.
Avoid literal images
⢠istockphoto is great for abstract searches (but the
pictures can be really expensive, and it drives me
CRAZY that their photo credits expire.
⢠Not just for focus groups; Works great in 1:1
interviews, too. Also, works both online & in-person.
Iâve used them in all sorts of methodologies with great
success.
ď§ There are lots more projective techniques, but
this is one of the most common and least
terrifying to use.
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 16
17. How to Choose a Methodology?
ď§ Always start with your objectives: What are you trying to learn?
Corporate See behavior Consider
User Tactical
Usability Identity (logo, in action
Experience Feedback
name) (observational)
IDI (1:1)
Explore drivers /
To help you
Understand Consider
When you donât barriers / behavior/decision
Fuel for innovation know what you perceptions /
prepare for quant
drivers, longitudinal Focus
research (identify
donât know attitudes / brand behaviors Groups
issues & language)
perceptions (reflective)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 17
18. More Methodology Considerations
ď§ How important is team observation?
⢠In-lab can be more efficient (complete more sessions per day)
⢠You think going to a focus group is weird? Having 2 researchers & a videographer come into your home is even weirder!
Observation Less important â Observation More important â
do your research in-context do your research in a lab
⢠If you donât have observers, you can go in- ⢠Labs are great for team observation. When else do
context! In-homes, street you get to spend a full day as a group in the presence
of your customers?
⢠Use a videographer
⢠Cram up to 20 observers in the back room. Great
⢠Consider a highlights reel (people have bang for your research buck. (facilitate the
good intentions to watch videos, but they observers!)
donât) (mic your participant)
Best practice:
Hybrid! (capitalize on the upside / strength of each approach)
ď§ Other considerations: Timing / budget / need for geographic reach
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 18
19. Focus Group Moderating Toolbox
ď§ Whatâs in my toolbox?
Moderating to the Max: A full-tilt guide to Dominators, Cynics & Wallflowers:
creative, insightful focus groups and depth Practical Strategies for Moderating
interviews (Bystedt, Lynn and Potts) Meaningful Focus Groups (Kahle)
⢠My favorite reference book on âhow toâ for projective ⢠Moderating groups is really, really hard.
techniques ⢠This book is a lifesaver for dealing with group
dynamics.
Secrets of a Master Moderator Riva Training Institute: Hands-on Training
(Henderson) ⢠Moderate in front of masters & your peers, get
⢠Naomi Henderson is a brilliant moderator, & the critiqued (rinse, repeat)
founder of the RIVA Training Institute ⢠Rockville, MD (unless you organize a custom course)
⢠Fundamentals of Moderating course: 3 days:
$3600, worth every penny (really)
QRCA (Qualitative Research Consultants Impulse Survey of Focus Facilities
Association) ⢠Find the best facilities in every market! Rated by
⢠Webinars recruit, location, value, etc.
⢠Free QRCA Views Magazine ⢠Aim for an overall rating of 85 with lots of reviewers.
⢠Find a Researcher ⢠Online version is $65
⢠Great annual conference (research suppliers only)
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 19
20. A word on recruitingâŚ
ď§ You didnât ask me about recruiting, but I canât resist a chance to talk about it
ď§ Getting harder & harder to find quality participants who truly qualify (and arenât oddballs)
ď§ Lots of âfriends and familyâ research â is that really your target?
ď§ Craigslist can be a problematic source! Easy to find warm bodies, but hard to find quality recruits
⢠Tend to be in it for the money (why else would you be trolling for focus groups on CraigsList?)
⢠Lots of dishonesty -- i.e., Woman in SF with >75 aliases, fake Facebook profiles, etc.
⢠According to one recruiter, 50% of her leads from Craigslist try to fake their way into research.
⢠Be careful! Many professional recruiters have database they build in a variety of ways â use them, use your own customer
lists, and use Craigslist sparingly & carefully (i.e., for niche recruits)
Itâs very easy to âlieâ your way into
research when the posting or email
blast says exactly the type of
participant they are looking forâŚ
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 20
21. Tips for Finding Great Research Participants
Open 2-way
Use carefully created Find cheaters / Ensure qualifications &
communication with the
materials! repeaters engagement
recruiter
⢠Use a professional ⢠Hide your topic in the ⢠Does your recruiter ⢠Maximize open-ended
(itâs worth it. Budget email blast (approve have a âwall of screening (engage in
$75-$150 per or write it yourself) shameâ? Shared with conversation, donât
consumer recruit, more ⢠Hide your topic in the other recruiters? give them a list of
for specialized B2B) pre-screener survey ⢠Use âdummyâ brands/behaviors to
⢠Bring the recruiter onto (approve or write it questions to sniff out click on)
the project team yourself) dishonesty (ask about ⢠Make it easy for
before you start work ⢠(donât bias your flavors / features that people to be honest
on the screener research before it donât exist (i.e., âweâre looking for
even starts. People ⢠Require ID & write a mix, so please be
⢠Explain the type of do, and will, cheat check to that exact honestâ)
person you are looking their way into name ⢠Re-screen: confirm
for, & agree upon
strategies to find
research) ⢠Verify identities key questions (pretend
⢠B2B: Call employer itâs important to âget it
these people exactly right,â i.e., âIâm
⢠(In the dog example: and ask for them by
⢠Agree what âresearch with pet title (esp. if they sorry I didnât take good
sources/approaches ownersâ should be wonât share their notes on thisâ)
will & will not be enough, especially if company email ⢠Homework! (send me
used (i.e., when/how you make people address) a picture of your music
to use Craigâs List, collection, see
prove they have a ⢠Look at their
email blasts, Survey dog via lots of Pinterest page, see
LinkedIn Profile (but
Monkey) pictures, & bring their your kitchen, a
beware of fake
⢠Include original treats into the profiles). For senior screenshot of your
participant source on group⌠pictures of execs, look at the iPhone appsâŚ)
recruiting profiles dogâs gnarly teeth?) corporate website.
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 21
22. Great Recruiters
Hagen Sinclair
â˘Cynthia Hagen-Cross, President & CEO
â˘National recruiting, general pop & hard to find niche populations, online & in-person methodologies
â˘www.hagensinclair.com/ / hagenc@hagensinclair.com / (530) 899-3140
Blarry House Research
â˘Annaliese Furnas, CEO
â˘National recruiting, general pop & hard to find niche populations, online & in-person methodologies
â˘www.blarry.com / annaliese@blarry.com / 415-648-2196
Satellite Research
â˘Claudia Page, Owner & Research Specialist
â˘National recruiting, general pop & hard to find niche populations, medical, online & in-person methodologies
â˘www.satelliteresearch.net/ / claudia@satelliteresearch.net / (888) 639-5312
Intact Qualitative
â˘Richard Ngo, Partner
â˘SF recruiting only (& SOMA loft facility on 3rd Street)
â˘Great for SF Bay Area in-home or contextual interviews
â˘www.intactqualitativeresearch.com/ / richard@intactqualitativeresearch.com / 415.400.5945
Your Research Facility
â˘Doing research in a hired research facility? Use their team
â˘First, check Impulse Survey to make sure they are highly reviewed on recruiting
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 22
23. Your Turn!
BellaVia Research ⢠User Experience & Market Research ⢠Santa Cruz CA ⢠831.454.8217 ⢠info@bellaviaresearch.com 23
24. Want to talk methodology?
Need a recommendation on a recruiter,
videographer, transcriptionist, facility�
Ping me!
Julie Francis
Principal & Founder, BellaVia Research
www.bellaviaresearch.com
@BellaVia
831.454.8217
BellaVia Research Š2011 24
Editor's Notes
Steve Jobs didnât believe in focus groups for design. âIt's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them.â Steve Jobs, BusinessWeek, May 25 1998Focus groups shouldnât be used to ask people for design insights / solutions. Instead, they should be used to identify pain points so the strategists & designers can come up with the solutions!
For innovation? Yes, for early stage. For design iteration? Nope. Famous henry Ford quote:
Focus Groups are great when you know relatively little about a topic. Need to know how people perceive your brand / service / product? Want to understand what issues are most important? How people talk about something (language)? Need to define topics or develop questions for quantitative survey research? Need to listen for what you donât even know aboutâŚas fodder for innovation?Examples:Thoughts / needs / feelings: Tequila â woman casually mentions that she does shots only to cut down on calories, bc margaritas have too many caloriesBrand perception & personality: People have a hard time articulating the differences between brands. Itâs not a simple âask & answer.â But by using projective techniques, you can cut straight to the heart of it. (colleges, amusement parks)Innovation & Use cases:Party planningâŚHow do you use the internet to help you plan parties? (also did usability testing, and lots of it, but it was focus groups that provided big picture fuel for innovation)Decision process & factors: How do you decide which piece of art to buy? (to fuel site development)Communication: campaign for new alcoholic beverageWhat you donât know: Why use us? Why are we losing customers? Situations where I wish I had done 1:1 interviews:Naming (highly subjective)Logos (highly subjective)Advertising (agencies are terrified of group think)âjust project the site and see what they thinkâ (only good for brand impressions, nothing about usability)
Also, great for projectives. Get below the high level, gut responses.