BellaVia Research: Mozilla: When to use focus groups

To Focus Group, or Not To Focus Group: This is the QuestionJulie Francis / BellaVia ResearchJuly 2012 / Presentation to Mozilla 1

Who Am I? Julie Francis  Founder and principal of BellaVia Research  20 years experience  Market Research Background & Usability/UX since 2000 • Creative Good (NYC & SF): online UX strategy consultant • Before that: Socratic Technologies, Audits & Surveys Worldwide  Range of work: • Usability & User Experience • Mobile / online / multi-channel • Contextual inquiry / “ethnography-lite” • Persona creation  “Agnostic” when it comes to methodology • What is the best way to answer the research question? • What methodology best fits budget, need for observation, desired geographic reach, timing, etc. • Huge range of methodologies / approaches  Arizona Public Service  Diageo  Sony  Arhaus.com  Groupon.com  Southern California Edison  Art Institute of Chicago  Interbrand  StubHub.com  ArtThatFits.com  Motorola  thebar.com  Bank of America  OPOWER  Walmart.com  Busch Gardens  Planned Parenthood  Wetpaint.comBellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 2

How we’ll spend our time today…  60 minutes  ½ me talking • Level setting – What the heck is a focus group? • When to use focus groups • When not to use focus groups • About recruiting….  ½ us talking together • What‟s on your mind?BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 3

Research Methodologies: Decisions, Decisions  We’re inundated with methodology options…  Qualitative Research approaches:  In person • Focus groups • IDIs / one-on-one interviews • Mini-groups / Dyads / Triads • Contextual? In-lab?  Online/Remote • Bulletin boards (open or closed) • Diary studies, including mobile phone / text diary studies • Remote moderated phone/online • Social media “listening”  And more…  Hybrid methodologies • Infinite combinations of the above • Hybrid is the current trend – No methodology is perfect. All have strengths & weaknesses… How to choose? • a hybrid approach lets us capitalize on upside of various approachesBellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 4

Baseline: What’s a Focus Group? Focused Group Discussion  Moderated (trained human being)  1.5 – 2 hours (usually)  6-10 people (usually) • Historically groups were larger (10 – 12) • Current trend: Smaller groups (6-8 ) - Why? Deeper inquiry (the whole point of qual) • Also Dyads & Triads (go really deep, but still get the benefit of dialogue you miss in a one-on-one interview…) (M&Ms are the official food of focus groups)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com Photo credit Willie Lunchmeat 5

Baseline: What’s a Focus Group?  Pre-recruited participants • Professional recruiters (facility or independent) • Client or facility-lists • Strangers (usually) • As with any kind of research, good recruiting is a must (weed out “cheaters & repeaters”) (be careful with Craig‟s List & Friends/Family recruiting) • Recruited at least in part by phone (online surveys & email blasts may also be involved) (definitely use the phone – casual conversation helps ensure the person is a good fit and helps identify cheaters) • Over-recruit in anticipation of no-shows  Paid to participate (almost always) • Lures „em in, & thanks people for their time • Helps with “show rate” • Addresses participation bias  Tangent: for bulletin board and other online/written research, (We want busy people not just broke people) use a written open-ended screening question for articulation. • $75 & up Can the person express themselves well in writing with minimal prodding? • Better incentive  better show rateBellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 6

Baseline: Logistics – Lab with Conference Table  Most groups are held in research facilities using a conference table setup & a one-way mirror… Got lots of written exercises or paper to juggle? Use a conference table setup… watchLAB One way mirror in facility  lots of real time observation (get the most bang for your buck by inviting team to watch real-time) Budget conscious? Research doesn‟t need to be “blind”? Use your in-house lab. In rural markets we sometimes set up in hotels. Amplify, PleasantonBellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 7

Baseline: Logistics – Lab with Living Room Setup  There‟s a growing trend towards Living Room Setups at research facilities watchLAB A living room setup can feel more comfortable & less foreign to people who aren‟t used to sitting around conference tables. GREAT for body language (much of which is hidden by conference table) watchLAB, San Francisco Not good if there are lots of papers to juggle (unless a clipboard will suffice)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 8

Baseline: Logistics – in Context  A lot of research (including focus groups) takes place in- context, in the “real world” • Street intercept -- a great way to find “fresh” participants (beware of intercept bias) • In-homes / Contextual – always good to see people in context – Friend Groups, Mom‟s groups • But… you lose the ability for your team to observe (one of the greatest benefits of research) TecEd, Inc. Ethnographic Field Study Susan Hosking & Jon Littel (photo from MomsSupport.org)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 9

Overheard Recently… “I don‟t believe in focus groups” (followed by) “I‟m thinking about getting together a group of people to talk about this topic.”  (Um…call it what you will… but that’s what I call a focus group)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 10

Focus Groups are getting a bad rap “Focus groups are dead. The paradigm has shifted.” “it‟s observational vs. directed market research” “awkward tactic”  Do they deserve this bad rap? IMHO, most “problem groups” stem from unrealistic expectations, improper use, DIY moderators who lack training/ experience  Clients tell researchers “I want to do some focus groups” when what they mean is “I want to understand my target” (and focus groups aren’t always the right fit)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 11

Let’s not bury focus groups All approaches can work well when used appropriately There is a role for focus groups It can be incredibly insightful to hear a group of people banter about your product/service/category See Francesca Johnson, Research Arts: http://www.researcharts.com/2011/01/in-praise-of-focus-groups-and-moderators/BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 12

When to use focus groups?  IMHO, Focus Groups are best when you want/need to: Explore needs / Explore thoughts / feelings Explore brand consumer perceptions language / issues (fuel for innovation, not (yours, competitors) iteration) (to inform/explain survey research) Understand Uncover what Explore use decision process you don‟t cases / factors know… (why, where, when, & how…)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 13

Focus Groups are not my first choice for…  IMHO, Focus Groups are not the best fit for… Usability Advertising impact / (product / website / Design iteration communication mobile – don’t do it) Understanding Situations where what people really you worry about do (instead of what people influencing they say they do) each other  Sometimes we use groups anyway. Why?  Efficiency / Speed (You can interview 24 people in groups much faster than you can in 1:1)  Budget (if it’s faster, it’s often cheaper too)  Client comfort (Clients ask for focus groups because it’s what they are used to)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 14

Focus Groups are Best When… They are open & exploratory • You don’t know what you don’t know • Use projective techniques to get below the surface (don‟t knock it until you‟ve tried it!) Banter happens! • You use the group dynamic to its full advantage! (and use a trained moderator, to help manage the chaos) • Let people interact, banter, build on each other‟s ideas, trigger each other with experiences/questions, etc. That‟s where some of the richest insights come from… The guide isn‟t too packed • Allow conversation to happen! Give people time to talk, banter, build, interact – let the magic unfold • Litmus test: Aim to wonder “Uh oh, is this enough to fill 90 minutes?” • (FYI, we’re seeing a trend towards clients packing guides…which diminishes our ability to get good quality insights.) There is no counting • It‟s not quant. It‟s qual. It‟s all about what drives opinion (the WHY), & the range of opinion/experience. It‟s not definitive. It‟s often followed by quant research, for good reason.BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 15

What do I mean by projective technique?  One example: picture selection • i.e., “Which of these images best describes how you feel about _______________”? How and why does that fit? (then probe, probe, and probe some more) • This seemingly bizarre approach is brilliant for cutting through all the superficial, high level words down to the core issue. Trust me, it‟s brilliant. • Each pictures should have multiple interpretations. Avoid literal images • istockphoto is great for abstract searches (but the pictures can be really expensive, and it drives me CRAZY that their photo credits expire. • Not just for focus groups; Works great in 1:1 interviews, too. Also, works both online & in-person. I‟ve used them in all sorts of methodologies with great success.  There are lots more projective techniques, but this is one of the most common and least terrifying to use.BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 16

How to Choose a Methodology?  Always start with your objectives: What are you trying to learn? Corporate See behavior Consider User Tactical Usability Identity (logo, in action Experience Feedback name) (observational) IDI (1:1) Explore drivers / To help you Understand Consider When you don‟t barriers / behavior/decision Fuel for innovation know what you perceptions / prepare for quant drivers, longitudinal Focus research (identify don‟t know attitudes / brand behaviors Groups issues & language) perceptions (reflective)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 17

More Methodology Considerations  How important is team observation? • In-lab can be more efficient (complete more sessions per day) • You think going to a focus group is weird? Having 2 researchers & a videographer come into your home is even weirder! Observation Less important – Observation More important – do your research in-context do your research in a lab • If you don‟t have observers, you can go in- • Labs are great for team observation. When else do context! In-homes, street you get to spend a full day as a group in the presence of your customers? • Use a videographer • Cram up to 20 observers in the back room. Great • Consider a highlights reel (people have bang for your research buck. (facilitate the good intentions to watch videos, but they observers!) don‟t) (mic your participant) Best practice: Hybrid! (capitalize on the upside / strength of each approach)  Other considerations: Timing / budget / need for geographic reachBellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 18

Focus Group Moderating Toolbox  What‟s in my toolbox? Moderating to the Max: A full-tilt guide to Dominators, Cynics & Wallflowers: creative, insightful focus groups and depth Practical Strategies for Moderating interviews (Bystedt, Lynn and Potts) Meaningful Focus Groups (Kahle) • My favorite reference book on “how to” for projective • Moderating groups is really, really hard. techniques • This book is a lifesaver for dealing with group dynamics. Secrets of a Master Moderator Riva Training Institute: Hands-on Training (Henderson) • Moderate in front of masters & your peers, get • Naomi Henderson is a brilliant moderator, & the critiqued (rinse, repeat) founder of the RIVA Training Institute • Rockville, MD (unless you organize a custom course) • Fundamentals of Moderating course: 3 days: $3600, worth every penny (really) QRCA (Qualitative Research Consultants Impulse Survey of Focus Facilities Association) • Find the best facilities in every market! Rated by • Webinars recruit, location, value, etc. • Free QRCA Views Magazine • Aim for an overall rating of 85 with lots of reviewers. • Find a Researcher • Online version is $65 • Great annual conference (research suppliers only)BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 19

A word on recruiting…  You didn‟t ask me about recruiting, but I can‟t resist a chance to talk about it  Getting harder & harder to find quality participants who truly qualify (and aren‟t oddballs)  Lots of “friends and family” research – is that really your target?  Craigslist can be a problematic source! Easy to find warm bodies, but hard to find quality recruits • Tend to be in it for the money (why else would you be trolling for focus groups on CraigsList?) • Lots of dishonesty -- i.e., Woman in SF with >75 aliases, fake Facebook profiles, etc. • According to one recruiter, 50% of her leads from Craigslist try to fake their way into research. • Be careful! Many professional recruiters have database they build in a variety of ways – use them, use your own customer lists, and use Craigslist sparingly & carefully (i.e., for niche recruits) It‟s very easy to “lie” your way into research when the posting or email blast says exactly the type of participant they are looking for…BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 20

Tips for Finding Great Research Participants Open 2-way Use carefully created Find cheaters / Ensure qualifications & communication with the materials! repeaters engagement recruiter • Use a professional • Hide your topic in the • Does your recruiter • Maximize open-ended (it‟s worth it. Budget email blast (approve have a “wall of screening (engage in $75-$150 per or write it yourself) shame”? Shared with conversation, don‟t consumer recruit, more • Hide your topic in the other recruiters? give them a list of for specialized B2B) pre-screener survey • Use “dummy” brands/behaviors to • Bring the recruiter onto (approve or write it questions to sniff out click on) the project team yourself) dishonesty (ask about • Make it easy for before you start work • (don’t bias your flavors / features that people to be honest on the screener research before it don‟t exist (i.e., “we‟re looking for even starts. People • Require ID & write a mix, so please be • Explain the type of do, and will, cheat check to that exact honest”) person you are looking their way into name • Re-screen: confirm for, & agree upon strategies to find research) • Verify identities key questions (pretend • B2B: Call employer it‟s important to “get it these people exactly right,” i.e., “I‟m • (In the dog example: and ask for them by • Agree what “research with pet title (esp. if they sorry I didn‟t take good sources/approaches owners” should be won‟t share their notes on this”) will & will not be enough, especially if company email • Homework! (send me used (i.e., when/how you make people address) a picture of your music to use Craig‟s List, collection, see prove they have a • Look at their email blasts, Survey dog via lots of Pinterest page, see LinkedIn Profile (but Monkey) pictures, & bring their your kitchen, a beware of fake • Include original treats into the profiles). For senior screenshot of your participant source on group… pictures of execs, look at the iPhone apps…) recruiting profiles dog’s gnarly teeth?) corporate website.BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 21

Great Recruiters Hagen Sinclair •Cynthia Hagen-Cross, President & CEO •National recruiting, general pop & hard to find niche populations, online & in-person methodologies •www.hagensinclair.com/ / hagenc@hagensinclair.com / (530) 899-3140 Blarry House Research •Annaliese Furnas, CEO •National recruiting, general pop & hard to find niche populations, online & in-person methodologies •www.blarry.com / annaliese@blarry.com / 415-648-2196 Satellite Research •Claudia Page, Owner & Research Specialist •National recruiting, general pop & hard to find niche populations, medical, online & in-person methodologies •www.satelliteresearch.net/ / claudia@satelliteresearch.net / (888) 639-5312 Intact Qualitative •Richard Ngo, Partner •SF recruiting only (& SOMA loft facility on 3rd Street) •Great for SF Bay Area in-home or contextual interviews •www.intactqualitativeresearch.com/ / richard@intactqualitativeresearch.com / 415.400.5945 Your Research Facility •Doing research in a hired research facility? Use their team •First, check Impulse Survey to make sure they are highly reviewed on recruitingBellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 22

Your Turn!BellaVia Research • User Experience & Market Research • Santa Cruz CA • 831.454.8217 • info@bellaviaresearch.com 23

Want to talk methodology?Need a recommendation on a recruiter,videographer, transcriptionist, facility…? Ping me! Julie FrancisPrincipal & Founder, BellaVia Research www.bellaviaresearch.com @BellaVia 831.454.8217 BellaVia Research ©2011 24

BellaVia Research: Mozilla: When to use focus groups

by BellaVia Research on Jul 25, 2012

Accessibility

Categories

Upload Details

Usage Rights

Statistics

BellaVia Research: Mozilla: When to use focus groups Presentation Transcript