The document is a practical guide for conducting actionable audience research presented at the NAMP Conference 2018. It outlines an audience research strategy that includes defining research questions, choosing appropriate research methods, and utilizing data effectively to drive decision-making. By the end of the guide, participants should be able to plan and execute audience research projects that lead to tangible organizational changes.

1. A PRACTICAL GUIDE
TO ACTIONABLE
AUDIENCE RESEARCH
NAMP Conference 2018 | #NAMPC
@devonvsmith | www.measurecrea?ve.com

2. HI! I’M DEVON.
I’m the co-founder of Measure Crea4ve.

3. AGENDA
• Audience research strategy
• Ac4vity #1: Framing your research ques4on
• Research methods
• Ac4vity #2: Map out your plan
• Making decisions with data
• Q&A

4. BY THE END, YOU SHOULD BE ABLE TO:
• Plan for an audience research project
• Overcome common challenges and pivot your approach
• Translate research findings into prac4cal change
www.slideshare.net/devonvsmith

5. STRATEGY

6. WHY AUDIENCE RESEARCH?
When we don’t know:
• Who our audiences are, or could be
• What our audiences want, think, or respond to
• Where the opportunity is for something new
• How much to invest in changing our current approach

7. Ac4onable research depends on precisely defining the
ques4on you’re trying to answer.
Tips:
• Be specific
• Narrow your focus
• Make sure the ques4on is answerable
CHOOSING A RESEARCH QUESTION

8. Examples:
• How are new audiences this year demographically dis4nct
from current audiences?
• What are the 3 most important ideas to communicate to
poten4al donors?
• How do La4nX audiences use our space differently than
other audiences?
• What type of content on our website convinces 4cket
buyers to purchase?
CHOOSING A RESEARCH QUESTION

9. Ac4onable research depends on precisely defining the
ques4on you’re trying to answer.
Tips:
• Narrow your focus
• Establish a baseline
• Lean toward skep4cs and those with barriers
DEFINING YOUR AUDIENCES

10. Examples:
• Donors giving under $50 during the past two years
• Poten4al sponsors, not yet commi_ed
• Survey respondents who were unaware we did _______
• La4nX a_endees to _______ exhibit
• Families with kids aged 8-18 and financial means
DEFINING YOUR AUDIENCES

11. ACTIVITY #1: FRAME
YOUR OWN RESEARCH
QUESTION

12. FRAME YOUR OWN RESEARCH QUESTION
Begin your audience research plan by framing your goals.
Step #1: Choose and frame the research ques4on
• Be: specific, narrow, answerable
Step #2: Define the audience
• Be specific, use comparison, focus on problem areas
Step #3: Pay a_en4on in the next sec4on on research methods
• Is this right for my research ques4on and audience type?
• Does this fit with my budget, 4meline, and situa4on?

13. RESEARCH METHODS

14. CHOOSING THE RIGHT METHOD
Take a balanced approach to quan4ta4ve and qualita4ve methods.
Quan%ta%ve
Answers “who,” “what,” and “how many”
Qualita%ve
Answers “why” and “what else”
Scalabe Not representa/ve
Fewer resources Higher investment
Less subjec/ve Difficult to find trends
More truthful answers Not as reliable
Predetermined, limited range of ques/ons Dynamic process
Slower process Faster results
Shallow responses Address sensi/ve issues
Data can by dry Stories are more memorable

15. Choose the method(s) that best fit your needs.
7 COMMON METHODS
Mostly quan4ta4ve Mixed Mostly qualita4ve
Exis/ng data sources Observa/onal study Interviews
Surveys User tes/ng Focus groups
A/B tes/ng

16. Use the data you already
have access to.
Try it when you:
• Don’t have money or 4me
• Want to compare your
audience to others
• Previously collected data
METHOD #1: EXISTING DATA SOURCES
Format:
Timeline:
Your data, community
data, peer data
1 day - 1 week

17. Sources:
• Google Analy4cs, Facebook, Twi_er, YouTube analy4cs
(demographics)
• Yelp, Foursquare, 4cke4ng site reviews (percep4ons)
• Surveys/interviews from prior audience research
• DataUSA.io (census)
• Na4onal Archive of Data on Arts & Culture (analysis)
• Arts.gov: Arts Data Profiles (par4cipa4on, outcomes)
• SMU/DataArts: Cultural Data Profile (peers)
METHOD #1: EXISTING DATA SOURCES

18. METHOD #2: SURVEYS
The most common (and
ohen misused) method.
Try it when you:
• Have many ques4ons to ask
• Want to represent an en4re
popula4on
• Want to benchmark data
Timeline:
Incen4ves:
Length:
Format: Online, iPad,
interac4ve, paper
Number: 1,000 responses
(1-10% response rate)
Tools: SurveyMonkey, Google
Surveys, HotJar
10 min
2-4 weeks
Usually unnecessary

19. Tips:
• Have 3 colleagues take the survey first
• Leave online surveys open for 1-2 weeks, promote it 3x
• Make the first ques4on easy, last ques4on open-ended
• For sensi4ve ques4ons, ask what a friend would do
• Ask for emails of those with interest in future research
opportuni4es
• An4cipate cross-tab analysis opportuni4es
METHOD #2: SURVEYS

20. Watch out for:
• Overlapping or missing answer choices
• Biased ranking answer choices
• 2+ consecu4ve open-ended ques4ons
• 5+ answer choice op4ons
• “Double barreled” ques4ons (asking 2 things in 1 ques4on)
• Undefined 4me periods
METHOD #2: SURVEYS

21. Defined choice quesBon examples:
• Think of the most recent 4me you _______. Which of the following
reason(s) best represent what mo4vated you to _______?
(reason 1, reason 2, reason 3)
• How likely are you to do the following within the next year?
(Unlikely, somewhat likely, very likely)
• How important to you is it that _______ does the following?
(This is a big factor in my decision to support; This plays a small role in my
decision to support; This doesn’t play any role in my decision to support; I
wasn’t aware ______ did this.)
METHOD #2: SURVEYS

22. Open-ended quesBon examples:
• How would you describe _______ to a friend who was unfamiliar
with them?
• What is one thing _______ could do to reach more of your friends?
• What is one thing _______ could do or tell you that would make
you consider becoming a member?
• If _______ could do one thing to improve _______, what would it
be?
METHOD #2: SURVEYS

23. METHOD #3: A/B TESTING
Audience A tries version A;
Audience B tries version B;
which gets the best results?
Try it when you:
• Have stakeholders that
disagree on solu4on
• Can build a quick, cheap
prototype
• An4cipate a singular, “majority
wins” outcome
Format:
Timeline:
Resources:
Tools:
Number:
Email, social ads,
website, in-person
1,000+ per variant
(online),
50+ (in-person)
1 day - 4+ weeks
Google Op4mize
“A/B Tes4ng” by Dan
Siroker, Op4mizely.com

24. Tips:
• Have a single difference between variants
• Ensure similar audiences are trying version A and B
• Don’t run mul4ple A/B tests in parallel
• Before: write down your hypothesis
• Aher: summarize test varia4ons and results for reference later
• Not all tests will have a clear “winning” result
METHOD #3: A/B TESTING

25. METHOD #4: OBSERVATIONAL STUDIES
Watch audience behavior while
they’re in your space or another
space.
Try it when you:
• Want to understand new
audiences or behaviors
• Want to improve an exis4ng
process
• Expect easy to interpret
behavior
Format:
Timeline:
People:
Tools:
Length:
Number:
Dwell 4me, naviga4on
path, public spaces
50+ par4cipants
1-4 hours
1+ observers
1 day - 1 week
Phone or paper

26. Tips:
• Maintain inter-observer consistency through training
• Create an “observa4onal protocol”
• Track 3-10 elements of behavior or demographics
• Observer should be unobtrusive
• Consider informed consent (signage)
METHOD #4: OBSERVATIONAL STUDIES

27. METHOD #5: USER TESTING
Watch audiences while they
use a simplified, early version
of a poten4al solu4on.
Try it when you:
• Have a working prototype
• Have a simple task for a user to
complete
• Are comfortable sharing in-
progress
Format:
People:
Length:
Number:
In-person, screen-share
10-20 par4cipants
5-30 minutes
Moderator (+ video)
Timeline: 1 day - 1 week
Incen4ves: Jus4fied (gih card,
4cket, merch)

28. Tips:
• Give the par4cipant a clearly defined task
• Define a list of user behaviors you expect to see
• Ask par4cipants to narrate their thought process
• Watch for moments of confusion
• Resist the tempta4on to step in
METHOD #5: USER TESTING

29. METHOD #6: INTERVIEWS
People like to be asked their
opinions, you’re (probably)
not bothering them.
Try it when you:
• Need to gather stories,
situa4ons, characters
• Have poten4ally sensi4ve or
complex topics to discuss
Format:
Timeline:
People:
Incen4ves:
Length:
Number:
Phone, video, in-person,
recorded kiosk
3-5 interviewees per
type (10-25% response)
5-30 minutes
1 interviewer
(+ record audio)
1-4 weeks
Usually unnecessary

30. Tips:
• Have 5-10 ques4ons planned, but stay flexible
• Be transparent with interview purpose, a_ribu4on
• Give interviewees an “out” for difficult ques4ons
• Ask interviewees to ar4culate a problem or a situa4on,
not to find a solu4on
• Plan for no-shows, re-schedules, and “bad” interviews
METHOD #6: INTERVIEWS

31. Interview quesBon examples:
• Tell me about the first 4me you learned about _______.
• What was your reac4on when _______ did _______?
• Walk me through the moments just before your most recent
decision to _______.
• I’m going to give you a short descrip4on of six different quali4es
about _______. We want to hear your take on how each resonates
with you.
• What are the benefits that you value most from _______?
METHOD #6: INTERVIEWS

32. METHOD #7: FOCUS GROUPS
When audiences interact
with each other, new
conversa4ons arise.
Try it when you:
• Have audiences in-person
• Want to hear diverse or
conflic4ng opinions, astudes
• Want to go deep on 1-3 topics
4-7 par4cipants
per group
Number:
Timeline:
People:
Incen4ves:
Length:
Snacks, merch
1-2 weeks
Moderator and note-
taker (+ record audio)
60-90 minutes

33. Tips:
• Consider power dynamics of the group
• Mix individual ques4ons, group conversa4on, and an ac4vity
• Run at least one prac4ce session
• Ask the group to reflect or react to each other’s answers
• Summarize what you hear from the group and see if they concur
METHOD #7: FOCUS GROUPS

34. Example agenda:
• Solo ques4on: What was your awareness of _______ when we first
contacted you?
• Scenario 1: If you saw this ad on Facebook, what would your first
reac4on be? Your next step?
• Group conversa4on: Can anyone share an example of a recent 4me
that they _______? How similar is everyone else’s experience?
• Scenario 2: If you visited this sec4on of the website, what would you
expect to find?
• Group ac4vity: sor4ng items into categories; walk me through why
you chose these categories.
METHOD #7: FOCUS GROUPS

35. RESEARCH PLAN: SMALL SIZE EXAMPLE
Research quesBon(s):
• Which features are cri4cal to
priori4ze in the prototype?
• Are the instruc4ons included in
the prototype clear?
Target audiences:
• Devised theatre-makers
Research methods:
• User tes4ng (15 par4cipants)
Outputs:
• Feature roadmap
• Content map
Timeline:
• Week 1-2: define scope
• Week 3-4: user tes4ng, synthesis,
and delivery of outputs
Budget:
• $200 (lunch)
+ 25 hours of consultant fees

36. RESEARCH PLAN: MEDIUM SIZE EXAMPLE
Research quesBon(s):
• What are the 3 most important
ideas to communicate about our
organiza4on to new audiences?
Target audiences:
• Poten4al a_endees, with low
awareness of us
• Regular a_endees, with poten4al
to give
• First 4me major donors
Research methods:
• Exis4ng data sources (1 prior email
survey)
• 3 focus groups
• 15 audience interviews
• 1 email survey

37. RESEARCH PLAN: MEDIUM SIZE EXAMPLE
Outputs:
• Brand guidelines
Timeline:
• Month 1: define scope
• Month 2-3: focus groups
• Month 4-5: audience interviews
• Month 6: survey
• Month 7: synthesis and delivery of
outputs
Budget:
• $0 + 225 hours of consultant fees

38. RESEARCH PLAN: LARGE SIZE EXAMPLE
Research quesBon(s):
• What are exis4ng audience
astudes toward our ac4on-taking
messaging?
• How are new audiences this year
demographically different?
• What do new audiences need from
us to stay engaged in year two?
• Which segments are most likely to
subscribe, donate, or take ac4on?
Target audiences:
• New donors this year
• New ac4on-takers this year
• Email list subscribers that have not
yet donated or taken ac4on
• Donor/ac4on-takers (two-year
window)
• Unaware audiences

39. RESEARCH PLAN: LARGE SIZE EXAMPLE
Research methods:
• Exis4ng data sources (Google
Analy4cs, prior email surveys)
• 4 email surveys
• 1 website survey
• 20 phone interviews
• 4 focus groups
Outputs:
• Conversion profiles
• A/B tes4ng plans
Timeline:
• Month 1: define scope
• Month 2: analyze exis4ng data
• Month 3-5: surveys
• Month 4-6: phone interviews
• Month 8-9: focus groups
• Month 9-10: synthesis and outputs
Budget:
• $1,000 (merch, snacks, prin4ng)
+ 400 hours of consultant fees

40. ACTIVITY #2: MAP OUT
YOUR RESEARCH PLAN

41. MAP OUT YOUR RESEARCH PLAN
Finish your research plan by choosing methods, outputs, a 4meline and a
budget.
Step #4: Decide on your research methods
• Mix quan4ta4ve and qualita4ve
Step #5: Decide what output (document) you need
• Connected to your research ques4on
Step #6: An4cipate 4meline and budget
• Based on research methods and your availability

42. MAKING DECISIONS
WITH DATA

43. SYNTHESIZING
When you feel overwhelmed, return to your research ques4on.
Tips:
• Talk about the data as a team
• Summarize as you go, find meaning at the end
• Claim hypotheses, then try to prove them
• Weight responses of key segments
• Look for trends and outliers, analyze cross-tabs,
find evoca4ve quotes

44. SHARING WITH STAKEHOLDERS
The data doesn’t ma_er if no one understands it.
Tips:
• Work backwards from decision-makers’ needs
• Answer the research ques4on first, then demonstrate “proof”
• Report by audience type, not by research method
• Interpret data, don’t (just) share raw numbers
• Mix visuals, stories, and graphs

45. SHARING WITH STAKEHOLDERS
No really, how you share the data is as important as what you
discovered.
Tips:
• Research needs its own internal marke4ng plan
• Separate “clear and obvious” from “needs more research”
• Suggest next steps and 4melines
• Plan for con4nued reminders about your findings
• Prominently post notable findings in analog form

46. IMPLEMENTING YOUR RECOMMENDATIONS
The data doesn’t ma_er if no one uses it.
Tips:
• Include short and long-term recommenda4ons
• Conduct a research post-mortem
• Evaluate the changes you implement
• Hang on to your data
• Training in-house audience researchers will pay future dividends

47. Q&A
@devonvsmith | www.measurecrea?ve.com | #NAMPC