Here are the slides from the kick-off research4impact workshop, entitled "Experiments in the Public Interest" held at the National Press Club in Washington DC on January 30, 2017.
10. LEARNINGS
Reflecting on Experience
ground the study in a
longer-term
relationship; avoid
transactional studies
RELATIONSHIPS
identify questions that
are of compelling
strategic interest to the
org & the researcher
SHARED INTEREST
do a few things well;
break the big
questions down into
smaller pieces
LESS IS MORE
11. PICO'S
RESEARCH NEEDS
Clarify & sharpen
the leadership choices the
organization is making
Identify best practices/tactical
improvements
Uncover trends that shape the
organization's strategic context
Help campaigns identify strategic
targets
12. Robyn S(les, LSU & Ali Akbar, BCF
◦ 2014 midterms; Louisiana was in play (Landrieu v. Cassidy)
◦ BCF interested in tes?ng outreach to eligible African American voters
◦ 18 precincts in EBRP, 96%+ African American
◦ 4,400 households; 6,000 voters
◦ Represented 20% of registered, likely African American voters
◦ Outreach through direct mail, telephone outreach, precinct walking
◦ Tes?ng pro-Cassidy, an?-Landrieu and Republican plaSorm messaging
13. Background & Expecta(ons
◦ Introduced by chance
◦ Worked together to create DM pieces, write walk and phone scripts, iden?fy
and segment precincts
◦ Previous experience in campaigns; if not, would be more important to sit down and
establish baseline expecta?ons, goals of both sides
◦ First solo field experiment -- helpful to have access to someone with field
experiments background as sounding block
◦ Have conversa?ons/rela?onships built pre-funding
◦ Make sure vendors/staff internally understand what’s happening
◦ Control over variables as much as possible (including ?me)
◦ Expected something unique, useful, applied, wanted ROI
29. “If you’re not looking at
your data, then you’re not
listening to your members.
And that probably makes
you kind of an asshole.” –
senior analytics staffer
30. Main takeaways
! Experimentally-informed research improves how we make
decisions. But it does not make the best strategic option
simple, easy, or obvious.
! Much of the value comes not from any single experiment,
but from the strategic objects created through a broader
culture of testing.
! Use experiments to make better decisions. Don’t rely on
experimental results to avoid making decisions altogether.
! (As we learned this cycle…) Moneyball doesn’t always win.
31. Discussion Questions
1. What research questions do you want to prioritize?
2. What do you want to gain from a collaborative effort?
3. What can you/your organization commit to
contributing to the endeavor?
