Here are the slides from the kick-off research4impact workshop, entitled "Experiments in the Public Interest" held at the National Press Club in Washington DC on January 30, 2017.

1. Experiments in the Public
Interest
January 30, 2017
Adam Seth Levine
Don Green
Jake Bowers

2. Mission
To make it easier to connect people from the
academic, nonprofit, and governance spaces
who are interested in collaboraKng on research
projects.

3. Vision
A LinkedIn-like online plaOorm
+
In-person workshops

4. Vision
A LinkedIn-like online plaOorm
+
In-person workshops
“Experiments in the Public Interest”, DC, Jan 2017
Coming Soon: West Coast version & others

5. Vision
A LinkedIn-like online plaOorm
(coming very soon!)

6. Today’s Workshop
• 105 people signed up! Holy smokes!
• Six flash talks on the process of collaboraKng
• (Very) brief introducKons to the room
• Table-wide discussions led by Dave Karpf
• Happy hour

7. Thanks to our sponsors

8. E X P E R I M E N T S I N T H E P U B L I C I N T E R E S T
HAHRIE HAN JOY CUSHMAN

9. 2014
Integrated Voter
Engagement RCT
Leadership
Development
Survey
THE STUDIES
2016
Relational
Organizing RCT
Issue + GOTV RCT
IVE RCT
IVE Case Study

10. LEARNINGS
Reflecting on Experience
ground the study in a
longer-term
relationship; avoid
transactional studies
RELATIONSHIPS
identify questions that
are of compelling
strategic interest to the
org & the researcher
SHARED INTEREST
do a few things well;
break the big
questions down into
smaller pieces
LESS IS MORE

11. PICO'S
RESEARCH NEEDS
Clarify & sharpen
the leadership choices the
organization is making
Identify best practices/tactical
improvements
Uncover trends that shape the
organization's strategic context
Help campaigns identify strategic
targets

12. Robyn S(les, LSU & Ali Akbar, BCF
◦ 2014 midterms; Louisiana was in play (Landrieu v. Cassidy)
◦ BCF interested in tes?ng outreach to eligible African American voters
◦ 18 precincts in EBRP, 96%+ African American
◦ 4,400 households; 6,000 voters
◦ Represented 20% of registered, likely African American voters
◦ Outreach through direct mail, telephone outreach, precinct walking
◦ Tes?ng pro-Cassidy, an?-Landrieu and Republican plaSorm messaging

13. Background & Expecta(ons
◦ Introduced by chance
◦ Worked together to create DM pieces, write walk and phone scripts, iden?fy
and segment precincts
◦ Previous experience in campaigns; if not, would be more important to sit down and
establish baseline expecta?ons, goals of both sides
◦ First solo field experiment -- helpful to have access to someone with field
experiments background as sounding block
◦ Have conversa?ons/rela?onships built pre-funding
◦ Make sure vendors/staff internally understand what’s happening
◦ Control over variables as much as possible (including ?me)
◦ Expected something unique, useful, applied, wanted ROI

14. Best Prac(ces
o Set forth expecta?ons at the beginning
o Randomiza?on
o Need for control group
o What project will encompass
o Who can write about this, publish; can organiza?on be named?
o Be accessible throughout the process (both sides)
o Wrap-up memo (NOT paper drad)
o What you did, when, how, results, implica?ons
o Par?cularly useful for donors; special projects
o Can be more formal (in-house use only) or wrieen directly for broader audience

19. Chrysanthi Hatzimasoura (Lab @
DC and GWU)
Flash Talk

23. → → →

24. Adam Levine (Cornell) and Sarah
Alexander (M+R)
Flash Talk

26. “Well, let’s test it.”

27. A Media Theory of Movement
Power
Activist resources
Media system
affordances
Opponent/target
vulnerabilities

28. Incorporating a Culture of Testing

29. “If you’re not looking at
your data, then you’re not
listening to your members.
And that probably makes
you kind of an asshole.” –
senior analytics staffer

30. Main takeaways
! Experimentally-informed research improves how we make
decisions. But it does not make the best strategic option
simple, easy, or obvious.
! Much of the value comes not from any single experiment,
but from the strategic objects created through a broader
culture of testing.
! Use experiments to make better decisions. Don’t rely on
experimental results to avoid making decisions altogether.
! (As we learned this cycle…) Moneyball doesn’t always win.

31. Discussion Questions
1. What research questions do you want to prioritize?
2. What do you want to gain from a collaborative effort?
3. What can you/your organization commit to
contributing to the endeavor?

32. Next Step
A LinkedIn-like online pla,orm
(coming very soon!)