Presentation by Christopher L. Griffin, Research Director, Access to Justice Lab, Harvard Law School, at the event on Governing better through evidence-informed policy making, 26-27 June 2017. The event was organised by the OECD Directorate for Public Governance in cooperation with the European Commission’s Joint Research Centre (JRC), the Campbell Collaboration and the International Network for Government Science Advice (INGSA). For further information please see http://www.oecd.org/gov/evidence-informed-policy-making.htm

1. PRIORITIES IN
EVIDENCE-BASED
ACCESS TO JUSTICE POLICIES
Christopher L. Griffin, Jr.
Research Director
The Access to Justice Lab at Harvard Law
School

2. Who we are
• researchers at Harvard Law
School
• focused on understanding
what works toward
enhancing access to justice
What we do
• conduct rigorous evaluations
through randomized studies
• adapt lessons and apply
methods from other fields to the
law
2
The Access to Justice Lab

3. Topic 1: Evidence Supply and Demand
• Supply
 Well understood: legal subject matter areas
o Consumer debt
o Eviction
o Certain family law issues (e.g., legal custody)
o Pre-trial release and sentencing
Much less understood is what solutions work and
which do not.
3

4. Topic 1: Evidence Supply and Demand
• Demand
 In the Lab’s experience, demand tends to focus on
short-term, pilot solutions
 Evaluations may or may not be included
 We seek to shift the “demand curve” so that
institutions partner with researchers to generate
more rigorous evidence of successful access to
justice initiatives
4

5. Topic 2: Presenting Evidence
• Who’s Afraid of Quantitative Evaluation?
 Preference for evidence based on statistical
inference rather than sense impressions or
qualitative assessments
 Such evidence presentations should account for,
among other issues:
o External validity concerns
o Technical concerns: e.g., confidence intervals vs.
estimates
At the end of the day, evidence must come in a
variety of forms; we prioritize causal relationships
over more descriptive formats.
But the former may not always be available.
5

6. Topic 3: Using and Linking Data
• We find this task to be the most pressing and
difficult in our A2J research
 Too many government data repositories do not
“talk to” each other
 Need for personal identifiers not based on pre-
existing data that can be linked across databases
• Other priorities
 Making use of non-adjudicatory (e.g., procedural
fairness) data
 Focusing on prospective data generation rather
than backward-focused data analysis
6

7. Topic 4: Impact Assessment
• When ethically defensible, we exclusively adopt the
randomized control trial as the methodology for
impact assessment
• Randomly allocate units to treatment 1 or treatment
2 (or 3, etc.), measure outcomes
• Only way to tell if treatment causes outcomes
 Confounding effects
 Reverse causation
7

8. Topic 4: Impact Assessment
• What do researchers have to consider?
 How and when to assign to treatment and control
groups
 Carefully applying intervention to treatment
participants
 Avoiding pitfalls
• Why randomize?
 Gold standard for causality
 Ethical allocation method
8

9. Topic 5: Role of the OECD
• Promote Randomized Evaluation
 The Organisation is in a unique position to embrace
and expand on a methodology that is severely
under-utilized in the social sciences
 Encouraging governments to think of ethical ways
in which to understand what policies really work
starting with small interventions
• Promote better practices in data storage and
management
 See Topic 3
9

10. Examples: Design Theory
10

11. Examples: Design Theory
11

12. THANK YOU
cgriffin@law.harvard.edu
a2jlab.org @a2jlab
facebook.com/a2jlab