This roundtable discussed how the use of qualitative data from a study on the experiences of homeless families accessing early childhood education influenced program and policies in Georgia and Connecticut; how communities used Homeless Management Information System data to drive systems transformation in Memphis, Phoenix and San Diego; how data on suicide prevention efforts in Alaska was used to inform policy makers and stakeholders, and how effective data visualization and story-telling tools have been used to inform program and policy change. Credit: - Lindsey Barranco, Ph.D. - Jamie Taylor, Ph.D.

1. Solutions for Health, Housing and Land ● www.cloudburstgroup.com
Using Data to Influence
Programs and Policy
Lindsey Barranco, Ph.D.
Jamie Taylor, Ph.D.
American Evaluation Association
Chicago, IL
November 14, 2015

2. 2
Session Objectives
1. To identify innovative ways existing data is used to influence
program and policy directions
2. To examine multiple ways data results can be used for mid-
course corrections in programs and long-term policy impacts
3. To understand the use of data visualization tools to promote
data utilization, stakeholder discussion and new policy
directions

3. 3
Using Administrative Data to
Drive Systems Change and
Improve Programs

4. 4
Background
• Homeless Management Information System (HMIS) is a locally
administered, electronic data collection system that stores
longitudinal person-level information about persons who
access the homeless service system.
• HMIS is HUD’s response to a Congressional Directive to capture
better data on homelessness
• Can also provide communities with a comprehensive view of
the nature and response to homelessness AND foster
collaboration
• HUD is now requiring communities use HMIS data to report on
system performance toward ending homelessness

5. 5
HUD System Performance Measures
5

6. 6
Benefits of Analyzing HMIS Data
• Provides HUD way to benchmark measure progress
around ending homelessness
• Provides communities with an indicator of their
success and challenges
• Provides agencies with information about how their
program is contributing to overall system
performance
• Allows communities to look at the needs of the
homeless population and what is and is not working
• Allows community to monitor performance of
agencies

7. 7
2014Annual HomelessAssessment Report – Published11/13/15
HMIS data entered at client level,
aggregated quarterly for AHAR.

8. 8
2014Annual HomelessAssessment Report – Published11/13/15

9. 9
Using the Data to Understand the
Local Homeless Service System

10. 10
• National trends in homelessness can be tracked to monitor
investments made in targeted efforts, i.e. Ending Veterans
homelessness and Ending Chronic Homelessness by 2016
• Great example of efforts to really USE the data collected as part of
Federal reporting requirements
• Complicated process at the federal level determining how to define
each measure accurately enough to ensure all communities are
measuring performance in the same manner
• Communities need support in understanding how to USE their data
• Communities need support in understanding what programmatic or
policy changes will “move the needle”
Lessons Learned

11. 11
RRH Analytics Project
GOAL OF PROJECT: Capacity building to empower
community leaders to use data to meet local and federal
policy goals to end homelessness. Policy focus on Rapid
Re-Housing (RRH) – a housing assistance approach that
provides time-limited rent payments to quickly move
households out of shelter and back into housing.
RRH DATA ANALTICS PROJECT:
Homeless Management Information
System (HMIS) data analyzed in six
sites across the country.
Community leaders engaged in
learning community: weekly
meetings to review data quality,
share system learning around
program/provider variation &
promote inquiry for RRH system
level improvement plans.
RRH Data Analytics
Project: 2
states, 3 cities, 1
county

12. 12
RRH Data Analytics Project
Four-month Learning Community Process
HMIS Data
Pull
Data
Preparation
Data
Analysis
Iterative
Data
Reviews
Data
Dashboards
12
• Weekly Meetings with Project Leadership
– HMIS Data Pull, Data Analysis Design
– Data Preparation, Data Reviews, Data Visualization
• Collaborative assessment with community partners ensured
analysis congruent with expected program data
• Theory of Change based on Data Literacy Intervention

13. 13
Theory of Change to End Homelessness
has Changed
Person falls into
homelessness
Person
sheltered
Person enters
Transitional
Housing
Person “ready”
to re-enter
community
End of Person
Homelessness

14. 14
Need for Rapid Re-Housing Evidence Base

15. 15
RAPID CYCLE EVALUATION on RRH IMPACTS
Propensity Score Matching (PSM) employed to assess the effect of
RRH on reducing the risk of return to homelessness. With
Phoenix/Maricopa County HMIS data, comparison groups that
statistically looked the same were created to assess the true
effects of RRH assistance.
Households were matched on:
age, type of household, single parent,
education, income, previous shelter stay,
race, ethnicity, mental health disability,
physical disability, substance use disability
explanatory variables.

16. 16
Evidence
supports the
claim that
RRH reduces
the risk of
return to
homelessness
16

17. 17
Additional PSM Analysis: Families vs. Singles
Single RRH Households Family RRH Households
*Returns to homelessness were significantly lower for households receiving
RRH than for similar households that received usual care. Significance at 5%
level of significance

18. 18
Lessons Learned
System transformation impacts of RRH data analytics
project:
• Accelerated state and local shift towards collective
understanding of local RRH impact on ending
homelessness
• Creative data visualization of analysis motivated broad
stakeholder engagement, system-wide program
improvements
• Data analytic results were used by leaders to move
support from temporary/transitional housing to RRH
approach
• System-Rapid cycle evaluation technique (PSM)
engaged social investment, motivated policy review
based on evidence of RRH effects on ending

19. 19
Displaying Surveillance Data
to Influence Resource
Allocation

20. 20
Background
Suicide Deaths in Alaska – One of Highest in the Country

21. 21
Data and Methods
Quantify the Problem, Quantify the Solution

22. 22
Data and Methods
Quantify the Problem, Quantify the Solution

23. 23
Lessons Learned
Data reports developed using effective visualization tools
promoted positive results of training initiative in decreasing
suicide risk
Communities, agencies, organizations and school systems now
expanding their use of suicide prevention trainings for all staff
New suicide prevention policies and procedures are being
developed locally to increase community awareness and actions
on suicide risk, and prevent youth suicide death.

24. 24
Other experiences using
existing data to
influence policy?

25. 25
Recommendations
• To empower communities to use data for program and policy
improvements, rapid evaluation methods and data literacy
development is critical
• Knowledge sharing across multi-sector stakeholders requires inquiry,
and the capacity to review what is and is not working with data as
neutral evidence
• To motivate policy change around complex public health issues, both
rigorous research evidence and the effective use of data
visualization tools are critical to promote cross-sector understanding
and political will
• Data-driven decision making is necessary for continuous, system-
wide improvement planning that is geared towards continual mid-
course corrections

26. 26
Recommendations
• Evaluation does not have to cost a fortune!
• Often there is a wealth of data available to communities for
planning at little to no cost
• Look for opportunities – up front and throughout – to apply
analysis results in a way that creates meaningful change
• When using administrative data – ensure you have a full
understanding of the pros and cons – and what data cleaning
may be required
• Include community stakeholders or practitioners in the
analysis process

27. 27
Questions?
Lindsey Barranco
Lindsey.stillman@cloudburstgroup.com
Jamie Taylor
Jamie.taylor@cloudburstgroup.com