1. MEASURING SOCIAL IMPACT OF BLIGHTED
HOUSING REMEDIATION
RICHMOND COMMUNITY FOUNDATION
RICHMOND HOUSING RENOVATION PROGRAM
REPORT TO JIM BECKER
CHIEF EXECUTIVE OFFICER
RICHMOND COMMUNITY FOUNDATION
PREPARED BY
KAITLYN ADLER, DANIEL BARTH, ROXANNE KAMALU, PATRICK WOO
DR. JULIET MUSSO, PRINCIPAL
UNIVERSITY OF SOUTHERN CALIFORNIA PRICE SCHOOL OF PUBLIC POLICY
APRIL 22, 2016
AN EVALUATION DESIGN FOR THE
RICHMOND HOUSING RENOVATION PROGRAM
3. This report has been developed for the Richmond Community Foundation to propose a design for evaluation of the
Richmond Housing Renovation Program. The program’s intervention strives to reduce the incidence of vacant and
blighted single-family houses in vulnerable low-income neighborhoods of Richmond, California. Program partners,
community stakeholders and observers of this new and innovative social impact bond program seek an assessment of the
social impacts derived from blighted property remediation.
The evaluation will measure if homeownership of formerly blighted houses improves the lives of residents living on
the block and in the neighborhoods of the properties. Public records’ statistics and resident survey data will be used to
compare pre- and post-intervention conditions. Three participatory approaches are proposed to augment the evaluation
process.
The report may be used to secure funds to execute the subsequent phases and activities related to the evaluation
process, whose results may support future social impact bond rounds for program scale-up across Richmond and to
support replication of the program’s innovative model by other community foundations in the US where blight conditions
are pervasive.
FORWARD
(Marshall, 2014)
4.
5. Forward..........................................................................................................................................3
Executive Summary…………………………………………………………………..............…………6
University of Southern California Student Consulting Team……………………...............….....8
Richmond Community Foundation Overview........................………………….....................….10
Social Impacts of Unstable Housing & Neighborhoods……………………….….................….12
Program Intervention: Richmond Housing Renovation Program…………............….............16
Program Logic Model…………………….………………………….…………...…...........….….......20
Program Implementation & Evaluability…………….……………………………..................…....24
Overview of Proposed Evaluation Design……………………….......…………….............…...…28
Evaluation Schematic...................................…………………….……………….....…...............…32
Key Performance Indicators of Neighborhood Charactericstics............................................38
Survey Measures of Qualitative Neighborhood Conditions....................................................44
Quasi-Experimental Design........................................................................................................48
Participatory Approaches: Program Activities For Community Engagement.......................54
Evaluation Design Strengths & Limitations..............................................................................58
Recommendations………………………………………………………………………..…............…62
Conclusions………………………………………………………………………….…............…..…..64
Appendix A: Research Design Matrix …………………..…………………………..................…..66
Appendix B: Key Informant Interviews.................................……………………........................70
Appendix C: Program Implementation & Evaluability..............................................................74
Appendix D: Crime Codes & Neighborhood Crime...…………………….....…....................…..84
Appendix E: Survey Instrument: “Survey of Residents on the Block”..................................90
Appendix F: Survey Instructions................................................................…..……...................94
Appendix G: Theoretical Constructs of Perceived Personal & Neighborhood
Quality-of-Life........................………………………..……..…....................................................…98
Appendix H: Institutional Review Board..................................………..…............…............…100
References…………………………………………………………………………..……................…102
TABLE OF CONTENTS
7. 7
Richmond California is a working class industrial city that continues to experience foreclosures, which leads to more
blighted homes occurring throughout neighborhoods. At a community summit convened by the Richmond Community
Foundation (RCF), the issue of blighted single-family houses was identified as a major concern among residents.
Community agencies and individuals brainstormed about how blight could be addressed. The result was the Richmond
Housing Renovation Program (RHRP).
The RHRP is an innovative intervention being employed in Richmond, California to address the incidence of vacant and
blighted single-family houses in vulnerable low-income neighborhoods. It is operated through the RCF via the Richmond
Housing Foundation. The RCF has utilized social impact bonds (SIB) to fund the program to renovate targeted houses
in specific neighborhoods for purchase by first-time homebuyer graduates of the SparkPoint financial education centers.
The RHRP acquires, rehabilitates, and sells properties to mitigate blight conditions and their adverse impacts.
While reducing blight and providing for homeownership opportunities, the RHRP is expected to produce the
“neighborhood spillovers” of reduced crime and city services costs to maintain the blighted properties, and increased
property tax revenues and property values. In addition, the RCF anticipates that the program will achieve a social
impact, most notably at the block level, for new homeowners and their immediate neighbors. Evaluation of the affected
blocks and neighborhoods will analyze data that measure neighborhood stability and perceived resident quality-of-life.
Participatory engagement processes will augment this evaluation.
This consulting team has designed tools that may be implemented by subsequent evaluators to provide regular
performance evaluation of the RHRP. Key informants participated in evaluation development. Residents and community
stakeholders will be engaged in forthcoming evaluation cycles. The methodology is robust and feasible, and would be
implemented by the RCF in the first, third, and final years of the initial five-year funding round. The design included
refinement of a program logic model; selection of key performance indicators (KPIs) for data gathering; and design of a
quasi-experimental comparison of blocks in the target neighborhoods that have experienced the remediation of blight with
untreated blighted blocks and blocks without blight.
The evaluation’s approaches align with the foundation’s mission to build social equity and improve the lives of Richmond’s
residents. Data collection tools and associated methodological protocols will add value to existing place-based efforts
and will support neighborhood stabilization efforts through resident engagement to increase neighborhood cohesion
and community vitality. Demonstration of this innovative program’s social impact will support the program’s scale-up in
Richmond and replication in cities where the problems of foreclosure and blight are pervasive.
9. 9
The RCF has commissioned an evaluation design to be created by a team of four graduate students from the University
of Southern California’s Master of Public Administration program:
• Kaitlyn Adler earned a Bachelor’s Degree in Business Administration - Accounting and Management from the
University of Michigan. She currently works for the C.S. Mott Foundation in Flint, Michigan as a Program Assistant
on place-based grantmaking.
• Daniel Barth has 27 years of nonprofit experience in community and housing development related to homelessness
and earned his Bachelor’s Degree from State University of New York at Albany in English and Geology Sciences.
• Roxanne Kamalu has 21 years of public sector experience working in Executive and Legislative Branches in Hawaii
and earned her Bachelor’s Degree from the University of Hawaii, Manoa in Economics.
• Patrick Woo earned a Bachelor’s Degree in Business Administration - Information Systems and Marketing from the
University of Southern California. He is currently an analyst at the Los Angeles Department of Water and Power’s
Corporate Performance Division.
To accomplish this study, the team used a combination of literature review, documentary data collection, and stakeholder
interview strategies to develop the frameworks in this report. The team conducted 14 stakeholder interviews. Each
team member was responsible for gathering 12-24 research documents each week through the first 8 weeks, followed
by additional research to “fill in the holes” through the completion of the Report. This resulted in the team reviewing,
analyzing, and compiling data from several hundred research documents to develop the Report for the Richmond
Community Foundation.
11. 11
The Richmond Community Foundation (RCF) was founded in 2000 and serves the residents of Richmond, CA with
the mission to build a healthy, sustainable community and to leverage assets that will help create change throughout
Richmond (Richmond Community Foundation, 2016a). As a community foundation, the RCF helps to advise community-
minded investors as they decide how to make charitable contributions. The foundation also operates its own mission-
based programming. The RCF’s work aligns with the five pillars of effective community development (Richmond
Community Foundation, 2016a), resulting in place-based revitalization.
Within these five pillars, RCF operates many different programs, including:
• Nystrom United Revitalization Effort;
• SparkPoint Contra Costa;
• Ensuring Opportunity Campaign;
• Campaign for Grade Level Reading; and
• Richmond Housing Renovation Program.
The Richmond Housing Renovation Program (RHRP) is the focus of this report.
Figure 1
13. 13
The City of Richmond currently contains some 1,000 blighted properties, of which 150-200 are blighted single-family
units that the city maintains. The blight places at risk neighborhood-level economic, social, and environmental conditions;
perpetuate low property values; and impede private investment (Goins, 2014). Tax delinquency erodes the city’s tax
base and the city incurs additional police and code enforcement costs (J. Becker, personal communication, January 29,
2016). Blighted properties cost an average of $7,000 annually per house to maintain (Richmond Community Foundation,
2016b).
The problem of blighted housing most severely affects immediate neighbors who are exposed to boarded up and broken
windows, peeling paint, dislodged gutters, graffiti, trash, abandoned vehicles, unkempt landscaping and trees, and insect
and rodent infestation (J. Becker & J. Knox, personal communications, January 29 & February 23, 2016). Abandoned
properties affect neighborhood safety, morale and quality of life, and result in increased crime on blocks compared
to blocks without the blighted properties (Spelman, 1993). Blight also has negative public health impacts that lead to
additional municipal mitigation costs (Vacant Properties Research Network, 2015).
While the nation’s foreclosure inventory has decreased by nearly a quarter from one year ago (CoreLogic, 2016) and
the number of vacant or “zombie” houses has decreased by 58% in two years (Henderson, 2016), the 2007 foreclosure
crisis continues to haunt Richmond. Foreclosures in Richmond have risen by 25% in two years (Palomino, 2013;
Realty Trak, 2015), with 40 new foreclosures per month (Moore, Barhoum & Franco, 2015). Housing divestment due to
foreclosure has negatively impacted neighboring home prices, and has had contagion effects on neighboring properties
that increase the incidence of foreclosures and the prevalence of blight (Immergluck & Smith, 2006). The impacts on
neighboring houses are likely substantial. A study of nearby Sacramento found that property values near foreclosed
properties fell by 32% (Wassmer, 2011).
Housing divestment due to foreclosure and housing speculative de-
occupancy has resulted in much of the blighting of vacant properties
in Central and North Richmond neighborhoods. Communities of color
and lower-income communities are hardest-hit by foreclosure (Drier,
Bhatti, Call, Schwartz & Squires, 2014). Central and North Richmond
has 87 percent of its residents of African American or Latino race/
ethnicity and 38 percent below 150 percent of the federal poverty line
(Pastor, Ito, Sanchez, Wander & Perez, 2013).
Housing divestment hinders neighborhood stability, safety, access
to jobs and quality schools (Reece, Rogers, Gambhir, Martin,
Baek & Lee, 2012). Such neighborhoods also experience high rates of resident out-migration, with consequent loss of
social cohesion and sense of belonging (Moore, Gambhir & Tseng, 2015). Households that are displaced from familiar
neighborhoods lose social networks and supports, and remaining households enjoy fewer networks, services, and
neighborhood businesses (Association of Bay Area Governments, 2015). This divestment reduces social participation,
social integration, and power (Berman & Phillips, 2000). Paradoxically, divestment can also advance gentrification by
offering real estate that is inexpensive relative to the regional market (Phillips, Flores & Henderson, 2014).
As a relatively lower income and working class industrial city located in the San Francisco Bay Area, Richmond
experiences complex dynamics around divestment and gentrification. As some neighborhoods in the city gentrify, adjacent
lower-income neighborhoods may experience accelerating housing prices, placing residents at risk of displacement
(Moore et al., 2015). This population loss due to blight may be a potential harbinger of population changes that adversely
affect low-income residents (Phillips et al., 2014; Plyer, Ortiz & Pettit, 2010).
“The vacant properties are called
zombie houses because their ownership
is in limbo; mortgage holders have
already left, but banks haven’t yet taken
possession through foreclosure, leaving
the properties abandoned and often
decaying.”
(Henderson, 2016, para. 3)
14. Hardest hit by recessionary forces include the Richmond neighborhoods of Coronado, Santa Fe, Iron Triangle, and
North Richmond (Figure 2). These specific neighborhoods show the very highest need when the cumulative effects
of social factors including poverty, unemployment, less than high school education, and health risks are aggregated,
compared with Contra Costa County as a whole (Pastor et al., 2013). Comprehensive analysis of Richmond’s educational,
economic, and neighborhood and housing quality indicators find that 89 percent of census tracts provide low or very low
opportunity for its residents, with Central and North Richmond neighborhoods all providing very low opportunity (Moore,
Barhoum & Franco, 2015; Reece et al., 2012).
Central and North Richmond neighborhoods experience accelerating population shifts with a loss of their poorest
residents at twice the County’s rate and an increase in homeownership by exogenous populations arriving as newcomers
(Pastor et al., 2013; Moore, Barhoum & Franco, 2015). While homeownership levels have slightly increased since 2000
(Pastor et al., 2013), low-income renter households comprise more
than 80 percent of residents in most of these neighborhoods (Moore,
Gambhir & Tseng, 2015).
The presence of vacant properties significantly increases crime, and
blight utilizes an inordinate amount of police resources (National Vacant
Properties Campaign, 2005). One analysis found that abandoned
properties had the highest correlation to crime of the variables tested.
SOCIAL IMPACTS OF UNSTABLE HOUSING
& NEIGHBORHOODS
Figure 2
A 2015 survey of Richmond residents
indicated that crime reduction was the
top priority and blighted properties
placed seventh.
(City of Richmond, 2016)
15. 15
Another study found that 41 percent of abandoned properties could be entered without use of force and 83 percent of
these properties showed evidence of illegal use by prostitutes, drug dealers and property criminals (National Vacant
Properties Campaign, 2005). Richmond remains one of the most dangerous cities in the Bay Area. In his 2016 State of
the City presentation, Richmond Mayor Butt noted the city’s increase
in crime in 2015. A 2015 survey of Richmond residents indicated that
crime reduction was the top priority and blighted properties placed
seventh (City of Richmond, 2016). A common theme that emerged
from key informant interviews with the student consulting team was that
residents and neighbors perceived a strong correlation between blight
and criminal activity, representing a lowered sense of safety.
In addition to improving neighborhood sense of safety and increasing
nearby property values, the renovation and re-occupancy of blighted
properties in Central Richmond may improve neighborhood quality of
life, represented by resident sense of community and neighborhood
cohesion, and new homeowner engagement with residents to improve
the neighborhood’s social quality (P. Jen, personal communication,
Feb 24, 2016). Re-occupancy of blighted properties may have little or
no impact on such perceptions if only a handful of blighted properties
are remediated while others remain vacant. Achieving social impact,
where residents find confidence in their neighborhood’s revitalization,
must reconcile the challenges of reaching scale in blighted property
remediation and addressing the dynamics of continuing foreclosures,
blight, and resulting neighborhood instability (Kopf & Vance, 2012).
Impacts of Blight in the Community:
• 150-200 blighted homes
maintained by the city, out of
1,000 city-maintained blighted
properties
• 40 new foreclosures every month
• Costs to the city $7,000 per home
annually to maintain blighted
properties
• Economic, social and
environmental divestment in
neighborhoods
• Increased crime in and around
blighted properties
(Richmond Community Foundation,
2016b)
(Kantor, 2014)
17. 17
The City of Richmond is undertaking blight remediation by means of a
social impact bond (SIB) program (J. Becker, personal communication,
January 29, 2016; Goins, 2014). The program is one of the state’s first
“pay-for-success” programs, and the first blight remediation program
in the US that is financed through social impact bonds (Hernandez &
Genser, 2014).
The program leverages private capital to invest in public acquisition
and renovation of housing. The model can be considered an enabling
financing mechanism where community-minded investors offer their role
as a secondary housing market solution provider (National Community
Stabilization Trust, 2016). If the program is successful, it will both repay
investors and result in municipal cost savings (Anthony, Bauer, Dixon,
Fisch, Griffin, Legaux, Maulhardt et al., 2014).
Pay-For-Success. Pay for Success programs can (Third Sector Capital Partners, 2016):
1. Measurably improve the lives of residents who struggle in low-opportunity constraints;
2. Develop an outcomes-focused and financially feasible way to address social urgencies;
3. Support scaling of evidence-based practices; and,
4. Increase accountability by directing resources toward effective programs.
Investor Motivations. The Community Reinvestment Act (CRA) requires federal financial supervisory agencies
(Mechanics Bank, 2013) to fund community-based projects, essentially investing private dollars into public programs. As
the result, many private financial institutions
competed for the initial SIB funding round (J.
Becker, personal communication, January
29, 2016). If the program is evaluated to
be effective by demonstrating social impact
without capital loss, it will likely attract
investors willing to fund future rounds and
promote model replication.
Place and People-Based. The model includes the sale of renovated blighted houses to first-time homebuyers. As
such, RHRP can be understood as an impact investment in people rather than in place as it aims directly to affect
specific residents’ lives (Roman, 2013). Yet the RCF seeks to affect a social impact that is place-based in the goals for
neighborhood stability.
Affordable Housing and Homeownership. SIBs will raise funds to rehabilitate blighted housing that will sustainably
produce affordable housing to low and middle-income families facing barriers to entry into this housing market. This
will result in decreased blight, decreased displacement, and improved neighborhood stability through homeownership
occupancies.
Performance Targets. The design proposed in this study provides a feasible evaluation strategy for RCF to take five
steps for SIB program implementation. Two of the key steps are identified as setting reasonable performance targets and
evaluating program performance goals (Roman, Walsh, Bieler & Taxy, 2014).
Social Impact. Collaborative resources are convened by the foundation to engage the intervention activities. Outputs
provide financial return to investors and economic benefits to individuals, neighborhoods and the City. The program’s
The RCF began as a neighborhood-based revitalization
effort that combined people and place-based approaches, a
“place-based people strategy” (Ladd, 1994, p. 195). Creating
opportunity for people communities of color is as urgent
as physical revitalization (Blackwell, 2006, p. 102). The
community development model is tangibly represented in
RHRP’s design and implementation.
Four Steps of Program Intervention
The RHRP intervention activities are
summarized as:
1. Obtain SIB funding
2. Purchase blighted property
3. Renovate home
4. Sell home to first-time homebuyer
18. ultimate impact is expected to improve the quality-of-life for Richmond residents. Such impact constitutes the program’s
“social return on investment”, which is the focus of this evaluation design.
• Collaborative Resources - Convened by RCF
• Intervention Activities - 4 Steps of Program Intervention
• Program Outputs - Financial & Economic Return on Investment
• Impact - Social Return on Investment
Logic Model. This report provides a program logic that models the design of key performance indicators (KPIs)
for evaluation data gathering and performance measurement, qualitative assessment tools, and participatory approaches.
A logic model provides a systematic presentation among the RHRP’s constituent resources and relationships, research
activities, and the expected outcomes of the research activities (W. K. Kellogg Foundation, 2004). The team has
developed the model based on information gathered from interviews with institutional stakeholders and from literature
reviews. The logic model will help the RCF focus and direct both current and future evaluations on progress toward
desired goals.
*See page 101 for larger view.
19. 19
The RCF has the capability to measure the program’s financial return on
investment and the extent of blighted property remediation (J. Becker,
personal communication, January 29, 2016; T. Higares, personal
communication, February 25, 2016). This report outlines performance
monitoring evaluative processes and tools to enable RCF to gather and
analyze quantitative and qualitative data that determine the social impact
and return on investment of rehabilitated, blighted housing on first-time
homebuyers, residents on the block, and the overall neighborhood.
An independent social impact measurement will additionally determine
whether (GECES Sub-group on Impact Measurement, 2013):
• Investment delivers acceptable social return;
• Assess whether the program’s innovative policy meets objectives; and,
• Help institutional stakeholders to understand the value gained by the intervention.
To achieve this, the process of evaluation design has strived to (GECES Sub-group on Impact Measurement, 2013):
See Appendix A for a Research Design Matrix that guided the evaluation design process.
See Appendix B for a list of Key Informant Interviews that were conducted with select stakeholders. Included are the
informants interviewed, others who may be interviewed by future evaluation teams, and recommended areas for follow-up.
Housing provides the locus for
a neighborhood within the larger
construct of a community. Programs
that help natural residents stay in
and enjoy their neighborhoods will
allow for them to benefit as property
values increase and community
opportunities grow.
(Chapple, 2009; Salsich, 2012)
Figure 3
21. 21
The program’s logic focuses on outputs that achieve financial return to
investors as well as economic benefits to individuals, neighborhoods
and the City. The program’s ultimate impact is expected to improve
the quality-of-life for Richmond residents. Such impact constitutes
the program’s “social return on investment”, which is the focus of this
evaluation design.
The model offers program flexibility to operate under difficult
constraints. For example, if the barriers to acquire a blighted home
are too great, the RCF can turn its focus to another property. Flexibility
helps the RCF to remain efficient in its use of finite organizational resources.
Another asset is RHRP’s network of partners, pro bono advisors, and institutional stakeholders supporting it.
Stakeholders share a strong sense of commitment to its success and the hope that an evaluation provides leverage
for program scale-up and replication. Collective enthusiasm exemplifies a willingness to adjust the program model, if
necessary, to improve its performance.
Political support for the foundation and the program are substantial. Key financial stakeholders of the RHRP are the
RCF, City of Richmond, and Mechanics Bank. Yet each of these parties are under financial constraints that create an
urgency for the program to demonstrate successes beyond those
of financial concern. Many factors constrain RCF from successfully
purchasing, rehabilitating, and selling blighted homes above the initial
pro forma rate, which could limit the ROI to investors. The RCF can
only renovate a small number of properties at any given time, so it will
be important for the program to demonstrate its level of neighborhood
impact to policymakers and future investors.
The program is highly evaluable because it has a model with a well-
defined purpose and structure, a broad network of partners, and
collaborative support from stakeholders who will improve the program
and its enabling structures (Rossi, Lipsey, & Freeman, 2004).
See Appendix C for a detailed discussion on Program Implementation
& Evaluability.
Study of interventions to address the pervasive conditions of blight
across the US have been conducted in older and larger cities whose
problem is widespread and scale much greater than Richmond’s:
• Rust Belt cities such as Flint and Cleveland
• Eastern cities such as Pittsburgh and Baltimore
• Southern cities such as New Orleans
Richmond’s intervention operates without the formal structures of large-scale projects. In New Orleans, a larger
percentage of the city’s blighted properties are owned by government entities (Plyer, Ortiz & Pettit, 2010) and the
intervention engaged a regional and formalized stakeholder analysis process (Gulf Coast Community Design Studio,
2012), both of which enabled structural efficiencies at great depth. Cleveland utilized three responses to blight: housing
court, land banking, and the community development corporation (CDC) models (Keating & Lind, 2012).
The Richmond Community Foundation
“harnesses the power of philanthropy
to build healthy, sustainable
communities throughout Richmond
and Contra Costa County.” The
Richmond Community Foundation
acts as a convener to engage
stakeholders and leverage knowledge
and resources to effect positive
change in the community.
(Richmond Community Foundation,
2016a, para. 1)
The Richmond Housing Renovation
Program provides an innovative
approach to reduce blight.The
program has a straightforward model:
the RCF buys a blighted home,
renovates it, and sells it, then repeats
the cycle with profit reinvestment.
22. Studies have explored the impacts of the Neighborhood Stabilization Program (NSP) federal intervention in response to
the 2007 housing market downturn. The NSP involved rehabilitation by CDCs of foreclosed and abandoned properties,
including North Richmond’s Community Housing Development Corporation (CHDC). NeighborWorks America has
similarly led foreclosure and blight remediation activities across the US, of which CHDC is an affiliated member. Both
efforts have studied qualitative measures of neighborhood stability as “primary” sources and quantitative public records of
neighborhood conditions as “secondary” (Kopf & Vance, 2011; Graves & Shuey, 2013). This suggests that neighborhood-
level data of resident perceptions are main data sources for this evaluation. They place confidence in neighbors and
neighborhoods as the focus of resident perceptions.
The RHRP seeks a social remedy as it focuses on physical conditions dictated by housing divestment and blight. The
program’s twin goals need to be evaluated in tandem, looking at neighborhood characteristics that improve through the
intervention and neighborhood conditions that result in improvements to people’s lives.
The evaluation will test the constraints of assessing the social impacts of a small-scale program that can only invest
incrementally to address the 150-200 blighted properties. Little has been studied of the social impacts of blight
remediation. A study of a similarly small-scaled program in Boston suggested that local homeownership and community
integration engagement strategies would improve impacts that remediation alone failed to accomplish (Graves & Shuey,
2013).
Use of the social impact bond model for blight remediation in a place
and people-based approach has been only conducted in the United
Kingdom, for which no empirical demonstration of its success has been
published. Evaluative study of the program’s social impact bond model
will be a first of its kind. If the model demonstrates success it promises
to be a trendsetter for replication by other localities.
The RHRP’s approach will benefit
from, and also be limited by, the
informality within the collaborative
framework convened by the RCF.
The model relies upon the collective
managerial efforts crafted by shared
motivations from respective agencies
and groups.
PROGRAM IMPLEMENTATION &
EVALUABILITY
25. 25
Foreclosures and blight trends continue in Richmond’s neighborhoods.
Intervention must not only involve property remediation but also provide
benefits that improve neighborhood stability. The social value of re-
occupying blighted properties can be gauged by how the intervention
directly affects residents on the block - their confidence, perception of
community, neighborhood cohesion, and the neighborhood’s discerned
quality of life (Kopf & Vance, 2011). Secondary improvements, such as
in areas of crime and property values, are important for neighborhood
stabilization. Belief that neighborhoods are turning around may require
community engagement and organizing processes to focus on the
program’s accomplishments as a vehicle for increasing resident
involvement.
The program intervention is expected to produce “neighborhood spillovers” of reduced crime and city services costs to
maintain the blighted properties, and increased property tax revenues and property values.
In addition to collecting these descriptive statistics on the neighborhood’s characteristics from quantitative public record
sources, resident perceptions will also be measured to provide an evaluation of the closest causal relationship between
intervention and impact. Analysis of the data collected from surveys and public records involve arranging data sets
to measure such variables as “Homeownership Effects” and “Block Confidence”. Analysis can then be made about
the block, of groups of blocks, between these “treatment” and “control” groups, of the neighborhood, and between
neighborhoods.
EVALUATION OBJECTIVES
An evaluation of the program’s social impacts needs to address the following five objectives:
The Richmond Community Foundation
has commissioned an evaluation
design of the Richmond Housing
Renovation Program to measure if
homeownership of formerly blighted
houses will improve the lives of
residents living on the block of the
properties and in the neighborhoods.
Figure 4
26. Primary Evaluation Question. Does renovation and first-time homebuyer occupancy of blighted housing lead to
improvements in neighborhood social conditions, such as in levels of perceived sense of community, attachment to place,
and social capital (Graves & Shuey, 2013)?
OUTCOME MEASURES
The chart below is a brief guide to correlate the overall data points with theoretical constructs, or variables, from which
the measures are derived and discussed in detail in Appendix G: Theoretical Constructs for Perceived Personal &
Neighborhood Quality-of-Life.
Quantitative Measures
Collected from public
records
Qualitative Measures
Resident perceptions collected from 22
questions in the “Survey of Residents on
the Block”
Personal Qualities Population Characteristics Personal Quality-of-Life, Residential Tenure,
Homeownership Effects, Psychological Sense of
Community, Place Attachment, Block Confidence
Neighborhood
Characteristics
Objective levels of: Crime,
Property Values, Property Tax
Revenue, Costs to City
Perceived levels of: Neighborhood Disorder, Block
Satisfaction
Neighborhood
Conditions
Informal Social Control, Block Cohesion, Collective
Efficacy, Social Capital, Community Competence
PRE- & POST-INTERVENTION COMPARISON
For the evaluation to demonstrate valid changes to blocks and neighborhoods, the program will measure conditions
before the program renovates the blighted housing (“pre-test”) and after the program completes all renovations (“post-
test”). The term “quasi-experimental design” refers to the comparison of the pre- and post-tests. Both pre-test/post-test
phases include a qualitative survey that is distributed to residents on select blocks in the three neighborhoods targeted for
intervention.
In the pre-test phase, surveys will be sent to 125 blocks in a treatment group and 125 in a control group of similar, non-
blighted blocks. The RCF may choose to increase the number of select blocks to include all city maintained single-family
units. In the post-test phase, surveys will be sent to 60-75 blocks (depending on how many total homes are renovated)
in a treatment group and two control groups consisting of blocks that experience blight but will not be treated, and blocks
that do not contain blighted properties. Control groups will be constructed through the process of matching for equivalence
on outcome-related characteristics and experiences between the compared neighborhood (Rossi, Lipsey, & Freeman,
2004).
• Phase 1 - Baseline data is collected and documented in this report.
• Phase 2 - “Pre-test” refers to analyses before the RHRP’s renovations and must be conducted in the first year of the
five-year program.
• Phase 3 - “Post-test” refers to analyses that occur after the renovations are completed and new homeowners occupy
the house. The post-test is conducted at the five-year program’s conclusion.
OVERVIEW OF PROPOSED EVALUATION
DESIGN
Figure 5
27. 27
UNITS OF ANALYSIS
The program’s intervention seeks to increase neighborhood stability as measured from objective data sources that
quantify neighborhood characteristics, and also to positively impact residents’ perceived well-being and social identity
as measured from subjective data sources about neighborhood conditions. The evaluation is focused on data collection
from the block’s residents who live nearest to each blighted house.
The Block. Personal and social quality improvements may be localized
at the level of only a few houses or the block. For the purpose of this
study, a “block” consists of the immediate neighbors (Figure 6). These
immediate neighbors are those who are in the two houses on each side
of the target property and the five houses directly across the street from
the property, for a total of nine houses surveyed per block. Self-selection
will produce a smaller sample size for each “block.” Subsequent to the
program’s intervention, the renovated house will also be included in
this block definition, for a total of ten houses surveyed per block. This
purposive use of the term “block”, focusing on the adjacent properties
most likely to be affected, is smaller than a census block. A census block
is a full a city block bounded on all sides by streets or in some cases
railroad rights-of-way (Rossiter, 2011).
Five Neighborhoods. The evaluation design proposed in this report
targets the three neighborhoods of the initial SIB round.
These three initially targeted neighborhoods are, with street boundaries:
• Santa Fe: 2nd Street, Ohio Avenue, Harbour Way, 580 Freeway
• Coronado: Ohio Avenue, Harbour Way, 580 Freeway, 24th St
• Iron Triangle: Richmond Parkway, Ohio Avenue, Union Pacific right-of-way extension of Carlson Boulevard (and
does not include bottom of triangles: 2nd Street and MacDonald Avenue; 16th and MacDonald Avenue)
Two additional neighborhoods will be targeted in future SIB rounds:
• Belding/Woods: 13th Street, Dunn Avenue, 23rd Street, MacDonald Avenue, Union Pacific right-of-way extension of
Carlson Boulevard
• Pullman: Carlson Boulevard, Ohio Avenue, 38th Street, Cutting Boulevard
SURVEY OF RESIDENTS ON THE BLOCK
A survey will be employed in treatment and control groups to engage residents living on each block and the homeowners
among these residents. A self-administered survey will be mailed to potential respondents. The survey cover letter will
be sent from the most identifiable neighborhood organization, such as from the neighborhood council, and its purpose is
to provide information that enhances belief in the survey’s purpose and importance. A gift incentive of a household good,
such as from Home Depot, will encourage response to the survey.
In the pre-test, 2,250 surveys will be disseminated. The post-test will mail 1,800 surveys. Door-to-door surveying may be
necessary for some blocks to achieve a 50% response rate. The pre- and post-test surveys will cost approximately $3,500
and $2,500 respectively, plus staff and volunteer time and costs. The survey will be informally field-tested when the RCF
meets with neighborhood council meetings to gain acceptance of partnership activities related to the evaluation phases.
Figure 6
29. 29
Data-gathering and evaluation will be conducted in three phases, as shown:
Below is a breakdown of each evaluation phase:
DATA ON NEIGHBORHOOD CHARACTERISTICS
The RCF will be provided baseline quantitative data in the five RHRP neighborhoods regarding burglary, drugs,
prostitution, trespassing, and vandalism, which are identified as possible crime indicators that can be correlated to
blight. In addition, the median value of homes and current tax revenue received in the five geographical areas. The cost
Figure 7
Figure 8
30. for city services is averaged at $7,000.00 annually per blighted property. The information will support examination of
the program’s implementation and assist in improving the program for future funding rounds. These specific data sets
and their correlation with other impacts from blight remediation will lend a depth analysis to make critical programmatic
recommendations that piecemeal descriptive statistics cannot provide.
For the evaluation to demonstrate valid changes to blocks and neighborhoods, data must be collected before the
program’s intervention (what’s called a “pre-test”) and after renovations have been completed (or “post-test”).
PRE-INTERVENTION COMPARISON
An assessment of pre-intervention neighborhood conditions is necessary for post-intervention comparison of
improvements to the neighborhood. It is anticipated that 60-75 blighted houses will be renovated during the program’s
initial five-year period. To effectively capture each block where properties may potentially be renovated, 125 blighted
and 125 non-blighted blocks will be identified for collection of pre-intervention data that captures existing objective
neighborhood characteristics and perceived quality-of-life neighborhood conditions. A comparison between blighted and
non-blight blocks will inform a pre-intervention assessment of blight’s impact on the block. Data will be sampled from 20
percent of these blocks for the initial SIB round.
PROGRAM ACTIVITIES
EVALUATION SCHEMATIC
Figure 9
Figure 10
31. 31
Augmenting the RHRP program, additional reflective activities will support the evaluation’s objectives, promote the
program in Richmond, and raise attention given to the program’s potential and realized social impacts.
As blighted houses are remediated, this evaluation design additionally proposes that the RCF and its partners engage in
discrete activities that focus attention upon new homeowners at the block-level as each house returns to productive use,
and at the community-level at the program’s third and fifth year.
A neighborhood welcoming event is proposed to encourage new homeowners to be actively engaged in the community
and will help them to feel as though they are part of the community. New neighbors could become acquainted, with
the potential to widen the new homeowner’s network of support. It is important for new homeowners to feel accepted
and supported as part of the community. Seeing a family move into the neighborhood could have positive implications
for block morale and can strengthen local economic conditions. A summit will help to identify ways in which the
neighborhood can strengthen its social network and cohesion in a holistic manner.
Upon the conclusion of the SIB round and after loan repayment, a community-wide summit of institutional and program
stakeholders is proposed to be convened by the RCF to collectively assess the program’s outcomes and challenges,
make appropriate recommendations for the program, and support development of institutional actions that may enable
improved future implementation.
These additional activities on the affected blocks can deepen the impact of blight remediation by involving residents,
neighborhood councils, City, and other organizations to substantively address the problem of blight and scale-
up program activities. Resources applied to the blocks will magnify impact that single-house interventions will not
accomplish through renovation alone.
POST-INTERVENTION COMPARISON
After the five-year program period is completed, a post-intervention evaluation will collect data to measure the program’s
impact on objective neighborhood measures and resident perceived well-being from perspectives of the homeowner, the
immediate resident, the block, and the neighborhood to the extent that treated blocks are clustered nearby. To validate
the perceived changes relative to treated blocks, a comparison will be drawn with untreated blight blocks and non-blight
blocks.
Figure 11
33. 33
A number of research publications provide KPIs on blight and impacts of reducing blighted properties on neighborhoods
and communities. The consulting team reviewed the literature to identify meaningful indicators as exemplars that are
considered best practices based upon research and key informant recommendations (Myers, Smith & Martin, 2004)
related to crime, property values, property tax revenue, and costs to the City of Richmond related to maintaining and
responding to blighted property conditions. The data are publicly available and can be attained easily on a yearly basis in
the absence of a quasi-experiment.
SOCIAL RETURN ON INVESTMENT
Social impact assessment is important for monitoring performance, resource acquisition, mission reinforcement, and
stakeholder accountability (Pathak & Dattani, 2014). Analysis typically involves cost-benefit or cost-effective analysis,
but seldom SROI (Lingane & Olsen, 2004). Cost-benefit analysis normally frames benefits and costs as trade-offs and
doesn’t help with planning or prioritizing. SROI allows managers to maximize both social and financial benefits (Lingane
& Olsen, 2004). SROI should base its numbers on sound data and plan for continuous collection and management of the
data. This makes SROI useful to managers and investors, helps them see where the impacts are weak, and encourage
them to plan to optimize social impact in the future (Lingane & Olsen, 2004).
The key performance indicators that can be monetized and calculated as social return on investment are: crime, costs to
the City of Richmond, property values, real property tax revenue.
CRIME
According to the Richmond Police Department the types of crime that typically affect blighted neighborhoods in the city
are burglary, drugs, prostitution, trespassing and vandalism (Milam, 2016). The department found that the number of
responses to and incidents of each of these types of crimes is likely to increase due to the presence of blighted properties
in the neighborhood. These types of crimes are consistent with those identified in Vacant Properties: The True Costs to
Communities by the National Vacant Properties Campaign in 2005.
Explanations of Typical Crimes Affecting Blighted Neighborhoods (Milam, 2016):
• Burglary. Individuals illegally enter into blighted properties to
commit theft. Properties adjacent to blighted properties are
susceptible to burglary as well.
• Drugs. Blighted properties are frequently the site of drug use and
drug transactions.
• Prostitution. Blighted properties are often used for the act
prostitution.
• Trespassing. Trespassing can be particularly difficult for law
enforcement to respond to because an individual may set up a
utility account for the property. With a utility bill for the property, law
enforcement cannot necessarily force the individual to leave.
• Vandalism. Individuals illegally enter blighted properties and
further destroy the property.
City of Richmond Crime
• “Richmond witnessed a recent
increase in crime in 2015.”
• “Unfortunately Richmond remains
one of the Bay Area’s most
dangerous cities.”
• A 2015 Richmond Resident
Priorities Survey indicated that
crime reduction was the top
priority and blighted properties
palced seventh.
(Butt, 2016)
34. The Richmond Police Department provided three years of historical
crime data for the program’s five targeted neighborhoods - Coronado,
Iron Triangle, Santa Fe, Belding Woods, and Pullman. The historical
crime data can be aggregated to illustrate the past trend for typical
crimes affecting the blighted neighborhoods. The Richmond Police
Department can provide updated crime data for each year of the RHRP
for comparison and trend analysis by future evaluators. However, the
program’s impact on the crime trends cannot be accurately derived
from neighborhood crime data due to other confounding factors such as
economic development and purchases by private real estate investors.
In order to more accurately and verifiably measure the RHRP’s impact
on typical crime affecting blighted neighborhoods, obtaining the crime data at the block level will be necessary.
The Richmond Police Department has the capability to provide historical and current crime data at the block level. The
Richmond Police Department indicated that the organization experienced several issues with crime reporting software.
Due to these issues, the agency was not publishing crime reports to the public until the software issues are resolved. It
expects the software issues to be resolved by April 2016. The crime data can be requested directly from the Richmond
Police Department. However, delivery may be delayed due to limited staffing resources (Milam, 2016). The RCF has
an established relationship with the Richmond Police Department’s Code Enforcement Division. RCF should utilize this
relationship to collaboratively and routinely obtain block level crime data for evaluators to perform accurate and verifiable
RHRP crime impact analysis without negatively impacting code enforcement and crime analysis operations.
See Appendix D for data on Crime Codes & Neighborhood Crime.
COSTS TO CITY OF RICHMOND
Assuming that 60 houses are renovated during the five-year period, for a City savings associated with maintenance per
house at $7,000 per year, the City will save $420,000 annually when all homes are renovated. With $420,000 as an upper
limit, if that amount is discounted in perpetuity, at a five percent discount rate, the present value of the blight savings
would be about $8 million. A further assumption is that the homes would not have been renovated in the absence of the
program. If the RCF renovates 75 homes, the blight savings would
increase. Some costs that the City incurs are: supplies including wood,
hardware and paint, biohazard removal and disposal, weed, vegetation
removal, staff and equipment time per property, and regular monitoring
(T. Higares, personal communication, March 29, 2016).
PROPERTY VALUES
Remediation of blight and increased homeownership rates are expected
to contribute to increased property values on the block. Property values
in a neighborhood are a reflection of the neighborhood’s desirability
and its economic stability (Rohe & Stewart, 1996). Where several
abandoned houses are restored and re-occupied in close vicinity of
a few blocks, the neighborhood’s characteristics, including property
values, can be expected to improve. Data collection can measure the
change in the median values of single-family properties over time (Rohe
& Stewart, 1996).
KEY PERFORMANCE INDICATORS
OF NEIGHBORHOOD CHARACTERISTICS
Future evaluators will need to
obtain the crime data by the block
level where blighted properties are
purchased. Future evaluators will then
utilize the block level data combined
with a quasi-experimental study
approach to measure the program’s
impact – described later in this report.
City of Richmond Budget, Planning
and Finances
• “Top Priority: Find and secure
$8.7 million in expense
reductions and new revenues.”
• “The current revenue and
expenditure projections do not
support a sustainable budget.
Changes have to be made.”
• “Funding the reserve from one-
time monies without a plan to
reduce the budget will result
in a continued deterioration of
reserves.”
(Butt, 2016)
35. 35
Data was retrieved on current median sales prices for the five target
neighborhoods for the one-year period of April 2015 through March
2016 to serve as baseline data. To calculate the median sales price,
data was retrieved, the outlier’s deleted, i.e., lowest and highest sales
price and calculated.
Coronado $275,000
Iron Triangle $240,000
Santa Fe $250,000
Belding Woods $315,000
Pullman $290,000
As the program’s initial SIB round continues until 2020, property values can be tracked.
If the percentage increase of home sales from 2014 to 2015 continues, as long as Richmond adjusts property values
relative to sales and in a timely manner, the property values for those neighborhoods can be:
Blighted homes in Belding Woods and Pullman will not be renovated for at least the first three years of the RHRP and will
be target neighborhoods in future years.
PROPERTY TAX REVENUE
If property tax rates remain the same, and the same numbers of property owners pay their taxes, the increase in property
values will ensure that property taxes also rise. Richmond is still considered to have low property taxes despite the
increase in median sales price. In 2016, property taxes will be 28.2% of total general fund tax revenue, where sales and
utility users contribute the difference (City of Richmond, 2016). Renovation of blighted properties will increase property
values quicker and property taxes will comprise a larger chunk of total general fund tax revenue. As the program scales-
up, actual tax revenue will be tracked.
SUMMARY
The RCF will collect descriptive statistics on a yearly basis to gather data on the neighborhood characteristics on crime,
property values, property tax revenue, and City service costs. Analysis by block, by block group (treatment and control),
and by neighborhood can be incorporated. The percentage of the city’s totals in each area can be shown for each KPI
(Kolko & Neumark, 2010). These data will be correlated with the qualitative data on perceived changes at these same
levels of analysis.
Median home value in Richmond in
2014 was $229,800, and appreciation
over the previous year was 22.3
percent (City of Richmond, 2015a).
Median sales price in December 2015
was $360,000, which was a 9 percent
increase since December 2014 (City of
Richmond, 2016).
2015 2016 2017 2018 2019 2020
CORONADO $275,000 $299,750 $326,728 $356,133 $388,185 $423,122
IRON
TRIANGLE
$240,000 $261,600 $285,144 $310,807 $338,780 $369,270
SANTA FE $250,000 $272,500 $297,025 $323,757 $352,895 $384,656
BELDING
WOODS
$315,000 $343,350 $374,252
PULLMAN $290,000 $316,100 $344,549
Figure 12
36. Key data points from both neighborhood conditions and resident quality-of-life can be mapped on PolicyMap, combining
the secondary data with the data collected at the block level (Kopf & Vance, 2011). This would provide evaluators a
different perspective when analyzing data and would organize the data in a visually appealing way. Another source for
geo-spatial mapping can be obtained by researcher of the USC Program for Environmental & Regional Equity (Pastor,
Ito, Sanchez, Wander & Perez, 2013) for the Healthy Richmond collaborative. The collaborative is focused on the
neighborhoods also targeted by the RHRP.
The effects of divestment and blight is most significantly felt closest to home, suggesting that variations in crime and other
aspects of neighborhood disorder across blocks are important to assess (Brown, Perkins & Brown, 2003). The qualitative
KPIs proposed here offer objective measures of the neighborhood characteristics, which can be used alongside the
qualitative measures, proposed below, that assess resident quality-of-life using variables that constitute the neighborhood
conditions on the affected blocks.
The key performance indicators of neighborhood characteristics provide quantitative descriptive data sets. These data
however are not sufficient measures of the impacts of blight and its remediation. Neighborhood disorder, which includes
social and physical disorders, can include such conditions as street-corner drug dealing, prostitution, graffiti, incivility,
abandoned property and trash, which are not captured sufficiently in public records data, since residents typically
experience a fear of reporting these situations, and vulnerable neighborhoods may lack informal social control to confront
them. A survey instrument to measure the perceptions of neighborhood residents is useful for gaining such missing
information.
KEY PERFORMANCE INDICATORS
OF NEIGHBORHOOD CHARACTERISTICS
(Cellania, 2014)
39. 39
To augment these objective neighborhood data from public sources, it is important to learn about how residents
themselves perceive their block and neighborhood. Resident perceptions can provide the closest causal relationship
between the intervention and the desired impact. A survey will be distributed to residents on select blocks in the
program’s three target neighborhoods.
Evaluation of the program’s social impacts will focus on the qualitative assessment of resident perceptions to gauge
improvements to the neighborhood. Surveying residents to assess perceived benefits from the RHRP intervention
will focus on the measures of neighborhood membership, influence, integration and fulfillment of needs, and shared
emotional connection (McMillan & Chavis, 1986). By improving this perception of belonging, residents matter to one
another and to the neighborhood with a shared belief and mutual commitment to meet interpersonal needs (McMillan &
Chavis, 1986).
SURVEY INSTRUMENT: “SURVEY OF RESIDENTS ON THE BLOCK”
This report’s main evaluation question prompts an assessment of the impact on neighborhood social conditions from
housing renovation and first-time homebuyer occupancy. The program’s intervention effects will be measured by
collecting and analyzing survey data and the neighborhood data outlined in the previous section.
Public records on crime, property tax revenues, property values, city services costs, and population characteristics will
not provide information that is specific to the intervention, and thus are secondary data sources. Personal and social
quality improvements may be localized at the level of only a few houses or the block. Additionally prescribed program
intervention activities may intensify the focus of existing neighborhood resources, and resident perceptions, on the
affected blocks to increase the impact that single-house interventions cannot accomplish through renovation alone
(Graves & Shuey, 2013).
Creation of the survey instrument is driven by research on theoretical
constructs from social psychology, explored in detail in Appendix G:
Theoretical Constructs of Personal & Neighborhood Quality-of-Life.
The table below summarizes each of the 22 survey questions and their
theoretical constructs derived from the research. These questions,
or “data points”, provide measures of these theoretical constructs as
“variables”, especially when questions are combined and correlated
through analysis to develop findings on measurable changes in resident
quality-of-life. The rating scales and other ranges of answer options
offer survey respondents with discrete “attributes” to choose from as
units of measure.
See Appendix E for the Survey Instrument: “Survey of Residents on the Block.”
The survey instrument is derived
primarily from two existing survey
tools:
• NeighborWorks America’s
“Resident Confidence in
Community” (Kopf & Vance,
2011);
• Federal Reserve Bank of Boston’s
“Sense of Community in High
Foreclosure Neighborhoods”
(Graves, 2012).
40. SURVEY MEASURES
OF NEIGHBORHOOD CONDITIONS
Question “Data points” Answer Options
“Attributes”
Theoretical
Construct
“Variables”
Relates to
1 How long have you lived on this block? Category ranges Residential Tenure Personal Quality-of-
Life
2 If you are a homeowner, how long have
you owned your home?
Category ranges Homeownership 1st Time Homebuyers
& Non-Homebuyers
3 What are the things that you like best
about living on this block?
Multiple choice Locus of Resident’s
Needs and Interests
Place Attachment
4 What are the things that you like least
about living on this block?
Multiple choice Locus of Resident’s
Needs and Interests
Place Attachment
5 If you had the choice, would you
continue to live on this block?
Yes-No; open-ended Sense of Community Place Attachment
6 Overall, considering everything, how
satisfied would you say you are living
on this block?
1-5 scale Block Satisfaction Place Attachment
7 You can recognize most of the people
who live on your block.
1-5 scale Sense of Community Neighboring and
Cohesion
8 If there is a problem on your block,
people who live here can get it solved.
1-5 scale Collective Efficacy Neighborhing; Sense
of Community
9 Do people on your block watch after
each other and help out when they can?
1-5 scale Neighboring and
Cohesion
Sense of Community
10 It is important to you to feel a sense of
community.
1-5 scale Sense of Community Neighboring and
Cohesion
11 You feel a sense of community with the
residents on your block.
1-5 scale Neighboring and
Cohesion
Sense of Community
12 How connected would you say you feel
to the neighborhood?
1-5 scale Sense of Community Neighboring and
Cohesion
13 How safe would you say you feel in
each of the following places?
1-5 scale Neighborhood
Disorder/Order
Block Satisfaction
14 How safe do you feel the following
people are in this neighborhood?
1-5 scale Neighborhood
Disorder
Block Satisfaction
15 Indicate how likely you think it is that
people on the block would help out if the
following occurred:
1-4 scale Neighboring and
Cohesion
Collective Efficacy
16 You care about what your neighbors
think of your actions.
1-5 scale Sense of Community Neighboring and
Cohesion
17 In the past 12 months, how often have
you participated in these neighborhood
groups?
Category ranges; Yes-
No; open-ended
Collective Efficacy Community
Competence
18 Thinking about your quality of life:
questions about attitude, confidence, life
conditions, control over life situations.
1-5 scale Personal Quality-of-
Life
Sense of Community
19 Compared with 3 years ago, how has
the neighborhood changed?
1-5 scale; open-ended Block Confidence Personal Quality-of-
Life
Figure 13
41. 41
SURVEY INSTRUCTIONS
Some 20,000 residents live in the neighborhoods of Coronado, Santa Fe, and Iron Triangle. In the pre-intervention
phase of the evaluation, 2,250 surveys will be disseminated to treatment and control groups in the program’s year one.
The post-intervention survey distribution will be approximately 1,800 surveys, depending on the number of houses
renovated over the five year period. Each survey will be coded by house and block for anonymous tracking. The survey
cost is approximately $6,000, plus staff and volunteer time/costs. A 50 percent response rate is adequate. A prominent
local organization’s letterhead, such as from the neighborhood council,
should provide the cover letter in a personalized tone that stresses the
survey’s relevance and the program’s importance to the community.
The proposed survey will be informally field-tested when the RCF
meets with neighborhood council meetings.
If the response rate is 50 percent and each pre-intervention block has
up to nine households who may respond, approximately 225 surveys
will be analyzed from the two block groups. The sampled post-intervention respondents will approximate 300 surveys. All
survey responses will be individually coded, themed, and inputted into data analysis software.
See Appendix F for the Survey Instructions.
ANALYTICAL APPROACH TO ASSESS INTERVENTION EFFECTS USING
SURVEY AND NEIGHBORHOOD
DATA
The theoretical constructs, and their questions
on the survey, constitute a set of dependent
variables. “Dependent variables” are measures that
assess what is affected by the intervention, or the
program’s effects (Trochim, 2006c). The RHRP is the
independent variable affecting the change, which will
be measured utilizing the variables listed in Figure 5.
Modeled on a study by Adams (1992), this evaluation
will assess the contributions made by the program’s
intervention on measures that may indicate relative
Question “Data points” Answer Options
“Attributes”
Theoretical
Construct
“Variables”
Relates to
20 Thinking about the next 3 years, how
much is the neighborhood likely to
change?
1-5 scale Block Confidence Locus of Resident’s
Needs and Interests
21 You expect to live on this block for a
long time.
1-5 scale Sense of Community Place Attachment
22 Questions about you and your
household.
Multiple choice Personal
Characteristics
Personal
Characteristics
A “sample”, or subset of the collected
surveys, will provide an accurate
reflection of the resident population
being studied.
The study’s three dependent variables are organized by:
• Perceived affective component of “Personal
Quality-of-Life” measures;
• Perceived social component of “Neighborhood
Conditions”; and
• Objective and cognitive aspects of “Neighborhood
Characteristics”.
(Unger & Wandersman, 1985)
42. neighborhood stability by examining the impact on qualities that a resident most values on the block. Analyzing the data
collected in the evaluation phases involve arranging data sets to create the dependent variables.
First the treatment and control block groups will be compared along each data point. The sample’s “mean”, or the average
of the data point, and the “standard deviation”, or the degree the data varies from the mean, will be measures by block
group. Then these dependent variable data sets will provide a comparison of Personal Quality-of-Life, Neighborhood
Conditions, and Neighborhood Characteristics components for each block group.
EXAMPLE: BLOCK CONFIDENCE
The construct of block confidence is represented in the survey questions #19, #20 and #21. A brief overview of the
analytic process exploring block confidence provides a simple and clear illustration of the manner in which this survey can
support this evaluation design:
1. Identify a theoretical construct that supports the research objective;
2. Determine its utility as a measurable variable;
3. Identify questions that may justifiably measure the variable;
4. Construct a data analysis strategy.
Block confidence is the resident’s confidence in the future of the block and neighborhood, and is closely linked to
psychological sense of community (PSOC) (Long & Perkins, 2007), which is discussed in detail in Appendix G. An
individual’s and community’s collective level of confidence drives the community’s prognosis. A useful metaphor is
homeownership: with confidence a homeowner invests in the property, in relationships, in building mutuality and problem-
solving. Lacking the confidence, the divestment is akin to the problem of blight itself.
The RHRP program and its stakeholders also suits this metaphor, as these efforts represent community confidence.
Richmond has long-struggled with its confidence and in the reciprocity of image and reality problems, with blight as a
serious symptom. It is clear that block confidence is an important theoretical construct to support this evaluation.
Three questions were identified as important enough to include on the survey related to Block Confidence:
19 Compared with 3 years ago, how has the neighborhood changed?
20 Thinking about the next 3 years, how much is the neighborhood likely to change?
21 You expect to live on this block for a long time.
By using a five-point scale and asking an important open-ended question to give increased amount of reflection to the
answer, data can be collected that clearly and also substantively help to measure block confidence.
There are several ways that the “data points” that these questions elicit can be correlated. One recommended way is
adapted from Long & Perkins (2007):
• Take the “mean”, or average, of questions #19 and #20 - this is a basic block confidence measure;
• Compare the means of these two questions with the measure derived in question #21 - this might say something
important about such issues as personal confidence in relation to block confidence;
• Determine the utility of the qualitative data from the open-ended question, based on the trends observed in the coded
SURVEY MEASURES
OF NEIGHBORHOOD CONDITIONS
43. 43
findings;
• Compare the means of questions #19 and #20 in the treatment and control groups during pre-test;
• Compare the means of these two questions in the treatment, non-treatment, and control groups during post-test;
• Compare pre-test to post-test results;
• Determine the utility of comparing these results with question #21 and the qualitative data from the open-ended
question, based on the trends observed in step #6;
• Use these findings to correlate with other findings from the other constructs, variables, questions, and analysis,
such as for Personal Quality-of-Life, Neighborhood Conditions, and Neighborhood Characteristics components.
SUMMARY
A survey instrument will assess changes in resident perceptions of their experiences on the block and neighborhood for
the households adjacent to the blighted property. It is hypothesized that:
• Residents will perceive improvements in personal quality-of-life, neighborhood attachment, sense of community,
and observed improvements to the block and neighborhood;
• First-time homebuyers will experience improvements in self-esteem, life satisfaction, and community integration as
a result of homeownership.
Data will be gathered from residents and homeowners who have experienced the intervention, and compared with
those who have not experienced the intervention but are either impacted by blighted properties or do not experience
blight. The data from the survey and statistics from public records will look at the Personal Qualities, Neighborhood
Characteristics, and the perceptions about Neighborhood Conditions. When conducted as a quasi-experiment, this
analysis will provide the best feasible evaluation of the social impact derived from the RHRP’s intervention.
(Duffrin, 2010)
45. 45
For the evaluation to demonstrate valid changes to blocks and neighborhoods, the program will measure conditions
before the program renovates the blighted housing (“pre-test” or pre-intervention) and after the program completes all
renovations (“posttest” or post-intervention). The term “quasi-experimental design” refers to the comparison of the pre-
and post-tests.
PRE-INTERVENTION COMPARISON OF BLIGHTED BLOCKS WITH NON-BLIGHT
BLOCKS
A pre-intervention comparison, or “pre-test”, is necessary to analyze neighborhood conditions and resident perceptions
about the quality of life in their neighborhoods prior to the intervention (Rossi, Lipsey, & Freeman, 2004). Since the
intervention will affect residents’ perceptions, a pre-test will provide a baseline measure with which to validly assess
survey responses (Trochim, 2006). Data collection should be conducted prior to the completion of the program’s first few
renovations. A pre-test is thus needed in the program’s first year and will include quantitative and qualitative elements.
Quantitative neighborhood characteristics provide baseline data on neighborhood stability through measures of crime
levels, property values, property tax revenues, city service costs, and population demographics. Qualitative data will be
obtained resident responses to mailed surveys.
Residents who live on blocks within the boundaries of the three
currently targeted neighborhoods - Coronado, Santa Fe, Iron Triangle
- will potentially be asked to participate in a pre-intervention survey.
The treatment group will include participants who live on neighborhood
blocks that contain blighted homes likely to receive the intervention.
The control group, comprised of residents who will not experience the
intervention, will include participants who live on blocks that are in the
same neighborhoods as the treatment group, but where no blighted
homes are located.
Except for the absence of blighted properties on the block, control blocks, while nonequivalent, are equivalent on
outcome-related characteristics and outcome-related experiences except for the difference in program exposure (Rossi
et al., 2004). The process of matching will take place when constructing the control group (Rossi et al., 2004). The
intervention, or treatment, group will be identified by locating blighted properties that are likely targeted by the RCF for
acquisition. The control group will be chosen based on blocks that have very similar characteristics to the treatment blocks
but lack the presence of blighted homes likely to receive treatment. One selection criterion may be to identify a block that
adjoins the treatment block but contains no blight.
Pre-intervention surveys will be mailed to homes located on 125 blocks with blighted homes and 125 blocks without
blighted homes. The 125 pre-intervention blighted properties are non-randomly selected by the RCF and the City’s Code
Enforcement Department based upon property characteristics that the department identifies as properties likely to be
purchased and acquired. Prior to the commencement of the year one pre-test evaluation, 125 non-blighted blocks will be
selected by the evaluators, who determine blocks adjacent to or in close proximity to the treatment blocks that reflect the
very similar neighborhood conditions. Surveys will be sent in equal number to treatment and control groups, as outlined in
the above survey discussion.
Quantitative baseline data collection will also occur as part of the year one evaluation, and yearly thereafter, within the
five identified Richmond neighborhoods. Baseline data points will include indicators of neighborhood characteristics: crime
levels, property values, property tax revenues, and city service costs. These data will provide a sense of quality of life in
these neighborhoods before the intervention and will give a reputable point of comparison for the post-test conducted in
year five.
To gain a better understanding of
how blighted properties and their
remediation may affect residents’
perceptions, qualitative data collection
will involve treatment and comparison
groups.
46. POST-INTERVENTION COMPARISON OF BLIGHTED TREATMENT BLOCKS WITH
BLIGHTED UNTREATED BLOCKS AND NON-BLIGHT BLOCKS
A post-intervention measure is necessary to analyze perceptions about the quality of life in the neighborhoods as a result
of the intervention. Similar to the pre-intervention comparison, there will be qualitative and quantitative elements to this
comparison, which will take place in the program’s year five of the initial SIB round. The quantitative element will provide
the same data on neighborhood characteristics as provided in the pre-test. The qualitative element will include resident
responses to surveys that are mailed. The post-test will survey three different block groups in order to measure how the
renovation, sale and occupancy of previously blighted properties may affect resident perceptions. Evaluators will use the
same strategy of identifying immediate neighbors on respective block groups, and the target property’s household will be
included in the “block”.
Since it is expected that the program will remediate 60-75 blighted
properties in the five years of the initial SIB program round, the three
block groups will each include 60-75 blocks.
This comparison will gather data from residents who no longer live
near a blighted home and have experienced the occupancy of a new
homeowner household, and compare this with data from residents who
continue to live immediately proximate to a blighted home, and those
who do not live near a blighted home. The comparison will provide
an evaluation of the perceptions of quality of life in the three targeted
neighborhoods with and without the intervention. Because this is the
post-test phase, the data will also be compared to the pre-test data as
the treated, untreated, and non-blight block groups are subsets of the
original 125 blighted and 125 non-blight blocks. This constitutes a “re-
test” of those same residents.
Evaluators will analyze the data and determine if correlations can be made. Post-test quantitative data will be compared
with pre-test data to determine if correlations can be made. Specifically, the evaluators will analyze the data to understand
the neighborhood’s objective characteristics and perceived conditions that indicate neighborhood stability after the
intervention.
The block groups are comprised of:
• Blocks that contain blighted
homes that have successfully
received the intervention;
• Blocks that contain blighted
homes have not yet received the
intervention; and
• Blocks with similar neighborhood
characteristics that do not
contain blighted homes.
QUASI-EXPERIMENTAL DESIGN
47. 47
(By the People: Participatory Democracy, Civic Engagement and Citizenship Education, 2015)
49. 49
The RCF’s community-based collaboration and its collectively-driven RHRP program can, by design, provide for due
participation of its diverse stakeholders. A participatory approach has been the hallmark of the foundation’s role in
building community competence.
The program’s strategic milestones are being achieved in three successive collaborative phases:
1. Pre-implementation relationships - RCF convened experts and advocates to conceive of and develop the program
and its activating systems;
2. Program implementation - coordinated key actors engage in operations;
3. Evaluation - Proposed in this report, key initiating participatory processes can normalize a system of regular and
ongoing measure of the program’s impact through stakeholder engagement processes that support the evaluation’s
objectives, inform program adjustments, and justify program scale-up.
Participatory approaches offer an informal means for brainstorming and feedback in areas of pivotal importance to the
program: with block-level residents, homeowners, and strategically engaged community-minded stakeholders.
The evaluation’s participatory approaches are consistent with the collaborative convening model that the RCF typically
engages. These approaches will increase attention to the program, improve the program’s responsiveness, enrich
the information available to evaluators, and build community capacity (Rossi, Lipsey & Freeman, 2004; Zukoski &
Luluquisen, 2002). Future evaluators may study stakeholder inquiry processes if it seeks to refine the proposed
approaches for stakeholder engagement associated with evaluation.
BLOCK-LEVEL HOMEOWNER WELCOMING AND CELEBRATION
Since the residential block is the level at which this analysis is focused, it seems important that a participatory approach
to the evaluation begin here and at the time when new homeowners occupy the treated houses. A block-level outreach
process would utilize the programmatic milestone as an occasion to draw together new homeowners and block neighbors,
modeling the program’s sought social impacts related to sense of community, social cohesion, and neighborly relations
(McMillan & Chavis, 1986), especially as these new households begin
the transition from rental to owner responsibilities.
Without explicit attention to new homeownership at this milestone,
attention to the achievement would be deferred until year five’s post-
intervention block survey. Neighbors are likely to be motivated by the
pride and role modeling of the new homeownership (J. Fisher, personal
communication, March 3, 2016).
The RCF and neighborhood councils can initiate a community
engagement process that supports the program’s social impact goals.
A welcoming event will mark the accomplishment of blight remediation
for future reflection by block members during the evaluation phase
in year five. Neighborhood councils in Coronado, Santa Fe, and Iron
Triangle have been ardently striving for improvement to neighborhood
conditions for more than ten years. It is anticipated that these leaders
and their block captains will be motivated to initiate the celebration of
the achievement of homeownership.
Among the potential achievements of
a welcoming event:
• Homeowners identify a tangible
sense of block membership;
• Neighbors increase their
motivation and influence to
improve block stability;
• Participants of the event share
emotional connection and
neighboring; and,
• Residents integrate and fulfill
personal needs through improved
interdependencies.
50. The RCF can present this plan to neighborhood councils to gain acceptance of their leadership role to initiate the block’s
welcoming. A mailing can notify block neighbors of the renovated blight property’s completion and sale. An identified block
leader is empowered to communicate through door-knocking and organizing a small event with the new household.
HOMEOWNER AND PROSPECTIVE HOMEOWNER SUMMIT
At the program’s year three, a Homeowner Summit and Mentoring Day is proposed to engage program partners, including
the RCF, SparkPoint, Community Housing Development Corporation (CHDC), and the United Way. The summit can
be integrated with the United Way’s regional Rise Together and/or Contra Costa County’s Insuring Opportunity poverty
reduction initiatives. The CHDC has experience in convening focus groups, improving homeownership readiness, and
providing subsequent coaching support to help households make quality-of-life improvements during their transition as
homeowners (M. G. Cantrell, personal communication, March 23, 2016). With the RCF’s role as convener and organizer,
partners will design the day’s activities. Homeowners, recent graduates, and current enrollees of the Sparkpoint financial
literacy program can engage in problem-solving, brainstorming, and mentoring activities. The summit can provide RCF
and program partners with data for use in future program implementation.
At the summit, homeowners are provided a framework for mentorship about homeowner experiences. New homeowners
provide important feedback on homeowner satisfaction, address the turbulent everyday emotional tensions that buyers
confront as new homeowners (Boatright, 2015), and discuss challenges in the transition from renting to owning.
Homeowner mentorship is an often-requested unmet need of the SparkPoint program (M. G. Cantrell, personal
communication, March 23, 2016). First-time homeowners struggle to understand their feelings about homebuying, which
may impede effective preparation and decision-making (Boatright, 2015). Homeowners can help their mentees identify
their fears and receive first-hand information about common pitfalls. The process of property purchase will be competitive,
as ten prospective homebuyers are expected to compete for each renovated house, adding stress from the outset.
Low to mid-income homeowners are challenged to attend such
a summit. Incentives, such as household gifts, can be offered to
participants to support attendance. The RCF may recruit event
sponsors, especially from businesses and associations related to
homeownership.
Current and prospective homeowners participate in brainstorming based on their perceived
challenges and opportunities related to homeownership (Turner, 2014):
• Individual tables with a trained facilitator are given specific discussion questions;
• The table’s participants decide on issues valuable to them specific to the discussion topic;
• The facilitator reports to the larger group;
• The summit organizer documents each of the issues reported, and posts each issue;
• Each participant votes with dot-stickers on the issues they value most;
• Priority issues identified by participants and documented by the organizer;
• The RCF analyzes the issues and creates strategies to subsequently address them.
PARTICIPATORY APPROACHES: PROGRAM
ACTIVITIES FOR COMMUNITY ENGAGEMENT
SparkPoint experiences increased
demand for its first-time homebuyer
program. As resources become
rationed, peer-focused mentorship
is an alternative means to provide
support to prospective homebuyers.
51. 51
COMMUNITY SUMMIT OF PROGRAM ACHIEVEMENT AND FUTURE
IMPLEMENTATION
Engagement of institutional and community stakeholders is proposed as an opportunity to share and celebrate program
achievements at the conclusion of the initial five year SIB round. Brainstorming and strategic planning will produce
recommendations for program adjustments, provide a forum to discuss enabling structural improvements, and envision
and establish buy-in for scale-up and replication. Here is a sample illustration of a visioning session:
Deepen Learning. Partners and other participating stakeholders can experience new learning opportunities as a result
of this innovative program. Institutional partners feel their way through the program’s initial round. A summit will provide a
forum for reflection upon the achievements and challenges, which may result in active experimentation as new ideas are
introduced and adopted (Kusters, van Vugt, Wigboldus, Williams & Woodhill, 2011). The process of property acquisition
is foremost among the challenges that collaborative learning and planning may benefit. Agencies can work together
and, through enabling legislation and improved structural changes, expedite negotiation and expropriation processes to
support system efficiencies and program expansion.
In Flint, Michigan residents are engaged through community forums and summits, and neighborhood councils are
encouraged to participate in any of these seven activities: waste removal, boarding, demolition, mowing, vacant lot reuse,
building rehabilitation and redevelopment, and code enforcement (Pruett, 2015). In 2013 community groups in Flint were
responsible for mowing 12,803 lots, boarding 175 vacant houses, and removing 905 tons of garbage from vacant homes
and lots (Pruett, 2015).
(Drawing Change, 2013)
52. In New Orleans, neighborhood organizations tracked down owners of
blighted properties, helped to connect property owners to pro-bono
legal services to clear succession issues, go door-to-door collecting
data about neighborhood issues, and help Code Enforcement to
prioritize the properties in most need of remediation (Plyer et al., 2010).
In both cities, neighborhood groups and residents are paramount
to blight elimination strategies and provide models for community
cohesion through continuous learning together.
Model a Transformational Process. The RCF is at the center of
an elaborate array of community-based collaborations in all its work
throughout Richmond. For this project the RCF brings many Richmond
partners together in a way that affords efficiency with an effectiveness that could not otherwise be achieved, showcasing
its extensive community competence.
The RCF will facilitate how stakeholders contribute to the summit’s process and encourage them to provide new
information, offer critical reflection and feedback, and think through and present recommendations (Kusters et al., 2011).
Deepen Place-Baseness. The RCF has an opportunity to deepen the program’s place-based approach in collaboration
with and by leveraging Richmond’s existing strengths:
• Impact Volunteering on vacant lots or other needs for amenity
improvements on the block, for example through Richmond’s
Cities of Service initiative, Richmond ESC;
• Community Policing and Neighborhood Safety efforts of the
Richmond Police Department, focused on the affected blocks as
neighbors increase informal social control;
• Integration with Healthy Richmond, a Building Healthy
Communities initiative, to embrace the RHRP’s goal of blight
remediation, to increase resident interest in health and
neighborhood stability activities in collaboration with existing local
nonprofit efforts;
• Neighborhood Revitalization activities can model from the RCF’s Nystrom United Revitalization Effort (NURVE)
in block areas where blight remediated houses are clustered and residents and institutional stakeholders identify
additional neighborhood needs.
While the program’s logic models the leverage of financial return
outcomes, the program’s theory of change is socially focused. The RCF
practices community development in its initiatives, and the participatory
approaches will advance its theory of scale. The augmented
program activities will initiate the further enhance Richmond’s target
neighborhoods by increasing and promoting social impact objectives.
Community-based collaboration and place-based approaches to blight
elimination are feasible as investments that co-occur with the program’s
evaluation phases.
PARTICIPATORY APPROACHES: PROGRAM
ACTIVITIES FOR COMMUNITY ENGAGEMENT
Neighborhood and community
representatives can be positioned
to offer testimony on how individual
and relational improvements have
impacted collective patterns of
perception at block and neighborhood
levels, which can inspire structural
changes to advance the program’s
model.
In Flint’s example, a city with very
limited resources, the community-
wide strategy to remediate blight
and build upon existing assets is a
necessary approach. Flint created
a Blight Elimination Framework
including municipal agencies,
philanthropic support, neighborhood
groups, nonprofit organizations, and
individuals.
A number of other communities, such
as New Orleans, LA and Flint MI,
have approached the issue of blight
with a community focus, using the
assets and abilities of many different
agencies and stakeholders to do much
more than simply renovate homes
(Pruett, 2015; Plyer, Ortiz & Pettit,
2010).
55. 55
Conducting an evaluation of an innovative intervention is a complex process, especially because the intervention has
broken new ground that lacks prior testing. Here evaluation matters most, where watchful eyes seek to have analysis
that demonstrates the program’s worth and its impact. The RCF may lack capacity to fund and dedicate resources
to execute an independent evaluation. Leadership from institutional stakeholders, investors, funders, or a pro bono
independent evaluator will be needed to support the effort. It is likely that the RCF will champion the evaluation’s
initiative to demonstrate social impact for potential funders of subsequent program phases. If an independent evaluation
is able to show positive social impacts, it is likely that it will draw external attention leading to increased scale and the
replication of similar SIB models to remediate blight in other communities.
LIMITATIONS
The ability to retrieve the data at the block level will require effort by
evaluators and stakeholders. For example, to effectively measure
changes in crime statistics at the block level, crimemapping.com
provides the data for only six months. If the RCF can retrieve the data
every six months and retain it for the evaluation, future evaluation
teams can have access to such data.
A distinct limitation of the analysis is that it involves resident
perceptions, which may result in reporting bias when inferring on
and associating personal and environmental contexts (Robinson,
Lawton, Taylor & Perkins, 2003). Thus, a comparative study of these
dependent variables, as proposed in the quasi-experimental design,
may be useful.
There is a considerable degree of uncertainty when measuring “sense”, “attachment”, “satisfaction”, “quality-of-life”, etc.
They are qualitative, and thus subjective, imperfect, and lacking a degree of reliability. Perceptions that are sought in the
survey involve complicated variables, and the feelings that are elicited may produce contradictory results (Hur & Morrow-
Jones, 2008). Surveys in themselves involve the interpretation of questions that will differ among residents with differing
perspectives, which limit the ability of the evaluator to measure the constructs that are sought and thus the precision of
comparisons between constructs and groups (Marans & Rodgers, 1975).
A survey is by design brief, limiting, and relies on measurements taken from one or a few questions to evaluate a
construct. The diversity of the residents inherently misses perspectives and embeds a bias through the limitations of the
questions. Further, personal information about the respondents themselves is limited and lacks measures that may be
useful such as age, household size, income distribution, housing type, questions to renters, robust variables to assess
neighborhood disorder, etc.
Valuable questions were omitted to provide economy that supports response rates from those who are surveyed.
Nonetheless, the survey is longer than is optimal, even as the length is consistent with those of the surveys that this one
is modeled from.
This small-scale program’s intervention may not bring about a significant change in objective and perceived conditions
of neighborhood stability, even at the block level. The 10-15 homes renovated per year across three neighborhoods may
prove difficult to draw clear relationships between the intervention and resident perceptions to determine the program’s
effects. Nonetheless, it is valuable to ascertain resident feelings about quality of life and the impacts that blighted homes
have at individual, block, and neighborhood levels. If the data find that blight remediation has specific positive outcomes,
these are important measures of social impact.
Many similar studies have been
conducted at community-wide and
census tract levels. A strength of
this evaluation is that the objective
and perceived data will be taken at
the block level, with an opportunity
to anchor perceptions on tangible
daily experiences and interactions.
Evaluation can harness an
independent voice for people rooted in
the neighborhoods.
56. The program’s impact on crime and property values may also be difficult if the properties are widely dispersed. Evaluators
must account for a number of “confounding variables” that would impact the results of the quantitative analysis.
It can be anticipated that the percentage of renters and homeowners who respond to the survey reflects the housing
tenure of the population in the neighborhood, and this can pose a “selection bias”, where certain patterns of respondent
characteristics lead to biased results. Also, selection bias may result from the socio-economic challenges that working
class families struggle with in target neighborhoods. Those who hold multiple jobs to make ends meet or who struggle with
family obligations may not find the time to the complete the survey.
“Internal threat to validity” regards the fact that no matter how careful evaluators are in identifying which homes and
blocks are placed in treatment and control groups, these groups will never be completely equal, leading to selection bias
(Trochim, 2006).
“External threat to validity” refers to the extent to which the intervention and subsequent outcomes can be replicated in
another place with different people, otherwise known as its “generalizability” (Trochim, 2006). While it is expected that
a similar SIB program along this model can be replicated in other cities, each city comes with unique circumstances,
challenges, and residential dynamics, making it impossible to exactly replicate the program model, leading to some
degree of this program’s threat to this program’s external validity.
STRENGTHS
While the RCF’s may lack existing resources to convene constituent and stakeholder summits, the foundation’s
experience in this area is a significant strength that can contribute to the evaluation. These summits will provide residents,
neighbors, and the community-at-large to identify and address issues related to blight that are of highest priority. Such
activities will lead the RCF to propose strategies that address these issues, including program modifications.
Gathering data from residents on their perceptions of neighborhood
conditions can provide an alternative to public record data sources.
In tandem, the use of objective and subjective measures can help
evaluators to identify causes and effects (Robinson, Lawton, Taylor &
Perkins, 2003).
This study’s sampling frame of Central Richmond’s three target
neighborhoods requires a sample of the population and a subsample
of residents on the target blocks who complete the survey. Targeting
this subsample of 50 percent of sample block residents who receive
the survey mailing will be sufficiently large enough to satisfy this
evaluation’s goals to measure the program effects and generalizability.
SUMMARY
The program’s intervention is complex, is difficult to measure, and lacks previous models. The evaluation provides a
triangulated approach that uses indicators of both descriptive statistics and social indicators of perceived well-being.
There is no “easy way” to successfully measure social impact. Key stakeholders indicated a sufficient shared interest
and motivation in program evaluation. Additional research has led to identification of the approaches proposed in this
report. A quasi-experimental design will be required to ensure validity. The resulting design minimizes the limitations of an
evaluation at the greatest economy possible.
When the community is engaged
in solutions to a shared problem
individuals feel valued, leading them
to participate and further impact
solutions. This community-minded
focus, or community competency,
will support improved buy-in for the
intervention, sense of community, and
strengthened support systems for
residents.
EVALUATION DESIGN STRENGTHS &
LIMITATIONS
59. 59
The RCF has initiated an innovative model that builds a case for demonstrated success, program scale-up, and
replication to other jurisdictions. The RCF is advised to make evaluation a priority within the program’s first year to
ensure validity by a comparative study of pre- and post-intervention measures. A watchful public will seek an evidence-
based assessment, and future funders will be interested in either profit or tangible social impact. An effective evaluation
will be an instrument to secure future funds and to gain community support in future and sustained intervention cycles.
Figure 14
