Joint presentation by the DC Promise Neighborhood Initiative (DCPNI) and Urban Institute staff at the Eastern Evaluation Research Society's Annual Conference in 2014. Presentation focuses on DCPNI's neighborhood survey - a community wide data collection project. The slides offer tips and suggestions on how to make the process as smooth as possible without compromising data collection rigor.
Addressing Obesity In The Latino Community Through Community-based Advocacy In Baldwin Park
Alfred Mata, Local Policy Specialist, California Center for Public Health Advocacy.
Tourism in TFCAs and Guidelines, Anna SpenceleyAnna Spenceley
A presentation given at a World Parks Congress on 14 November 2014 during a session on Tourism in Transfrontier Conservation Areas: A vehicle for development and sustainable life.
Program Evaluation: Can You Prove You Are Making a Difference?Isaac Castillo
Workshop for NetworkPeninsula - Nonprofit EXcellence in Training (NEXT) Workshop series
Most nonprofits assume they are doing good work and changing the lives of their service recipients, but how do you really know? In this workshop, participants will learn some important concepts, approaches, and techniques to utilize when attempting to measure nonprofit effectiveness. Common mistakes and challenges will also be discussed.
Addressing Obesity In The Latino Community Through Community-based Advocacy In Baldwin Park
Alfred Mata, Local Policy Specialist, California Center for Public Health Advocacy.
Tourism in TFCAs and Guidelines, Anna SpenceleyAnna Spenceley
A presentation given at a World Parks Congress on 14 November 2014 during a session on Tourism in Transfrontier Conservation Areas: A vehicle for development and sustainable life.
Program Evaluation: Can You Prove You Are Making a Difference?Isaac Castillo
Workshop for NetworkPeninsula - Nonprofit EXcellence in Training (NEXT) Workshop series
Most nonprofits assume they are doing good work and changing the lives of their service recipients, but how do you really know? In this workshop, participants will learn some important concepts, approaches, and techniques to utilize when attempting to measure nonprofit effectiveness. Common mistakes and challenges will also be discussed.
Definition of assessment,
ASSESSMENT AND TESTING
EDUCATIONAL DECISION
FACTORS WHY WE PLAN ASSESSMENT DEVICES,
Criteria for selecting Assessment instrument
,PURPOSE OF ASSESSMENT,
Assessment can do more than simply diagnose and identify students’ learning needs; it can be used to assist improvements across the education system in a cycle of continuous improvement:
PRINCIPLES OF ASSESSMENT
TYPES OF ASSESSMENT
When technology hits the sidewalk empowering community residents through 21s...Isaac Castillo
The conversation around 21st century data collection methods continues to evolve, but little work has been done to employ these methods to empower community stakeholders as part of the data collection process. This session will detail how DCPNI conducted a representative community survey using innovative data collection methods within an inclusive evaluation framework by employing community members as agents of community data collection. We will address the challenges faced in implementing a survey of significant scope, including time constraints, unknown literacy levels of survey participants, and inconsistent access to the internet, and detail how these challenges were overcome. The session will include a step-by-step demonstration of DCPNI's use of Key Survey Software to administer surveys on tablet devices with offline capability. We will discuss the strengths and weaknesses of our approach compared to alternatives and outline how the collected data will be utilized to benefit the community going forward.
Process Documentation Research and Impact of ICRISAT Village GrantICRISAT
ICRISAT village grant was provided to six villages: two in Telangana and four in Maharashtra state. One time grant of US$ 7,000 was provided to each community adopting Community - Driven Development approach.
Transforming Transportation 2015: Smart Cities for Shared Prosperity is the annual conference co-organized by the World Resources Institute and the World Bank.
Community Engagement for Complete CommunitiesRPO America
In our everyday lives, we interact with people who look, act, and think like us. In the course of our jobs as planners, we interact with the public—that entire collection of human variability that exists in communities. To engage effectively, we are required to get outside our comfort zone and reach out to the entire collection of residents that make a complete community. This workshop held at the National Regional Transportation Conference (June 2019, Columbus, OH) presented in an interactive format how a planner can develop a community engagement strategy based upon the five I’s of public involvement—Identify, Invite, Inform, Involve, and Improve. Workshop participants were briefed on the concepts of environmental justice, recent public involvement tools and strategies, and then interact in groups to develop a public participation plan for a community scenario.
Social Impact of Open Data - hosted by Center for Data Innovation and Sunligh...Sandra Moscoso Mills
For Social Impact of Open Data - hosted by Center for Data Innovation and Sunlight Foundation
Focus of social impact of open data on education and international development.
http://www.eventbrite.com/e/the-social-impact-of-open-data-tickets-11775253099
Definition of assessment,
ASSESSMENT AND TESTING
EDUCATIONAL DECISION
FACTORS WHY WE PLAN ASSESSMENT DEVICES,
Criteria for selecting Assessment instrument
,PURPOSE OF ASSESSMENT,
Assessment can do more than simply diagnose and identify students’ learning needs; it can be used to assist improvements across the education system in a cycle of continuous improvement:
PRINCIPLES OF ASSESSMENT
TYPES OF ASSESSMENT
When technology hits the sidewalk empowering community residents through 21s...Isaac Castillo
The conversation around 21st century data collection methods continues to evolve, but little work has been done to employ these methods to empower community stakeholders as part of the data collection process. This session will detail how DCPNI conducted a representative community survey using innovative data collection methods within an inclusive evaluation framework by employing community members as agents of community data collection. We will address the challenges faced in implementing a survey of significant scope, including time constraints, unknown literacy levels of survey participants, and inconsistent access to the internet, and detail how these challenges were overcome. The session will include a step-by-step demonstration of DCPNI's use of Key Survey Software to administer surveys on tablet devices with offline capability. We will discuss the strengths and weaknesses of our approach compared to alternatives and outline how the collected data will be utilized to benefit the community going forward.
Process Documentation Research and Impact of ICRISAT Village GrantICRISAT
ICRISAT village grant was provided to six villages: two in Telangana and four in Maharashtra state. One time grant of US$ 7,000 was provided to each community adopting Community - Driven Development approach.
Transforming Transportation 2015: Smart Cities for Shared Prosperity is the annual conference co-organized by the World Resources Institute and the World Bank.
Community Engagement for Complete CommunitiesRPO America
In our everyday lives, we interact with people who look, act, and think like us. In the course of our jobs as planners, we interact with the public—that entire collection of human variability that exists in communities. To engage effectively, we are required to get outside our comfort zone and reach out to the entire collection of residents that make a complete community. This workshop held at the National Regional Transportation Conference (June 2019, Columbus, OH) presented in an interactive format how a planner can develop a community engagement strategy based upon the five I’s of public involvement—Identify, Invite, Inform, Involve, and Improve. Workshop participants were briefed on the concepts of environmental justice, recent public involvement tools and strategies, and then interact in groups to develop a public participation plan for a community scenario.
Social Impact of Open Data - hosted by Center for Data Innovation and Sunligh...Sandra Moscoso Mills
For Social Impact of Open Data - hosted by Center for Data Innovation and Sunlight Foundation
Focus of social impact of open data on education and international development.
http://www.eventbrite.com/e/the-social-impact-of-open-data-tickets-11775253099
Crowdsourcing Metadata Practices at USUAndrea Payant
USU Libraries’ Cataloging and Metadata Unit has successfully investigated several methods to engage the public to involve them in the creation of metadata for USU’s Digital History Collections. Most, if not all the techniques we have tested have yielded positive results and have improved the relevancy and accuracy of our descriptive metadata.
In December 2015, the Social Development Centre Waterloo Region was fighting the city staff recommendation to cut its community information services funding. We lost the appeal in Council on December 14th with 6 to 4 vote despite community voices speaking up and delegations presenting before the Council. Lear more at www.waterlooregion.org
A webinar from the Annie E. Casey and William T. Grant foundations explores how partnerships between researchers and child welfare professionals can be a valuable resource for agencies serving kids and families.
Spokane Regional Health District uses Providence Health Services grant funding to increase access to early intervention services. The Ask Now. Babies and Kids Can't Wait campaign utilizes mass media strategies to empower parents to do developmental screening at home, share tools with pediatricians and also do online Ages and Stages Questionnaire through WithinReach/ParentHelp123
The Future of Logic Models: Logic Models in 3DIsaac Castillo
The time has come for us to break out of our two dimensional representations of programmatic interventions. We need to re-think representations of program interventions and explore how logic models can be crafted in three dimensions (3D) to better respond to the evolving and dynamic nature of our programs and society.
Session at the American Evaluation Association Virtual Conference - October 28, 2020
The Logic Model Repair Shop: An Introduction to 3D Logic Models.Isaac Castillo
Presentation at EERS 2018 on #3DLogicModels by Elizabeth Grim, Isaac Castillo, and Elena Pinzon O'Quinn
http://eers.org/session/the-logic-model-repair-shop-why-most-logic-models-are-broken-and-how-we-can-fix-them/
Collective impact in 3D - Collective Impact Convening Short TalkIsaac Castillo
We can think of collective impact initiatives in new ways if we break out of a traditional approach to logic models and consider building logic models in three dimensions. #3DLogicModels
Workshop for NetworkPeninsula - Nonprofit EXcellence in Training (NEXT) Workshop series
How do you know which outcomes to measure as part of your collective impact effort, and how do you come to a consensus on those outcomes? This workshop will offer practical and theoretical advice on how a collective impact effort can identify, choose, and refine outcome areas as part of their work. Practical challenges (where will the data come from, who is going to analyze it, what will be done with the data) will also be discussed, along with some recommendations.
Evaluation Blooers and How to Make the Most of Your MistakesIsaac Castillo
Presentation focusing on common evaluation mistakes and how audience members can avoid similar situations. Slides contain a list of audience generated take aways and lessons learned.
Castillo high quality program evaluation in nonprofitsIsaac Castillo
Presentation for Tidewater Community College Workforce Solutions school - the Academy for Nonprofit Excellence.
http://www.tccworkforce.org/non-profit-management
Can your nonprofit prove you are making a difference?Isaac Castillo
Slides from class taught at Tidewater Community College's Academy for Nonprofit Excellence. Focuses on the basics of data collection, outcome measurement, logic models,and performance management.
Sharpen existing tools or get a new toolbox? Contemporary cluster initiatives...Orkestra
UIIN Conference, Madrid, 27-29 May 2024
James Wilson, Orkestra and Deusto Business School
Emily Wise, Lund University
Madeline Smith, The Glasgow School of Art
Have you ever wondered how search works while visiting an e-commerce site, internal website, or searching through other types of online resources? Look no further than this informative session on the ways that taxonomies help end-users navigate the internet! Hear from taxonomists and other information professionals who have first-hand experience creating and working with taxonomies that aid in navigation, search, and discovery across a range of disciplines.
Acorn Recovery: Restore IT infra within minutesIP ServerOne
Introducing Acorn Recovery as a Service, a simple, fast, and secure managed disaster recovery (DRaaS) by IP ServerOne. A DR solution that helps restore your IT infra within minutes.
0x01 - Newton's Third Law: Static vs. Dynamic AbusersOWASP Beja
f you offer a service on the web, odds are that someone will abuse it. Be it an API, a SaaS, a PaaS, or even a static website, someone somewhere will try to figure out a way to use it to their own needs. In this talk we'll compare measures that are effective against static attackers and how to battle a dynamic attacker who adapts to your counter-measures.
About the Speaker
===============
Diogo Sousa, Engineering Manager @ Canonical
An opinionated individual with an interest in cryptography and its intersection with secure software development.
This presentation by Morris Kleiner (University of Minnesota), was made during the discussion “Competition and Regulation in Professions and Occupations” held at the Working Party No. 2 on Competition and Regulation on 10 June 2024. More papers and presentations on the topic can be found out at oe.cd/crps.
This presentation was uploaded with the author’s consent.
This presentation, created by Syed Faiz ul Hassan, explores the profound influence of media on public perception and behavior. It delves into the evolution of media from oral traditions to modern digital and social media platforms. Key topics include the role of media in information propagation, socialization, crisis awareness, globalization, and education. The presentation also examines media influence through agenda setting, propaganda, and manipulative techniques used by advertisers and marketers. Furthermore, it highlights the impact of surveillance enabled by media technologies on personal behavior and preferences. Through this comprehensive overview, the presentation aims to shed light on how media shapes collective consciousness and public opinion.
Media as a Mind Controlling Strategy In Old and Modern Era
A Neighborhood Survey in the Nation’s Capital: Balancing Rigor, Resources, and Respect
1. A Neighborhood Survey in the Nation’s Capital:
Balancing Rigor, Resources, and Respect
www.dcpni.org
@dcpni
1
www.urban.org
@urbaninstitute
Eastern Evaluation Research Society
37th Annual Conference
April 28, 2014
2. Today’s Panelists
2
Isaac Castillo
Director of Data and Evaluation
DC Promise Neighborhood
Initiative
@isaac_outcomes
Samantha Greenberg
Data and Evaluation Specialist
DC Promise Neighborhood
Initiative
@eval_revolution
Maia Woluchem
Research Assistant
Metropolitan Housing and
Communities Policy Center
Urban Institute
Megan Gallagher
Senior Research Associate
Metropolitan Housing and
Communities Policy Center
Urban Institute
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
3. A Neighborhood Survey
in the Nation’s Capital
• The DC Promise Neighborhood Initiative’s
Neighborhood Survey
• It Takes a Neighborhood to Design a Survey
• Getting Community Residents Excited About a
Neighborhood Survey.
• Data Collection in the 21st Century
3April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
4. The DC Promise Neighborhood Initiative’s
Neighborhood Survey:
When Community, Research, Evaluation,
and Rigor All Come Together
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
4
Isaac Castillo
Director of Data and Evaluation
DC Promise Neighborhood
Initiative
Twitter: @Isaac_outcomes
5. As part of the White House Neighborhood
Revitalization Initiative, Promise Neighborhoods
align federal funding streams that invest in
transforming neighborhoods of concentrated
poverty into neighborhoods of opportunity.
What is the Promise Neighborhood Approach?
The INSPIRATION….
Harlem Children’s Zone (HCZ) inspired the U.S.
Department of Education’s Promise
Neighborhoods program, which launched in 2010.
5April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
6. What is DCPNI?
• DCPNI is one of 12 communities awarded
funding from the U.S. Department of
Education.
• DCPNI’s focus is on the Kenilworth-Parkside
neighborhood of Washington, DC.
• Founded as an independent nonprofit in 2012
shortly after receiving federal funding.
6April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
7. Kenilworth-Parkside by the Numbers
• 5,725 people – 1,800 children
• 98% African American, 55% are female
• 50% of adults live in poverty
• 49% of K-P residents experience food insecurity
• 70% of K-P residents have a high school
education or higher (compared to 88% in DC)
• 85% of households with children are headed by
single females
7April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
8. DCPNI’s Vision
To end intergenerational
poverty in the
Kenilworth-Parkside
community.
8April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
10. What does DCPNI do?
10
Fund
Community-based organizations
and schools to provide effective
and research based programming
Coordinate
All services provided in the
Promise Neighborhood to
maximize effectiveness and avoid
duplication
Connect
Community residents and
students to available services
provided by partners
Train
Everyone in the Promise
Neighborhood to utilize
information and advocate for
themselves and their neighbors
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
11. DCPNI’s Work at a Glance
11
Fund
Coordinate
Provide
Train
Schools
Partners
Parents
Children
=+
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
12. How Does DCPNI Measure Progress
on Each Promise?
• DCPNI has 15 federal indicators that we are
required to track by the Department of
Education.
• Developing additional indicators on issues that
are unique to Kenilworth-Parkside.
– Teen birth rate
– Mother’s educational attainment
– Food insecurity
• These indicators help us determine if we are
progressing towards the Promises.
12April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
13. Sample of DCPNI Indicators
• Parents read to their 0-5 year old
children.
• Increased feelings of safety among
community residents.
• Children ages 0-5 that have a medical
home.
13April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
These data points need to be collected
directly from community residents.
14. The Need for a Neighborhood Survey
• Collect information directly from community
residents.
• Limited other publically (or privately) available data
sets for the Kenilworth-Parkside population.
• Desire to measure community level change over
time.
• What little data that was available was old, limited
to parts of the community, or collected using low
rigor methods.
14April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
15. The Most Significant Challenges
• Very short timeframe – needed to complete entire
process (from survey design to final analysis of
data) in 9 months.
• Low levels of literacy in the community (and low
comfort level with technology).
• High levels of suspicion of outsiders (and even
suspicion of those from another part of the
community).
• High levels of distrust of government and
government-like entities.
15April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
16. It Takes a Neighborhood to Design a Survey:
The Methodology Behind the Kenilworth-
Parkside Neighborhood Survey
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
16
Megan Gallagher
Senior Research Associate
Metropolitan Housing and
Communities Policy Center
Urban Institute
17. Methodology Overview
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
17
• Instrument Design
• IRB Review
• Sampling
• Sampling frame
• Strata, or key groupings
• Random selection
• Response Rates
• Population Weights
18. Instrument Design
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
18
• Create a baseline assessment of the neighborhood
from which to assess change
• Many considerations, including
• Department of Education Government Performance and
Results Act (GPRA) indicators
• DCPNI programmatic priorities
• DCHA Choice Neighborhoods data needs
• Urban Institute suggestions
• Resident interests and concerns
• Instrument length, nature of questions
19. Instrument Design
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
19
• Topics include:
• Housing
• Neighborhood amenities
• Access to Food
• Neighborhood Supports
• Neighborhood Conditions and Safety
• Household Composition
• Travel, Education, Employment and Public Assistance
• Child Health, Education, and Well-being
20. IRB Review
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
20
• Institutional Review Board (IRB) makes sure that
research minimizes risks to human subjects
• Reviewed and approved the plan for:
• Requiring confidentiality pledges from staff
• Obtaining consent from research subject
• Roles for community resident and City Year corps
member
• Providing incentives for completed surveys
• Keeping data confidential
21. Sampling
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
21
• Used a list of addresses for 2 census tracts that
make up Kenilworth Parkside
• Separated them into 3 sub-neighborhoods, or strata
• Group 1: Kenilworth Courts
• Group 2: KPRMC, Mayfair, Paradise, Lotus Square and
Victory Square
• Group 3: Eastland Gardens and Paradise
• Randomly select addresses within each sub-
neighborhood (SAS Proc Surveyselect)
• Oversampled Group 1
23. Population Weights
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
23
Sub-
neighborhood Population
% of
Population
Completed
Surveys
% of
Surveys Weight
Group 1:
Kenilworth
Courts
283 12% 133 30%
=12%/30%
=.3857
Group 2: KPRMC,
Mayfair,
Paradise, Lotus
Square and
Victory Square
1620 68% 217 49%
=68%/49%
=1.4012
Group 3:
Eastland Gardens
and Paradise
476 20% 94 21%
=20%/21%
=.9731
Total 2379 100% 444 100% 1.0000
24. Doing the Impossible:
Getting Community Residents Excited
About a Neighborhood Survey
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
24
Samantha Greenberg
Data and Evaluation Specialist
DC Promise Neighborhood
Initiative
Twitter: @eval_revolution
25. April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
25
Be inclusive
Gain community
support
Build
relationships
Be culturally
sensitive
Add unique
value
Goals for Kenilworth-Parkside Survey:
26. April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
26
Survey Administration Teams:
Kenilworth-
Parkside
Community
Resident
• Navigated the neighborhood
• Greeted person at door
• Introduced survey
• Documented households visited
• Handled incentives
• Consent process
• Asked survey questions
• Documented answers on tablet
• Handled technology
Step 1: Build an Inclusive Process
27. 27
Community residents:
• Familiar with neighborhood
• Have relationships with survey takers
• Knowledgeable about community history and assets
Step 1: Build an Inclusive Process
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
28. 28
City Year corps team members :
• Versed in technology
• Able to offer confidentiality (not from K-P)
• Experienced serving D.C. communities
Step 1: Build an Inclusive Process
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
29. 29
Community residents:
• Recruitment and screening
• Background checks
• Mandatory training session
City Year Corps Team:
• Partnership with City Year
• Recruitment at full team event
• Mandatory training session
Hiring Process
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
30. 30
NEIGHBORHOOD SURVEY TRAINING
30
• Structure of each
work day
• Safety precautions
• How to introduce
survey
• How to document
responses from
potential survey
respondents
Training Process
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
31. 31
Kenilworth-
Parkside
Community
Resident
Advantages of Two-person Teams
• Increased likelihood of “open doors”
• Decreased # of survey days needed
• Easy navigation of community
• Division of labor
• Two people available to answer questions
• Balance between confidentiality and trust
• Increased safety
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
32. 32
Challenges of Two-person Teams
• Massive recruiting and hiring efforts
• DCPNI staff capacity
• Difficult to ensure fidelity
• Downtime during surveys
• Answering questions about survey when
approached on street
Kenilworth-
Parkside
Community
Resident
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
33. 33
Step 2: Prioritize cultural sensitivity
• Survey questions tested with community
• Community feedback encouraged throughout
• Survey administrators
participated in focus
group
• Concerns from
community members
noted for next survey
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
34. 34
Step 3: Add Unique Value
• I heart data booklet
provides data
snapshot
• Data now exists on
topics for which
there are no other
sources
• Ongoing
conversations with
the K-P community
about data
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
35. 35April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
Step 4: Build Relationships
36. 36April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
• Impacts of Kenilworth-Parkside neighborhood survey:
Inclusive data
collection
Gained community
support
Built
relationships
Culturally
sensitive survey
Unique data
collected
Results: Community Support for Survey:
37. Data Collection in the 21st Century:
The Use of Tablet Computers for a
Neighborhood Survey
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
37
Maia Woluchem
Research Assistant
Metropolitan Housing and
Communities Policy Center
Urban Institute
mwoluchem@urban.org
38. Challenges for Our Survey
• Several teams approaching hundreds of
doorsteps over several points in time
• Each respondent was different from the last
• Each interviewer is different from the last
• Presenting several challenges
– Monetary costs
– Confidentiality
– Standardization
– Flexibility
– Ease
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
38
39. Paper vs Plastic
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
39
Feature Paper Surveys Tablet Surveys
Money
Printing supplies,
associated materials, post
survey dataentry
$200 per tablet,KeySurvey
subscription
Confidentiality
Maintaining security of
documentsin field, post
survey
Tablet’s secure connection
and privacy capabilities
Standardization Interview cues
Flexibility May involve skipping pages Skip logic, conditionality
Ease
40. Program
• Used KeySurvey as
our survey platform
– Allowed ability to build conditional sections and
complicated skip patterns
– Allowed personalization based on household
traits
• Cost-reducing in terms of time, money and flexibility
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
40
41. Back-End v. Front-End
• Level one: 159 questions and hundreds of
answer choices
• Level two: Variable names
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
41
42. Back-End v. Front-End
• Level Three: Skip patterns
– Sometimes a simple skip:
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
42
43. Back-End v. Front-End
• Sometimes it’s something else entirely:
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
43
44. Back-End v. Front-End
• Everything was automated, making things
easy for everyone—
• Setting:
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
44
45. Back-End v. Front-End (Consent)
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
45
49. Administration:
Who conducted the survey?
• AmeriCorps members and college volunteers
– Read aloud the questions to the householder
– Householder responds verbally
• Why?
– Literacy rates among the community
– Unfamiliarity with the technology
– Standardization of questions and answers
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
49
50. Training
• Back-end was sophisticated but front-end was
very user friendly.
• Training session has some ground rules:
– Trust the programming
– Don’t be pushy
• And some golden rules:
– Read the exact wording of the question as presented
– Obtain consent
– Work with your partner
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
50
51. Partnerships in Action
• Morning Briefing (DCPNI Offices)
– Community Resident:
• Receive ten addresses, gift cards (4), survey receipts,
door hangers, and a walkie talkie
– Interviewer:
• Receive same ten addresses and a tablet computer
• Walk
– Resident: Fill out tracking form to document
address condition
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
51
52. Partnerships in Action
• Doorstep
– Community Resident:
• Explain survey and present opportunity for giftcard
• Answer questions householder may have
• Document what happens
– Interviewer:
• Obtain consent
• Administer survey
• Administration (Interviewer & Community Resident)
– At a comfortable place
– Resident Role
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
52
53. Partnerships in Action
• Post-Survey
– Community Resident:
• Offer respondent choice of incentives
• Everyone signs receipt
• What if no one is home?
– Resident:
• Leave door hanger
• Document address so that another
team could make attempt
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
53
54. Tablets are not Without Challenges
• What isn’t done by hand must be done by
computer
– Careful time-intensive programming necessary
• Technical difficulties
– Potential for tablet failure (need for backup tablets)
• Post-launch errors are data errors
– Examples of missing answers or misplaced variables
– Typing answers in leads to some data quality errors
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
54
55. But the Benefits?
• Benefits greater than challenges (in our case)
• Data/Tablet Security
– End of day wipe and upload to UI SFTP
– Confidential drives at UI
– Cerberus for tablet security
• Substantial results
– Over four weekends in October and November,
got 444 responses, a response rate of 59% overall
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
55
56. Audience Q&A
April 28, 2014
DC Promise Neighborhood Initiative and
The Urban Institute
56
Isaac Castillo
Director of Data and Evaluation
DC Promise Neighborhood
Initiative
@isaac_outcomes
Samantha Greenberg
Data and Evaluation Specialist
DC Promise Neighborhood
Initiative
@eval_revolution
Maia Woluchem
Research Assistant
Metropolitan Housing and
Communities Policy Center
Urban Institute
Megan Gallagher
Research Associate
Metropolitan Housing and
Communities Policy Center
Urban Institute
Editor's Notes
Importance of validated items, items suggested by the PNI TA team
The neighborhood survey instrument includes the following domains: HousingNeighborhood amenitiesAccess to Food Neighborhood Supports Neighborhood Conditions and SafetyHousehold CompositionTravel, Education, Employment and Public AssistanceChild Health, Education, and Well-being
For analysesUse population weights to reflect full neighborhoodBe conscious of sample sizes
Differences:Respondents—family types are different, some are more/less literate, some have kids and others are singletonsInterviewers—Although everyone undergoes training,…
444 surveys with 757 eligible households, which is equivalent to 59% response rate.