This document discusses challenges with collecting accurate and complete data in Homeless Management Information Systems (HMIS). It notes that data quality is influenced by many personalities, including clients, end users entering data, and the variables themselves. Specific issues are discussed for variables like name, social security number, date of birth, race, gender, disability status, income, and housing status. Accuracy depends on properly training staff on interview techniques, understanding skip patterns and response options, and updating records over time. The document emphasizes that while data is used for important purposes, the limitations of self-reported data from a hard-to-reach population must also be acknowledged.

1. Don’t Know/Refused:
A Researcher’s Perspective on
HMIS Data Collection Challenges
By Susan Walker
Rhode Island Coalition for the Homeless
Bowman Systems Collaborate 2016

2. What do I Know About Homelessness and Data?
 Occupy Providence 2011-2012, back when I used to call
constituents “friends”.
 Antiques Dealer since 1998, alongside homeless and
marginalized people.
 Studied Medical Billing & Coding at a for-profit online
university. Learned about records and predatory lending.
 Currently enrolled at Brown School of Public Health,
learning and doing data collection and analysis.
 Recently hired by my dream employer.
 I’m new! If you pay close attention, you’ll probably catch
my mistakes!

3. Data
Collection
Concepts
 Hard to Reach Population
 Question Ordering
 Coding
 Skip Patterns
 Prompting with Answer Choices
 Probing
 Look backs
 Data Collection Points
 Non-Verbal Communication
 Interviewer Bias
 Client disposition/Setting
 Sample Size
 Bias
 ERROR!

4. Personality
Data Quality is influenced by the
personalities of…
 The HMIS end user
 The client
 The provider’s office culture
 The funding managers
 Government officials
 The HMIS administrators
 Variable itself

5. Form Versus
Questionnaire
 The HUD Intake form has issues.
 You ask a client for their name.
 You don’t ask a client for their name data quality.
 “Has the client been continuously homeless (i.e., on the street, in an emergency
shelter, or safe haven) for at least one year? … Still on the HUD Template.
 Our Entry in HMIS is in a very different order.
 A form gives the impression that the data is somehow uniform, and fits easily
into the boxes.
 A questionnaire, with instructions, communicates that gathering data requires
conversation, and entering data requires judgment.

6. Entry Date
 The Entry Date and data entry date are
often different: a source of error.
 There can be overlapping ES entries on
the same date.
 Backdating is great, but can lead to
absurdities.
 I emphasize the importance of the
relationship between date and data:
 By entering a date, they are
confirming that all the data in the
record is true.
 If they haven’t taken care to verify
it, there is a problem.

7. Name
 Street Names… Duplicate Clients
 Data Quality Descriptors?
 We have found out that we sometimes
have 2 clients to one ID because they
have the same name.
 We train Case Managers to enter a name
data quality descriptor, but not to ask
about it.
 It is unclear which HMIS Data Elements
need to be asked of the client and which
do not.
ALIAS

8. Social Security
Number
 There is a disincentive for
sharing this data.
 HUD’s rationale:
unduplicated client counts,
and SSN necessary for
applications.
 Data Quality Descriptor errors: If the Case Manager enters “doesn’t know” or
“refused”, then the blank SSN shouldn’t flag.
 Some shelters were using 4 digits as a matter of procedure.
 In the event of a security breach, our clients would be vulnerable.

9. Date of Birth
 Date of Birth is a variable with personality!
 It’s a number does not express a quantity.
 We use it to calculate age.
 If it’s not entered properly, it’s hard to calculate age.
 This is about the only trouble free variable.
 Our case managers find “Date of Birth Type” confusing, which is about the only
thing I would consider changing in ServicePoint.

10. Veteran Status
• Self report might be in conflict with actual Veteran Status.
• Often missing, but I don’t understand the skip pattern.
• Why do you think people skip this question?
• In Rhode Island, Veteran Status as defined in the data standards would exclude
people from programs.
• We train to accept self report. The VA verifies later on.

11. Primary Race
 Frustration with these categories is widespread.
 Positive –They allow for meaningful comparison with all other race
data.
 Negative- Our clients do not like categorizing themselves this way.
 The HMIS Race Data Standards are based on the OMB’s Standards.
 Unless the Census changes, these categories are unlikely to change.

12. Secondary Race
 “Allow clients to identify as many
racial categories as apply (up to 5).”
 The 2000 Census was the first to
offer more than one race category,
with a view to understanding
multi-racial populations.
 The secondary race option is a good faith effort to assess racial diversity.
 Stay tuned for revised OMB standards which will likely impact HUD data
standards.

13. Ethnicity
 “In the 2010 census, about 20 percent of
Hispanics left the race question completely
blank.”
 Ethnicity is about culture, customs and
language.
 Different races can have the same ethnicity.
People of one race can be many different
ethnicities.
 We discovered in training that people
answer “Latino” for race, and trained to
enter the ethnicity and probe for primary
and secondary race.

14. Gender
 A simple, yet awkward question.
 Prefacing the whole interview with “I need to
ask every question, even though some may seem
obvious” can help.
 The data standards changed recently.
 It’s highly likely that case managers nationwide
answer this question without asking clients.

15. Disabling Condition
 We can’t decide where to put this question.
 Personality: Often in conflict with Disability HUD Verifications.
 Case managers report hearing “I don’t know”.
 Case managers need training on how to probe by specific conditions.
 How do you ask, and preserve the dignity of the client?
 Disability is a huge term, but the HUD criteria are specific.
 There’s a conflict if your program prohibits asking about disability before
housing placement.
 Confusion surrounding the “self report” standard, and the documentation
required for housing eligibility.

16. Residence Prior to Project Entry
 It’s not clear from HMIS that this needs to be
updated every entry.
 It often does not get updated every entry.
 Our Case Managers want a “getting evicted” option.
 We need an error report that shows that this was
not updated at entry.
 HMIS Data Standards 5.1 devotes 10 pages to this
element.
 New Option “Interim Housing”
 It can be very challenging to code this information
properly even if the client is properly interviewed.

17. Length of Time
 We tried to put this into plain English.
 This field is only as good as the
previous field.
 It would be nice if error reports would
flag when these fields are neglected.
 There is no way to match the length of time to the previous place to see that the
length of time actually corresponds to the previous place.

18. Head of Household
 Frequently skipped.
 What household?
 Even our plain English questions do not make sense in most settings.
 Long instructions in the Data Standards.
 Skipping seems to be associated with the awkwardness of the question, and the
amount of judgment needed to answers the question.
 In our COC many case managers enter the client with the disability as HoH.
 After that, we select the woman before the man, as men commonly exit PH-FAM.
 Personality-How case managers answer this question reflects uncomfortable
truths about our society.

19. Entering from Street or Emergency Shelter
 Our data completeness is good with
this field.
 No way of knowing if it is updated at
each entry.
 Length of Time Homeless is an
extremely important metric.
 We don’t have Safe Havens in RI.
 “Look Back” data is challenging.

20. How Many Times?
 Our data is complete for this field.
 Hard to know if it’s accurate.
 In training we discovered that more than
one case manager would count 4 years as 4
times.
 We trained them to count the breaks.
 We had a case manager act like a client, and he did a fantastic job
of not answering any question straight.
 We can train to collect and enter data, but training in judgment
is very difficult.

21. Total Number of Months 1 month, ah ah ah…
2 months ah ah ah…
1 day = 1 month???

22. Income
 “Projects collecting data through client
interviews should ask clients whether they
receive income from each of the sources
listed rather than asking them to state the
sources of income they receive.”
 We struggle to get PH providers to update
this information annually.
 Our case managers are not entering
“Under the Table” income.
 We are encouraging copious use of
Updates so that the data is already there at
Annual Assessment.
 Clients are not sure what the “right”
answer is.

23. Non-Cash Benefits
 TANF: A variable with an alias.
 Some benefits have amounts and some
don’t.
 Plagued by duplicates.
 I used this screenshot in a newsletter to
show what perfect end dates look like.
 We are training that it is ok to use files
and case notes to make these updates in
the Annual Assessment.

24. Health Insurance

25. Domestic Violence
 The accuracy of this data element is entirely
dependent on the situation.
 What do you do when a client answered YES at
an earlier entry, and answers NO at a current
entry interview?
 People enter NO, then enter “Client doesn’t
know”
 The purpose of this element is to determine if
the client needs protection: a unique personality
trait.
 The data standards have a specific interview
protocol, unique to this element.
 Also unique data entry protocols.

26. Is the Client Permanently Housed?
• Personality: Answered in error
all over our HMIS.
• Too much data rather than not
enough.
• A cut and dried answer that is
easy to determine.
• It seems like it should be more
important than it is.
• A misunderstood variable. It does
not serve the purpose people
think it does.

27. Error(s)
• Error and analysis are two sides of
the same concept.
• Individual errors are addressed.
• I don’t know if Confidence Intervals
and Standard Deviations are
accounted for in ART Reports.
• Our data is used for eligibility,
funding, reporting and
understanding, but the limitations
of the data are rarely discussed.
• A calculation for average Length of
Time Homeless needs a Confidence
Interval.

28. Validity and Reliability

29. Motivation &
Data Quality
 Winning buy-in from end users is probably the biggest challenge.
 Federal and local funding depend on data.
 Eligibility depends on data.
 Providing the right services depends on data.
 Entering high quality data makes you a “good citizen of the HMIS”.
 HMIS gives voice to people who have no voice.
 HMIS tells their story, and people really do listen.

30. References
Bunzli, L. (2016). The Definition of Chronic Homelessness: Applications and Implications for Policy and Practice Part II:
Mixed Methods Study.
Cohn, D. (2015). Census Considers New Approach to Asking About Race - By Not Using the Term at All. Pew Research
Center. Retrieved from http://www.pewresearch.org/fact-tank/2015/06/18/census-considers-new-approach-to-asking-
about-race-by-not-using-the-term-at-all/
HUD. (2014). HMIS Data Collection Template for Project ENTRY – CoC Program.
HUD. (2016). 2014 HMIS Data Standards Data Manual . HUD.
Matt Warfield, B. D. (2016). How has the ACA Medicaid Expansion Affected Providers Serving the Homeless Population:
Analysis of Coverage,Revenues, and Costs. Retrieved from http://kff.org/medicaid/issue-brief/how-has-the-aca-medicaid-
expansion-affected-providers-serving-the-homeless-population-analysis-of-coverage-revenues-and-costs/
Prewitt, K. (2013, August 21). Fix the Census’ Archaic Racial Categories. The New York Times. Retrieved from
http://www.nytimes.com/2013/08/22/opinion/fix-the-census-archaic-racial-categories.html?_r=0

31. Image Credits
Adams, S. Dilbert.com. Retrieved from http://dilbert.com/terms/.
Hargreaves, R. Mr. Men. Fabbri.
Sesame Street.

32. Questions?