This document compares Medicaid reporting in 4 national surveys to administrative data from the Medicaid Statistical Information System (MSIS). It finds that all surveys undercount Medicaid enrollment compared to MSIS. However, when survey data is linked to MSIS, the percentage of "correct reports" of Medicaid is 77% or higher for all surveys. The percentage of misreports is lower for children and low-income individuals. Misreports are more likely to report having another form of coverage rather than being uninsured. The implications are that care should be taken when adjusting Medicaid counts in surveys, as misreports may offset each other between Medicaid and uninsurance.

1. Medicaid Reporting Errors in Four
National Surveys: ACS, CPS, MEPS, and
NHIS
Brett Fried
State Health Access Data Assistance Center (SHADAC)
University of Minnesota
JSM, Boston
August 6, 2014

2. Acknowledgments
• Funding for this work is supported by the
Census Bureau
• Coauthors:
– Brett O’Hara: Census Bureau
– Kathleen Call, Michel Boudreaux, Joanna
Turner: SHADAC
2

3. Background
• Administrative data on public assistance
programs are not sufficient for policy making
– Not always timely
– Do not measure percent uninsured
– Limited number of high quality covariates
– Do not include policy relevant outcome
measures
• Population surveys fill these gaps
– Yet they universally undercount public program
enrollment described in administrative data
3

4. The problem
• Every survey said to “undercount” Medicaid
enrollment
• If counts of Medicaid are biased downward,
counts of other coverage and uninsurance
are biased upward
• The size of the undercount varies by survey
• How does the ACS undercount and survey
response error compare to other federal
surveys?
4

5. Data and methods
• Four National Surveys
– 2009 American Community Survey (ACS)
– 2005 Current Population Survey (CPS)
– 2002 National Health Institute Survey (NHIS)
– 2003 Medical Expenditure Panel Survey/Household
Component (MEPS/HC)
• Medicaid Statistical Information System (MSIS)
• Compare enrollment counts: all values
• Compare linked data: explicit values only
5

6. Compare National Surveys: Medicaid
Question
6
Reference Period Recall Period
Includes
Institutionalized
Pop.
Can exclude
state CHIP
ACS
Point in time
(PIT)
NA Y N
CPS
Ever during the
year
15-17
months
N Y
NHIS
Point in time
(PIT)
NA N Y
MEPS/HC
Monthly/Ever
during the year
3-5 months N N

7. COUNTS PRIOR TO LINKING:
ALL VALUES
7

8. Counts: Why do we care about counts?
–What public see when they see estimates
of enrollment
– Illustrates the problem of concept
alignment
– How the undercount is generally reported
8

9. Counts: Adjustments to align content
between data sources
• Administrative Data:
– Exclude most partial benefit Medicaid enrollees
– Remove duplicates and deceased individuals
– Exclude institutionalized (except ACS)
– Exclude CHIP & SCHIP (except ACS & MEPS)
• Survey Data:
– CPS & NHIS - Exclude CHIP, SCHIP and other
public program enrollees
– ACS and MEPS cannot exclude CHIP, SCHIP
or other means tested public program enrollees
9

10. Counts: Compare to MSIS: All Values
10
Percent different from MSIS
Raw Adjusted
ACS 2009 .2% 16%
CPS 2005 -38% -32%
NHIS 2002 -39% -26%
MEPS/HC 2003 -25% -6%
Sources: SNACC reports and 2009 American Community Survey and 2009 Medicaid Statistical Information
System

11. LINKED FILE RESULTS:
WHAT PEOPLE ACTUALLY
REPORT (EXPLICIT VALUES
ONLY)
11

12. Linked data: How do we link the data?
• Use linking methodology developed by the
Census Bureau’s Center for Administrative
Records Research and Applications
– Personal Identification Key (PIK)
• Research conducted at the MN Census
Research Data Center located at the
University of Minnesota
– http://mnrdc.umn.edu/
12

13. Linked and unlinked data and misreports
13

14. Accuracy of Medicaid Reporting: What they
report in the ACS linked to MSIS
14
“Correct Reports”
Sources: SNACC reports and 2009 ACS and MSIS.
“Misreports”
(False Negatives)
Medicaid
Any Other
Coverage
Uninsured
ACS 2009 77% 13% 10%
CPS 2005 59% 23% 17%
NHIS 2002 67% 24% 9%
MEPS/HC 2003 84% 8% 8%

15. Percent Linked: Age (“correct reports”)
0-17 18-64 65+
15
ACS 2009* 81%
(0-18)
Sources: SNACC reports and 2009 ACS and MSIS.
• The age groups used for the ACS analyses are slightly different.
72%
(19-64)
71%
CPS 2005 62% 56% 60%
NHIS 2002 68% 65% 66%
MEPS/HC 2003 86% 81% 85%

16. Percent Linked: Percent of poverty (“correct
repots”)
0-124% 125-199% 200%+
16
ACS 2009* 83%
(0-138%)
Sources: SNACC reports and 2009 ACS and MSIS
* The percent of poverty categories used for the ACS were slightly different.
72%
(139-200%)
64%
(201+%)
CPS 2005 66% 52% 40%
NHIS 2002 73% 56% 41%
MEPS/HC 2003 88% 83% 73%

17. Linked and unlinked data and misreports
17

18. Percent of those not linked: Misreports of
those who are not linked
18
Sources: SNACC reports and 2009 ACS and MSIS
Percent of those not linked
ACS 2009* unknown
CPS 2005 3%
NHIS 2002
1%
MEPS/HC 2003 unknown

19. Summary of results
• The percent of “misreports” in the ACS appears
in line with other surveys
• The percent of “misreports” is lower in all four
surveys for
– Children and
– Low income individuals
• The percent of “misreports” in all four surveys
that are reported as uninsured is lower or equal
to the percent reported as other sources of
coverage
19

20. Implications
• Care should be taken in making any
adjustments to Medicaid counts because it is
important to consider
– survey and administrative data universes and
definitions,
– “misreports” of uninsurance as compared to other
sources of coverage,
– offsetting “misreports” of Medicaid among those not
linked.
• ACA health reforms present new challenges.
– Differentiating Medicaid from exchange enrollment.
– Large increase in the number of Medicaid
enrollees.
20

21. Key Contact
Brett Fried
bfried@umn.edu
612.624.1406
www.shadac.org
@shada
c