Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Coverage Nonresponse Trade-Off

281 views

Published on

Undercoverage plaques many frames - housing units are missed by listers or do not appear on the postal service list; persons with tenuous connections to households are not captured in rosters; persons hide their eligibility during screener interviews. The literature on undercoverage suggests several methods for improving the coverage of such frames, via a missed housing unit procedure, or detailed probes about household members, or disguising the target population in survey questions. However, each of these solutions introduces additional costs into the survey process. In this way, survey designers face a coverage-cost trade-off. In addition, there is increasing evidence that the cases found via these coverage-improvement measures are disproportionately nonresponders to the survey request. Thus there appears to be a coverage-nonresponse trade-off as well. Together these points raise the question of how much effort we should put into increasing coverage, when such efforts increase costs and nonresponse? This presentation will review empirical evidence for these trade-offs and search for clues to the mechanisms underlying the connection between nonresponse and undercoverage.

Published in: Science
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Coverage Nonresponse Trade-Off

  1. 1. The Coverage-Nonresponse Trade-Off JSM 2014 Stephanie Eckman Frauke Kreuter
  2. 2. Motivation  Reducing undercoverage means: ‐ Adding cell numbers to RDD frame ‐ Including homeless, institutionalized in general population surveys ‐ Providing tablet & internet access  Efforts are costly  Are the people included with these efforts disproportionate nonresponders? 2
  3. 3. Examples  RDD + mobile phone surveys ‐ Lower response rates among mobile only HHs (AAPOR Cell Phone Task Force Report)  LISS online panel ‐ Lower recruitment rates among cases without internet (Leenheer & Scherpenzeel 2013)  Random walk ‐ Interviewers may skip HHs that look like NRs (Alt et al 1991, Manheimer & Hyman 1949 )  Half open interval procedure ‐ Interviewers may fail to cover units that look like NRs (Eckman & O’Muircheartaigh 2011) 3
  4. 4. Screening Study  2 versions of screener questions ‐ Direct “Is anyone in this household 35-55?” ‐ Full HH roster Age of all adults in HH Condition Screener Completion Rate Eligibility Rate Interview Completion Rate Response Rate Yield Direct 59.3 31.8 86.3 32.3 285 Roster 53.5 45.1 71.5 23.9 361  Trade-off: High response rate or high coverage? Tourangeau, Kreuter & Eckman 2013 4
  5. 5. Mechanisms Behind Trade-Off  Respondent side ‐ Burden ‐ Learning to use internet/computer difficult ‐ Survey on cell phone annoying ‐ Hidden refusals: Respondents screen out rather than refuse  Interviewer side ‐ Judged by response rate, not coverage ‐ Coverage & response are different skill sets 5
  6. 6. Choice Faced by Survey Designers 6  Choose A or B?  High RR in A hides low coverage rate  Cost considerations Coverage Rate High Low Response Rate High A * Low B
  7. 7. Screening Study  Coming back to this example: ‐ Direct: high RR, low coverage ‐ Roster: low RR, high coverage Condition Screener Completion Rate Eligibility Rate Interview Completion Rate Response Rate Yield Direct 59.3 31.8 86.3 32.3 285 Roster 53.5 45.1 71.5 23.9 361 Tourangeau, Kreuter & Eckman (2013) 7
  8. 8. Propensity Models  Logit models, run separately for 2 screener conditions  IVs: female, age (squared), # calls (squared), party identification, postal code, interviewer payment, refusal flag, mobile indicator  Coverage Model ‐ Case base: expected to be eligible (n=2,904) ‐ DV: screened & eligible ‐ Pseudo-R2: 15% direct; 10% roster  Response Model ‐ Case base: found to be eligible (n=735) ‐ DV: complete main interview ‐ Pseudo-R2: 20% direct; 33% roster 8
  9. 9. Propensity Models  Logit models, run separately for 2 screener conditions  IVs: female, age (squared), # calls (squared), party identification, postal code, interviewer payment, refusal flag, mobile indicator  Coverage Model ‐ Case base: expected to be eligible (n=2,904) ‐ DV: screened & eligible ‐ Pseudo-R2: 15% direct; 10% roster  Response Model ‐ Case base: found to be eligible (n=735) ‐ DV: complete main interview ‐ Pseudo-R2: 20% direct; 33% roster 9 }} CPdirect CProster RPdirect RProster
  10. 10. Coverage Propensities Correlation 0.79 10
  11. 11. Response Propensities Correlation 0.88 11
  12. 12. Overall Inclusion Propensity Correlation 0.86 12
  13. 13. Overall Inclusion Propensity Correlation 0.86 13
  14. 14. First Look at Bias 14
  15. 15. Research Agenda  Mechanisms of connection between them ‐ Interviewers? Respondents?  Bias due to undercoverage & NR  How much should we spend to increase coverage, if it only increases nonresponse? 15
  16. 16. www.iab.de Thanks – comments & ideas welcome stephanie.eckman@iab.de Website: stepheckman.com
  17. 17. References 17 Alt, C., Bien, W. & Krebs, D. (1991). “Wie zuverlässig ist die Verwirklichung von Stichprobenverfahren? Random route versus Einwohnermeldeamtsstichprobe”. ZUMA Nachrichten, 28, 65-72. American Association for Public Opinion Research (2010). “Cell Phone Task Force Report: New Considerations for Survey Researchers When Planning and Conducting RDD Telephone Surveys in the U.S. With Respondents Reached via Cell Phone Numbers”. Eckman, S. & O’Muircheartaigh, C. (2011). “Performance of the Half-Open Interval Missed Housing Unit Procedure”. Survey Research Methods, 5(3), 125-131. Hainer, P. (1987). “A Brief and Qualitative Anthropological Study Exploring the Reasons for Census Coverage Error Among Low Income Black Households”. In Report prepared under contract with the Census Bureau. Leenheer, J. & Scherpenzeel, A. (2013). “Does it Pay Off to Include Non-Internet Households in an Internet Panel?” International Journal of Internet Science 8 (1), 17–29. Manheimer, D. & Hyman, H. (1949). “Interviewer Performance in Area Sampling”. The Public Opinion Quarterly, 13(1), 83-92. Tourangeau, R., Kreuter, F. & Eckman, S. (2012). "Motivated Underreporting in Screening Interviews“. Public Opinion Quarterly 76(3), 453- 469.
  18. 18. Coverage & Nonresponse Propensities Correlation 0.43 18
  19. 19. Coverage & Nonresponse Propensities Correlation 0.67 19
  20. 20. www.iab.de Thanks – comments & ideas welcome stephanie.eckman@iab.de Website: stepheckman.com

×