The use of GIS tools in analyzing and conducting large-scale surveys has increased in the last several years and will likely continue to do so as the technologies become less expensive and easier to use. Starting with the Total Survey Error framework, this talk will discuss how GIS tools can help us measure and reduce different error sources, such coverage, nonresponse and measurement error. In addition, the tools can increase interviewer efficiency and reduce data collection costs. As we embrace these tools, survey researchers should maintain a healthy skepticism about their role. The talk will review the errors that GPS devices and GIS software can introduce; privacy and confidentiality concerns are also important.

1. Uses of GIS in Survey Data Collection
AAPOR 2014
Stephanie
Eckman

2. What Can GIS Do for Us?
 Management & analysis of spatial data
 GPS / navigation tools
 Geocoding
‐ Assign coordinate to address
 Relate coordinates to blocks/tracts
 Proximity
 Interpolation
‐ Model air pollution from observation stations
2

3. GIS & Total Survey Error Framework
3
Construct
Response
Edited Response
Target Population
Sampling Frame
Sample
Respondents
Postsurvey Adjustment
Survey Statistic
Measurement

4. Measurement
 Where respondent is at time of survey
‐ Via smartphone, IP address
 Where respondent lives
‐ Ask respondent to click on home (web)
‐ Ask respondent for address to geocode
 Where respondent travels to
‐ Transportation surveys (Chung et al 2005)
‐ Zoo study (Czaplicki 2011)
4

5. Data Processing & Editing
 Merge in additional info about neighborhood
‐ Distance to nearest grocery store
‐ Frequency of traffic accidents within 2 miles
‐ Number of violent crimes within 5 miles
 Falsification detection through GPS
‐ Manchin & de Keulenaer 2013
5

6. Sampling
 Reverse geocoding
‐ Select points, addresses close to these points
 Random Geographic Cluster Sampling
‐ Survey of Pastoralists in Ethiopia (Himelein et al 2013)
‐ Loaded selected circles onto GPS devices
‐ All persons within circles interviewed
6

7. What Can Go Wrong?
 Privacy concerns
‐ Release of geocoded data
‐ NLSY-G use restricted
‐ Privacy preserving geocodes (Schnell, Christen)
‐ Typing addresses into Google maps, navis prohibited by Census
Bureau (?)
 Errors in GIS
‐ User error
‐ Variability in GPS coordinates
‐ Geocoding Error
7

8. What is Geocoding?
 Software assigns addresses to points
 Assigns points to Census geographies
8
Block 2001 Block 2002
Block 2003 Block 2004
Block 2005
Block 2006

9. Geocoding Quality
 Hit Rate – how many addresses geocode?
 Distance from true location
 Assigned to correct block
 Precision of geocode
‐ Based on code returned by software
‐ Correlated with correct block / distance
Eckman & English 2012 9

10. Different Methods of Making Frames
 Listing Method
‐ Use DSF addresses to make frame inside blocks
 Westat Method
‐ Use DSF addresses where they geocode
 Postal Method
‐ Use ZIPs, ZIP+5s as selection geographies
10

11. “Listing” Method
 Select segments PPS
‐ MOS is Census HU count
 All addresses that geocode inside the selected areas are HU
frame
‐ Any found to be outside the segment are OOS
‐ Any inside segment which were missed, added via missed unit
procedure
 Goal: frame of HUs truly inside segments
11

12. Westat Method
 Goal: every HU associated with 1 & only 1 block, but not
necessarily inside selected segments
 Missed HU procedure adds HUs inside blue area, not on frame
Dohrmann et al 2012 12

13. Postal Method
 PSUs: 2-digit ZIP codes
 Segments: 5-digit ZIP codes
 No geocoding needed, not susceptible to geocoding error
 What stratification variables are available at the ZIP code level?
‐ CB publishes data for ZCTAs
‐ Can GIS help solve this problem?
13

14. Coverage Rates for 3 Methods
14
 According to our calculations, coverage rates for these methods:
‐ If not supplemented with listing
Net coverage
rate
Missed unit
procedure?
Segment
size
Design
Effect
Listing 86.7% Yes, HOI Small Large
Westat 92.3% Yes, CEP Small Large
Postal 92.3% Not possible Big Small
Evidence suggests missed unit procedures do not capture many missed units
(Kish & Hess 1958; McMichael et al 2008; Eckman & O’Muircheartaigh 2012)
Eckman & English 2012

15. Next Steps
 New uses of GIS tools to reduce errors and costs
‐ But maintain skepticism
 Research needed on effects of inaccuracy in GIS on:
‐ Measurement
‐ Nonresponse
‐ Sampling
‐ Editing & processing
15

16. References
16
Christen, P. (2006). “Privacy-Preserving Data
Linkage and Geocoding: Current Approaches and
Research Directions” Proceedings of the Sixth
IEEE International Conference on Data Mining-
Workshops, IEEE Computer Society, 497-501.
Chung, E.-H. & Shalaby, A. (2005). “A Trip
Reconstruction Tool for GPS-based Personal Travel
Surveys” Transportation Planning and Technology,
28(5), 381-401.
Czaplicki, A. (2011). “GPS in der Markt- und
Sozialforschung: Herausforderungen und Chancen”
In Faulbaum, F., Stahl, M. & Wiegand, E. (eds.).
Qualitätssicherung in der Umfrageforschung,
Wiesbaden, 147-159.
Dohrmann, S., Kalton, G., Montaquila,J., Good, C. &
Berlin, M. (2012). “Using address-based sampling
frames in lieu of traditional listing: A new approach”
Proceedings of the American Statistical
Association, Section on Survey Research Methods,
3729-3741.
Eckman, S., & English, N. (2012). “Creating Housing Unit
Frames from Address Databases Geocoding Precision
and Net Coverage Rates” Field Methods, 24(4), 399-
408.
Eckman, S., & O’Muircheartaigh, C. (2011). “Performance
of the Half-Open Interval Missed Housing Unit
Procedure ” Survey Research Methods, 5(3), 125-131.
Kish, L. & Hess, I. (1958). “On Noncoverage of Sample
Dwellings” Journal of the American Statistical
Association, 282(53), 509-524.
McMichael, J. P., Ridenhour, J. L., Mitchell, S., Fahrney,
K., & Stephenson, W. (2008). “Evaluating the use and
effectiveness of the half-open interval procedure for
sampling frames based on mailing address lists in urban
areas” Proceedings of the American Statistical
Association, Section on Survey Research Methods,
4251-4257.
Schnell, R., Bachteler, T., & Reiher, J. (2009). “Privacy-preserving
record linkage using Bloom filters” BMC
Medical Informatics and Decision Making, 9(1), 41.

17. www.iab.de
Thanks – comments & ideas welcome
stephanie.eckman@iab.de
Website: stepheckman.com