MEASURE EvaluationMEASURE Evaluation works to improve collection, analysis and presentation of data to promote better use of data in planning, policymaking, managing, monitoring and evaluating population, health and nutrition programs.
Presentation from the May 2015 webinar on GIS and sampling.
MEASURE EvaluationMEASURE Evaluation works to improve collection, analysis and presentation of data to promote better use of data in planning, policymaking, managing, monitoring and evaluating population, health and nutrition programs.
1. John Spencer
Peter Lance
Aiko Hattori
MEASURE Evaluation
University of North Carolina at Chapel Hill
May 2015
Webinar
Apowerful partnership:
GIS and Sampling
6. Geographic Information Systems (GIS) is a system designed to
capture, store, manipulate, analyze, manage, and present data
using a geographic context.
Adapted from wikipedia
7. GIS makes it possible to link data using a common geography.
This leads to better understanding of the context of data.
44. Probability Sampling
1. Each member of the population is in
the “list” from which we will select
2. We know each member of the
population’s probability of selection
from the list
45. So how in practice do we select in
accordance with probability sampling????
We usually do so from a list of the
population called a sampling frame.
That list must contain all members of the
population (sampling units) to satisfy
probability sampling
46. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian
2 Andre Jones 302 Lyon Lane African American
3 Michelle M. Lee 101 Jay Street Asian American
4 Dan T. Waters 402 Smith Street Caucasian
5 Molly McGuire 111 Main Street Caucasian
6 Ron S. Swanson 0 Opsec Ave Caucasian
7 Lisa Levine 3 Phillips Ave Caucasian
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
47. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American
3 Michelle M. Lee 101 Jay Street Asian American
4 Dan T. Waters 402 Smith Street Caucasian
5 Molly McGuire 111 Main Street Caucasian
6 Ron S. Swanson 0 Opsec Ave Caucasian
7 Lisa Levine 3 Phillips Ave Caucasian
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
48. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American
4 Dan T. Waters 402 Smith Street Caucasian
5 Molly McGuire 111 Main Street Caucasian
6 Ron S. Swanson 0 Opsec Ave Caucasian
7 Lisa Levine 3 Phillips Ave Caucasian
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
49. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian
5 Molly McGuire 111 Main Street Caucasian
6 Ron S. Swanson 0 Opsec Ave Caucasian
7 Lisa Levine 3 Phillips Ave Caucasian
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
50. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian
6 Ron S. Swanson 0 Opsec Ave Caucasian
7 Lisa Levine 3 Phillips Ave Caucasian
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
51. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian
7 Lisa Levine 3 Phillips Ave Caucasian
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
52. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
53. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
54. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
55. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian No
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
56. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian No
10 Jennifer Lownes 14c Avenue A African American Yes
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
57. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian No
10 Jennifer Lownes 14c Avenue A African American Yes
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
58. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian No
10 Jennifer Lownes 14c Avenue A African American Yes
11 Anna A. Anderson 148d Avenue A African American No
12 Gary Park 55 Park Avenue Asian American
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
59. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian No
10 Jennifer Lownes 14c Avenue A African American Yes
11 Anna A. Anderson 148d Avenue A African American No
12 Gary Park 55 Park Avenue Asian American No
… … … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
60. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian No
10 Jennifer Lownes 14c Avenue A African American Yes
11 Anna A. Anderson 148d Avenue A African American No
12 Gary Park 55 Park Avenue Asian American No
… … … … …
10548 Peter R. Martinez 404 Main Street Latino No
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
61. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian No
10 Jennifer Lownes 14c Avenue A African American Yes
11 Anna A. Anderson 148d Avenue A African American No
12 Gary Park 55 Park Avenue Asian American No
… … … … …
10548 Peter R. Martinez 404 Main Street Latino No
10549 Faith B. Rogers 201 Cedar Street African American No
10550 Will Billings 402 Buford Place Caucasian
62. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian No
2 Andre Jones 302 Lyon Lane African American No
3 Michelle M. Lee 101 Jay Street Asian American No
4 Dan T. Waters 402 Smith Street Caucasian No
5 Molly McGuire 111 Main Street Caucasian Yes
6 Ron S. Swanson 0 Opsec Ave Caucasian No
7 Lisa Levine 3 Phillips Ave Caucasian No
8 Andrea Rice 14 Main Street African American No
9 Hunter Lewis 29 9th Avenue Caucasian No
10 Jennifer Lownes 14c Avenue A African American Yes
11 Anna A. Anderson 148d Avenue A African American No
12 Gary Park 55 Park Avenue Asian American No
… … … … …
10548 Peter R. Martinez 404 Main Street Latino No
10549 Faith B. Rogers 201 Cedar Street African American No
10550 Will Billings 402 Buford Place Caucasian Yes
63. No Name Address Race Selected?
1 Burt T. Macklin 100 April Street Caucasian
2 Andre Jones 302 Lyon Lane African American
3 Michelle M. Lee 101 Jay Street Asian American
4 Dan T. Waters 402 Smith Street Caucasian
5 Molly McGuire 111 Main Street Caucasian
6 Ron S. Swanson 0 Opsec Ave Caucasian
7 Lisa Levine 3 Phillips Ave Caucasian
8 Andrea Rice 14 Main Street African American
9 Hunter Lewis 29 9th Avenue Caucasian
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
12 Gary Park 55 Park Avenue Asian American
… … … …
10548 Peter R. Martinez 404 Main Street Latino
10549 Faith B. Rogers 201 Cedar Street African American
10550 Will Billings 402 Buford Place Caucasian
64. No Name Address Race Selected?
2 Andre Jones 302 Lyon Lane African American
8 Andrea Rice 14 Main Street African American
10 Jennifer Lownes 14c Avenue A African American
11 Anna A. Anderson 148d Avenue A African American
… … … …
10549 Faith B. Rogers 201 Cedar Street African American
65. No Name Address Race Selected?
2 Andre Jones 302 Lyon Lane African American No
8 Andrea Rice 14 Main Street African American No
10 Jennifer Lownes 14c Avenue A African American Yes
11 Anna A. Anderson 148d Avenue A African American No
… … … …
10549 Faith B. Rogers 201 Cedar Street African American Yes
85. What We Have Learned
• Representative estimates typically require probability sampling
• Probability sampling is in practice most commonly performed
with lists from the entire population called a sampling frame
• Sampling can be stratified by dividing the frame into mutually
exclusive and exhaustive separate frames for different
subpopulations
• You can still form a representative estimate for a subpopulation
even if the selected units from it are scattered across frames
for different strata
• In practice sampling is usually multistage
91. Now, we know an important survey prior:
History suggests that we will find dengue
cases in only around 4.9 percent of
clusters.
92. Let’s Talk Costs
$200: The cost to monitor each selected
study cluster
$2000: The fieldwork cost per cluster
with a dengue outbreak
This means an expected cost per cluster
of
$200+.049 x $2000 ≈$298
93. Uh Oh
At that price, we can afford to select
only
$300,000/$298≈1006 clusters
From these we would expect only
1006 X .049≈49 clusters
to have a dengue outbreak.
174. Two Strata of Clusters
• Low Risk: 95% of clusters have a Low Risk
of Dengue Outbreak (2% risk of outbreak
during study)
• High Risk: 5% of clusters have a High Risk
of Dengue Outbreak (60% risk of outbreak
during study)
Note that this is consistent with 4.9%
overall risk across clusters:
.95 X .02 + .05 X .6 ≈ .049
179. The Power of Sampling and GIS
One Possible $300,000 Scheme:
Sample 180 High Risk clusters with expected
yield of 108 with dengue outbreaks
AND
Sample 200 Low Risk clusters with a expected
yield of 4 dengue outbreaks
180. The Payoff
• Without Stratification: We expect to
observe 49 clusters with dengue outbreaks
• With Stratification: We expect to observe
112 clusters with dengue outbreaks
181. Let’s consider a counterfactual: how much
survey budget we would need to obtain
112 dengue clusters without stratification
into high and low risk clusters?
112
.049
𝑋 $298 = $681,142.86
182. Let’s consider a counterfactual: how much
survey budget we would need to obtain
112 dengue clusters without stratification
into high and low risk clusters?
112
.049
𝑋 $298 = $681,142.86
183. Let’s consider a counterfactual: how much
survey budget we would need to obtain
112 dengue clusters without stratification
into high and low risk clusters?
112
.049
𝑋 $298 = $681,142.86
184. Let’s consider a counterfactual: how much
survey budget we would need to obtain
112 dengue clusters without stratification
into high and low risk clusters?
112
.049
𝑋 $298 = $681,142.86
185. Let’s consider a counterfactual: how much
survey budget we would need to obtain
112 dengue clusters without stratification
into high and low risk clusters?
112
.049
𝑋 $298 = $681,142.86
186. Fieldwork Costs to Get 112 Clusters with
Outbreak
With GIS Without GIS
$ 300,000.00 $681,142.86
188. A GIS Isn’t Free…
It Costs Something to Build One
189. Difference in survey fieldwork
costs
Without GIS: $681,142.86
With GIS: $300,000.00
$381,142.86
222. MEASURE Evaluation is funded by the U.S. Agency for
International Development (USAID) under terms of
Cooperative Agreement AID-OAA-L-14-00004 and
implemented by the Carolina Population Center,
University of North Carolina at Chapel Hill in partnership
with Futures Group, ICF International, John Snow, Inc.,
Management Sciences for Health, and Tulane University.
The views expressed in this presentation do not
necessarily reflect the views of USAID or the United
States government.
www.measureevaluation.org