AAPOR 2013 Langer Research-AbtSRBI: Hurricane Sandy

563 views

Published on

Published in: News & Politics, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
563
On SlideShare
0
From Embeds
0
Number of Embeds
334
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

AAPOR 2013 Langer Research-AbtSRBI: Hurricane Sandy

  1. 1. Gregory Holyk, Langer Research AssociatesSeth Brohinsky, Abt-SRBIDean Williams, Abt-SRBIDamla Ergun, Langer Research AssociatesGary Langer, Langer Research AssociatesJulie Phelan, Langer Research AssociatesPolling in the Midst of a Natural Disaster:The ABC News/Washington Post 2012Election Tracking Poll and Hurricane Sandy
  2. 2. What Could Possibly Go Wrong? Sandy made landfall on the NY/NJ coast the night of Oct. 29,2012 (night 12 of our tracking poll in the presidential election) Presented an unprecedented situation in our experience Intense internal discussion: Empirical question: Can we continue to obtain a sufficiently reliablenational sample? Ethical question: Should we continue to call affected areas in theaftermath of a natural disaster? If we proceed, how will we diagnose our sample is acceptable anddetermine whether or not to proceed?2
  3. 3. How We Proceeded Some said the storm made it impossible to continue samplingand obtain a representative national estimate. Our approach: Test it and see. Steps we implemented: Changed our intro language in the affected areas to be sensitiveto the situation:“…. We know it’s a difficult time in much of the Northeast becauseof the storm damage. But we’re doing an opinion poll on interestingsubjects in the news and would appreciate your participating. Wouldthat be OK?” Evaluate sample dispositions. Compare each night’s results (demographics, attitudes) vs. theirpre-Sandy averages and ranges.3
  4. 4. Field Procedures and Affected Areas On the night the storm hit the final NE interview was completedat 7:56 p.m. – before landfall. Call centers (Ft. Myers, FL and Huntington, WV) not affected. Received hourly status updates from field site staff to projectmanagement team during every interviewing shift. Project managers debriefed nightly by field staff. Northeast region = 18% of the country’s LVs. Census divisions: Middle Atlantic (NJ, NY, PA) = 13% New England (CT, ME, MA, NH, RI, VT) = 5% South Atlantic (DE, DC, FL, GA, MD, NC, SC, VA, WV) = 20% Hardest hit states (NY and NJ) = 8% But impact was coastal, not statewide4
  5. 5. Night-by-night Diagnosis
  6. 6. NE Data; Post-Sandy W12 n=57Pre-Sandy Post-SandyMin. Max. Avg. Wave 12Obama 50% 65% 57% 69%Romney 33% 46% 41% 31%Democrat 31% 49% 40% 43%Republican 16% 30% 21% 20%Independent 21% 43% 35% 30%Leaned Dem. 48% 64% 56% 63%Leaned Rep. 26% 45% 36% 31%Liberal 18% 34% 26% 46%Moderate 25% 61% 42% 33%Conservative 19% 40% 30% 19%Men 37% 57% 47% 41%Women 43% 63% 53% 59%6
  7. 7. NE Data; Post-Sandy W13 n=55Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13Obama 50% 65% 57% 69% 60%Romney 33% 46% 41% 31% 35%Democrat 31% 49% 40% 43% 34%Republican 16% 30% 21% 20% 16%Independent 21% 43% 35% 30% 39%Leaned Dem. 48% 64% 56% 63% 60%Leaned Rep. 26% 45% 36% 31% 29%Liberal 18% 34% 26% 46% 24%Moderate 25% 61% 42% 33% 46%Conservative 19% 40% 30% 19% 27%Men 37% 57% 47% 41% 41%Women 43% 63% 53% 59% 59%7
  8. 8. NE Data; Post-Sandy W14 n=52Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14Obama 50% 65% 57% 69% 60% 66%Romney 33% 46% 41% 31% 35% 28%Democrat 31% 49% 40% 43% 34% 41%Republican 16% 30% 21% 20% 16% 25%Independent 21% 43% 35% 30% 39% 23%Leaned Dem. 48% 64% 56% 63% 60% 59%Leaned Rep. 26% 45% 36% 31% 29% 37%Liberal 18% 34% 26% 46% 24% 38%Moderate 25% 61% 42% 33% 46% 35%Conservative 19% 40% 30% 19% 27% 27%Men 37% 57% 47% 41% 41% 32%Women 43% 63% 53% 59% 59% 68%8
  9. 9. NE Data; Post-Sandy W15 n=94Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14 Wave 15Obama 50% 65% 57% 69% 60% 66% 59%Romney 33% 46% 41% 31% 35% 28% 35%Democrat 31% 49% 40% 43% 34% 41% 38%Republican 16% 30% 21% 20% 16% 25% 23%Independent 21% 43% 35% 30% 39% 23% 34%Leaned Dem. 48% 64% 56% 63% 60% 59% 52%Leaned Rep. 26% 45% 36% 31% 29% 37% 35%Liberal 18% 34% 26% 46% 24% 38% 27%Moderate 25% 61% 42% 33% 46% 35% 38%Conservative 19% 40% 30% 19% 27% 27% 24%Men 37% 57% 47% 41% 41% 32% 35%Women 43% 63% 53% 59% 59% 68% 65%9
  10. 10. NE Data, Overall Pre- vs. Post-Sandy ComparisonPre-Sandy (n=630) Post-Sandy (n=580)Min. Max. Avg. Avg. Diff. Nights out of rangeObama 50% 65% 57% 59% +2 pts 3Romney 33% 46% 41% 37% -4 3Democrat 31% 49% 40% 41% +1 1Republican 16% 30% 21% 20% -1 0Independent 21% 43% 35% 32% -3 0Leaned Dem. 48% 64% 56% 58% +2 0Leaned Rep. 26% 45% 36% 33% -3 0Liberal 18% 34% 26% 29% +3 2Moderate 25% 61% 42% 40% -2 0Conservative 19% 40% 30% 27% -3 0Men 37% 57% 47% 41% -6 2Women 43% 63% 53% 59% +6 210
  11. 11. NE Data; Post-Sandy W12 n=57Pre-Sandy Post-SandyMin. Max. Avg. Wave 12White 66% 89% 80% 73%Black 0% 14% 7% 6%Hispanic 0% 17% 8% 12%18-29 0% 22% 10% 15%30-39 0% 25% 16% 5%40-49 9% 27% 15% 20%50-64 26% 43% 36% 39%65+ 13% 30% 23% 21%HS or less 26% 43% 34% 34%Somecollege10% 24% 19% 20%College deg. 13% 37% 26% 26%Post-grad 14% 29% 22% 20%11
  12. 12. NE Data; Post-Sandy W13 n=55Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13White 66% 89% 80% 73% 69%Black 0% 14% 7% 6% 20%Hispanic 0% 17% 8% 12% 4%18-29 0% 22% 10% 15% 19%30-39 0% 25% 16% 5% 7%40-49 9% 27% 15% 20% 28%50-64 26% 43% 36% 39% 26%65+ 13% 30% 23% 21% 20%HS or less 26% 43% 34% 34% 38%Some college 10% 24% 19% 20% 20%College deg. 13% 37% 26% 26% 28%Post-grad 14% 29% 22% 20% 14%12
  13. 13. NE Data; Post-Sandy W14 n=52Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14White 66% 89% 80% 73% 69% 70%Black 0% 14% 7% 6% 20% 11%Hispanic 0% 17% 8% 12% 4% 16%18-29 0% 22% 10% 15% 19% 6%30-39 0% 25% 16% 5% 7% 28%40-49 9% 27% 15% 20% 28% 25%50-64 26% 43% 36% 39% 26% 24%65+ 13% 30% 23% 21% 20% 17%HS or less 26% 43% 34% 34% 38% 21%Some college 10% 24% 19% 20% 20% 28%College deg. 13% 37% 26% 26% 28% 33%Post-grad 14% 29% 22% 20% 14% 14%13
  14. 14. NE Data; Post-Sandy W15 n=94Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14 Wave 15White 66% 89% 80% 73% 69% 70% 80%Black 0% 14% 7% 6% 20% 11% 11%Hispanic 0% 17% 8% 12% 4% 16% 4%18-29 0% 22% 10% 15% 19% 6% 8%30-39 0% 25% 16% 5% 7% 28% 23%40-49 9% 27% 15% 20% 28% 25% 10%50-64 26% 43% 36% 39% 26% 24% 34%65+ 13% 30% 23% 21% 20% 17% 25%HS or less 26% 43% 34% 34% 38% 21% 33%Some college 10% 24% 19% 20% 20% 28% 18%College deg. 13% 37% 26% 26% 28% 33% 24%Post-grad 14% 29% 22% 20% 14% 14% 25%14
  15. 15. NE Data, Overall Pre- vs. Post-Sandy ComparisonPre-Sandy (n=630) Post-Sandy (n=580)Min. Max. Avg. Avg. Diff. Nights out of rangeWhite 66% 89% 80% 72% -8 pts 3Black 0% 14% 7% 11% +4 1Hispanic 0% 17% 8% 9% +1 118-29 0% 22% 10% 11% +1 030-39 0% 25% 16% 16% 0 140-49 9% 27% 15% 21% +6 150-64 26% 43% 36% 31% -5 065+ 13% 30% 23% 21% -2 0HS or less 26% 43% 34% 33% -1 1Some college 10% 24% 19% 24% +5 3College deg. 13% 37% 26% 23% -3 1Post-grad 14% 29% 22% 19% -3 015
  16. 16. NE Data; Post-Sandy W12 n=57Pre-Sandy Post-SandyMin. Max. Avg. Wave 12<$50K 26% 50% 39% 39%$50-100K 20% 46% 31% 39%$100K+ 23% 44% 30% 21%Protestant 19% 50% 32% 21%Catholic 26% 52% 38% 38%Other non-Christ. 3% 24% 10% 14%None 3% 25% 13% 19%Contact - Obama 16% 40% 27% 41%Contact - Romney 15% 26% 19% 27%‘08 vote - Obama 57% 83% 65% 77%‘08 vote - McCain 17% 42% 33% 23%Folo - very closely 43% 76% 62% 69%Folo - smwt closely 21% 55% 34% 21%16
  17. 17. NE Data; Post-Sandy W13 n=55Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13<$50K 26% 50% 39% 39% 42%$50-100K 20% 46% 31% 39% 38%$100K+ 23% 44% 30% 21% 20%Protestant 19% 50% 32% 21% 38%Catholic 26% 52% 38% 38% 27%Other non-Christ. 3% 24% 10% 14% 10%None 3% 25% 13% 19% 21%Contact - Obama 16% 40% 27% 41% 35%Contact - Romney 15% 26% 19% 27% 20%‘08 vote - Obama 57% 83% 65% 77% 70%‘08 vote - McCain 17% 42% 33% 23% 26%Folo - very closely 43% 76% 62% 69% 60%Folo - smwt closely 21% 55% 34% 21% 31%17
  18. 18. NE Data; Post-Sandy W14 n=52Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14<$50K 26% 50% 39% 39% 42% 33%$50-100K 20% 46% 31% 39% 38% 34%$100K+ 23% 44% 30% 21% 20% 32%Protestant 19% 50% 32% 21% 38% 29%Catholic 26% 52% 38% 38% 27% 39%Other non-Christ. 3% 24% 10% 14% 10% 11%None 3% 25% 13% 19% 21% 17%Contact - Obama 16% 40% 27% 41% 35% 21%Contact - Romney 15% 26% 19% 27% 20% 9%‘08 vote - Obama 57% 83% 65% 77% 70% 71%‘08 vote - McCain 17% 42% 33% 23% 26% 27%Folo - very closely 43% 76% 62% 69% 60% 57%Folo - smwt closely 21% 55% 34% 21% 31% 37%18
  19. 19. NE Data; Post-Sandy W15 n=94Pre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14 Wave 15<$50K 26% 50% 39% 39% 42% 33% 42%$50-100K 20% 46% 31% 39% 38% 34% 23%$100K+ 23% 44% 30% 21% 20% 32% 35%Protestant 19% 50% 32% 21% 38% 29% 27%Catholic 26% 52% 38% 38% 27% 39% 30%Other non-Christ. 3% 24% 10% 14% 10% 11% 13%None 3% 25% 13% 19% 21% 17% 20%Contact - Obama 16% 40% 27% 41% 35% 21% 31%Contact - Romney 15% 26% 19% 27% 20% 9% 20%‘08 vote - Obama 57% 83% 65% 77% 70% 71% 64%‘08 vote - McCain 17% 42% 33% 23% 26% 27% 31%Folo - very closely 43% 76% 62% 69% 60% 57% 54%Folo - smwt closely 21% 55% 34% 21% 31% 37% 36%19
  20. 20. NE Data, Overall Pre- vs. Post-Sandy ComparisonPre-Sandy (n=630) Post-Sandy (n=580)Min. Max. Avg. Avg. Diff. Nights out of range<$50K 26% 50% 39% 39% 0 pts 0$50-100K 20% 46% 31% 34% +3 0$100K+ 23% 44% 30% 27% -3 2Protestant 19% 50% 32% 29% -3 0Catholic 26% 52% 38% 36% -2 0Other non-Christ. 3% 24% 10% 10% 0 0None 3% 25% 13% 18% +5 0Contact - Obama 16% 40% 27% 31% +4 1Contact - Romney 15% 26% 19% 22% +3 3‘08 vote - Obama 57% 83% 65% 68% +3 0‘08 vote - McCain 17% 42% 33% 29% -4 0Folo - very closely 43% 76% 62% 60% -2 1Folo - smwt closely 21% 55% 34% 31% -3 320
  21. 21. A National “Sandy bump” for Obama?50%50%47%47%20%30%40%50%60%70%80%Obama RomneyObama vs. Romney, 2012Among all likely votersABC News/Washington Post pollsHurricane Sandy10/21 and later: four-day rolling average21
  22. 22. Sample Overview
  23. 23. Sample Facts Field Period: Oct. 18-Nov. 4, 2012 Non-overlapping RDD and CPO design Max of 6 call attempts per record Mix of fresh and live sample released every day Nights 1-14: 440 total x’s per night Nights 15-18: 800 total x’s per night LL sample: 66,414 unique numbers dialed 224,922 call attempts made to complete n=7,874 interviews Cell sample: 22,750 unique numbers dialed 70,665 call attempts made to complete n=1,193 interviews All respondents were cell-phone only All phone numbers were dialed manually23
  24. 24. Landline Sample
  25. 25. Landline Sample - Overview Landline Sample Controlled by Census Division Nine total census divisions nationwide.Sandy’s StormPath25
  26. 26. Landline Sample -Post-Sandy: Numbers Dialed / CompletePre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14 Wave 15Total 20.7 37.1 30.5 26.7 30.1 31.8 21.8Censdiv 1 21.8 58.9 35.4 22.1 33.2 19.4 30.8Censdiv 2 20.8 41.5 30.6 18.1 48.7 44.2 20.1Censdiv 3 18.2 39.1 29.2 30.8 28.2 29.9 19.9Censdiv 4 16.1 48.8 27.9 22.1 18.7 27.3 19.2Censdiv 5 20.0 43.6 32.1 19.7 31.7 28.9 22.6Censdiv 6 19.0 39.9 27.2 17.9 23.4 32.0 14.1Censdiv 7 26.1 49.7 34.4 24.0 33.7 38.1 21.0Censdiv 8 19.1 37.6 27.1 42.8 19.9 34.3 28.1Censdiv 9 16.3 51.1 29.4 39.5 22.9 26.5 23.126
  27. 27. 0510152025303540Night1Night2Night3Night4Night5Night6Night7Night8Night9Night10Night11Night12Night13Night14Night15Night16Night17Night18Numbers Dialed per Completed InterviewLandline SampleBurn Rate No significant increase in total #’s dialed / complete: Burn rate for the entire tracking poll: 28.6 Burn rate on night 13: 30.1Hurricane Sandy27
  28. 28. 010203040506070Night1Night2Night3Night4Night5Night6Night7Night8Night9Night10Night11Night12Night13Night14Night15Night16Night17Night18Number of Dials per Completed Interviewby Census DivisionCensdiv 1Censdiv 2Censdiv 3Censdiv 4Censdiv 5Censdiv 6Censdiv 7Censdiv 8Censdiv 9Landline Sample – By Census DivisionHurricane Sandy28
  29. 29. Landline Sample -Post-Sandy; Percent BusyPre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14 Wave 15Total 0.2% 3.8% 2.2% 4.3% 9.4% 5.3% 3.9%Censdiv 1 0% 5.0% 1.7% 5.0% 4.3% 6.1% 2.4%Censdiv 2 0% 8.0% 3.2% 6.7% 27.2% 15.7% 13.3%Censdiv 3 0% 2.2% 1.3% 4.2% 5.2% 2.9% 2.5%Censdiv 4 .2% 10.8% 3.2% 4.2% 4.2% 2.3% 2.2%Censdiv 5 .1% 4.2% 2.2% 6.8% 6.3% 4.5% 3.1%Censdiv 6 0% 6.4% 2.1% 5.6% 2.9% 2.4% 1.9%Censdiv 7 .4% 6.9% 3.0% 3.9% 2.4% 2.7% 2.0%Censdiv 8 0% 4.0% 1.6% 3.3% 5.4% 1.1% 3.1%Censdiv 9 0% 3.5% 1.6% 2.2% 4.1% 1.6% 2.6%29
  30. 30. Landline Sample Cont’d Percentage of busy numbers by night: Average percentage of busy numbers: 3.2% of total dialed Night 12: 4.3%; night 13: 9.4%; night 14: 5.3%0%5%10%15%20%25%30%35%40%Night1Night2Night3Night4Night5Night6Night7Night8Night9Night10Night11Night12Night13Night14Night15Night16Night17Night18Busy Numbers as % of Total SampleHurricane Sandy30
  31. 31. Landline Sample – By Census Division0%5%10%15%20%25%30%35%40%45%50%Night1Night2Night3Night4Night5Night6Night7Night8Night9Night10Night11Night12Night13Night14Night15Night16Night17Night18Busy as a Percentage of Total NumbersDialed by Census DivisionCensdiv 1 Censdiv 2 Censdiv 3Censdiv 4 Censdiv 5 Censdiv 6Censdiv 7 Censdiv 8 Censdiv 9Hurricane Sandy31
  32. 32. Pre-Sandy Busy DispositionsNight 11 Night 1232
  33. 33. Night 13 Busy Dispositions33
  34. 34. Post-Sandy Busy DispositionsNight 14 Night 1534
  35. 35. Landline Sample – Respondent Participation0%5%10%15%20%25%Night1Night2Night3Night4Night5Night6Night7Night8Night9Night10Night11Night12Night13Night14Night15Night16Night17Night18Respondent Participation by NightCensus Division 2 TotalHurricane Sandy35
  36. 36. Cell Phone Sample
  37. 37. Cell Phone vs. Landline Samples Main differences: LL sample uses autodialer, cell phone does not. Cell sample dispositions are manually entered by a liveinterviewer based on his/her interpretation of the outcome. The following disposition codes were used to determine“busy” equivalent in cell sample: Answering machine/voicemail – unknown eligibility Busy Incomplete call/line problems (temporary) No answer Not In service/disconnected37
  38. 38. Cell Phone Sample -Post-Sandy Burn RatePre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14 Wave 15Total 26.1 57.5 38.8 39.1 36.1 34.4 24.2Northeast 15.7 61.1 39.7 64.4 29.5 87.8 27.5Midwest 18.5 72.0 36.7 33.1 39.5 24.2 34.6South 23.5 64.6 41.6 38.8 27.0 26.9 16.4West 16.6 62.6 35.4 33.6 51.6 30.7 25.838
  39. 39. Cell Phone Sample No significant increase in total #s dialed / complete: Burn rate for the entire tracking poll: 36:1, same as night 13010203040506070Night 1Night 2Night 3Night 4Night 5Night 6Night 7Night 8Night 9Night 10Night 11Night 12Night 13Night 14Night 15Night 16Night 17Night 18Total Numbers Dialed per CompleteHurricane Sandy39
  40. 40. Cell Phone Sample – By Region Total #’s dialed / complete: cell average 36:1 Night 13: no outliers Night 14: NE jumps to highest burn rate of any region 88:10102030405060708090100Night 1Night 2Night 3Night 4Night 5Night 6Night 7Night 8Night 9Night 10Night 11Night 12Night 13Night 14Night 15Night 16Night 17Night 18Numbers Dialed per Complete by RegionNorthwestMidwestSouthWestHurricane Sandy40
  41. 41. Cell Phone Sample -Post-Sandy; Percent BusyPre-Sandy Post-SandyMin. Max. Avg. Wave 12 Wave 13 Wave 14 Wave 15Total 76% 85% 82% 81% 81% 78% 79%Northeast 69% 85% 78% 80% 85% 80% 70%Midwest 71% 84% 81% 82% 80% 77% 83%South 78% 85% 82% 84% 80% 75% 75%West 70% 87% 80% 80% 83% 74% 75%41
  42. 42. Cell Phone Sample Percentage of NIS/VM/busy numbers by night: Average percentage of NIS/VM/busy: 81% of total dialed Consistent % across all nights50%55%60%65%70%75%80%85%90%95%100%Night1Night2Night3Night4Night5Night6Night7Night8Night9Night10Night11Night12Night13Night14Night15Night16Night17Night18NIS/VM/Busy as a % of Total DialedHurricane Sandy42
  43. 43. Cell Phone – By Region Increase in NIS/VM/busy percentage on night 14 Night 14: NE hits 85% (749 NIS/VM/busy is tops for any night) Night 13: 237 and 80% of total60%65%70%75%80%85%90%95%100%Night1Night2Night3Night4Night5Night6Night7Night8Night9Night10Night11Night12Night13Night14Night15Night16Night17Night18Percent of #s NIS/unknownNorthwestMidwestSouthWestHurricane Sandy43
  44. 44. Cell Phone Sample – Participation Analysis0%5%10%15%20%Night1Night2Night3Night4Night5Night6Night7Night8Night9Night10Night11Night12Night13Night14Night15Night16Night17Night18Cell Phone Respondent ParticipationTotal NortheastHurricane Sandy44
  45. 45. Conclusions Post-Sandy dialings-to-completes in the affected areas werewithin the normal range. Demographic differences were generally within range. No evidence of a “Sandy bump” in the NE or nationally. Continuing tracking provided an important source of accurateongoing information about the campaign that otherwisewould have been absent.45

×