Your SlideShare is downloading. ×
Rossmann a new-approach_to_the_analysis_of_survey_drop-out_results_from_follow-up_surveys-183
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Rossmann a new-approach_to_the_analysis_of_survey_drop-out_results_from_follow-up_surveys-183

175

Published on

A GOR presentation

A GOR presentation

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
175
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. General Online Research ConferenceGOR 11, March 14-16, 2011, Heinrich Heine University, Düsseldorf, GermanyJoss Roßmann, GESIS - Leibniz Institute for the Social SciencesJan Eric Blumenstiel, Mannheim Centre for European Social Research (MZES)Dr. Markus Steinbrecher, University of MannheimA new approach to the analysis of survey drop-out. Results from a follow- up survey in the German Longitudinal Election Study (GLES)Contact: joss.rossmann@gesis.org
  • 2. Why study survey drop-out?• High drop-out rates in web surveys – GLES Online Tracking Surveys (wave 1 – 12): Up to 607 drop-outs (= 31.5% of initial respondents), Ø ≈ 341 drop-outs• Threat to data quality & survey inference if – drop-out is non-random – complete responders and drop-outs differ Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 2 Dr. Markus Steinbrecher (University of Mannheim)
  • 3. Results GOR10 I Life tables of tracking survey 2 to tracking survey 7 T2 T3 T4 T5 1Cumulated probability of survival T6 T7 0,9 0,8 0,7 0,6 110 115 100 105 120 125 130 135 140 145 150 155 160 165 170 175 180 185 190 195 200 205 15 55 10 20 25 30 35 40 45 50 60 65 70 75 80 85 90 95 0 5 Screen Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 3 Dr. Markus Steinbrecher (University of Mannheim)
  • 4. Results GOR10 II(1) Respondent characteristics: • Older respondents and respondents with higher interest in politics have a lower drop-out risk(2) Characteristics of the instrument: • drop-out risk decreases with progress in the survey • burdensome questions (complex matrix and open questions) increase drop-out risk(3) Characteristics of the instrument were relatively more important than respondent characteristics Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 4 Dr. Markus Steinbrecher (University of Mannheim)
  • 5. Why conducting a follow-up survey?• So far: correlations between respondent/page/questionnaire characteristics and drop-out• Follow-up survey provides substantial information about the drop-outs Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 5 Dr. Markus Steinbrecher (University of Mannheim)
  • 6. Survey & data I• Two surveys:(1) Online tracking survey (Fieldtime: 10/12/10 – 19/12/10)(2) Follow-up survey of drop-outs (Fieldtime: 20/12/10 – 30/12/10)• Source: Nonprobability online access panel (Respondi/Mingle)• (Additional follow-up surveys in March and May/June 2011) Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 6 Dr. Markus Steinbrecher (University of Mannheim)
  • 7. Survey & data II % of % of T12 % of T12 Follow-upParticipation n interview drop-outs interview starts startsT12 interview starts 1279 100T12 complete interviews 1144 89.4T12 drop-outs 135 10.6 100Follow-up interview starts 109 8.5 80.7 100Follow-up complete 94 7.3 69.6 86.2interviewsFollow-up drop-outs 15 1.2 11.1 13.8Follow-up non-participation 26 2.0 19.3 Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 7 Dr. Markus Steinbrecher (University of Mannheim)
  • 8. Interview recall Filter for 45 subsequent 40 questions on 35 participation in 30 T12 and reasons 25 for drop-outIn % 20 15 10 5 0 Very good Rather good Less good Not at all Interview Recall Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 8 Dr. Markus Steinbrecher (University of Mannheim)
  • 9. Distance between interviews 25 Correlation between 20 distance and interview recall: .15, p > .1 15in % 10 5 0 2 3 4 5 6 7 8 9 10 11 12 13 14 15 18 Days between interview and follow-up Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 9 Dr. Markus Steinbrecher (University of Mannheim)
  • 10. Interview situation Complete Drop-outs Gesamt respondersWorkplace 93.3% 6.7% 100%At home 93.4% 6.6% 100%Public place 81.8% 18.2% 100% 93.3% (1132) 6.7% (81) 100% (1213)χ² = 2.359, p > . 1 Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 10 Dr. Markus Steinbrecher (University of Mannheim)
  • 11. Subjective reasons for drop-out Closed Question Open Question 45% 40% 35% 30% 25%in % 20% 15% 10% 5% 0% Survey too Time Technical Disrupted No interest Problems Privacy Other long problems problems in politics to answer concerns reason questions Reasons for drop-out (multiple codings) Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 11 Dr. Markus Steinbrecher (University of Mannheim)
  • 12. Technical problems I Complete Drop-outs Total respondersYes 2.9% 22.2% 4.2%No 97.1% 77.8% 95.8%Total 100% (1134) 100% (81) 100% (1215)χ² = 70.115, p < .001 Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 12 Dr. Markus Steinbrecher (University of Mannheim)
  • 13. Technical problems II 6 5 4Frequency 3 2 1 0 Could not Connection to Browser did not Page(s) did not Computer crashed continue the internet failed work correctly load survey Technical problem Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 13 Dr. Markus Steinbrecher (University of Mannheim)
  • 14. Do complete responders and drop-outs differ? Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 14 Dr. Markus Steinbrecher (University of Mannheim)
  • 15. Demographics Complete Drop-outs Sig. respondersGender (% Women) 49.7 62.8 +Education 1.93 1.77 -Age 43.3 40.0 ++ Significant at 5%-level, - Insignificant Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 15 Dr. Markus Steinbrecher (University of Mannheim)
  • 16. Debriefing questions Complete Drop-outs Sig. respondersInterest in survey 4.00 2.83 +Burden (questions) 1.70 2.28 +Burden (answers) 1.91 2.67 +Accuracy 4.67 4.37 +Scientific surveys 3.96 3.50 +No. of surveys last month 5.12 5.33 -+ Significant at 5%-level, - Insignificant Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 16 Dr. Markus Steinbrecher (University of Mannheim)
  • 17. Personality traits I Complete Drop-outs Sig. respondersExtraversion 3.18 3.18 -Agreeableness 2.89 2.98 -Conscientiousness 3.70 3.78 -Neuroticism 2.65 2.67 -Openness 3.52 3.42 -+ Significant at 5%-level, - Insignificant Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 17 Dr. Markus Steinbrecher (University of Mannheim)
  • 18. Personality traits II Complete Drop-outs Sig. respondersNeed to evaluate 3.74 3.76 -Need for cognition 2.72 2.93 -Need for cognitive closure 3.49 3.29 ++ Significant at 5%-level, - Insignificant Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 18 Dr. Markus Steinbrecher (University of Mannheim)
  • 19. Political attitudes Complete Drop-outs Sig. respondersInterest in politics 3.23 2.71 +Political internet usage 2.49 2.13 -External efficacy 2.16 2.13 -Internal efficacy 2.94 2.73 +Duty to vote 3.87 3.78 -No party identification (%) 30.6 35.5 -Vote intention (%) 80.4 73.9 -Undecided (%) 12.3 31.9 ++ Significant at 5%-level, - Insignificant Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 19 Dr. Markus Steinbrecher (University of Mannheim)
  • 20. Vote choice (2nd vote) Completes Drop-outs 35% 30% 25% 20%in % 15% 10% 5% 0% CDU/CSU SPD FDP GRUENE DIE LINKE OTHER χ² = 8.503 p > .05 Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 20 Dr. Markus Steinbrecher (University of Mannheim)
  • 21. Conclusions I(1) Reasons for drop-out• Interview situation plays an important role in the subjective statements of drop-outs (Time problems, technical problems, disruptions)• Survey characteristics (duration) are also of major importance in the subjective evaluation• Characteristics of the respondents (interest in politics, problems with questions) are mentioned less frequently by the drop-outs Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 21 Dr. Markus Steinbrecher (University of Mannheim)
  • 22. Conclusions II(2) Do complete responders and drop-outs differ?• Very low N (until now), but…• Some differences in socio-demographics and political attitudes between complete responders and drop-outs• Personality less importantDrop-outs reduce the quality of the data becausethey induce additional systematic bias Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 22 Dr. Markus Steinbrecher (University of Mannheim)
  • 23. Further research• Inclusion of additional online tracking surveys• Drop-out analysis: Replication of survival analysis for additional trackings• Compute multivariate models Joss Roßmann (GESIS), Jan Eric Blumenstiel (MZES), 23 Dr. Markus Steinbrecher (University of Mannheim)

×