1. “Can Verbal Instructions Counteract Visual Context Effects in Web Surveys?” A Critique of Toepoel and Couper’s Research For EDET 780 Dawn Sudduth | Maymester 2011
2. Can Verbal Instructions Counteract Visual Context Effects in Web Surveys?” A 2009 study by Vera Toepoel and Mick Couper Published in Public Opinion Quarterly Research conducted in the Netherlands
3. The Research Question “Does effective question writing reduce the the visual context effects produced by pictures, as tested within a Web survey?”
4. The Literature Review An established body of research exists that explores how various aspects of the visual layout of survey questions affects respondent answers. Collectively, that research has established that answers can be influenced by question length, orientation, order, shape, and format.
5. The Hypothesis By adding verbal instructions, the effect of pictures should be diminished if verbal language is more powerful than visual language.
6. The Experiment A two-wave, Web-based survey was provided to just over 7,900 panel members who were already established participants in an existing monthly opinion polling structure in the Netherlands. The overall pool of panel members had been randomly selected and were representative of the Dutch speaking population age 16 and over. Of the 7,900+ solicited, just over 5,100 participated.
9. How many times in the past year did you take an overnight trip?
10. How many times in the past year did you attend a sporting event?
11.
12. The Results For all topics, the high-frequency picture prompted higher reporting on average than the low-frequency one with the no-picture condition in the middle. The scores for the low-frequency instructions were lower than questions with no verbal instructions and the ones with the most inclusive instructions --- meaning the verbal instructions served to limit the number of events recalled. While both main and interaction effects were significant, verbal instructions explained more of the variation in rates than did the pictures.
13. What Do the Findings Mean? Pictures used to supplement survey questions can systematically influence answers to those questions. Showing a high-frequency event had a similar effect as a verbal instruction to count both high and low-frequency events. Likewise, showing low-frequency pictures had an effect similar to a low-frequency verbal instruction.
14. What Do the Findings Mean? Pictures and verbal instructions affected respondentsin similar ways, but differed in magnitude as verbal instructions had stronger effects than those produced by the pictures. Restrictive instructions (“Do not count…”) appeared to be more powerful than the more inclusive instructions (“Count all…”) Verbal instructions took more time to process than visual ones. Participant ratings of how much they liked the various question formats didn’t support the idea that questions with pictures would be more enjoyable to answer.
15. Was their hypothesis supported? Yes – Effective question writing, with verbal instructions that make the question clear to respondent, reduces the effects of the pictures.
16. What were the study’s shortcomings and what further research is needed? The survey consisted of only behavior related questions. Further research is needed to determine if the same results would be obtained with attitudinal or knowledge questions. The respondent pool represented a single language and culture. Further research is needed to determine if the same results would be obtained within different cultures. Survey participants were experienced survey takers. Further research is needed to determine if the same would have been obtained with less experienced participants.
17. Is this research useful for me? Its direct applicability is limited because of the single question type studied and the unlikely prospect that I would perceive a need for images within the survey itself. This article did reinforce my personal view that clearly written instructions are critical to respondent understanding. The literature review within this article points to areas of further reading that I would like to explore prior to creating online surveys in the upcoming months.
18. Toepoel, V., & Couper, M. P. (2011). Can Verbal Instructions Counteract Visual Context Effects in Web Surveys?. Public Opinion Quarterly, 75(1), 1-18.
