This document provides an overview of a one day symposium on causes of mode effects on survey measurement. The symposium focused on how the mode of data collection can affect survey responses. It explored this topic through several presentations and discussions on how cognitive interviewing, visual/aural stimuli, interviewers, question format optimization, and question design affect comparability of responses across modes. The goal was to provide practical advice on improving portability of survey questions across different modes of data collection like telephone, web, and paper surveys.

1. Causes of mode effects
on survey measurement
One day symposium at the Royal Statistical Society, London
11 October 2011

2. Background
Background
• Increasing use of mixed modes
• Falling response rates
• Rising costs of data collection
• Risk of reduced data comparability
• Coverage error
• Non-response error
• Measurement error
• Need for practical advice to inform decisions
about when to mix modes and how

3. Mixing Modes and Measurement Error
• Funded under the ESRC Survey Design and
Measurement Initiative
• 3-year contract starting 1 Oct 2007
• Collaboration between NatCen, ISER and
independent survey methods consultant

4. Core research team
• National Centre for Social Research (NatCen)
• Gerry Nicolaas
• Steven Hope (now UCL)
• Margaret Blake
• Michelle Gray
• Institute for Social & Economic Research (ISER)
• Peter Lynn
• Annette Jäckle
• The Survey Coach (Independent Survey Methods Consultant)
• Pamela Campanelli

5. Main objective
• Practical advice on how to improve portability of
questions across modes
• Which mode combinations are likely to produce
comparable responses?
• Which types of questions are more susceptible to
mode effects?

6. Research design
• A literature review & framework of mixed modes
• develop a conceptual framework
• identify gaps in evidence base and formulate
hypotheses to address gaps
• Quantitative data analysis
• test hypotheses using existing datasets and new
experimental data
• Cognitive interviewing
• explore how respondents process questions in different
modes

7. Presentations & Discussion
1. How and when does the mode of data collection
affect survey measurement?
2. The use of cognitive interviewing methods to evaluate
mode effects
3. The role of visual and aural stimuli in producing mode
effects
4. The role of the interviewer in producing mode effects
5. Is it a good idea to optimise question format for mode
of data collection?
6. Designing questions for mixed mode surveys
7. Discussant (Patten Smith)
8. Open discussion