Recommended
More Related Content
Similar to ESRA 2017 How to encourage people to go online when using a different contact mode
Similar to ESRA 2017 How to encourage people to go online when using a different contact mode (20)
Recently uploaded
Recently uploaded (20)
ESRA 2017 How to encourage people to go online when using a different contact mode
- 1. © 2016 Ipsos. All rights reserved. Contains Ipsos' Confidential and Proprietary information and may not be disclosed or reproduced without the prior written consent of Ipsos. 1Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Gerry Nicolaas & Patten Smith ESRA 2017 Conference, Lisbon, Portugal A summary of 2 sessions at #ESRA17 How to encourage people to go online when using a different contact mode?
- 2. 2Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Presentations Session 1 1. Dr Don Dillman - The worldwide increase in use of web-push methods that start with a mail contact; what have we learned and where might we be going? 2. Dr Robert Neumann - Push2web or less is more? Experimental evidence from a mixed-mode population survey at the community level in Germany. 3. Sophie Nickson - Transformed respondent engagement strategy; an overview of the research at ONS to develop respondent materials. 4. Andrew Phelps - Quantitative testing of the most effective advance communication strategies for a mixed mode (including online) UK Labour Force Survey. 5. Patrice Mathieu - The 2016 Canadian Census: An Innovative Wave Collection Methodology to Maximize Self-Response and Internet Response. Session 2 1. Dr Gudbjorg Andrea Jonsdottir - Pushing to web in the ISSP 2. Dr Mónica Méndez - Notes from a Push2web(+mail) survey: the ISSP 2014 Citizenship survey in Spain 3. Alice Fitzpatrick - Address-based Online Surveying (ABOS): The impact of design features on response 4. Evangelia Kartsounidou - Pushing from telephone to web: a low-cost and effective way to conduct national election studies
- 3. 3Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Some emerging themes Increasing use of push-to-web survey designs No single magic bulletHighly variable response rates Mainly postal contact but also telephone
- 4. 4Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) A framework for understanding response to a push-to-web survey Different stages of gaining cooperation Participant perspective Survey design features that can be manipulated
- 5. 5Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Focusing on postal contact
- 6. 6Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Maximising push-to-web response rates Motivated to read the mailing Motivated to go online Motivated to take part in survey Motivated to open the mailing Motivated to complete questionnaire Personalisation Type of mailing; e.g. envelope with letter, postcard Appearance of mailing before opening; e.g. logo Personalisation Easy to read; e.g. length, font, vocabulary Appearance; e.g. important, professional Clarity about the purpose of the mailing Clarity about the survey request Use persuasive reasons for taking part Clear instructions for logging into the questionnaire Minimal effort needed to enter login details Multiple access methods; e.g. any internet-enabled device Use multiple mailings: • An optimum number of mailings • An optimum length of time between mailings • A diverse and yet coherent package of multiple mailings Landing page looks authentic with clear instructions Design for mobile; e.g. short, reduce clutter and text Avoid question types that are prone to break- offs Reduce cognitive burden
- 7. 7Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) What about telephone contact?
- 8. 8Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Maximising push-to-web response rates Motivated to engage with tel interviewer Motivated to read email & go online Motivated to take part & give email Motivated to pick up phone Motivated to complete questionnaire Landing page looks authentic with clear instructions Design for mobile; e.g. short, reduce clutter and text Avoid question types that are prone to break- offs Reduce cognitive burden
- 9. 9Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Maximising push-to-web response rates Motivated to engage with tel interviewer Motivated to read email & go online Motivated to take part & give email Motivated to pick up phone Time of calling Personalisation Maintaining contact Tailoring Clarity about the purpose of the contact Clarity about the survey request Use persuasive reasons for taking part Email is recognised as genuine Clear instructions & easy to start questionnaire Multiple access methods; e.g. any internet-enabled device Multiple contact attempts: • An optimum number & mix of phone calls & emails • An optimum timing of phone calls & emails • Refusal conversion
- 10. 10Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Fairly confident that the following has a positive impact on P2W response Personalisation Engagement – Tailoring & maintaining interaction with phone contact – Multiple and varied contact attempts with postal contact Communication style, complexity, length Use of incentives Reduce break-offs by avoiding the use of complex question formats & cognitively difficult questions Device-agnostic Offer alternative mode to web non-respondents
- 11. 11Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Some doubt/uncertainty about The type of incentives The type and look of the mailings Visual design of mailing content The optimal package of multiple contact attempts – Number of contact attempts, interval between contact attempts, strategic approach to persuasive messaging, mix of contact modes How to reduce the effort of moving from offline mode to online mode, particularly when relying on postal contact
- 12. 12Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Suggested priorities for further research Address-based samples – How to encourage people to open the envelope when we don’t have a name? – How to select respondent(s)? Postal contact: – What is the optimal strategy of multiple contacts? – How to reduce the burden of moving from postal contact to online questionnaire? Sequential phone and email contact: – How to encourage people by phone to provide email address? – How to encourage people to open and read the email?
- 13. 13Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Questions for discussion Pre-paid versus conditional incentives? Alternative mode for web non-respondents – Is it necessary for reducing non-response bias? – For all or some web non-respondents? Is mobile RDD+SMS an option in the near future? Are we converging to a single optimal push-to-web design? – Internet access will become (almost) universal – Digital communication is becoming the norm for public & private services – BUT the availability of sampling frames & contact information continues to vary by country
- 14. 14Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) Thank you.
Editor's Notes
- All postal except session 2 presentation 1 (various tests with p2w including some use of tel), and session 2 presentation 4 (RDD).
- Strongly influenced by TDM approach; overlap with postal methods but the additional step of moving from letter to online. The framework describes the response process temporally from the participant’s perspective, moving from opening and reading the postal contact, to understanding what is being requested and being persuaded to take part in the survey, to making the extra step from being persuaded by the postal contact to actually go online, and finally to making the effort to complete the web questionnaire. It presents an overview of survey design features at each of these stages which can be manipulated to improve response.
- Adapted the framework for telephone – Once contact is made, the tel interviewer has to quickly convince the listener that this is not a sales call but a genuine request to take part in something that is worth their time & effort – as for f2f, it is important to maintain contact and tailor the communication Need to persuade them to provide an email address or provide login details over the phone which is prone to recording error Although collecting email addresses over the phone can be problematic, once you have the email address you can send email invites with a direct link to the questionnaire which overcomes the barrier of moving from offline mode to online mode Emails are often not opened and read – special attention for the subject heading and the sender’s address Making telephone contact – the need to make multiple calls on different days, at different times But if you have collected email addresses, then we are moving from a single offline contact mode to a sequential mixed contact modes and as demonstrated by Don Dillman using more than one mode of contact is better – need to use a mix of multiple telephone and email contacts, possibly going back to telephone contact to convert non-respondents This framework for telephone & the previous one for postal can be used to identify what we already know about maximising online response rates
- Adapted the framework for telephone – Once contact is made, the tel interviewer has to quickly convince the listener that this is not a sales call but a genuine request to take part in something that is worth their time & effort – as for f2f, it is important to maintain contact and tailor the communication Need to persuade them to provide an email address or provide login details over the phone which is prone to recording error Although collecting email addresses over the phone can be problematic, once you have the email address you can send email invites with a direct link to the questionnaire which overcomes the barrier of moving from offline mode to online mode Emails are often not opened and read – special attention for the subject heading and the sender’s address Making telephone contact – the need to make multiple calls on different days, at different times But if you have collected email addresses, then we are moving from a single offline contact mode to a sequential mixed contact modes and as demonstrated by Don Dillman using more than one mode of contact is better – need to use a mix of multiple telephone and email contacts, possibly going back to telephone contact to convert non-respondents This framework for telephone & the previous one for postal can be used to identify what we already know about maximising online response rates
- Ample evidence on positive impact of personalisation Multiple & varied contact attempts: Telephone contact – multiple calls at different times of the day and on different days of the week will increase the likelihood of making contact, additional calls for refusal conversion If using phone contact, then tailoring and maintaining interaction are important Postal contact – cooperation increases with the number of contacts, this is more effective when subsequent contact attempts look and feel different Communication style, complexity & length: e.g. old Peter Lynn study showed that short informal letter was better than long formal letter; but Dillman would argue that it should still look professional (evidence?). Perhaps more evidence needed on visual design/look (see next slide). Incentives – ample evidence; unconditional is better than conditional, monetary is better than non-monetary, response increases with the value but with diminishing returns; BUT unconditional incentives can be prohibitively expensive if the online response rate is relatively low; AND delivery of conditional incentives online can be cumbersome (in some countries when offering a choice of retailers) Complex question formats will result in higher levels of break-off- e.g. traditional grids, ranking Cognitively difficult questions will result in higher levels of break-off- e.g. recall of non-salient events over a long period, computations, reliance on looking up information from external sources (e.g. pay slips, bank account) Smartphones are becoming the most important device for accessing the internet; for some groups the smartphone could be their only means of personal internet access – and this varies by country (e.g. smartphone access to the internet is particularly prevalent in developing countries) Alternative mode for offliners and those who are online but haven’t made the effort to take the letter and go online
- Across all traditional modes there is ample evidence that upfront incentives to all are better than incentives that are only given to those who take part – however, we are seeing mixed results for push-to-web surveys – Dillman, Spanish study, UK experience Type & look of the mailings – evidence is mixed. For example, sometimes logos work and sometimes they don’t (possibly related to named samples versus address samples. Colour and size of envelopes may vary by country depending on country-specific conventions with regard to official mail and spam. Advance letters work for postal surveys but cost-effectiveness is less obvious for push-to-web surveys. A postcard thankyou/reminder works in postal surveys but generally not recommended to print usernames and passwords on postcards unless these can be folded. Our own experimental work using alternative mailing types such as pressure-sealed letters is not very promising. Visual design of the content – how can we use visual design to encourage the recipient to read the letter and how can we ensure that the recipient reads the key messages even when they are only skim reading? At the same time, the visual design should not give the impression that the letter is marketing, sales, etc. Although we know that multiple and varied contacts have a positive impact on response, we know less about how to design a strategy of using multiple carefully designed and timed contacts that reflect the characteristics of the survey and population, as well as tapping into different motivations for taking part in the survey. Respondent selection instructions can be complicated and off-putting. It is easier to instruct any adult to go online but this will introduce selection bias. Evidence from the ESS and the UK community life survey show that (quasi) random selection such as last/next birthday and kish methods are ignored with about 25% of respondents not being the intended respondent. Asking all household members to complete the questionnaire when combined with incentives can result in respondent falsification. Asking “up to two adults” to complete the questionnaire exploits the fact that only a small proportion of households have more than two adults (at least in the UK) and reduces the risk of respondent falsification, but the instructions are non-intuitive and complicated which may put people off. We know of two studies (UK LFS & EU FRA) that are experimenting with asking any adult to go online and then asking this adult to provide information about household members so that a random selection can take place. The effectiveness of this approach needs to be demonstrated. For both postal and telephone contact methods, no/limited experimental evidence on the best way of providing login instructions that are secure; e.g. username & password, or just a single password, length of username/password, the use of QR codes and SMS short codes, the number of steps needed to get to the first question. Not sure about the last bullet point – probably mixed??? We know that short and simple questionnaires that use limited graphics etc are best. But there is ongoing research into the conversion of offline question formats into device-agnostic formats; e.g. traditional grids being changed into either collapsible grids or carousel grids. Perhaps this is more about the conversion of offline questionnaires into device-agnostic web questionnaires while maintaining measurement equivalence over time and across surveys???