SlideShare a Scribd company logo
Mobile Phone Surveys from a Total Survey
          Quality Perspective

                        Peter Lynn
       Institute for Social and Economic Research
                  University of Essex, UK




                Mobile Research Conference
                       19 April 2011
                        © Peter Lynn
1.




Accuracy of Survey Estimates (Variance and Bias)
2.




       Accuracy of Survey Estimates (Sources)
                                                                                                 Mean Square Error


                                                 Variance                                                                                                     Bias


           Errors of Non-observation                                 Observational Errors
                                                                                                                        Errors of Non-observation                               Observational Errors

Coverage          Sampling        Non-response      Interviewer   Respondent      Instrument   Coder
                                                                                                             Coverage          Sampling        Non-response    Interviewer   Respondent      Instrument   Coder
3.




Total Survey Quality
Relevance
Accuracy
Timeliness
Accessibility
Interpretability
Coherence

(Statistics Canada, 2009)
4.




Role of Mobile Phone Surveys
Focus here on interviewing, not self-completion
i.e. respondent is listening and talking, not reading and keying
Stand-alone mode or within multi-mode
Focus is on accuracy of survey estimates
5.




Coverage Error
Initial interest was in mobile phone ownership as a problem:
Worsening coverage of landline frames (Finland, Slovenia etc in 1990s,
USA in 2000s; need for dual-frame approaches
Then some interest in improved coverage for phone surveys:
Sub-Saharan Africa, Eastern Europe
Now, opportunities for improving coverage in mixed-mode surveys:
Germany, UK, etc



(Blumberg and Luke 2007, Kuusela et al 2008, Tortora et al, 2008, Gallup Europe 2009,
Häder & Häder 2009)
6.




Coverage: UK Example
Since 1980s: Frames that only permit F2F have provided 98 – 99%
coverage. Coverage biased, but more important for some survey topics
(social exclusion) than others (purchasing)
But, F2F fieldwork has been becoming more expensive, more difficult, and
response rates falling
Fixed phone coverage reached 95 – 96% in 1990s, but falling since. Again,
coverage biased
Meanwhile, internet penetration grew rapidly
7.




UK General Population Surveys

None of fixed phones, mobile phones, internet provide good coverage
But around 99% of households have at least one of those three.
Points towards mixed-mode and/ or mixed-frame surveys to achieve good
coverage without reliance on F2F
8.




Role of Mobile Phones: Coverage
General Population Surveys: Permits high coverage rates when combined
with other frames/modes
Some Specialist Surveys: Adequate coverage rates as sole mode
Dual-Frame Issues: Properties of frames constantly changing, so
efficiencies change. Conflict between accurate estimation of selection
probabilities and over-loading the interview with seemingly meaningless
questions
9.




Sampling Error
Main feature affected if mobile phone numbers used as sampling elements
is selection probabilities
Personal Surveys:
     - Mobiles as single mode can help to reduce sampling error, through
       closer correspondence of persons to mobiles
     - Mobiles in multi-mode/frame less efficient (less close correspondence
       of persons to phone numbers) but still tends to reduce sampling error
       compared to landline-only surveys
Household Surveys:
     - Mobiles tend to increase sampling error
10.




Role of Mobile Phones: Sampling
Only relevant when mobile phone numbers used as sampling elements
Effect is generally positive (more accuracy) for surveys of individuals,
rather than households
Many uses of mobiles do not involve sampling mobile phone numbers –
e.g. if number is obtained from a sample selected in another way
11.




Non-Response
Early evidence that individuals less likely to participate in a survey if
approached by mobile rather than fixed phone
This now seems far less clear
Contact is quicker and contact rates higher with mobiles
But marginal effect typically more important than substitute effect
This can be positive or negative depending on the survey protocols
Likely to be positive when used as an adjunct to other modes/devices, e.g.
follow up to non-respondents
Can also be used as a mode for alerts/reminders


(Fuchs 2000, Lau 2004, Vehovar et al 2004, Bosnjak et al 2010, Lynn 2011)
12.




Measurement
Measurement differences between fixed and mobile phones are small
Those differences that have been detected point in the direction of less
social desirability bias with mobiles
This may be due to a) respondent having greater ability to move to
somewhere more private; b) removed risk of a family member ‘listening
in’ on another line
There may be other measurement advantages, such as ability to obtain
reactions promptly and on-location



(Kennedy 2010, Lynn & Kaminska 2011)
13.




Summary Effect of Mobiles on Accuracy
Coverage: Can be positive for mixed-mode surveys
Sampling: Either positive or neutral
Non-response: Can be positive for mixed-mode surveys with appropriate
protocols; may be negative if used as a substitute mode
Measurement: Either positive or neutral
14.




A Good News Story
Scientific survey literature often views mobile phones as a problem to be
overcome
Reality is that they bring many potential benefits, even in terms of
traditional statistical concepts of accuracy of estimates
And even for government and academic social surveys
But these benefits accrue only if mobiles are incorporated appropriately
into survey design and survey protocols
We maybe need better dissemination of good survey practice for mobile
phone interviews
Mobile Phone Surveys from a Total Survey
          Quality Perspective

                        Peter Lynn
       Institute for Social and Economic Research
                  University of Essex, UK

                plynn@essex.ac.uk
          www.iser.essex.ac.uk/people/plynn

More Related Content

Viewers also liked

Shulamit Perlamtter
Shulamit PerlamtterShulamit Perlamtter
Shulamit Perlamtter
crunyon2012w
 
'Groundtruth - Using mobile to Research Developing Markets' - txteagle (Mobil...
'Groundtruth - Using mobile to Research Developing Markets' - txteagle (Mobil...'Groundtruth - Using mobile to Research Developing Markets' - txteagle (Mobil...
'Groundtruth - Using mobile to Research Developing Markets' - txteagle (Mobil...
QuestBack AG
 
Salud AutopoiéTica
Salud AutopoiéTicaSalud AutopoiéTica
Salud AutopoiéTica
Hernando Vanegas
 
Historia Ligier
Historia LigierHistoria Ligier
Historia Ligier
antoc60
 
PojęCie Rozumienia
PojęCie RozumieniaPojęCie Rozumienia
PojęCie Rozumieniaguesta43bdd
 
'Mobile Exposure: Mobile media usage data' - Orange Advertising Network (Mobi...
'Mobile Exposure: Mobile media usage data' - Orange Advertising Network (Mobi...'Mobile Exposure: Mobile media usage data' - Orange Advertising Network (Mobi...
'Mobile Exposure: Mobile media usage data' - Orange Advertising Network (Mobi...
QuestBack AG
 
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
QuestBack AG
 
Globalpark Social Insight Connect Präsentation
Globalpark Social Insight Connect PräsentationGlobalpark Social Insight Connect Präsentation
Globalpark Social Insight Connect Präsentation
QuestBack AG
 
'Using mobile digital ethnography to measure the 2010 World Cup' - The Nielse...
'Using mobile digital ethnography to measure the 2010 World Cup' - The Nielse...'Using mobile digital ethnography to measure the 2010 World Cup' - The Nielse...
'Using mobile digital ethnography to measure the 2010 World Cup' - The Nielse...
QuestBack AG
 
'Minimising the experiential gap in today’s digital world' - Kantar (Mobile R...
'Minimising the experiential gap in today’s digital world' - Kantar (Mobile R...'Minimising the experiential gap in today’s digital world' - Kantar (Mobile R...
'Minimising the experiential gap in today’s digital world' - Kantar (Mobile R...
QuestBack AG
 
Parent
ParentParent
Parent
bre14
 
[ABCP 2016] A colaboração dos cidadãos na produção de leis: lições das consu...
[ABCP 2016] A colaboração dos cidadãos na produção de leis: lições das consu...[ABCP 2016] A colaboração dos cidadãos na produção de leis: lições das consu...
[ABCP 2016] A colaboração dos cidadãos na produção de leis: lições das consu...
Samuel Barros
 
Os desafios das consultas públicas online: lições do Marco Civil da Internet
Os desafios das consultas públicas online: lições do Marco Civil da InternetOs desafios das consultas públicas online: lições do Marco Civil da Internet
Os desafios das consultas públicas online: lições do Marco Civil da Internet
Samuel Barros
 

Viewers also liked (13)

Shulamit Perlamtter
Shulamit PerlamtterShulamit Perlamtter
Shulamit Perlamtter
 
'Groundtruth - Using mobile to Research Developing Markets' - txteagle (Mobil...
'Groundtruth - Using mobile to Research Developing Markets' - txteagle (Mobil...'Groundtruth - Using mobile to Research Developing Markets' - txteagle (Mobil...
'Groundtruth - Using mobile to Research Developing Markets' - txteagle (Mobil...
 
Salud AutopoiéTica
Salud AutopoiéTicaSalud AutopoiéTica
Salud AutopoiéTica
 
Historia Ligier
Historia LigierHistoria Ligier
Historia Ligier
 
PojęCie Rozumienia
PojęCie RozumieniaPojęCie Rozumienia
PojęCie Rozumienia
 
'Mobile Exposure: Mobile media usage data' - Orange Advertising Network (Mobi...
'Mobile Exposure: Mobile media usage data' - Orange Advertising Network (Mobi...'Mobile Exposure: Mobile media usage data' - Orange Advertising Network (Mobi...
'Mobile Exposure: Mobile media usage data' - Orange Advertising Network (Mobi...
 
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
Workshop: 'Self-administered Mobile Survey Workshop' - Dr Michael Bosnjak, Fr...
 
Globalpark Social Insight Connect Präsentation
Globalpark Social Insight Connect PräsentationGlobalpark Social Insight Connect Präsentation
Globalpark Social Insight Connect Präsentation
 
'Using mobile digital ethnography to measure the 2010 World Cup' - The Nielse...
'Using mobile digital ethnography to measure the 2010 World Cup' - The Nielse...'Using mobile digital ethnography to measure the 2010 World Cup' - The Nielse...
'Using mobile digital ethnography to measure the 2010 World Cup' - The Nielse...
 
'Minimising the experiential gap in today’s digital world' - Kantar (Mobile R...
'Minimising the experiential gap in today’s digital world' - Kantar (Mobile R...'Minimising the experiential gap in today’s digital world' - Kantar (Mobile R...
'Minimising the experiential gap in today’s digital world' - Kantar (Mobile R...
 
Parent
ParentParent
Parent
 
[ABCP 2016] A colaboração dos cidadãos na produção de leis: lições das consu...
[ABCP 2016] A colaboração dos cidadãos na produção de leis: lições das consu...[ABCP 2016] A colaboração dos cidadãos na produção de leis: lições das consu...
[ABCP 2016] A colaboração dos cidadãos na produção de leis: lições das consu...
 
Os desafios das consultas públicas online: lições do Marco Civil da Internet
Os desafios das consultas públicas online: lições do Marco Civil da InternetOs desafios das consultas públicas online: lições do Marco Civil da Internet
Os desafios das consultas públicas online: lições do Marco Civil da Internet
 

Similar to 'From a Total Survey Quality perspective' - University of Essex (Mobile Research Conference 2011)

Harnessing Mobile Technology to Draw Insights from Health Care Professionals ...
Harnessing Mobile Technology to Draw Insights from Health Care Professionals ...Harnessing Mobile Technology to Draw Insights from Health Care Professionals ...
Harnessing Mobile Technology to Draw Insights from Health Care Professionals ...
Merlien Institute
 
Malhotra06
Malhotra06Malhotra06
Malhotra06
Malhotra06Malhotra06
An evaluation of mobile phone usage restriction in selected public places in ...
An evaluation of mobile phone usage restriction in selected public places in ...An evaluation of mobile phone usage restriction in selected public places in ...
An evaluation of mobile phone usage restriction in selected public places in ...
Alexander Decker
 
Question 1
Question 1Question 1
Marketing Research PPT - III
Marketing Research PPT - IIIMarketing Research PPT - III
Marketing Research PPT - III
Ravinder Singh
 
Holding the Line: Customer Perceptions of Interactive Voice Response Systems
Holding the Line: Customer Perceptions of Interactive Voice Response SystemsHolding the Line: Customer Perceptions of Interactive Voice Response Systems
Holding the Line: Customer Perceptions of Interactive Voice Response Systems
Interactions Corporation
 
Coverage Nonresponse Trade-Off
Coverage Nonresponse Trade-OffCoverage Nonresponse Trade-Off
Coverage Nonresponse Trade-Off
Stephanie Eckman
 
Consumer_Insights_about_Gesture_Interaction_in_Vehicles
Consumer_Insights_about_Gesture_Interaction_in_VehiclesConsumer_Insights_about_Gesture_Interaction_in_Vehicles
Consumer_Insights_about_Gesture_Interaction_in_Vehicles
Visteon123
 
7027203.ppt
7027203.ppt7027203.ppt
7027203.ppt
Praveen Kumar
 
Paper 3 (iftikhar alam)
Paper 3 (iftikhar alam)Paper 3 (iftikhar alam)
Paper 3 (iftikhar alam)
Nadeem Shafique Butt
 
Consumer_Insights_about_Gesture_Interaction_in_Vehicles
Consumer_Insights_about_Gesture_Interaction_in_VehiclesConsumer_Insights_about_Gesture_Interaction_in_Vehicles
Consumer_Insights_about_Gesture_Interaction_in_Vehicles
Dr. Alexander van Laack
 
Making The Leap From Web To Mobile
Making The Leap From Web To MobileMaking The Leap From Web To Mobile
Making The Leap From Web To Mobile
Amy Buckner Chowdhry
 
Poster_final
Poster_finalPoster_final
Poster_final
Rushil Goyal
 
Data collection methods
Data collection methodsData collection methods
Data collection methods
Aanya Kumar
 
Sampling errors 8-12-2014
Sampling errors 8-12-2014Sampling errors 8-12-2014
Sampling errors 8-12-2014
Angel Mary George
 
Texting:The New Drunk Driving
Texting:The New Drunk DrivingTexting:The New Drunk Driving
Texting:The New Drunk Driving
xosweetbaby14
 
Topic 5 survey methods
Topic 5   survey methodsTopic 5   survey methods
Topic 5 survey methods
Dhani Ahmad
 
Distracted Driving
Distracted DrivingDistracted Driving
4 introduction to elementary sampling theory 17749
4 introduction to elementary sampling theory 177494 introduction to elementary sampling theory 17749
4 introduction to elementary sampling theory 17749
CHERUDUGASE
 

Similar to 'From a Total Survey Quality perspective' - University of Essex (Mobile Research Conference 2011) (20)

Harnessing Mobile Technology to Draw Insights from Health Care Professionals ...
Harnessing Mobile Technology to Draw Insights from Health Care Professionals ...Harnessing Mobile Technology to Draw Insights from Health Care Professionals ...
Harnessing Mobile Technology to Draw Insights from Health Care Professionals ...
 
Malhotra06
Malhotra06Malhotra06
Malhotra06
 
Malhotra06
Malhotra06Malhotra06
Malhotra06
 
An evaluation of mobile phone usage restriction in selected public places in ...
An evaluation of mobile phone usage restriction in selected public places in ...An evaluation of mobile phone usage restriction in selected public places in ...
An evaluation of mobile phone usage restriction in selected public places in ...
 
Question 1
Question 1Question 1
Question 1
 
Marketing Research PPT - III
Marketing Research PPT - IIIMarketing Research PPT - III
Marketing Research PPT - III
 
Holding the Line: Customer Perceptions of Interactive Voice Response Systems
Holding the Line: Customer Perceptions of Interactive Voice Response SystemsHolding the Line: Customer Perceptions of Interactive Voice Response Systems
Holding the Line: Customer Perceptions of Interactive Voice Response Systems
 
Coverage Nonresponse Trade-Off
Coverage Nonresponse Trade-OffCoverage Nonresponse Trade-Off
Coverage Nonresponse Trade-Off
 
Consumer_Insights_about_Gesture_Interaction_in_Vehicles
Consumer_Insights_about_Gesture_Interaction_in_VehiclesConsumer_Insights_about_Gesture_Interaction_in_Vehicles
Consumer_Insights_about_Gesture_Interaction_in_Vehicles
 
7027203.ppt
7027203.ppt7027203.ppt
7027203.ppt
 
Paper 3 (iftikhar alam)
Paper 3 (iftikhar alam)Paper 3 (iftikhar alam)
Paper 3 (iftikhar alam)
 
Consumer_Insights_about_Gesture_Interaction_in_Vehicles
Consumer_Insights_about_Gesture_Interaction_in_VehiclesConsumer_Insights_about_Gesture_Interaction_in_Vehicles
Consumer_Insights_about_Gesture_Interaction_in_Vehicles
 
Making The Leap From Web To Mobile
Making The Leap From Web To MobileMaking The Leap From Web To Mobile
Making The Leap From Web To Mobile
 
Poster_final
Poster_finalPoster_final
Poster_final
 
Data collection methods
Data collection methodsData collection methods
Data collection methods
 
Sampling errors 8-12-2014
Sampling errors 8-12-2014Sampling errors 8-12-2014
Sampling errors 8-12-2014
 
Texting:The New Drunk Driving
Texting:The New Drunk DrivingTexting:The New Drunk Driving
Texting:The New Drunk Driving
 
Topic 5 survey methods
Topic 5   survey methodsTopic 5   survey methods
Topic 5 survey methods
 
Distracted Driving
Distracted DrivingDistracted Driving
Distracted Driving
 
4 introduction to elementary sampling theory 17749
4 introduction to elementary sampling theory 177494 introduction to elementary sampling theory 17749
4 introduction to elementary sampling theory 17749
 

More from QuestBack AG

'How mobile research can use consumers' 'lost’ moments' - Valuewait B.V. (Mob...
'How mobile research can use consumers' 'lost’ moments' - Valuewait B.V. (Mob...'How mobile research can use consumers' 'lost’ moments' - Valuewait B.V. (Mob...
'How mobile research can use consumers' 'lost’ moments' - Valuewait B.V. (Mob...
QuestBack AG
 
Voice recognition in self-administered mobile surveys' - Free University of B...
Voice recognition in self-administered mobile surveys' - Free University of B...Voice recognition in self-administered mobile surveys' - Free University of B...
Voice recognition in self-administered mobile surveys' - Free University of B...
QuestBack AG
 
'Measuring the French mobile internet audience' - Médiamétrie (Mobile Researc...
'Measuring the French mobile internet audience' - Médiamétrie (Mobile Researc...'Measuring the French mobile internet audience' - Médiamétrie (Mobile Researc...
'Measuring the French mobile internet audience' - Médiamétrie (Mobile Researc...
QuestBack AG
 
Interpreting the Mobile Audience – Case Study from the US' - Zokem (MoInterpr...
Interpreting the Mobile Audience – Case Study from the US' - Zokem (MoInterpr...Interpreting the Mobile Audience – Case Study from the US' - Zokem (MoInterpr...
Interpreting the Mobile Audience – Case Study from the US' - Zokem (MoInterpr...
QuestBack AG
 
Mobile Research Conference 2010 Globalpark Surveys
Mobile Research Conference 2010 Globalpark SurveysMobile Research Conference 2010 Globalpark Surveys
Mobile Research Conference 2010 Globalpark Surveys
QuestBack AG
 
Mobile Market Research: Kundenmeinungen am Point of Experience
Mobile Market Research:   Kundenmeinungen am Point of ExperienceMobile Market Research:   Kundenmeinungen am Point of Experience
Mobile Market Research: Kundenmeinungen am Point of Experience
QuestBack AG
 
Globlapark Twitter Studie und die Top-5 der Social-Media-Nutzung
Globlapark Twitter Studie und die Top-5 der Social-Media-NutzungGloblapark Twitter Studie und die Top-5 der Social-Media-Nutzung
Globlapark Twitter Studie und die Top-5 der Social-Media-Nutzung
QuestBack AG
 
Globalpark Kundenpanels und Communities
Globalpark Kundenpanels und CommunitiesGlobalpark Kundenpanels und Communities
Globalpark Kundenpanels und CommunitiesQuestBack AG
 
Globalpark Befragungsmanagement
Globalpark BefragungsmanagementGlobalpark Befragungsmanagement
Globalpark BefragungsmanagementQuestBack AG
 
Mobile Market Research: Erfahrungen - Potenzial - Lösungen
Mobile Market Research: Erfahrungen - Potenzial - LösungenMobile Market Research: Erfahrungen - Potenzial - Lösungen
Mobile Market Research: Erfahrungen - Potenzial - LösungenQuestBack AG
 
Dmexco Vortrag Online Panels
Dmexco Vortrag Online PanelsDmexco Vortrag Online Panels
Dmexco Vortrag Online PanelsQuestBack AG
 
Surveytainment und Social Media: Innovative Befragungsformen für die Marktfor...
Surveytainment und Social Media: Innovative Befragungsformen für die Marktfor...Surveytainment und Social Media: Innovative Befragungsformen für die Marktfor...
Surveytainment und Social Media: Innovative Befragungsformen für die Marktfor...QuestBack AG
 
Customer Feedback Community Online Marktforschung Goes Social Media
Customer Feedback Community  Online Marktforschung Goes Social MediaCustomer Feedback Community  Online Marktforschung Goes Social Media
Customer Feedback Community Online Marktforschung Goes Social MediaQuestBack AG
 
Measurement options, measurement error, and usability in mobile surveys
Measurement options, measurement error, and usability in mobile surveysMeasurement options, measurement error, and usability in mobile surveys
Measurement options, measurement error, and usability in mobile surveys
QuestBack AG
 
Mastering Online Surveys
Mastering Online SurveysMastering Online Surveys
Mastering Online Surveys
QuestBack AG
 
Using flash type questions – stroke of luck or curse for data quality?
Using flash type questions – stroke of luck or curse for data quality?Using flash type questions – stroke of luck or curse for data quality?
Using flash type questions – stroke of luck or curse for data quality?
QuestBack AG
 

More from QuestBack AG (16)

'How mobile research can use consumers' 'lost’ moments' - Valuewait B.V. (Mob...
'How mobile research can use consumers' 'lost’ moments' - Valuewait B.V. (Mob...'How mobile research can use consumers' 'lost’ moments' - Valuewait B.V. (Mob...
'How mobile research can use consumers' 'lost’ moments' - Valuewait B.V. (Mob...
 
Voice recognition in self-administered mobile surveys' - Free University of B...
Voice recognition in self-administered mobile surveys' - Free University of B...Voice recognition in self-administered mobile surveys' - Free University of B...
Voice recognition in self-administered mobile surveys' - Free University of B...
 
'Measuring the French mobile internet audience' - Médiamétrie (Mobile Researc...
'Measuring the French mobile internet audience' - Médiamétrie (Mobile Researc...'Measuring the French mobile internet audience' - Médiamétrie (Mobile Researc...
'Measuring the French mobile internet audience' - Médiamétrie (Mobile Researc...
 
Interpreting the Mobile Audience – Case Study from the US' - Zokem (MoInterpr...
Interpreting the Mobile Audience – Case Study from the US' - Zokem (MoInterpr...Interpreting the Mobile Audience – Case Study from the US' - Zokem (MoInterpr...
Interpreting the Mobile Audience – Case Study from the US' - Zokem (MoInterpr...
 
Mobile Research Conference 2010 Globalpark Surveys
Mobile Research Conference 2010 Globalpark SurveysMobile Research Conference 2010 Globalpark Surveys
Mobile Research Conference 2010 Globalpark Surveys
 
Mobile Market Research: Kundenmeinungen am Point of Experience
Mobile Market Research:   Kundenmeinungen am Point of ExperienceMobile Market Research:   Kundenmeinungen am Point of Experience
Mobile Market Research: Kundenmeinungen am Point of Experience
 
Globlapark Twitter Studie und die Top-5 der Social-Media-Nutzung
Globlapark Twitter Studie und die Top-5 der Social-Media-NutzungGloblapark Twitter Studie und die Top-5 der Social-Media-Nutzung
Globlapark Twitter Studie und die Top-5 der Social-Media-Nutzung
 
Globalpark Kundenpanels und Communities
Globalpark Kundenpanels und CommunitiesGlobalpark Kundenpanels und Communities
Globalpark Kundenpanels und Communities
 
Globalpark Befragungsmanagement
Globalpark BefragungsmanagementGlobalpark Befragungsmanagement
Globalpark Befragungsmanagement
 
Mobile Market Research: Erfahrungen - Potenzial - Lösungen
Mobile Market Research: Erfahrungen - Potenzial - LösungenMobile Market Research: Erfahrungen - Potenzial - Lösungen
Mobile Market Research: Erfahrungen - Potenzial - Lösungen
 
Dmexco Vortrag Online Panels
Dmexco Vortrag Online PanelsDmexco Vortrag Online Panels
Dmexco Vortrag Online Panels
 
Surveytainment und Social Media: Innovative Befragungsformen für die Marktfor...
Surveytainment und Social Media: Innovative Befragungsformen für die Marktfor...Surveytainment und Social Media: Innovative Befragungsformen für die Marktfor...
Surveytainment und Social Media: Innovative Befragungsformen für die Marktfor...
 
Customer Feedback Community Online Marktforschung Goes Social Media
Customer Feedback Community  Online Marktforschung Goes Social MediaCustomer Feedback Community  Online Marktforschung Goes Social Media
Customer Feedback Community Online Marktforschung Goes Social Media
 
Measurement options, measurement error, and usability in mobile surveys
Measurement options, measurement error, and usability in mobile surveysMeasurement options, measurement error, and usability in mobile surveys
Measurement options, measurement error, and usability in mobile surveys
 
Mastering Online Surveys
Mastering Online SurveysMastering Online Surveys
Mastering Online Surveys
 
Using flash type questions – stroke of luck or curse for data quality?
Using flash type questions – stroke of luck or curse for data quality?Using flash type questions – stroke of luck or curse for data quality?
Using flash type questions – stroke of luck or curse for data quality?
 

Recently uploaded

-- June 2024 is National Volunteer Month --
-- June 2024 is National Volunteer Month ---- June 2024 is National Volunteer Month --
-- June 2024 is National Volunteer Month --
NZSG
 
The Influence of Marketing Strategy and Market Competition on Business Perfor...
The Influence of Marketing Strategy and Market Competition on Business Perfor...The Influence of Marketing Strategy and Market Competition on Business Perfor...
The Influence of Marketing Strategy and Market Competition on Business Perfor...
Adam Smith
 
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
bosssp10
 
How to Implement a Real Estate CRM Software
How to Implement a Real Estate CRM SoftwareHow to Implement a Real Estate CRM Software
How to Implement a Real Estate CRM Software
SalesTown
 
Observation Lab PowerPoint Assignment for TEM 431
Observation Lab PowerPoint Assignment for TEM 431Observation Lab PowerPoint Assignment for TEM 431
Observation Lab PowerPoint Assignment for TEM 431
ecamare2
 
In the Adani-Hindenburg case, what is SEBI investigating.pptx
In the Adani-Hindenburg case, what is SEBI investigating.pptxIn the Adani-Hindenburg case, what is SEBI investigating.pptx
In the Adani-Hindenburg case, what is SEBI investigating.pptx
Adani case
 
Understanding User Needs and Satisfying Them
Understanding User Needs and Satisfying ThemUnderstanding User Needs and Satisfying Them
Understanding User Needs and Satisfying Them
Aggregage
 
Recruiting in the Digital Age: A Social Media Masterclass
Recruiting in the Digital Age: A Social Media MasterclassRecruiting in the Digital Age: A Social Media Masterclass
Recruiting in the Digital Age: A Social Media Masterclass
LuanWise
 
Company Valuation webinar series - Tuesday, 4 June 2024
Company Valuation webinar series - Tuesday, 4 June 2024Company Valuation webinar series - Tuesday, 4 June 2024
Company Valuation webinar series - Tuesday, 4 June 2024
FelixPerez547899
 
Exploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social DreamingExploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social Dreaming
Nicola Wreford-Howard
 
Creative Web Design Company in Singapore
Creative Web Design Company in SingaporeCreative Web Design Company in Singapore
Creative Web Design Company in Singapore
techboxsqauremedia
 
Mastering B2B Payments Webinar from BlueSnap
Mastering B2B Payments Webinar from BlueSnapMastering B2B Payments Webinar from BlueSnap
Mastering B2B Payments Webinar from BlueSnap
Norma Mushkat Gaffin
 
ModelingMarketingStrategiesMKS.CollumbiaUniversitypdf
ModelingMarketingStrategiesMKS.CollumbiaUniversitypdfModelingMarketingStrategiesMKS.CollumbiaUniversitypdf
ModelingMarketingStrategiesMKS.CollumbiaUniversitypdf
fisherameliaisabella
 
BeMetals Investor Presentation_June 1, 2024.pdf
BeMetals Investor Presentation_June 1, 2024.pdfBeMetals Investor Presentation_June 1, 2024.pdf
BeMetals Investor Presentation_June 1, 2024.pdf
DerekIwanaka1
 
Digital Transformation and IT Strategy Toolkit and Templates
Digital Transformation and IT Strategy Toolkit and TemplatesDigital Transformation and IT Strategy Toolkit and Templates
Digital Transformation and IT Strategy Toolkit and Templates
Aurelien Domont, MBA
 
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdfikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
agatadrynko
 
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel ChartSatta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
➒➌➎➏➑➐➋➑➐➐Dpboss Matka Guessing Satta Matka Kalyan Chart Indian Matka
 
Best practices for project execution and delivery
Best practices for project execution and deliveryBest practices for project execution and delivery
Best practices for project execution and delivery
CLIVE MINCHIN
 
An introduction to the cryptocurrency investment platform Binance Savings.
An introduction to the cryptocurrency investment platform Binance Savings.An introduction to the cryptocurrency investment platform Binance Savings.
An introduction to the cryptocurrency investment platform Binance Savings.
Any kyc Account
 
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Zodiac Signs and Food Preferences_ What Your Sign Says About Your TasteZodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
my Pandit
 

Recently uploaded (20)

-- June 2024 is National Volunteer Month --
-- June 2024 is National Volunteer Month ---- June 2024 is National Volunteer Month --
-- June 2024 is National Volunteer Month --
 
The Influence of Marketing Strategy and Market Competition on Business Perfor...
The Influence of Marketing Strategy and Market Competition on Business Perfor...The Influence of Marketing Strategy and Market Competition on Business Perfor...
The Influence of Marketing Strategy and Market Competition on Business Perfor...
 
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
 
How to Implement a Real Estate CRM Software
How to Implement a Real Estate CRM SoftwareHow to Implement a Real Estate CRM Software
How to Implement a Real Estate CRM Software
 
Observation Lab PowerPoint Assignment for TEM 431
Observation Lab PowerPoint Assignment for TEM 431Observation Lab PowerPoint Assignment for TEM 431
Observation Lab PowerPoint Assignment for TEM 431
 
In the Adani-Hindenburg case, what is SEBI investigating.pptx
In the Adani-Hindenburg case, what is SEBI investigating.pptxIn the Adani-Hindenburg case, what is SEBI investigating.pptx
In the Adani-Hindenburg case, what is SEBI investigating.pptx
 
Understanding User Needs and Satisfying Them
Understanding User Needs and Satisfying ThemUnderstanding User Needs and Satisfying Them
Understanding User Needs and Satisfying Them
 
Recruiting in the Digital Age: A Social Media Masterclass
Recruiting in the Digital Age: A Social Media MasterclassRecruiting in the Digital Age: A Social Media Masterclass
Recruiting in the Digital Age: A Social Media Masterclass
 
Company Valuation webinar series - Tuesday, 4 June 2024
Company Valuation webinar series - Tuesday, 4 June 2024Company Valuation webinar series - Tuesday, 4 June 2024
Company Valuation webinar series - Tuesday, 4 June 2024
 
Exploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social DreamingExploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social Dreaming
 
Creative Web Design Company in Singapore
Creative Web Design Company in SingaporeCreative Web Design Company in Singapore
Creative Web Design Company in Singapore
 
Mastering B2B Payments Webinar from BlueSnap
Mastering B2B Payments Webinar from BlueSnapMastering B2B Payments Webinar from BlueSnap
Mastering B2B Payments Webinar from BlueSnap
 
ModelingMarketingStrategiesMKS.CollumbiaUniversitypdf
ModelingMarketingStrategiesMKS.CollumbiaUniversitypdfModelingMarketingStrategiesMKS.CollumbiaUniversitypdf
ModelingMarketingStrategiesMKS.CollumbiaUniversitypdf
 
BeMetals Investor Presentation_June 1, 2024.pdf
BeMetals Investor Presentation_June 1, 2024.pdfBeMetals Investor Presentation_June 1, 2024.pdf
BeMetals Investor Presentation_June 1, 2024.pdf
 
Digital Transformation and IT Strategy Toolkit and Templates
Digital Transformation and IT Strategy Toolkit and TemplatesDigital Transformation and IT Strategy Toolkit and Templates
Digital Transformation and IT Strategy Toolkit and Templates
 
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdfikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
 
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel ChartSatta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
Satta Matka Dpboss Matka Guessing Kalyan Chart Indian Matka Kalyan panel Chart
 
Best practices for project execution and delivery
Best practices for project execution and deliveryBest practices for project execution and delivery
Best practices for project execution and delivery
 
An introduction to the cryptocurrency investment platform Binance Savings.
An introduction to the cryptocurrency investment platform Binance Savings.An introduction to the cryptocurrency investment platform Binance Savings.
An introduction to the cryptocurrency investment platform Binance Savings.
 
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Zodiac Signs and Food Preferences_ What Your Sign Says About Your TasteZodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
Zodiac Signs and Food Preferences_ What Your Sign Says About Your Taste
 

'From a Total Survey Quality perspective' - University of Essex (Mobile Research Conference 2011)

  • 1. Mobile Phone Surveys from a Total Survey Quality Perspective Peter Lynn Institute for Social and Economic Research University of Essex, UK Mobile Research Conference 19 April 2011 © Peter Lynn
  • 2. 1. Accuracy of Survey Estimates (Variance and Bias)
  • 3. 2. Accuracy of Survey Estimates (Sources) Mean Square Error Variance Bias Errors of Non-observation Observational Errors Errors of Non-observation Observational Errors Coverage Sampling Non-response Interviewer Respondent Instrument Coder Coverage Sampling Non-response Interviewer Respondent Instrument Coder
  • 5. 4. Role of Mobile Phone Surveys Focus here on interviewing, not self-completion i.e. respondent is listening and talking, not reading and keying Stand-alone mode or within multi-mode Focus is on accuracy of survey estimates
  • 6. 5. Coverage Error Initial interest was in mobile phone ownership as a problem: Worsening coverage of landline frames (Finland, Slovenia etc in 1990s, USA in 2000s; need for dual-frame approaches Then some interest in improved coverage for phone surveys: Sub-Saharan Africa, Eastern Europe Now, opportunities for improving coverage in mixed-mode surveys: Germany, UK, etc (Blumberg and Luke 2007, Kuusela et al 2008, Tortora et al, 2008, Gallup Europe 2009, Häder & Häder 2009)
  • 7. 6. Coverage: UK Example Since 1980s: Frames that only permit F2F have provided 98 – 99% coverage. Coverage biased, but more important for some survey topics (social exclusion) than others (purchasing) But, F2F fieldwork has been becoming more expensive, more difficult, and response rates falling Fixed phone coverage reached 95 – 96% in 1990s, but falling since. Again, coverage biased Meanwhile, internet penetration grew rapidly
  • 8. 7. UK General Population Surveys None of fixed phones, mobile phones, internet provide good coverage But around 99% of households have at least one of those three. Points towards mixed-mode and/ or mixed-frame surveys to achieve good coverage without reliance on F2F
  • 9. 8. Role of Mobile Phones: Coverage General Population Surveys: Permits high coverage rates when combined with other frames/modes Some Specialist Surveys: Adequate coverage rates as sole mode Dual-Frame Issues: Properties of frames constantly changing, so efficiencies change. Conflict between accurate estimation of selection probabilities and over-loading the interview with seemingly meaningless questions
  • 10. 9. Sampling Error Main feature affected if mobile phone numbers used as sampling elements is selection probabilities Personal Surveys: - Mobiles as single mode can help to reduce sampling error, through closer correspondence of persons to mobiles - Mobiles in multi-mode/frame less efficient (less close correspondence of persons to phone numbers) but still tends to reduce sampling error compared to landline-only surveys Household Surveys: - Mobiles tend to increase sampling error
  • 11. 10. Role of Mobile Phones: Sampling Only relevant when mobile phone numbers used as sampling elements Effect is generally positive (more accuracy) for surveys of individuals, rather than households Many uses of mobiles do not involve sampling mobile phone numbers – e.g. if number is obtained from a sample selected in another way
  • 12. 11. Non-Response Early evidence that individuals less likely to participate in a survey if approached by mobile rather than fixed phone This now seems far less clear Contact is quicker and contact rates higher with mobiles But marginal effect typically more important than substitute effect This can be positive or negative depending on the survey protocols Likely to be positive when used as an adjunct to other modes/devices, e.g. follow up to non-respondents Can also be used as a mode for alerts/reminders (Fuchs 2000, Lau 2004, Vehovar et al 2004, Bosnjak et al 2010, Lynn 2011)
  • 13. 12. Measurement Measurement differences between fixed and mobile phones are small Those differences that have been detected point in the direction of less social desirability bias with mobiles This may be due to a) respondent having greater ability to move to somewhere more private; b) removed risk of a family member ‘listening in’ on another line There may be other measurement advantages, such as ability to obtain reactions promptly and on-location (Kennedy 2010, Lynn & Kaminska 2011)
  • 14. 13. Summary Effect of Mobiles on Accuracy Coverage: Can be positive for mixed-mode surveys Sampling: Either positive or neutral Non-response: Can be positive for mixed-mode surveys with appropriate protocols; may be negative if used as a substitute mode Measurement: Either positive or neutral
  • 15. 14. A Good News Story Scientific survey literature often views mobile phones as a problem to be overcome Reality is that they bring many potential benefits, even in terms of traditional statistical concepts of accuracy of estimates And even for government and academic social surveys But these benefits accrue only if mobiles are incorporated appropriately into survey design and survey protocols We maybe need better dissemination of good survey practice for mobile phone interviews
  • 16. Mobile Phone Surveys from a Total Survey Quality Perspective Peter Lynn Institute for Social and Economic Research University of Essex, UK plynn@essex.ac.uk www.iser.essex.ac.uk/people/plynn