Representativeness and quality of online sampling by Germán Loewe - Netquest 2011

Representativeness and qualityof online sampling
German Loewe
Founder & CEO

No method…
CONVENIENCE
SAMPLING
Unstructured data
Yes method…
Known inclusion probabilities
Known estimates error
PROBABILITY
SAMPLING
Unknown inclusion probabilities
Unknown error
Representation via quotas
QUOTA SAMPLING
Main sampling techniques

Sample Frame
Sample Frame exists = target population
1
Units sampled randomly (known distribution)
2
The three pillars of representativity…
Target Population
Population

Sample Frame
Sample Frame exists = target population
1
Units sampled randomly (known distribution)
2
Unbiased non-response
3
The three pillars of representativity…
Target Population
Population

Internet Population
Target Population
Population
No such thing as a Sample Frame exists
1
…no random (unknown inclusion probabilities)
2
…also non-response
3
Online’s methodological limbo…

Probability Sampling
Quota Sampling
So how is online working at all??
no sample frames available
low response rates
no time / money for design

Quota Sampling
…and what about off-line??
no sampleframesavailable
low response rates

Quota Sampling
Quota Sampling
Why is online so widely accepted
Fast
Cheap
Multimedia
Low incidence
Coverage??
low response rates
On-line
Fast??
Cheap??
Multimedia??
Low incidence?
Coverage
low response rates
Off-line

How big is the coverage disadvantage?
?
On-line
Off-line

Not connected to the Internet
Not visiting web sites were you recruit
Not wanting to participate in on-line surveys
Abandoning on-line surveys longer than X min
?
On-line
Not accessible via Phone or f2f
Not visiting places were you recruit
Not wanting to participate in off-line surveys
Abandoning off-line surveys longer than Y min
Off-line

?
On-line
Not visiting places were you recruit / not at home
Off-line

?
On-line
Abandoning on-line surveys longer than Y min
Off-line

?
On-line
Off-line

DEPENDS ON:
Typeofproductorserviceyou are askingabout:
A PDA device?
Oran ice cream?
So how much should you worry about coverage?

Measurement Error
4
PROBABILITY
SAMPLING
…what about the measurement error?
1
2
Coverage Error
Sampling Error
“Sample Frame does not match target population”
Statistical error
(under control)
3
Interviewer
Mode-effects
Social desirability
Situational representativity
Control via technology
Non-response
“Only people wanting to share opinions participate”
Online ok!!!

What is quality in online panels then?
No professional respondents
No monetary incentives
Maximum 2-3 surveys a month (different categories)
Updated information
Continuous automatic updates (mandatory)
No fraud
Lower duplicates thanks to no media buying
Higher control over recruiting campaigns

What is quality in online panels, then?
No self-selection
Actively recruited only panelists
Incentivize recruiting
No incentive or content announcement in the field
Reduced non-observation
(non-response + non-coverage)
Multisource (incentivized) recruiting
Incentivize proportional to time (including filter-outs)
High response rate due to sure incentives

If you cannot look inside,
how can you know find the quality providers??
“It’s the response rate, stupid”

If you cannot look inside,
how can you know find the quality providers??
“It’s the observation rate, stupid”

Thankyou!
German Loewe
gloewe@netquest.es

Representativeness and quality of online sampling by Germán Loewe - Netquest 2011

by Netquest

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 16

Statistics

Representativeness and quality of online sampling by Germán Loewe - Netquest 2011 Presentation Transcript