Various methods exist to improve CATI Fieldwork
with regard to Interviewer training, Dialer, incentives
etc. Often overlooked is the actual sampling: whether
it is business sample, consumer sample, client sample
or acquired sample. In this article we will look at various
options that exist to improve fieldwork efficiency by
developing a sampling strategy. This is a 3 part series
which explores B2B Sample, RDD sample and Lifestyle
sample.The article uses real cases as an example to
derive lessons learned and best practices.
1. 1Brochurename or the title
Improving your CATI Fieldwork
Mobile RDD
Sample
Pulsed Mobile RDD
Consumer Sample
Global Coverage with more than 250
key variables
RDD onDemand
RDD onDemand provides direct access to
our global RDD data base
Geocoding Services
Enrich Data with NUTS regions or
other socio-demographic data
Sample Solutions
P R E M I E R . S A M P L E . P R O V I D E R
PREMIER.SAMPLE.PROVIDER
SAMPLE SOLUTIONS
Landline RDD Sample
Pulsed landline RDD for more thann
140 countries
www.sample.solutions
2.
3. 3Brochurename or the title
Introduction
Various methods exist to improve CATI Fieldwork
with regard to Interviewer training, Dialer, incentives
etc. Often overlooked is the actual sampling: whether
it is business sample, consumer sample, client sample
or acquired sample. In this article we will look at various
options that exist to improve fieldwork efficiency by
developing a sampling strategy. This is a 3 part series
which explores B2B Sample, RDD sample and Lifestyle
sample.The article uses real cases as an example to
derive lessons learned and best practices.
4. CONTENT
4 Brochurename or the title
3
9
11
Introduction
Consumer
Sample
Blending
CATI with
SMS data
Contents
5 B2B
Sample
13Conclusion
5. NAME OF THE CHAPTER
5Brochurename or the title
B2B Sample
In many cases clients approach us with a specific SIC code range, employee
ranges and a sample is purchased and loaded into the dialer. In some cases, few
days later, a client would come back and explain that the sample does not meet
the requirements. In our case we would then usually go through the question-
naire and analyze the requirements. In many cases however, time and money
can be saved by looking into the questionnaire and the quotas beforehand: What
are the requirements, who is the contact person we need to speak to, what are
other boundary conditions? When explaining this to a sample provider, a sample
strategy can be developed prior to the actual fieldwork. Whether it is a small or
large scale project, a sampling strategy can be vital for the success of a project.
1. Pulsing
A fraction of businesses dies every year, changes their phone number, moves
addresses thus it is important to utilize a regularly updated database. Even
though business databases are updated frequently, we encounter 10-20% of
phone numbers not working. Before starting a project it might be worth puls-
ing these records to remove these non functioning numbers so that no time is
lost during fieldwork. Furthermore, it allows better judgment whether a specific
number of interviews can be reached.
6. NAME OF THE CHAPTER
6 Brochurename or the title
There shouldn’t be any
outdated contact people
in the data - changes and
updates need to be made regulary.
It is essential to have
multiple contacts – in
most B2B research in CATI
research several attempts are required
in order to complete an interview or
book an appointment for an interview.
Instead of neglecting
contact people because
of outdated information,
the info can be used to connect to the
correct follow-up person.
2. Enrichment
Going a step further from pulsing, we could enrich the disconnected phone
numbers with verified working phone numbers. Additionally, it is possible to add
other valuable information like employee count size, location, revenue or contact
person. By doing so, we can further narrow down the target market or specify
stratifications of our sample so that during the end of fieldwork it is possible
whether the target record is still required for any of the quotas.
In the current age, we have access to a plethora of sources of secondary data as
well as databases from private or public sources that can be accessed
3. Contacts
Contact names are essential for bypassing a gatekeeper. However, many data-
bases contain only the Managing Director or CEO of a company. The chances
that a good gatekeeper will pass you on to the CEO are fairly small. In many
cases having a few additional contacts can be beneficial.
Imagine you ask the gatekeeper to connect you to accounts payable – I am sure
you will be passed on. From there it is an easy step to ask to be connected to
the correct contact person.
Having outdated contact people in the data is a common problem. As much
as databases are kept up-to-date, changes frequently occur in our fast-paced
business environment.
7. NAME OF THE CHAPTER
7Brochurename or the title
4. Employee Size & Revenue
In many cases, the target audience is not just described
by means of an industry but also by employee size or
revenue band (minimum, range or maximum).
These extra filters can have a significant effect on the
available data and make fieldwork not feasible due to
the fact that the desired targets cannot be reached. In
the following example we explain how employee size
and turnover can be used to increase the available
sample frame:
Imagine you would like to survey manufacturing compa-
nies in Belgium and the Netherlands with a revenue of
EUR +10M. However, it seems we cannot get sufficient
records from data sources making it impossible to reach
the desired targets with the available sources. However,
we can use an estimate for the required employment
size. From which we then calculate around EUR 100.000
in turnover per employee, thus, we can work with an
estimate of 100 employees or more. Many databases will
contain a lot more information with regard to employee
sizes rather than revenue. Note, this method is of course
not completely accurate and will have an influence on
the incidence rate. We can improve it by increasing the
employee size level or decrease it by lowering it (though
increasing the sampling frame).
Note that the ratio between turnover and employee size
will vary from country to country, for the case of Belgium
and the Netherlands, these will be similar.
5. Fine-tuning your selection
By means of sampling a specific audience, we try to
create a dataset which has characteristics similar to
the desired audience. In this section, we will look at
some cases from the past and see how sampling is
approached.
Sampling of VETS: Narrowing down the target selection
Let us take the example of animal veterinarians (we do
not make use of veterinarians that work with small pets
but larger animals such as horses, cattle and so on). We
could start off with defining a category or a SIC code
and draw the sample from there. Nevertheless, to further
increase efficiency we could add some extra steps:
Remove all sample that contains the name “pet” or
anything that points to smaller animals.
Remove all VETs from urban areas and focus on
remote rural areas.
Prefer records which contain names like horse or
cattle and remove Veterinarians that contain dog or cat
care.
Even though the sample will still contain Veterinarians
which take care of smaller animals, nevertheless, we have
managed to increase the incidence rate significantly.
8. 8 Brochurename or the title
6. Sample Sources
Selecting sample sources does not only have an influ-
ence on the quality of the sample but also on the
response rate. The most common databases out there
require a lot of work and time to keep the records
updated. This is costly and can only be covered by suf-
ficient sales of these profiled records. How often do you
think Fortune 500 companies, companies with +500 M
revenue or IT decision makers of TOP-X companies of
a country are purchased? While quality might be good,
these people might have been contacted way too often
for the purpose of market research but also for sales.
In many cases it might, therefore, be handy to blend
data: Think of a government database, blended with
Google information and maybe LinkedIn contacts. While
upfront it may seem to be more work than a standard
database.
7. A/B Testing
Under the section of “Fine-tuning your selection” we
have discussed various options for sampling Veterinar-
ians – for larger studies, it is essential to identify the most
effective sampling strategy (= highest incidence rate) for
different methods of sampling. Therefore, it is recom-
mended to set quotas on the different sample sources.
The same holds true for different sample providers.
Different reasoning can be used for sampling an audi-
ence, nevertheless, only when you measure it, you can
manage it and increase your incidence rate.
“Measure it, to manage it!”
After completing about 10% of fieldwork, take a moment
to compare the different sample sources together with
refusal rates, IR and other important variables. Based on
this comparison, you can make a decision with which
sample source to continue using for the remainder of
your fieldwork.
Sample suppliers should not charge an extra setup fee
or high minimum order if you explain to them your case
and want advice for your sampling strategy needed for
a successful fieldwork period.
“Different methods can be used to achieve higher field
productivity. It is important that a sampling strategy
is developed which exceeds the standard “Some SIC
codes + employees ranges)”.
9. 9Brochurename or the title
1. Sample Type
For each consumer project it is essential to analyze the options for sampling.
For general population studies, for example, it is feasible to use RDD sample
since it is cost-effective and yields a large coverage while lifestyle sample would
be more expensive.
Nevertheless, if certain quotas need to be met (quota-sampling) a mix of RDD
sample and lifestyle sample may prove useful. Initial usage of RDD sample
(dual-frame) can cover the majority of the population. In order to reach special
groups at the end of a survey in specific areas or age groups, targeted lifestyle
sample can be used.
2. Screening techniques
Though a pulsed or verified sample is slightly
more expensive than a raw sample, it is worth
the investment as clean samples have a much
higher return on investment. Not only the
hit rate of the sample should be taken into
account, but also the dialing time. Modern
Predictive diallers can significantly reduce
waiting times for agents, nevertheless, many
B2C projects are still not dialed predictive but
rather through manual or automatic methods.
Using a pre-dialled sample reduces waiting
time.
Consumer Sample
Consumer projects can run on client sample (e.g. customer satisfaction studies), RDD sample
(e.g. public opinion polling) or lifestyle sample (e.g. brand awareness of a specific age group).
All these sample types can be used individually or combined.
10. 10 Brochurename or the title
3.Enrichment
Screening allows us to remove disconnected records while
enrichment adds more value to the sample. Through enrich-
ment additional information like names, age, locations and
more can be added to the sample.
Enrichment reduces the need for agents to ask for these vari-
ables during fieldwork and diminishes the chance for human
error. This also mitigates cost and also increases strike rate
per interviewer as the average questionnaire length can be
reduced.
4. Replicating
In projects where response rate is important, it might be use-
ful to replicate sample and use a maximum amount of tries
within a replicate before a new replicate is opened. By doing
so it is possible to achieve a higher response rate while part of
the sample remains untouched. However, at the same time, it
might have an influence on the efficiency of the sample.
5. A/B Testing
In many cases, it is not known beforehand which approach
for sampling yields the best possible outcome. Therefore, it is
recommended to set quotas on the different sample sources.
The same holds true for different sample providers. Various
reasoning can be used for sampling an audience, nevertheless,
only when you measure it, can you manage it and increase
your incidence rate. For instance, High-Net-Worth Individuals
(HNWI): can be targeted based on postcode, lifestyle sample
on income.
6. Geocoding
In many consumer projects aspects such as rural/urban topol-
ogy or size of the community (5000 people or more than
1,000,000 people), postcode, city or state are asked. Neverthe-
less, this kind of information can be precoded into the sample
which saves money by reducing interviewing time – thus
geocoding provides multiple advantages.
Additionally, statistical data for location and also demographics
(income level, ethnographic variables etc.) can be appended
in many cases.
11. 11Brochurename or the title
Blending CATI with SMS data collection
This article is the last in the series of discussing how CATI fieldwork, in general, can be enhanced by the usage
of SMS. In the previous pages, we discussed various methods used to improve CATI data collection by means
of advanced B2B and B2C sampling.Data collection in the future will converge more towards a mixed mode of
data collection. Therefore, a blended sampling approach assists with multi-mode data collection. Thus, we refer
to blended data collection as the use of a single sampling frame with multiple sources of data. In this article, we
focus on 1-way SMS messages which consist of the respondent receiving a link to a survey.
A. Blended Data Collection
Data blending begins with the gathering of data from different sources. Once these valid sources are located and loaded, the next
step is to merge the useful data into a functioning data set (sampling frame) while excluding the polluted noise of unnecessary data.
The sampling frame could be a client database or a national representative (NatRep) mobile phone frame in a population with high
penetration of mobile phone numbers.
In this case, one can use an initial wave of survey invitations via SMS. Since the opening rate of SMS is beyond 90% – most of the
people that respond to SMS surveys will complete the mobile survey within a few hours. By using the survey link it is possible to track
whether respondents have opened or begun the survey. Furthermore, an extra reminder can be sent out via SMS to respondents,
while the respondents which have not reacted can be contacted via CATI.
The benefits of this method are evident:
- Cost savings due to part of the interviews being completed online on a mobile device.
- Higher response rate
- Different characteristics of respondents (some prefer online while others telephone) therefore lower bias.
The timelines and a number of reminders can be chosen depending on the type of survey. In the initial phase of a survey, it is rec-
ommended to make use of an A/B split in order to reach maximum efficiency. Respondents can be targeted with SMS messages at
any specified time.
12. NAME OF THE CHAPTER
12 Brochurename or the title
C. SMS Follow-up questions
Some surveys require extra follow-up questions for clarifications or weighting. In many cases finding the right respondent is tedious.
Therefore, it is recommended to collect the mobile phone number of a respondent during the main fieldwork. Not only is it possible
to follow-up directly with the right respondent but also make use of SMS to collect extra data either through 1-way or 2-way SMS
messages.
This procedure reduces the probability of selecting the wrong contact person within an organization, decreases the cost due to less
time spent on agents having to find the right respondent or even removing the necessity for a CATI follow-up completely, as part
of the data can be collected via SMS.
B. Pre-Notification SMS
Another simple but very effective way to decrease cost while at the same time increasing response rate is the usage of pre-notification
SMS. With a pre-notification, it is possible to build up trust, inform the respondent of the purpose of the study and set a calling time.
D. Pre-Screening via SMS
In low incidence rate (IR) surveys, one of the biggest cost factors is pre-screening. An agent in the CATI facility will have to work
through the respondents and check for eligibility. With some surveys having an IR below 10% it is possible to imagine not just the
extra cost due to the low incidence but also the decreased motivation of the actual agent. By using SMS pre-screening one can make
use of a cost-effective method for finding the correct target population. Of course, pre-screening is more feasible for B2C projects.
Nevertheless,when using a client set, it is possible to add a pre-screening question. Pre-screening via 1-way or 2-way SMS can be
connected to CATI software via an API in order to collect the response directly from the CATI software.
E. Quality Check via SMS
Many CATI fieldwork agencies currently check around 10% of their interviews either via listening-in of call recordings or alternatively
via a follow-up call in which multiple questions are asked.
By using either 2-way SMS or 1-way SMS surveys some basic data can be verified automatically thus saving time and cost. During
the fieldwork period mobile phone numbers should be collected (if not already present) and furthermore respondents should be
informed about potential follow-up messages.
13. NAME OF THE CHAPTER
13Brochurename or the title
Conclusion
Researchers have a lot to gain from SMS integration and should move from only using email
follow-ups to integrating SMS, as this would provide for multiple modes of interaction with a
respondent and lead to a higher response rate. Email reading rate is currently below 1% and will
decrease even further while SMS open rates are beyond 90% and read within the first 3 minutes.
Feel free to get in contact with Sample Solutions (info@samplesolutions.eu) or your CATI software
provider to discuss how SMS can be integrated within your current CATI fieldwork processes.
www.sample.solutions