Survey costing is a complex process which balances an organization’s financial objectives against the expenses associated with achieving or maintaining the scientific standards which govern validity and reliability, or quality, of the final product. Achieving optimum balance between budgetary and scientific goals requires that researchers first understand how survey components are related to costs and how changing each influences both data quality and budgetary outcomes.

1. 1
Survey Costing
By Karl Feld, Research Manager
Survey costing is a complex process which balances an organization’s financial
objectives against the expenses associated with achieving or maintaining the scientific
standards which govern validity and reliability, or quality, of the final product.
Achieving optimum balance between budgetary and scientific goals requires that
researchers first understand how survey components are related to costs and how
changing each influences both data quality and budgetary outcomes.
It is important to separate survey costs from survey price, as price includes costs plus
profit. Profit does not necessarily have any relationship to cost other than being added to
it to create total price. As a result, profit and price are omitted from this discussion.
Managerial accounting defines the principles used to develop costs. It differs from
financial accounting in that it is intended to provide insight for better organizational
decision-making. As such, managerial accounting has no standards and its formulas and
data are often unique to an organization.1
Much of what goes into managerial accounting is built on an organization’s operations
and is often proprietary and confidential. However, there are some basic concepts which
are universal to costing in all survey operations. These concepts, and costing principles
derived from them, provide the foundation for any organization to understand and build
its own managerial accounting system for survey costing and budget management.
Understanding them helps researchers understand how their budget and costs are related
to survey quality.
Managerial accounting uses six basic concepts to define costs;
• Cost Object: The thing managers want to know how to cost
• Cost Driver: Any factor that affects costs
• Variable Cost: A cost that changes in proportion to a cost driver
• Fixed Cost: A cost that does not change in total
• Direct Cost: Those costs traced to a Cost Object. These are usually, but not
always, Variable Costs.
• Indirect Cost: Those costs allocated to a Cost Object. These are usually, but not
always, Fixed Costs.
1
To learn more about managerial accounting, see the website for the Institute of Management Accountants
at http://www.imanet.org.
D3 Systems, Inc.
8000 Towers Crescent Drive, Suite 1350, Vienna, VA 22182
phone: 703-255-0884 fax:703-255-6465
www.d3systems.com

2. 2
What constitutes a fixed or variable cost changes by survey mode. What constitutes a
direct or indirect cost can also change by mode, but usually less often. What is important
to remember is that the determination is made by the cost drivers and object(s).
For example, if the cost object is a CATI survey, cost drivers might include the following
list of items.
Some cost drivers are generally constant across all survey modes. General cost drivers
that cross survey modes include the following;
General Cost Drivers
• Number of completed interviews (n size)
• Topic
• Survey length
• Time in field
• Data manipulation
• Validation requirements
• Net Effective Incidence
• Desired response rate
• Analytical Labor
• Travel to client
For a specific survey mode, such as a CATI survey, there is a set of cost drivers that are
constant across different organizations, and include the following;
CATI Cost Drivers
• Length and complexity of CATI programming
• Number of open ends and associated coding
• Desired data outputs and associated reporting
Once the cost drivers have been identified for a particular cost object, a list of direct and
indirect costs associated with those cost drivers can be constructed. These cost categories
will also usually be constant across organizations, although the actual costs associated
with each can be highly variable. Using our CATI example, an organization might see
the following direct and indirect costs;
D3 Systems, Inc.
8000 Towers Crescent Drive, Suite 1350, Vienna, VA 22182
phone: 703-255-0884 fax:703-255-6465
www.d3systems.com

3. Direct Costs
• Interviewer labor
• Telephone long distance
• Production floor management
• CATI programming
• Cost of sample
• Data manipulation
• Manual reports
Indirect Costs
• Telephone line charges
• Management
• Rent
• Insurance
• Technical infrastructure
For many organizations some costs tend to remain fixed regardless of survey size. These could
include upper management, rent, insurance, technical infrastructure, manual reports, and data
manipulation.
A close look at the CATI example will demonstrate that the majority of direct and indirect costs
for this mode are variable costs, meaning that their amount goes up or down based on the size of
the survey. This is primarily due to the fluctuations in interviewer labor required. Interviewer
and floor management labor, telephone long distance and power are just a few of the variable
costs tied to labor. Other variable costs are tied to survey length and n size including cost of
sample and CATI programming.
In contrast, consider an Internet survey. Cost drivers for an Internet survey might include;
• General cost drivers outlined above
• Concurrency/bandwidth
• Data manipulation
• Manual reports
• Length and complexity of programming
It is important to remember that in some cases whether a cost is variable or fixed depends on the
organization’s managerial accounting approach. Similarly, whether a cost is considered direct or
indirect is a function of the accounting approach.
D3 Systems, Inc.
8000 Towers Crescent Drive, Suite 1350, Vienna, VA 22182
phone: 703-255-0884 fax:703-255-6465
www.d3systems.com

4. 4
Rate Calculations
Because of the dominance of variable costs in modes which rely on interviewing labor, formulas
to assist in calculating production accuracy have been developed to predict costs for these
modes. This is particularly true of survey modes which are applied to difficult to reach
populations or with large n sizes. In these cases variable, direct costs can fluctuate tremendously
with the slightest change in specification.
There are three fundamental formulas used to predict cost for surveys which include
interviewing labor. These are net effective incidence rate, respondent cooperation rate, and
production rate.
These three rates often have different names, depending on the country or even which
professional association’s standards are being employed within an organization. It is not the
intent here to suggest that any one association’s or organization’s labels are preferred.
Fortunately, the research industry in the United States has reached a point where the
mathematical formulas for calculating net effective incidence and respondent cooperation rate
are generally agreed upon. Production rates are derived using incidence and respondent
cooperation calculations. They are managerial accounting equations and as such are proprietary
and confidential. A universal standard for CATI surveys is offered below as an example and as a
baseline for any organization or researcher to use to build an accurate production rate.
Incidence is another way of saying frequency of a desired characteristic occurring in some group
of people. It is sometimes expressed as eligibility. There are as many names for incidence as
there are types of research.2
What is important to remember from a costing perspective is that there may be a difference
between incidence of a desired characteristic among the universe and among the selected
population from which the sample frame is drawn. If the sample frame is poorly drawn, there
may yet be another difference between the selected population and the sample frame.
Net effective incidence is one industry term used to mean the percentage of respondents known
to be eligible for the survey in the sample frame, meaning that they have been qualified through
contact.3
2
The term “incidence” is business language used primarily in commercial research, though the concept and
mathematics involved are universal to all forms of survey research.
3
Net effective incidence is the term used by companies whose business language is primarily influenced by the
standards of Marketing Research Association and the Council of American Survey Research Organizations, among
others. Eligibility calculations are the equivalent for those organizations whose business language is driven by the
American Association for Public Opinion Research formulae.
D3 Systems, Inc.
8000 Towers Crescent Drive, Suite 1350, Vienna, VA 22182
phone: 703-255-0884 fax: 703-255-6465
www.d3systems.com

5. 5
Net effective incidence is a critical variable in the production rate equation. It is therefore
important to accurately estimate what the net effective incidence rate will be when estimating
costs. The calculation for actual net effective incidence is effectively a function of multiplying
the percentage of the sample frame eligible for each desired criteria by the percentages for other
desired criteria. This can be expressed as;
Q1 * Q2 * Q3 * Q4 = Net Effective Incidence
It is important to note that for costing purposes net effective incidence is based on actual
eligibility rates of screened respondents only. When building initial estimates it must be based
on estimated or projected eligibility rate of screened respondents. This difference makes
knowing in advance the eligibility differences of the desired universe, population and sample
frame design used important to survey costing.
Net effective incidence can also be used to make estimates of screenings that will be required to
achieve minimum sample sizes by multiplying incidence by minimum sample size desired. This
is sometimes referred to as net incidence. It is a useful calculation to survey costing in predicting
sample and labor requirements for example. It is important to note however that achieved
cooperation rate also influences how much sample and labor is required. Cooperation rate is
discussed further below. The formula for the net incidence calculation, using a prediction of
cooperation rate, would be;
(Q1 * Q2 * Q3 * Q4) * Cooperation Rate = Net Incidence
For those organizations which have response rates as a cost driver, total projected interviewing
labor effort to meet a response rate goal can also be derived from this calculation by building on
net incidence.
“Cooperation Rate” is a term widely used throughout the research industry and has various
definitions based upon which association standards are being used. The use of different
definitions will yield different rates from the same formula, so it is important to note which
definition is being applied in any production formula.4 The definition used here is that of the
proportion of all cases interviewed of all eligible units ever contacted, or the Association for
American Public Opinion Research COOP1 formula expressed as;
4
The Marketing Research Association defines cooperation rate as the percentage of all qualified respondents who
agree to complete an interview. The American Association for Public Opinion Research has four definitions, all
derived from a definition that the rate is the proportion of all cases interviewed of all eligible units ever contacted.
The Council of American Survey Research Organizations, the Advertising Research Foundation and the Council on
Marketing and Opinion Research all use AAPOR definition.
D3 Systems, Inc.
8000 Towers Crescent Drive, Suite 1350, Vienna, VA 22182
phone: 703-255-0884 fax: 703-255-6465
www.d3systems.com

6. 6
COOP15= Interviews
_________________________
(Interviews + Partial Interviews) + Refusals + Other
Where live interviewing is involved, cooperation rates are influenced by the skill of the
interviewing staff, the topic of the survey, the length of the study, and the population being
interviewed. As a result, the rates themselves are unique to the operation conducting the work
and often proprietary. Cost estimates must be built using cooperation rate data from past
performance inside the same operational unit on previous, similar surveys with similar
populations. Once a survey is underway, more accurate projections can be made from the
survey’s own performance for cost control purposes.
Benchmarks are available for some types of general population research. The Council for
Marketing and Opinion Research publishes average industry cooperation rates annually for
CATI research.6
With cooperation rate and net effective incidence in hand, a researcher or organization can
accurately estimate production rate; the holy grail of labor-intensive, variable cost survey modes.
The values for variables in production rate formulas are highly unique to individual production
operations. They are based on management effectiveness, labor quality, operational standards,
quality control standards, even weather. They are also closely guarded to maintain
competitiveness.
However, the variables themselves in the production rate formula are universal. Below is a
CATI production rate formula as an example.
(60 minutes-B)
L + S ((1/I)-1) + W/(C*I) + X
B = break, brief, coach time in minutes (any idle time)
L = length of one survey in minutes (including screener)
S = average time to screen per survey in minutes
W = wait time between contacts in minutes
C = cooperation rate as %
I = net effective incidence as %
X = wrap time per survey in minutes
The nature of the variable labels make clear the degree to which operational and management
factors influence survey costing as much as survey design in projects using live interviewers.
5
Full definitions of all terms in the formula can be obtained at http://www.aapor.org.
6
For more information see http://www.cmor.org.
D3 Systems, Inc.
8000 Towers Crescent Drive, Suite 1350, Vienna, VA 22182
phone: 703-255-0884 fax: 703-255-6465
www.d3systems.com

7. 7
Break and coaching time are functions of operations management. Wait time is driven by
technology. Survey length is a function of design.
Production rates are designed to derive a number of completed surveys per hour (or in some
cases a number of hours per completed survey). Having a solid estimate or mid-term projection
of this figure allows for accurate cost estimating for variable data collection costs. Even in cases
where live interviewing labor is not required, one or more rates are often an appropriate method
for estimating some variable, direct costs (eg. cooperation rates for self-administered surveys via
mail or Internet to project mail out rate to receive sufficient returns on schedule).
There are live interviewing environments where these formulae need be modified or even
eliminated. Some countries of the world pay CATI or face-to-face interviews by the piece,
rather than per hour. In these cases net effective incidence and cooperation rate may become less
important cost drivers, affecting direct variable cost of management labor and sample acquisition
for example, but not interviewing labor. The use of these formulae, as with all other costing
calculations, is dependent on the cost drivers involved.
Estimating and Controlling Costs
Derived production rates per hour allow organizations and researchers to project estimated or
actual variable, direct costs for surveys. To turn those hours into a budget requires the
application of costs per hour. How those costs per hour are derived is as variable as there are
managerial accounting systems.
Fixed and indirect costs are added as line items to variable and direct costs to build an entire
budget. In others they are allocated by the hour. In some cases these costs are estimated at their
actual usage by a specific project. In some cases, they are allocated proportionally to entire labor
hours. In yet other cases, they are derived from estimated company expenses which are included
in a markup ratio to project expense.
A researcher needs to understand how their organization’s accounting rules affect the way their
costing is designed in order to control and modify their budgets most effectively. As a study
progresses, estimated rates and the budget built on them can be replaced with actual rates to
project final costs. When a budget shortfall is projected the researcher can then dig into all the
cost drivers that go into the costing equation to find ways to realign costs while balancing quality
considerations.
Some common approaches to budget realignment include;
• Increase field time, which can boost cooperation rates and boost production.
• Decrease total n size, screening criteria or response rate goals to reduce labor and sample
required
• Change the proportionality of samples which are not reflective of the population’s natural
characteristics, increasing net effective incidence.
D3 Systems, Inc.
8000 Towers Crescent Drive, Suite 1350, Vienna, VA 22182
phone: 703-255-0884 fax: 703-255-6465
www.d3systems.com

8. 8
• Reduce production quality control standards, reducing floor management labor costs
• Move work to an operation with less expensive costs
• Incentivize labor to produce at a more rapid rate
• Renegotiate price with operations supplier
Net effective incidence deserves special note here. The relationship between incidence and
production rate is not linear, but rather curved. Low incidences have much lower rate of
completion per hour than high incidences. As a result, the most effective way to increase
production rates in low incidence studies is to eliminate screener questions.
Of course, eliminating screeners, as well as most of the other options listed above have direct and
significant impacts on survey validity and reliability. The researcher must use his or her best
judgment and understanding of how costs are derived and controlled to design and manage cost
effective yet scientifically useful research.
Karl G. Feld, D3 Systems
Cross-References: Cooperation Rate, Eligibility, Interviewer Productivity, Mode of Data
Collection, Quality Control, Research Management, Total Design Method
Further Readings and References:
American Association for Public Opinion Research (2004). Standard Definitions: Final
Dispositions of Case Codes and Outcomes Rates for Surveys. Lenexa, KS: AAPOR.
Dawson, K. (2004). The Call Center Handbook. San Francisco, CA: CMP Books.
Council of American Survey Research Organizations/Marketing Research Association (1995).
Incidence Guidelines. Port Jefferson, NY and Rocky Hull, CT: MRA-CASRO.
Feld, K. and Haynes, D. (October 18, 2004). Costing Quantitative Live Interview and Web Data
Collection. Short course taught for the Marketing Research Association, Richmond, VA.
D3 Systems, Inc.
8000 Towers Crescent Drive, Suite 1350, Vienna, VA 22182
phone: 703-255-0884 fax: 703-255-6465
www.d3systems.com