This document provides a sketch of the parameters, variables, and functions for a computer-aided simulation of an online consumer panel. It describes models for calculating key metrics like average response rate and exposure rate. Panelists are assigned attributes like a response rate. The simulation tracks recruits, purges, the eligible universe, sampling, interviewing, and exclusion over iterative tracking periods to reach quotas. Exposure rate models the inability to reach the entire sample at once and is estimated using a goal-seeking method in a sample tracking experiment.

1. Computer-aided
Online Consumer Panel
Simulation
Sketch of First Version

2. Introduction
● Brief description of the parameters, variables
and functions (VP&F)
● Diagrams

3. Calculation of Main Parameters
Universe Baseline
Inputs:
- Desired average
response rate
- Baseline period
Preliminary
assignment of
response rates to
Panelist instances
Average
Response Rate
Equal to desired
one for several
iterations?
No
Yes
Recalculate
again
Outputs:
- Parameters to run
successive
universes so that rr
is constant
Exposure Rate Calculation
Inputs:
- Universe Baseline
- “Typical” project
- Expected outcome
of “Typical” project
Preliminary
sampling/interview
for a typical project
assuming days in
week using an
Exposure rate
Is there a rate
for which we
obtain the
Expected
outcome?
No
Yes
Recalculate
again
Outputs:
- An Exposure Rate
at asymptotic
behaviour

4. Sketch of events
at each tracking
iteration
Recruits
Purges
Excluded
Universe
Eligible
To Sampling
To Interview
Got a status?
Sample
Yes
No
Instance
lifetime
Panelist:
Attribute initialization
and assignment of
response rate
If sample still active
(not end of fieldwork)
and or target hasn’t
been reached...
(incidence, exposure, etc)
(response rate)
End of
Exclusion

5. ● Exploratory demo
○ First overview of model and potential
● Pending: better version with better data...
● More info about project:
○ slideshare.net/evaristoc
○ github.com/evaristoc
Remarks

6. Some Model’s Details

7. VP&F: Panelist; Response Rate
Class Panelist:
● Some attributes included:
○ unique identification (indid)
○ “response rate” assignment
○ status in panel: active or inactive (“purged”)
○ “response statuses” (completed survey, quotafull, screenout, etc)
○ “elimination” period
Response Rate model:
● arbitrary approximation to what have been observed in field
● simple concept (assumes business rules touches/invites); fixed
● assigned to each Panelist randomly following a sketched distribution:
○ Two betas: one for “bad” respondents and another for “good” ones

8. VP&F: Recruits; Purging
Recruits model:
● List of Panelist unique instances with assigned response rate
● Fixed amount per iteration cycle (eg 2000 initializations)
Purging model:
● A priority queue: FIFO and then importance sampling
○ Active universe (see Universe) ordered by entry, the “older” ones
evaluated first
○ Then purging by sampling over response rate: the higher the response
rate, the higher the chance of staying
● Assumes:
○ Older ones leave first but higher response => higher Loyalty
○ Number of Purged = Number of Recruits (i.e. fixed amount)

9. VP&F: Universe, Eligible
Universe model:
● List of active instances at each cycle of Recruits and Purgings
Universe baseline model:
● Used to calculate the parameters associated to Response Rate model,
particularly the relationship between the two beta distributions
○ arbitrary model for demo purposes
○ Calculated using an iterative goal-seeking method
○ Objective is to find consecutive universes which average response
rates stay around a requested value (asymptotic behaviour)
■ universe’s average response rate = (current universe size)-1
* Σ (rr of active instances)
Eligible model:
● The instances available for sampling at each iteration (see Exclusion)
○ It is not purging

10. VP&F: Sampling, Interviewing
Sampling model:
● Sampled instances extracted from Eligible
● Assumes:
○ Quota Sampling: surveying stops if a fixed number of completes
(“target”) is reached
○ Sample size calculation based on business rules and previous
performance (eg. see http://www.slideshare.net/drkellypage/pagek-marketing-researchwk61)
Interviewing model:
● Conditioned Bernoulli rules hierarchy to assign response statuses

11. VP&F: Exclusion
Exclusion (or Elimination) model:
● Temporary inactivation for sampling (“flagging”) of an active Panelist
instance
● Used in real practice to avoid recall bias
● Inactivation period usually defined following best practices rules and based
on last response status
○ In this work, implemented as a reverse counter that updates per
iteration and with Panelist response status as parameter
○ Arbitrary values for demonstration only

12. Tracking model:
● The core of the experiment; complex
● Describes typical operations to reach quota target during a determined
period of time (eg. DAYS of WEEK) called fieldwork
● Assumes:
○ Quota sampling over a specific demographic (eg. “young males”)
○ Each iteration represents a fieldwork; a nested one for days
○ Target reached either with one or more samples per fieldwork
○ Any additional sample during a fieldwork is called “batch” or “extra”
○ Eg. constraints that might enter the formula for sample size calc.:
■ Incidence (measures of the product penetration in universe;
usually given)
■ Exposure (see below)
VP&F: Tracking

13. VP&F: Exposure
Exposure model:
● A proposal for this project
● Measures the inability to reach the whole sample at once; supported by
empirical evidence
● Also a measure of technology effect over survey access
● Hypergeometric-family dist: instances in sample without a response status
will be resampled according to the exposure parameter until sample
turns inactive (in DAYS of fieldwork)
● Assumes:
○ Constant
○ Example of how to estimate it: Using a referential experiment
■ iterative procedure using a goal-seeking method over a simple tracking experiment
■ simple tracking experiment should be a typical one; the one given here is arbitrary