(RIYA)🎄Airhostess Call Girl Jaipur Call Now 8445551418 Premium Collection Of ...
10 information bias
1. 3/22/2011 Sources of error: Information bias 1
Sources of error: Information bias
Principles of Epidemiology for Public Health (EPID600)
Victor J. Schoenbach, PhD home page
Department of Epidemiology
Gillings School of Global Public Health
University of North Carolina at Chapel Hill
www.unc.edu/epid600/
2. 11/5/2001 Sources of error: Information bias 2
Abort, Retry, Fail
“Tips for safer drives: Never turn off a
PC or accessories while the computer
is on or the disk is active.”
— USA Today
[PC Magazine, 10/3/1996]
3. Chapter 1
THE HISTORIAN'S TASK:
Insight into the future
History, a record of things left behind
by past generations, started in 1815.
Thus we should try to view historical
times as the behind of the present.
Anders Henriiksson (ed), Non Campus Mentis, NY,
Workman Publishing Co., 2003
4. Non Campus Mentis
“History, as we know, is always bias,
because human beings have to be
studied by other human beings, not by
independent observers of another
species.”
Anders Henriiksson (ed), Non Campus Mentis, NY,
Workman Publishing Co., 2003, chapter 1
5. 3/22/2011 Sources of error: Information bias 5
Information bias
Information bias: a systematic distortion
or error that arises from the procedures
used for classification or measurement
of the disease, the exposure, or other
relevant variables.
6. 3/22/2011 Sources of error: Information bias 6
Information bias
• Classification or measurement
• Differential or nondifferential bias
• Direction of bias
• Misclassification of covariables
7. 3/22/2011 Sources of error: Information bias 7
Classification or measurement
• Data for epidemiologic studies consist
of classifications (e.g., “hypertensive” vs.
“normotensive”) or measurements (e.g., 120
mmHg systolic BP).
• Possible sources of measurement or
classification error include instrumentation,
laboratories, records, respondents; data
collectors, managers, analysts, and
interpreters.
8. 3/22/2011 Sources of error: Information bias 8
Sources of measurement error
Respondent (interview, questionnaire):
• inability to understand, recall, articulate;
• unwillingness to disclose
• social desirability influences
Can be influenced by wording of
questions and how they are asked.
9. 9
Example of misunderstanding
• Medico – Não consigo encontrar o
motivo das suas dores, meu caro. Só
pode ser por causa da bebida.
• Paciente – Não tem importãncia,
doutor. Eu volto outro dia que o senhor
estiver sóbrio.
De Luciana V. Paiva, Osasco - SP, en Bom Humor Nosso E Dos Leitores”,
Almanaque Brasil de Cultura Popular. Maio 2001;3(26)
(almanaquebrasil@uol.com.br). Exemplar de quem viaja TAM.
10. 3/22/2011 Sources of error: Information bias 10
How not to ask questions
“Has anyone ever tried to give you the
mistaken idea that sex intercourse is
necessary for the health of the young man?
(from a survey by the NC state health officer, circa 1926,
summarized in Kinsey et al., 1948)
Can you guess the right answer?
11. 3/27/2007 Sources of error: Information bias 11
Respondent cognitive processes
• Respondent cognitive processes: interpretation,
recall, judgment formation, response formatting,
editing
• Qualitative research on response processes, e.g.:
• “What types of physical activity or exercise did you
perform during the past month?”
• “What did you think we meant when we said ‘physical
activity’?”
• “Which, if any, of the following would you (also)
consider to be physical activity?
www.minority.unc.edu/institute/2000/materials/slides/RichardWarnecke-2000-06-08.ppt
12. 3/27/2007 Sources of error: Information bias 12
Cognitive testing - 2
Recalling and retrieving – Retrieval probes:
• Recall strategy
• Recall interval
• Search strategy (proximal, distal, anywhere)
• Long term recall - link to events to help
remember
• Recall frame of reference--what kinds of
things helped you remember?
www.minority.unc.edu/institute/2000/materials/slides/RichardWarnecke-2000-06-08.ppt
13. 3/22/2011 Sources of error: Information bias 13
Surveys and Questionnaires
• Survey validation
• Pretesting (wording, item sequence, time)
• Pilot testing (all steps - procedure, item
performance)
• Translation validation
Sources of error: Information bias 13
14. 3/27/2007 Sources of error: Information bias 14
Sources of measurement error
• Data collector: unclear or ambiguous
questions, lack of a neutral demeanor,
insufficiently conscientious, inaccurate
transcription, fraud
15. 11/5/2001 Sources of error: Information bias 15
Sources of measurement error
• Data managers: inaccurate transcription,
mis-reading, miscoding, programming
errors
• Data analysts: variable coding and
programming errors
• Data interpreters: inadequate
appreciation of the characteristics of the
measure or of the relations being studied
16. 11/5/2001 Sources of error: Information bias 16
Techniques for avoiding data collection errors
• Precise operational definitions of variables
• Detailed measurement protocols
• Repeated measurements on key variables
• Training, certification, and re-certification
• Data audits (of interviewers, of data centers)
• Data cleaning – visual, computer
• Re-running all analyses prior to publication
17. 11/5/2009 Sources of error: Information bias 17
Validation and agreement
• Sensitivity and specificity – used to evaluate
classifications
• When no validation standard, we measure
agreement
• Measures of agreement often correct for
“chance”
18. 11/5/2001 Sources of error: Information bias 18
Information bias – differential
or non-differential
• Important question for any kind of bias –
are error processes different for groups
being compared
• If no, “non-differential”
• If yes, “differential”
• Has implications for direction of bias
• In general, non-differential is safer
19. 3/22/2011 Sources of error: Information bias 19
Direction of bias
• “Upward”
• “Downward”
• “Towards the null”
• “Away from the null”
Null = 0 (for differences)
1 (for ratios)
20. 11/5/2001 Sources of error: Information bias 20
Direction of bias
In simple situation, information bias is
towards the null IF:
1. Dichotomous exposure and disease
2. Non-differential misclassification with
both sensitivity and specificity each
greater than 0.5; AND
3. Errors in one variable are independent
of errors in the other
21. 11/5/2001 Sources of error: Information bias 21
Errors in covariables
• It is almost always important to control for
other variables (e.g., age)
• Errors in measurement of these variables
hamper attempts to control for them
• Direction of bias is generally unpredictable
22. 3/22/2011 Sources of error: Information bias 22
Classic studies – 6 degrees of separation?
• Classic experiment by Yale psychologist Stanley
Milgram asked people in Kansas to forward a
letter to a target person in Massachusetts
• If did not know target person, then send it to
someone they thought might know him
• Milgram’s 1967 paper reported that it only took 5
jumps, on average, for letters to arrive
23. 3/29/2005 Sources of error: Information bias 23
Selection bias in a classic study
According to Judith Kleinfeld,
psychologist at the University of Alaska,
Fairbanks, archives reveal that only
30% of the letters actually reached their
destination!
(Gewolb, Josh. Random samples. Science 26
October 2001;294:777. See Kleinfeld, Judith S.
Society. Jan/Feb2002; 39(2):61-66)
24. 24
Dr. Kinsey and the Institute for Sex Research
Alfred S. Kinsey (photograph from Wardell B. Pomeroy,
Dr. Kinsey and the Institute for Sex Research)
25. 25
Sex research in the mid-20th
century
Alfred S. Kinsey (photograph from Wardell B. Pomeroy,
Dr. Kinsey and the Institute for Sex Research)
26. 11/7/2005 26
Kinsey et al. on selection and information bias
(Alfred C. Kinsey, Wardell B. Pomeroy, Clyde E.
Martin. Sexual behavior in the human male, Phila,
W.B. Saunders, 1948)
Alfred S. Kinsey (photograph from Wardell B. Pomeroy,
Dr. Kinsey and the Institute for Sex Research)
28. 3/22/2011 Sources of error: Information bias 28
Real-life example: Quit for Life
• Randomized trial of smoking cessation
interventions
• Self-reported “In the past 7 days, have you
smoked a cigarette, even a puff?”
• Attempted (unsuccessfully) to validate with
saliva cotinine
• People who did not give a time to be called
for validation had very high quit rates!
29. 29
Kinsey biography by Wardell Pomeroy
Pomeroy, Wardell B. Dr. Kinsey and the Institute for Sex
Research NY, Signet / New American Library, 1972: p136
30. Non Campus Mentis
“Hindsight, after all, is caused by a
lack of foresight.”
Anders Henriiksson (ed), Non Campus Mentis, NY,
Workman Publishing Co., 2003, chapter 1
Here’s some good advice from USA Today, courtesy of the late PC Magazine’s Abort, Retry, Fail:
“Tips for safer drives: Never turn off a PC or accessories while the computer is on or the disk is active.”
— USA Today [PC Magazine, 10/3/1996]
And now a few selections from Anders Henriiksson (ed), Non Campus Mentis, NY: Workman Publishing Co., 2003
From Chapter 1 THE HISTORIAN'S TASK:Insight into the future
“History, a record of things left behind by past generations, started in 1815. Thus we should try to view historical times as the behind of the present.”
Come to think of it, we do save a lot of time by starting in 1815.
And:
“History, as we know, is always bias, because human beings have to be studied by other human beings, not by independent observers of another species.”
Anders Henriiksson (ed), Non Campus Mentis, NY: Workman Publishing Co., 2003, chapter 1
There’s actually quite a bit of truth to that observation.
So now we turn to information bias. Information bias refers to a systematic distortion or error that arises from the procedures used for classification or measurement of the disease, the exposure, or other relevant variables.
Our overview of information bias is organized into four subtopics: classification or measurement, differential or non-differential bias, the direction of bias, misclassification of a variable other than the disease or primary exposure (a “covariable”). I’ll spend most of the time on the first one.
Data for epidemiologic studies consist of classifications, such as “hypertensive” vs. “normotensive”, or measurements, such as 120 mm Hg systolic blood pressure.
Any data source or means of data collection or transmission can be a source of error. So possible sources of measurement or classification error include instrumentation, such as sphygmomanometers, laboratory assays, medical records, interview responses, data collectors, data managers, data analysts, and data interpreters.
A great deal of epidemiologic data comes from respondents, either directly through interviews and questionnaires, or indirectly, through information provided by respondents during clinical encounters and recorded in the medical record. Respondents may misunderstand questions, may be unable to recall the requested information, may be unable to articulate the information precisely or understandably, may be unwilling to disclose the information, or may give a response they regard as more socially desirable than the unvarnished truth.
Poor quality or wrong information from respondents, whether from interviews, self-administered questionnaires, or audio - computer assisted self interviewing (A-CASI) is at least partly a function of the quality of the questions and the way they are asked.
Or, consider the following question from a study by the state health officer of North Carolina, published in 1926 and summarized in the first of the Kinsey Reports):
“Has anyone ever tried to give you the mistaken idea that sex intercourse is necessary for the health of the young man?”
(Hughes, W. L. 1926. Sex experiences of boyhood. J. Soc. Hyg. 12:262-273, summarized in Alfred C. Kinsey, Wardell B. Pomeroy, Clyde E. Martin. Sexual behavior in the human male, Phila, W.B. Saunders, 1948, p.26)
How would you respond?
Cognitive testing is a methodology for exploring how people interpret and respond to survey questions. Richard Warnecke gave an overview of some of the cognitive processes involved in a presentation for the Annual Summer Public Health Research Videoconference on Minority Health (his abstract, slides, and references are available at www.minority.unc.edu/institute/2000/materials/abstracts.htm#Richard ).
In his presentation, Warnecke identified five distinct steps in the process through which a respondent answers a question: interpretation, recall, judgment formation, response formatting, response editing. He also explained how to use qualitative research techniques to gain insight into how respondents carry out each of these steps. I’ll illustrate the first two steps.
For example, for the question “What types of physical activity or exercise did you perform during the past month?”, the researcher could ask a small group of people questions like:
“What did you think we meant when we said ‘physical activity’?”
and
“Which, if any, of the following would you (also) consider to be physical activity? what about . . . walking, housework, such as cleaning the bathroom or kitchen, work-related activity, such as construction, yard work, such as mowing the lawn or raking leaves?”
Once the respondent has interpreted (decoded) the question, the second step is to recall the requested information. The recall process involves retrieving relevant information from memory. To study the process of recalling and retrieving, the investigator can ask respondents to describe how they went about recalling the information for the answer. Probes would attempt to understand the strategy used, the interval over which recall was attempted (especially if the interval was not specified in the question), the search strategy (e.g., start in the present and move backward, start in the past and move forward, or try to think of events anywhere in the past), links to events to boost long term recall, and frames of references employed.
Investment in qualitative research for constructing the questions will improve the accuracy and consistency of the data and improve their interpretation.
If a survey instrument is used to collect information about the exposure, outcome, or other covariates, the questionnaire is typically reviewed several times to ensure accuracy, completeness, and “face validity” (i.e., the questions appear to be valid). Surveys are reviewed by knowledgeable colleagues and then pretested on a small group of participants to ensure that the questions are being interpreted correctly and estimate the length of the questionnaire. The questionnaire should then be pilot-tested to ensure that it works as it will be used in the field. Responses can be analyzed to get a preview of how well the items are working and discard or reformulate ones that are not useful (e.g., no variability).
As research becomes more global, often surveys need to be translated into other languages. In order to maintain the integrity of the survey (and ensure cultural appropriateness and linguistic correctness), the validation process may be repeated in the translated version. The translated version should also be translated back to the original language (called “back-translation”) to verify the accuracy of original translation.
Meanwhile, the data collector, e.g., an interviewer, may ask questions that are unclear or ambiguous, may lack a neutral demeanor thereby encouraging a socially-desirable response or discouraging disclosure, may be insufficiently conscientious in following the protocol, may transcribe information inaccurately, or may engage in outright fraudulent behavior. The last of these may come as a surprise to many of you. But the sad truth is that though most interviewers are conscientious and honest, as in so many areas there are people who are highly susceptible to the temptation to take shortcuts. It is much easier to fill out questionnaires in the comfort of one’s living room than to drive to people’s houses and try to get them to answer the questions or try to reach them on the telephone. It is therefore standard procedure to verify a percentage of interviews to detect such shortcuts. In one study with which I was familiar, a student research assistant was caught manufacturing data, and in another a survey contractor reported that one of their telephone interviewers was caught manufacturing telephone interviews.
Measurement error can come at any place where data are generated, edited, managed, coded, or interpreted. Instrumentation, for example, can be functioning in an unreliable or erroneous fashion. During data management, errors can arise through inaccurate transcription (“keying”), misreading, miscoding, and programming errors.
Data analysts can introduce errors through variable coding and programming (I once destroyed the meaning of a variable through a faulty programming recode in constructing an epidemiology final exam [not for EPID600 or EPID710, which I used to teach, I’m relieved to say; though perhaps this is why I’ve never taught EPID715!], the error presumably did not harm the exam and was fortunately detected before the data were used as a case-example in a professional presentation, but who knows what other data management errors I have made that I have not discovered). (No one is completely immune: see, for example, www.cdc.gov/nchs/data/nsfg/Cyc7MaleLSEXRLTN_errata.pdf ).
Data interpretation can be compromised by inadequate appreciation of the characteristics of the measure or of the relations being studied. In some circumstances misinterpretation can be regarded as information bias. For example, studies of racial/ethnic disparities often find that disparities remain even after controlling for socioeconomic status, prompting biological hypotheses. But problems such as the inadequacy of socioeconomic status measurement make such reasoning faulty. (E.g., see Kaufman JS, Cooper RS, McGee DL, Epidemiology 1997;8(6):621-628.)
Epidemiologic research uses various techniques to avoid or at least minimize these myriad sources of error. Such techniques include the use of precise, operational definitions for all study variables, use of detailed, written measurement protocols (for an elaborate example, see www.cscc.unc.edu/aric/), the practice of making repeated measurements on key variables; training, certification, and re-certification for data collectors; audits of data collected by interviewers and by data centers in a multi-center study; data cleaning by visual review and by computer checks; and independently re-running all analyses prior to publishing results.
Earlier in the course we described the use of sensitivity and specificity to evaluate the accuracy of a screening test. These same measures are used to evaluate the accuracy of classifications for other purposes
When there is no validation standard (often called a “gold standard”) then we can assess the classification only in regard to agreement – how often the two measurements on the same person agree – rather than for accuracy. Some agreement can occur by chance, of course, so we often use a measure of agreement, such as the kappa statistic, that adjusts for the amount of agreement expected by chance. Because many conditions studied by epidemiologists are rare, we would often expect a great deal of agreement to occur by chance.
The concept of whether errors are differential or non-differential in respect to groups being compared arises for both selection bias and information bias. The question concerns whether the processes that create the errors operate differently for the different groups being compared. If the processes do not operate differently, then the error is termed “nondifferential”. If the error processes do operate differently, then the error is termed “differential”.
For the most part, selection bias arises when selection processes (e.g., nonresponse) are differential. Information bias, in contrast, often occurs whether the error processes operate in the same way in all groups or not. But whether errors are differential or nondifferential may have implications for the direction of the bias that may result. For example, misclassification is nondifferential if the sensitivity (and specificity) for disease detection is the same in the exposed and unexposed groups and the sensitivity (and specificity) for classification of exposure is the same for cases and non-cases. (Note: “Nondifferential” does not mean that sensitivity equals specificity, as people sometimes assume.)
In general, nondifferential errors are somewhat less problematic, and one objective of study design and conduct is to make sure that where errors cannot be prevented at least they should be nondifferential. What masking of data collectors, laboratory technicians, and investigators achieves is to ensure that data errors are nondifferential.
In the selection bias lecture we talked about the different terms for describing the direction of a bias. Most selection biases are best characterized as “upward” or “downward.” It is for information bias that the terms “towards the null” and “away from the null” become relevant. Recall that the null value of a ratio measure of association is 1.0. If information bias causes the observed association to be closer to 1.0 than the true association, regardless of whether the true ratio is below 1.0 or above 1.0, then the situation is termed “bias towards the null”. Conversely, if the observed association is farther from 1.0 than is the true association, regardless of whether the true association is below 1.0 or above 1.0, then the situation is termed “bias away from the null”. The distinction between “negative bias” and “bias towards the null” is that, in principle, negative bias could make a truly positive association appear negative, whereas bias towards the null could conceal a truly positive association but could not reverse it (i.e., move the observed association “through the null”).
Since scientists prefer to be conservative (and especially prefer that other scientists be conservative), bias away from the null is generally regarded as a greater threat. If we can be certain that bias is towards the null, then when we find an association we can take comfort in the implication that the true effect must be stronger than the one we are reporting.
It turns out that in a somewhat simple situation, but one that does arise, we can be reasonably confident that information bias will be towards the null. This situation involves (1) a dichotomous (i.e., “yes” vs. “no”) disease and exposure variables, (2) misclassification that is nondifferential – errors in the classification of disease are not affected by the value of the exposure, and errors in the classification of exposure are not affected by the value of the disease variable – and sensitivity and specificity are both better than 50%, and (3) error processes that are independent – an error in measuring exposure is no more likely to occur if there is an error in measuring disease, and vice-versa. In this somewhat special situation, it can be shown that the observed ratio measure of association will be closer to 1.0 than the measure would have been if there were no bias. Since this influence will occur whether the true association is less than 1.0 or greater than 1.0, “towards the null” is the best way to characterize the direction of bias.
It is thus comforting to conclude that errors are nondifferential and independent, since the implication is that an observed association reflects a stronger one in the population. However, that conclusion is probably wishful thinking more often then we would like to believe. For example, if both disease and exposure are measured by interview, a bad interviewer will tend to introduce errors in both exposure and disease, so that errors processes will not be independent.
One final topic is errors in covariables – variables other than the disease and primary exposure. Epidemiologic studies almost always control for other variables, such as age, gender, education, and other risk factors for the disease being studied. Measurement of these variables is also susceptible to error, of course. Errors in measurement and classification of control variables hamper our attempts to control for them. The direction of information bias arising from errors in classification and measurement of control variables is generally unpredictable.
Information bias is probably one of the greatest challenges in conducting epidemiologic research.
Now I’ll offer some real-life examples of selection and information bias.
As I was writing the first version of this lecture I happened across an article that had just appeared in Science magazine. The article reported a re-analysis of data from a classic experiment by Yale psychologist Stanley Milgram. That experiment recruited a set of people in Kansas, gave each a letter, and asked the person to please forward the letter to a target person in Massachusetts. If the participant did not know the target person, as was generally the case, the instruction was to forward it to someone s/he thought might know the target person. In the paper, Milgram and colleagues report that it took only 5 jumps, on average, for the letters to arrive to the target people (this was the pre-Internet era). The finding generated considerable popular interest, since it reinforced the notion that we live in a “small world” in which people are basically connected.
According to the article in the October 2001 issue of Science, however, Judith Kleinfeld, a psychologist at the University of Alaska in Fairbanks, found from the archives that only 30% of the letters actually reached their destination. The remaining 70% had simply never arrived. So the “average of 5 jumps” was based on the highly selected subsample that had reached their destination, and so “5 jumps” greatly overstates the extent of connectedness. To an epidemiologist, this situation is a rather severe case of selection bias.
Gewolb, Josh. Random samples. Science 26 October 2001;294:777
Kleinfeld, Judith S. Society. Jan/Feb2002; 39(2):61-66
(For much more on what Judith Kleinfeld found in the Milgram archives, see her webpage at http://www.judithkleinfeld.com/ar_bigworld.html)
An area where I’ve carried some of my own research is sexual behavior, so I thought it would be interesting to examine some of the pioneering work in the area. Alfred Kinsey and his colleagues were true pioneers in the scientific study of sexuality in the U.S. While there is much to criticize from our present-day vantage point – and they were aware of many of the shortcomings of the methodology they used – the Kinsey Reports are a landmark in the scientific study of human sexuality and helped to pave the way for all that has come since its appearance. The magnitude of the contribution of the Kinsey studies is illustrated by the fact that when the AIDS epidemic appeared about 30 years after their publication, public health officials were having to rely on the Kinsey data for estimates of the prevalence of male homosexuality in the U.S.
In evaluating the Kinsey studies, it is important to keep in mind the social climate of the era in which these researchers worked. As they describe it in the Male volume, “There seem to have been two chief sources of these objections. Some of the psychologists contended that sexual behavior involved primarily psychological problems, and that no biologist was qualified to make such a study. Some of the sociologists felt that the problems were for the most part social, and that neither a biologist nor a psychologist was the right person to make a sex study. A few of the psychoanalysts felt that sexual behavior could not properly be studied by anyone but a psychoanalyst. One group of physicians objected that taking histories constituted clinical practice, and that all such studies should be made by clinicians inside of clinics.” (page 12)
The Kinsey researchers were quite aware of the hazards of selection bias. They adopted an interesting technique that they referred to as “hundred percent samples”:
“Since it is impossible to secure a strictly randomized sample, the best substitute is to secure one hundred percent of the persons in each social unit from which the sample is drawn. One hundred percent of the members of a family group, all the persons living in a particular apartment house, all the members of a college sorority or fraternity, all the persons in some service club, all the members of some Sunday School class or some other church organization, all the persons in a city block, all the persons in a rural township, all the inmates of some penal or other institution, all the persons in some other unit, provided that unit has not been brought together by a common sexual interest.”
Sexual behavior in the human male, p. 93
Other parts of the volume demonstrate their great attention to avoiding information bias as well. For example, training interviewers included putting the subject at ease, ensuring privacy, establishing rapport, carefully sequencing topics, recording information during the interview, using a standardized outline, asking direct questions without hesitation or bias, placing the burden of denial on the subject, avoiding compound questions, employing rapid-fire questioning with cross-checks for consistency, and avoiding any discussion of non-sexual controversial issues such as politics, religion, etc. (pp.47-58)
Before I began studying sexual behavior, I worked on two randomized trials to help people quit smoking. In one of these studies it appeared that information bias was a concern for the key outcome, quitting smoking. The study, called Quit or Life, was carried out in the late 1980s among policyholders of the North Carolina Mutual Life Insurance Company, headquartered in Durham, NC. The study recruited about 2,000 policyholders in 32 NC Mutual sales districts. These were people who said that they wanted to quit smoking and planned to quit in the next year.
The study was a randomized trial of self-help methods for quitting smoking. Agents in half of the districts presented a quit smoking packet to the participants, and agents in all 32 districts collected follow-up questionnaires 4 and 8 months later. The primary outcome variable was quitting status, as assessed by the standard item “In the past 7 days, have you smoked a cigarette, even a puff”.
We attempted to validate answers by collecting saliva specimens to test for levels of cotinine. Cotinine is a metabolic product of nicotine and a reasonably good indicator of whether someone has been exposed to nicotine, especially as an active smoker. However, we were unsuccessful in obtaining specimens from most self-reported quitters.
The questionnaire did ask, though, when would be a good time to schedule an appointment to collect a saliva specimen if we needed to do so. Most people answered the question by choosing “evenings” or “weekends”, but some people either wrote in “none” or “don’t call” or they left the question unanswered. Participants who did not provide a time to be called turned out to have very high reported rates of quitting, suggesting the possibility of information bias in their quitting status – or an unsuspected predictor of quitting!
(The Quit for Life study is described at http://www.epidemiolog.net/pub/qfl/ )
Finally, I thought that you might enjoy this anecdote from one of the co-authors of the two main Kinsey Reports, Sexual behavior in the human male and Sexual behavior in the human female:
“Early in the research I went to a cocktail party in New York with Kinsey, and a dozen or so psychiatrists were among the guests, accompanied by their wives or mistresses, as the case might be. We resolved to obtain the sexual histories of everyone there. I was only thirty years old at the time, and viewed the prospect with some anxiety, since I was a little overawed by the company.
“Spotting a pretty young girl in her twenties, I concluded that it would be easier for me to ask for her history than to confront one of the older and more formidable psychiatrists. Employing everything Kinsey had taught me, I appealed for her cooperation, citing the scientific approach, the anonymity and the need for the information we were collecting. Everyone at the party was going to help us, I told her.
“She listened attentively to everything I said, and when I paused she smiled sweetly and remarked, ‘But, Mr. Pomeroy, it’s been less than two weeks since you took my history.’
“Making a rapid recovery, I assured her, ‘Well, you see, there's the proof - we not only forget the histories after we record them, but we forget the people, too.’ I hoped she found this reassuring, if not flattering.”
Pomeroy, Wardell B. Dr. Kinsey and the Institute for Sex Research NY, Signet / New American Library, 1972: p136
And to close, here is another student answer that is humorous but also insightful.
![11/5/2001 Sources of error: Information bias 2
Abort, Retry, Fail
“Tips for safer drives: Never turn off a
PC or accessories while the computer
is on or the disk is active.”
— USA Today
[PC Magazine, 10/3/1996]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)