Download to read offline
![CHAPTER 1 SOURCES OF ERROR 3
Don’t think—use the computer. Dyke (tongue in cheek) [1997].
We cannot help remarking that it is very surprising that research
in an area that depends so heavily on statistical methods has
not been carried out in close collaboration with professional
statisticians, the panel remarked in its conclusions. From the
report of an independent panel looking into “Climategate.”1
STATISTICAL PROCEDURES FOR HYPOTHESIS TESTING,
ESTIMATION, AND MODEL building are only a part of the decision-
making process. They should never be quoted as the sole basis for making
a decision (yes, even those procedures that are based on a solid deductive
mathematical foundation). As philosophers have known for centuries,
extrapolation from a sample or samples to a larger, incompletely examined
population must entail a leap of faith.
The sources of error in applying statistical procedures are legion and
include all of the following:
1. a) Replying on erroneous reports to help formulate hypotheses
(see Chapter 9)
b) Failing to express qualitative hypotheses in quantitative form
(see Chapter 2)
c) Using the same set of data both to formulate hypotheses and
to test them (see Chapter 2)
Chapter 1
Sources of Error
Common Errors in Statistics (and How to Avoid Them), Fourth Edition.
Phillip I. Good and James W. Hardin.
© 2012 John Wiley & Sons, Inc. Published 2012 by John Wiley & Sons, Inc.
1
This is from an inquiry at the University of East Anglia headed by Lord Oxburgh. The
inquiry was the result of emails from climate scientists being released to the public.](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-19-2048.jpg)
![8 PART I FOUNDATIONS
Sample
A sample is any (proper) subset of a population. Small samples may give a
distorted view of the population. For example, if a minority group
comprises 10% or less of a population, a jury of 12 persons selected at
random from that population fails to contain any members of that
minority at least 28% of the time.
As a sample grows larger, or as we combine more clusters within a
single sample, the sample will grow to more closely resemble the
population from which it is drawn.
How large a sample must be to obtain a sufficient degree of closeness
will depend upon the manner in which the sample is chosen from the
population.
Are the elements of the sample drawn at random, so that each unit in
the population has an equal probability of being selected? Are the
elements of the sample drawn independently of one another? If either of
these criteria is not satisfied, then even a very large sample may bear little
or no relation to the population from which it was drawn.
An obvious example is the use of recruits from a Marine boot camp as
representatives of the population as a whole or even as representatives of
all Marines. In fact, any group or cluster of individuals who live, work,
study, or pray together may fail to be representative for any or all of the
following reasons (Cummings and Koepsell, 2002):
1. Shared exposure to the same physical or social environment;
2. Self selection in belonging to the group;
3. Sharing of behaviors, ideas, or diseases among members of the
group.
A sample consisting of the first few animals to be removed from a cage
will not satisfy these criteria either, because, depending on how we grab,
we are more likely to select more active or more passive animals. Activity
tends to be associated with higher levels of corticosteroids, and
corticosteroids are associated with virtually every body function.
Sample bias is a danger in every research field. For example, Bothun
[1998] documents the many factors that can bias sample selection in
astronomical research.
To prevent sample bias in your studies, before you begin determine all
the factors that can affect the study outcome (gender and lifestyle, for
example). Subdivide the population into strata (males, females, city
dwellers, farmers) and then draw separate samples from each stratum.
Ideally, you would assign a random number to each member of the
stratum and let a computer’s random number generator determine which
members are to be included in the sample.](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-24-2048.jpg)
![CHAPTER 1 SOURCES OF ERROR 9
SURVEYS AND LONG-TERM STUDIES
Being selected at random does not mean that an individual will be willing
to participate in a public opinion poll or some other survey. But if survey
results are to be representative of the population at large, then pollsters
must find some way to interview nonresponders as well. This difficulty is
exacerbated in long-term studies, as subjects fail to return for follow-up
appointments and move without leaving a forwarding address. Again, if
the sample results are to be representative, some way must be found to
report on subsamples of the nonresponders and the dropouts.
AD-HOC, POST-HOC HYPOTHESES
Formulate and write down your hypotheses before you examine the data.
Patterns in data can suggest, but cannot confirm, hypotheses unless these
hypotheses were formulated before the data were collected.
Everywhere we look, there are patterns. In fact, the harder we look the
more patterns we see. Three rock stars die in a given year. Fold the
United States twenty-dollar bill in just the right way and not only the
Pentagon but the Twin Towers in flames are revealed.3
It is natural for us
to want to attribute some underlying cause to these patterns, but those
who have studied the laws of probability tell us that more often than not
patterns are simply the result of random events.
Put another way, finding at least one cluster of events in time or in
space has a greater probability than finding no clusters at all (equally
spaced events).
How can we determine whether an observed association represents an
underlying cause-and-effect relationship or is merely the result of chance?
The answer lies in our research protocol. When we set out to test a
specific hypothesis, the probability of a specific event is predetermined. But
when we uncover an apparent association, one that may well have arisen
purely by chance, we cannot be sure of the association’s validity until we
conduct a second set of controlled trials.
In the International Study of Infarct Survival [1988], patients born
under the Gemini or Libra astrological birth signs did not survive as long
when their treatment included aspirin. By contrast, aspirin offered apparent
beneficial effects (longer survival time) to study participants from all other
astrological birth signs. Szydloa et al. [2010] report similar spurious
correlations when hypothesis are formulated with the data in hand.
3
A website with pictures is located at http://www.foldmoney.com/.](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-25-2048.jpg)
![10 PART I FOUNDATIONS
Except for those who guide their lives by the stars, there is no hidden
meaning or conspiracy in this result. When we describe a test as significant
at the 5% or one-in-20 level, we mean that one in 20 times we will get a
significant result even though the hypothesis is true. That is, when we test
to see if there are any differences in the baseline values of the control and
treatment groups, if we have made 20 different measurements, we can
expect to see at least one statistically significant difference; in fact, we will
see this result almost two-thirds of the time. This difference will not
represent a flaw in our design but simply chance at work. To avoid this
undesirable result—that is, to avoid attributing statistical significance to an
insignificant random event, a so-called Type I error—we must distinguish
between the hypotheses with which we began the study and those which
came to mind afterward. We must accept or reject our initial hypotheses at
the original significance level while demanding additional corroborating
evidence for those exceptional results (such as a dependence of an
outcome on astrological sign) that are uncovered for the first time during
the trials.
No reputable scientist would ever report results before successfully
reproducing the experimental findings twice, once in the original
laboratory and once in that of a colleague.4
The latter experiment can be
particularly telling, as all too often some overlooked factor not controlled
in the experiment—such as the quality of the laboratory water—proves
responsible for the results observed initially. It is better to be found wrong
in private, than in public. The only remedy is to attempt to replicate the
findings with different sets of subjects, replicate, then replicate again.
Persi Diaconis [1978] spent some years investigating paranormal
phenomena. His scientific inquiries included investigating the powers
linked to Uri Geller, the man who claimed he could bend spoons with his
mind. Diaconis was not surprised to find that the hidden “powers” of
Geller were more or less those of the average nightclub magician, down to
and including forcing a card and taking advantage of ad-hoc, post-hoc
hypotheses (Figure 1.1).
When three buses show up at your stop simultaneously, or three rock
stars die in the same year, or a stand of cherry trees is found amid a forest
of oaks, a good statistician remembers the Poisson distribution. This
distribution applies to relatively rare events that occur independently of
one another (see Figure 1.2). The calculations performed by Siméon-
4
Remember “cold fusion”? In 1989, two University of Utah professors told the newspapers
they could fuse deuterium molecules in the laboratory, solving the world’s energy problems
for years to come. Alas, neither those professors nor anyone else could replicate their
findings, though true believers abound (see http://www.ncas.org/erab/intro.htm).](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-26-2048.jpg)
![CHAPTER 1 SOURCES OF ERROR 13
population, the samples must be taken at random from, and be
representative of, that population.
The preliminary findings clearly called for an examination of surgical
procedures and of patient characteristics that might help forecast successful
surgery. But the generation of a p-value and the drawing of any final
conclusions had to wait for clinical trials specifically designed for that
purpose.
This does not mean that one should not report anomalies and other
unexpected findings. Rather, one should not attempt to provide p-values
or confidence intervals in support of them. Successful researchers engage
in a cycle of theorizing and experimentation so that the results of one
experiment become the basis for the hypotheses tested in the next.
A related, extremely common error whose correction we discuss at
length in Chapters 13 and 15 is to use the same data to select variables for
inclusion in a model and to assess their significance. Successful model
builders develop their frameworks in a series of stages, validating each
model against a second independent dataset before drawing conclusions.
One reason why many statistical models are incomplete is that they
do not specify the sources of randomness generating variability
among agents, i.e., they do not specify why otherwise
observationally identical people make different choices and have
different outcomes given the same choice.—James J. Heckman
TO LEARN MORE
On the necessity for improvements in the use of statistics in research
publications, see Altman [1982, 1991, 1994, 2000, 2002]; Cooper and
Rosenthal [1980]; Dar, Serlin, and Omer [1994]; Gardner and Bond
[1990]; George [1985]; Glantz [1980]; Goodman, Altman, and George
[1998]; MacArthur and Jackson [1984]; Morris [1988]; Strasak et al.
[2007]; Thorn et al. [1985]; and Tyson et al. [1983].
Brockman and Chowdhury [1997] discuss the costly errors that can
result from treating chaotic phenomena as stochastic.](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-29-2048.jpg)
![CHAPTER 2 HYPOTHESES: THE WHY OF YOUR RESEARCH 17
individuals rather than populations [Hagood, 1941]. Finally, note that
until someone other than Thurber succeeds in locating unicorns, the
hypothesis, “80% of unicorns are white” is not testable.
Formulate your hypotheses so they are quantifiable, testable, and statistical in
nature.
HOW PRECISE MUST A HYPOTHESIS BE?
The chief executive of a drug company may well express a desire to test
whether “our antihypertensive drug can beat the competition.” The
researcher, having done done preliminary reading of the literature, might
want to test a preliminary hypothesis on the order of “For males over 40
suffering from chronic hypertension, there is a daily dose of our new
drug that will lower diastolic blood pressure an average of 20 mm Hg.”
But this hypothesis is imprecise. What if the necessary dose of the new
drug required taking a tablet every hour? Or caused liver malfunction?
Or even death? First, the researcher would need to conduct a set of
clinical trials to determine the maximum tolerable dose (MTD).
Subsequently, she could test the precise hypothesis, “a daily dose of
one-third to one-fourth the MTD of our new drug will lower diastolic
blood pressure an average of 20 mm Hg in males over 40 suffering from
chronic hypertension.”
In a series of articles by Horwitz et al. [1998], a physician and his
colleagues strongly criticize the statistical community for denying them (or
so they perceive) the right to provide a statistical analysis for subgroups
not contemplated in the original study protocol. For example, suppose
that in a study of the health of Marine recruits, we notice that not one of
the dozen or so women who received a vaccine contracted pneumonia.
Are we free to provide a p-value for this result?
Statisticians Smith and Egger [1998] argue against hypothesis tests of
subgroups chosen after the fact, suggesting that the results are often likely
to be explained by the “play of chance.” Altman [1998; pp. 301–303],
another statistician, concurs noting that, “. . . the observed treatment
effect is expected to vary across subgroups of the data . . . simply through
chance variation,” and that “doctors seem able to find a biologically
plausible explanation for any finding.” This leads Horwitz et al. to the
incorrect conclusion that Altman proposes that we “dispense with
clinical biology (biologic evidence and pathophysiologic reasoning)
as a basis for forming subgroups.” Neither Altman nor any other
statistician would quarrel with Horwitz et al.’s assertion that physicians
must investigate “how do we [physicians] do our best for a particular
patient.”](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-32-2048.jpg)
![18 PART I FOUNDATIONS
Scientists can and should be encouraged to make subgroup analyses.
Physicians and engineers should be encouraged to make decisions based
upon them. Few would deny that in an emergency, coming up with
workable, fast-acting solutions without complete information is better than
finding the best possible solution.1
But, by the same token, statisticians
should not be pressured to give their imprimatur to what, in statistical
terms, is clearly an improper procedure, nor should statisticians mislabel
suboptimal procedures as the best that can be done.2
We concur with Anscombe [1963] who writes, “. . . the concept of
error probabilities of the first and second kinds . . . has no direct relevance
to experimentation. . . . The formation of opinions, decisions concerning
further experimentation, and other required actions, are not
dictated . . . by the formal analysis of the experiment, but call for
judgment and imagination. . . . It is unwise for the experimenter to view
himself seriously as a decision-maker. . . . The experimenter pays the piper
and calls the tune he likes best; but the music is broadcast so that others
might listen. . . . ”
A Bill of Rights for Subgroup Analysis
• Scientists can and should be encouraged to make subgroup
analyses.
• Physicians and engineers should be encouraged to make
decisions utilizing the findings of such analyses.
• Statisticians and other data analysts can and should
rightly refuse to give their imprimatur to related tests of
significance.
FOUND DATA
p-values should not be computed for hypotheses based on “found data” as
of necessity all hypotheses related to found data are after the fact. This
rule does not apply if the observer first divides the data into sections.
One part is studied and conclusions drawn; then the resultant hypotheses
are tested on the remaining sections. Even then, the tests are valid only
if the found data can be shown to be representative of the population
at large.
1
Chiles [2001; p. 61].
2
One is reminded of the Dean, several of them in fact, who asked me to alter my grades.
“But that is something you can do as easily as I.” “Why Dr. Good, I would never dream of
overruling one of my instructors.” See also Murder at Oklahoma by J. M. Bickham.](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-33-2048.jpg)
![CHAPTER 2 HYPOTHESES: THE WHY OF YOUR RESEARCH 19
NULL OR NIL HYPOTHESIS
A major research failing seems to be the exploration of
uninteresting or even trivial questions. . . . In the 347 sampled
articles in Ecology containing null hypotheses tests, we found few
examples of null hypotheses that seemed biologically plausible.—
Anderson, Burnham, and Thompson [2000].
We do not perform an experiment to find out if two varieties of
wheat or two drugs are equal. We know in advance, without
spending a dollar on an experiment, that they are not equal.—
Deming [1975].
Test only relevant null hypotheses.
The null hypothesis has taken on an almost mythic role in contemporary
statistics. Obsession with the null (more accurately spelled and pronounced
nil), has been allowed to shape the direction of our research. We have let
the tool use us instead of us using the tool.3
Virtually any quantifiable hypothesis can be converted into null form.
There is no excuse and no need to be content with a meaningless nil.
For example, suppose we want to test that a given treatment will
decrease the need for bed rest by at least three days. Previous trials have
convinced us that the treatment will reduce the need for bed rest to some
degree, so merely testing that the treatment has a positive effect would
yield no new information. Instead, we would subtract three from each
observation and then test the nil hypothesis that the mean value is zero.
We often will want to test that an effect is inconsequential, not zero but
close to it, smaller than d, say, where d is the smallest biological, medical,
physical, or socially relevant effect in our area of research. Again, we would
subtract d from each observation before proceeding to test a null
hypothesis.
The quote from Deming above is not quite correct as often we will wish
to demonstrate that two drugs or two methods yield equivalent results. As
shown in Chapter 5, we may test for equivalence using confidence
intervals; a null hypothesis is not involved
To test that “80% of redheads are passionate,” we have two choices
depending on how “passion” is measured. If “passion” is an all-or-none
phenomena, then we can forget about trying to formulate a null
hypothesis and instead test the binomial hypothesis that the probability p
3
See, for example, Hertwig and Todd [2000].](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-34-2048.jpg)
![24 PART I FOUNDATIONS
Of course, we would! In determining the significance level in the
present example, we must add together the total number of tables that lie
in either of the two extremes or tails of the permutation distribution.
The critical values and significance levels are quite different for one-
tailed and two-tailed tests and, all too often, the wrong test has been
employed in published work. McKinney et al. [1989] reviewed some
70-plus articles that appeared in six medical journals. In over half of these
articles, Fisher’s exact test was applied improperly. Either a one-tailed test
had been used when a two-tailed test was called for or the authors of the
paper simply had not bothered to state which test they had used.
Of course, unless you are submitting the results of your analysis to a
regulatory agency, no one will know whether you originally intended a
one-tailed test or a two-tailed test and subsequently changed your mind.
No one will know whether your hypothesis was conceived before you
started or only after you had examined the data. All you have to do is lie.
Just recognize that if you test an after-the-fact hypothesis without
identifying it as such, you are guilty of scientific fraud.
When you design an experiment, decide at the same time whether you
wish to test your hypothesis against a two-sided or a one-sided alternative.
A two-sided alternative dictates a two-tailed test; a one-sided alternative
dictates a one-tailed test.
As an example, suppose we decide to do a follow-on study of the cancer
registry to confirm our original finding that men diagnosed as having
tumors live significantly longer than women similarly diagnosed. In this
follow-on study, we have a one-sided alternative. Thus, we would analyze
the results using a one-tailed test rather than the two-tailed test we applied
in the original study.
Determine beforehand whether your alternative hypotheses are ordered or
unordered.
Ordered or Unordered Alternative Hypotheses?
When testing qualities (number of germinating plants, crop weight, etc.)
from k samples of plants taken from soils of different composition, it is
often routine to use the F-ratio of the analysis of variance. For
contingency tables, many routinely use the chi-square test to determine if
the differences among samples are significant. But the F-ratio and the
chi-square are what are termed omnibus tests, designed to be sensitive to
all possible alternatives. As such, they are not particularly sensitive to
ordered alternatives such “as more fertilizer equals more growth” or
“more aspirin equals faster relief of headache.” Tests for such ordered
responses at k distinct treatment levels should properly use the Pitman](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-39-2048.jpg)
![CHAPTER 2 HYPOTHESES: THE WHY OF YOUR RESEARCH 25
correlation described by Frank, Trzos, and Good [1978] when the data
are measured on a metric scale (e.g., weight of the crop). Tests for
ordered responses in 2 × C contingency tables (e.g., number of
germinating plants) should use the trend test described by Berger,
Permutt, and Ivanova [1998]. We revisit this topic in more detail in the
next chapter.
DEDUCTION AND INDUCTION
When we determine a p-value as we did in the example above, we apply a
set of algebraic methods and deductive logic to deduce the correct value.
The deductive process is used to determine the appropriate size of resistor
to use in an electric circuit, to determine the date of the next eclipse of
the moon, and to establish the identity of the criminal (perhaps from the
fact the dog did not bark on the night of the crime). Find the formula,
plug in the values, turn the crank and out pops the result (or it does for
Sherlock Holmes,6
at least).
When we assert that for a given population a percentage of samples will
have a specific composition, this also is a deduction. But when we make an
inductive generalization about a population based upon our analysis of a
sample, we are on shakier ground. It is one thing to assert that if an
observation comes from a normal distribution with mean zero, the
probability is one-half that it is positive. It is quite another if, on observing
that half the observations in the sample are positive, we assert that half of
all the possible observations that might be drawn from that population will
be positive also.
Newton’s Law of Gravitation provided an almost exact fit (apart from
measurement error) to observed astronomical data for several centuries;
consequently, there was general agreement that Newton’s generalization
from observation was an accurate description of the real world. Later, as
improvements in astronomical measuring instruments extended the range
of the observable universe, scientists realized that Newton’s Law was only
a generalization and not a property of the universe at all. Einstein’s
Theory of Relativity gives a much closer fit to the data, a fit that has not
been contradicted by any observations in the century since its formulation.
But this still does not mean that relativity provides us with a complete,
correct, and comprehensive view of the universe.
In our research efforts, the only statements we can make with God-like
certainty are of the form “our conclusions fit the data.” The true nature of
the real world is unknowable. We can speculate, but never conclude.
6
See “Silver Blaze” by A. Conan-Doyle, Strand Magazine, December 1892.](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-40-2048.jpg)
![26 PART I FOUNDATIONS
LOSSES
In our first advanced course in statistics, we read in the first chapter of
Lehmann [1986] that the “optimal” statistical procedure would depend
on the losses associated with the various possible decisions. But on day
one of our venture into the real world of practical applications, we were
taught to ignore this principle.
At that time, the only computationally feasible statistical procedures
were based on losses that were proportional to the square of the difference
between estimated and actual values. No matter that the losses really
might be proportional to the absolute value of those differences, or the
cube, or the maximum over a certain range. Our options were limited by
our ability to compute.
Computer technology has made a series of major advances in the past
half century. What forty years ago required days or weeks to calculate
takes only milliseconds today. We can now pay serious attention to this
long-neglected facet of decision theory: the losses associated with the
varying types of decision.
Suppose we are investigating a new drug: We gather data, perform a
statistical analysis, and draw a conclusion. If chance alone is at work
yielding exceptional values and we opt in favor of the new drug, we have
made an error. We also make an error if we decide there is no difference
and the new drug really is better. These decisions and the effects of
making them are summarized in Table 2.2.
We distinguish the two types of error because they have quite different
implications, as described in Table 2.2. As a second example, Fears,
Tarone, and Chu [1977] use permutation methods to assess several
standard screens for carcinogenicity. As shown in Table 2.3 their Type I
error, a false positive, consists of labeling a relatively innocuous compound
as carcinogenic. Such an action means economic loss for the manufacturer
and the denial to the public of the compound’s benefits. Neither
consequence is desirable. But a false negative, a Type II error, is much
TABLE 2.2. Decision making under uncertainty
The Facts Our Decision
No Difference No Difference Drug is Better
Type I error:
Correct Manufacturer wastes money
developing ineffective drug.
Drug is Better Type II error:
Manufacturer misses
opportunity for profit.
Correct
Public denied access to
effective treatment.](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-41-2048.jpg)
![28 PART I FOUNDATIONS
sociologically) significant effect takes place, but what the size of the effect
is and the extent to which the effect varies from instance to instance. We
would also want to know what factors, if any, will modify the size of the
effect or its duration.
We may not be able to address all these issues with a single dataset. A
preliminary experiment might tell us something about the possible
existence of an effect, along with rough estimates of its size and variability.
Hopefully, we glean enough information to come up with doses,
environmental conditions, and sample sizes to apply in collecting and
evaluating the next dataset. A list of possible decisions after the initial
experiment includes “abandon this line of research,” “modify the
environment and gather more data,” and “perform a large, tightly
controlled, expensive set of trials.” Associated with each decision is a set of
potential gains and losses. Common sense dictates we construct a table
similar to Table 2.2 or 2.3 before we launch a study.
For example, in clinical trials of a drug we might begin with some
animal experiments, then progress to Phase I clinical trials in which, with
the emphasis on safety, we look for the maximum tolerable dose. Phase I
trials generally involve only a small number of subjects and a one-time or
short-term intervention. An extended period of several months may be
used for follow-up purposes. If no adverse effects are observed, we might
decide to pursue a Phase II set of trials in the clinic, in which our
objective is to determine the minimum effective dose. Obviously, if the
minimum effective dose is greater than the maximum tolerable dose, or if
some dangerous side effects are observed that we did not observe in the
first set of trials, we will abandon the drug and go on to some other
research project. But if the signs are favorable, then and only then will we
go to a set of Phase III trials involving a large number of subjects
observed over an extended time period. Then, and only then, will we hope
to get the answers to all our research questions.
Before you begin, list all the consequences of a study and all the actions you
might take. Persist only if you can add to existing knowledge.
TO LEARN MORE
For more thorough accounts of decision theory, the interested reader is
directed to Berger [1986], Blyth [1970], Cox [1958], DeGroot [1970],
and Lehmann [1986]. For an applied perspective, see Clemen [1991],
Berry [1995], and Sox, Blatt, Higgins, and Marton [1988].
Over 300 references warning of the misuse of null hypothesis testing
can be accessed online at http://www.cnr.colostate.edu/∼anderson/
thompson1.html. Alas, the majority of these warnings are ill informed,](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-43-2048.jpg)
![CHAPTER 2 HYPOTHESES: THE WHY OF YOUR RESEARCH 29
stressing errors that will not arise if you proceed as we recommend and
place the emphasis on the why, not the what, of statistical procedures. Use
statistics as a guide to decision making rather than a mandate.
Neyman and Pearson [1933] first formulated the problem of hypothesis
testing in terms of two types of error. Extensions and analyses of their
approach are given by Lehmann [1986] and Mayo [1996]. Their approach
has not gone unchallenged, as seen in Berger [2003], Berger and Berry
[1988], Berger and Selke [1987], Berkson [1942], Morrison and Henkel
[1970], Savage [1972], Schmidt [1996], Seidenfeld [1979], and Sterne,
Smith, and Cox [2001]. Hunter and Schmidt [1997] list and dismiss
many of their objections.
Guthery, Lusk, and Peterson [2001] and Rozeboom [1960] are among
those who have written about the inadequacy of the null hypothesis.
For more guidelines on formulating meaningful primary hypotheses, see
Selike, Bayarri, and Berger [2001]. Clarity in hypothesis formulation is
essential; ambiguity can only yield controversy; see, for example, Kaplan
[2001].
Venn [1888] and Reichenbach [1949] are among those who have
attempted to construct a mathematical bridge between what we observe
and the reality that underlies our observations. Such efforts to the
contrary, extrapolation from the sample to the population is not merely a
matter of applying Sherlock Holmes-like deductive logic but entails a leap
of faith. A careful reading of Locke [1700], Berkeley [1710], Hume
[1748], and Lonergan [1992] is an essential prerequisite to the application
of statistics.
See also Buchanan-Wollaston [1935], Cohen [1990], Copas [1997],
and Lindley [2000].](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-44-2048.jpg)
![CHAPTER 3 COLLECTING DATA 31
GIGO: Garbage in; Garbage out.
“Fancy statistical methods will not rescue garbage data.”
Course notes of Raymond J. Carroll [2001].
THE VAST MAJORITY OF ERRORS IN STATISTICS (and, not
incidentally, in most human endeavors) arise from a reluctance (or even an
inability) to plan. Some demon (or demonic manager) seems to be urging
us to cross the street before we have had the opportunity to look both
ways. Even on those rare occasions when we do design an experiment, we
seem more obsessed with the mechanics than with the underlying
concepts.
In this chapter, we review the fundamental concepts of experimental
design, the choice of primary and secondary variables, the selection of
measurement devices, the determination of sample size, the assumptions
that underlie most statistical procedures along with the precautions
necessary to ensure they are satisfied and that the data you collect will be
representative of the population as a whole. We do not intend to replace a
text on experiment or survey design, but to supplement one, providing
examples and solutions that are often neglected in courses on the subject.
PREPARATION
The first step in data collection is to have a clear, preferably written
statement of your objectives. In accordance with Chapter 1, you will have
defined the population or populations from which you intend to sample
and have identified the characteristics of these populations you wish to
investigate.
Chapter 3
Collecting Data
Common Errors in Statistics (and How to Avoid Them), Fourth Edition.
Phillip I. Good and James W. Hardin.
© 2012 John Wiley & Sons, Inc. Published 2012 by John Wiley & Sons, Inc.](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-45-2048.jpg)
![32 PART I FOUNDATIONS
You developed one or more well-formulated hypotheses (the topic of
Chapter 2) and have some idea of the risks you will incur should your
analysis of the collected data prove to be erroneous. You will need to
decide what you wish to observe and measure, and how you will go about
observing it. We refer here not only to the primary variables or endpoints,
but to the secondary variables or cofactors that may influence the former’s
values. Indeed, it is essential that you be aware of all potential sources of
variation.
Good practice is to draft the analysis section of your final report based
on the conclusions you would like to make. What information do you
need to justify these conclusions? All such information must be collected.
The next section is devoted to the choice of response variables and
measuring devices, followed by sections on determining sample size and
preventive steps to ensure your samples will be analyzable by statistical
methods.
RESPONSE VARIABLES
Know What You Want to Measure
If you do not collect the values of cofactors, you will be unable to account
for them later.
As whiplash injuries are a common consequence of rear-end collisions,
there is an extensive literature on the subject. Any physician will tell you
that the extent and duration of such injuries will depend upon the sex,
age, and physical condition of the injured individual as well as any prior
injuries the individual may have suffered. Yet we found article after article
that failed to account for these factors; for example, Krafft, Kullgren,
Ydenius, and Tingvall [2002], Kumar, Ferrari, and Narayan [2005], and
Tencer, Sohail, and Kevin [2001] did not report the sex, age, or prior
injuries of their test subjects.
Will you measure an endpoint such as death or a surrogate such as the
presence of HIV antibodies? A good response variable takes values over a
sufficiently large range so that they discriminate well [Bishop and Talbot,
2001].
The regression slope describing the change in systolic blood pressure (in
mm Hg) per 100 mg of calcium intake is strongly influenced by the
approach used for assessing the amount of calcium consumed (Cappuccio
et al., 1995). The association is small and only marginally significant with
diet histories (slope −0.01 (−0.003 to −0.016)) but large and highly
significant when food frequency questionnaires are used (slope −0.15
(−0.11 to −0.19). With studies using 24 hour recall, an intermediate result
emerges (slope −0.06 (−0.09 to −0.03). Diet histories assess patterns of](https://image.slidesharecdn.com/93324-250509022939-3e8ff4c2/75/Common-errors-in-statistics-and-how-to-avoid-them-4ed-Edition-Phillip-I-Good-46-2048.jpg)
Common errors in statistics and how to avoid them 4ed. Edition Phillip I. Good
- 1. Common errors instatistics and how to avoid them 4ed. Edition Phillip I. Good download https://ebookgate.com/product/common-errors-in-statistics-and- how-to-avoid-them-4ed-edition-phillip-i-good/ Get Instant Ebook Downloads – Browse at https://ebookgate.com
- 2. Get Your DigitalFiles Instantly: PDF, ePub, MOBI and More Quick Digital Downloads: PDF, ePub, MOBI and Other Formats Common Errors in Statistics and How to Avoid Them Third Edition P. I. Good https://ebookgate.com/product/common-errors-in-statistics-and- how-to-avoid-them-third-edition-p-i-good/ Common Errors in Statistics 3rd Edition Phillip I. Good https://ebookgate.com/product/common-errors-in-statistics-3rd- edition-phillip-i-good/ Common Mistakes at First Certificate and How to Avoid Them Tayfoor https://ebookgate.com/product/common-mistakes-at-first- certificate-and-how-to-avoid-them-tayfoor/ The 106 Common Mistakes Homebuyers Make and How to Avoid Them 3rd edition Edition Eldred G.W. https://ebookgate.com/product/the-106-common-mistakes-homebuyers- make-and-how-to-avoid-them-3rd-edition-edition-eldred-g-w/
- 3. Top Screwups DoctorsMake and How to Avoid Them Joe Graedon https://ebookgate.com/product/top-screwups-doctors-make-and-how- to-avoid-them-joe-graedon/ Applying statistics in the courtroom a new approach for attorneys and expert witnesses 1st Edition Phillip I. Good https://ebookgate.com/product/applying-statistics-in-the- courtroom-a-new-approach-for-attorneys-and-expert-witnesses-1st- edition-phillip-i-good/ 25 Biggest Mistakes Teachers Make and How to Avoid Them 2nd Edition Dr. Carolyn M. Orange https://ebookgate.com/product/25-biggest-mistakes-teachers-make- and-how-to-avoid-them-2nd-edition-dr-carolyn-m-orange/ Best Laid Plans The Tyranny of Unintended Consequences and How to Avoid Them 1st Edition William A. Sherden https://ebookgate.com/product/best-laid-plans-the-tyranny-of- unintended-consequences-and-how-to-avoid-them-1st-edition- william-a-sherden/ Analyzing the Large Number of Variables in Biomedical and Satellite Imagery 1st Edition Phillip I. Good https://ebookgate.com/product/analyzing-the-large-number-of- variables-in-biomedical-and-satellite-imagery-1st-edition- phillip-i-good/
- 5. COMMON ERRORS INSTATISTICS (AND HOW TO AVOID THEM)
- 6. COMMON ERRORS INSTATISTICS (AND HOW TO AVOID THEM) Fourth Edition Phillip I. Good Statcourse.com Huntington Beach, CA James W. Hardin Dept. of Epidemiology & Biostatistics Institute for Families in Society University of South Carolina Columbia, SC A JOHN WILEY & SONS, INC., PUBLICATION
- 7. Cover photo: GaryCarlsen, DDS Copyright © 2012 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data: Good, Phillip I. Common errors in statistics (and how to avoid them) / Phillip I. Good, Statcourse.com, Huntington Beach, CA, James W. Hardin, Dept. of Epidemiology & Biostatistics, University of South Carolina, Columbia, SC. – Fourth edition. pages cm Includes bibliographical references and index. ISBN 978-1-118-29439-0 (pbk.) 1. Statistics. I. Hardin, James W. (James William) II. Title. QA276.G586 2012 519.5–dc23 2012005888 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1
- 8. CONTENTS v Contents Preface xi PARTI FOUNDATIONS 1 1. Sources of Error 3 Prescription 4 Fundamental Concepts 5 Surveys and Long-Term Studies 9 Ad-Hoc, Post-Hoc Hypotheses 9 To Learn More 13 2. Hypotheses: The Why of Your Research 15 Prescription 15 What Is a Hypothesis? 16 How Precise Must a Hypothesis Be? 17 Found Data 18 Null or Nil Hypothesis 19 Neyman–Pearson Theory 20 Deduction and Induction 25 Losses 26 Decisions 27 To Learn More 28 3. Collecting Data 31 Preparation 31 Response Variables 32 Determining Sample Size 37 Fundamental Assumptions 46 Experimental Design 47
- 9. vi CONTENTS Four Guidelines49 Are Experiments Really Necessary? 53 To Learn More 54 PART II STATISTICAL ANALYSIS 57 4. Data Quality Assessment 59 Objectives 60 Review the Sampling Design 60 Data Review 62 To Learn More 63 5. Estimation 65 Prevention 65 Desirable and Not-So-Desirable Estimators 68 Interval Estimates 72 Improved Results 77 Summary 78 To Learn More 78 6. Testing Hypotheses: Choosing a Test Statistic 79 First Steps 80 Test Assumptions 82 Binomial Trials 84 Categorical Data 85 Time-To-Event Data (Survival Analysis) 86 Comparing the Means of Two Sets of Measurements 90 Do Not Let Your Software Do Your Thinking For You 99 Comparing Variances 100 Comparing the Means of K Samples 105 Higher-Order Experimental Designs 108 Inferior Tests 113 Multiple Tests 114 Before You Draw Conclusions 115 Induction 116 Summary 117 To Learn More 117 7. Strengths and Limitations of Some Miscellaneous Statistical Procedures 119 Nonrandom Samples 119 Modern Statistical Methods 120 Bootstrap 121
- 10. CONTENTS vii Bayesian Methodology123 Meta-Analysis 131 Permutation Tests 135 To Learn More 137 8. Reporting Your Results 139 Fundamentals 139 Descriptive Statistics 144 Ordinal Data 149 Tables 149 Standard Error 151 p-Values 155 Confidence Intervals 156 Recognizing and Reporting Biases 158 Reporting Power 160 Drawing Conclusions 160 Publishing Statistical Theory 162 A Slippery Slope 162 Summary 163 To Learn More 163 9. Interpreting Reports 165 With a Grain of Salt 165 The Authors 166 Cost–Benefit Analysis 167 The Samples 167 Aggregating Data 168 Experimental Design 168 Descriptive Statistics 169 The Analysis 169 Correlation and Regression 171 Graphics 171 Conclusions 172 Rates and Percentages 174 Interpreting Computer Printouts 175 Summary 178 To Learn More 178 10. Graphics 181 Is a Graph Really Necessary? 182 KISS 182 The Soccer Data 182 Five Rules for Avoiding Bad Graphics 183
- 11. viii CONTENTS One Rulefor Correct Usage of Three-Dimensional Graphics 194 The Misunderstood and Maligned Pie Chart 196 Two Rules for Effective Display of Subgroup Information 198 Two Rules for Text Elements in Graphics 201 Multidimensional Displays 203 Choosing Effective Display Elements 209 Oral Presentations 209 Summary 210 To Learn More 211 PART III BUILDING A MODEL 213 11. Univariate Regression 215 Model Selection 215 Stratification 222 Further Considerations 226 Summary 233 To Learn More 234 12. Alternate Methods of Regression 237 Linear Versus Nonlinear Regression 238 Least-Absolute-Deviation Regression 238 Quantile Regression 243 Survival Analysis 245 The Ecological Fallacy 246 Nonsense Regression 248 Reporting the Results 248 Summary 248 To Learn More 249 13. Multivariable Regression 251 Caveats 251 Dynamic Models 256 Factor Analysis 256 Reporting Your Results 258 A Conjecture 260 Decision Trees 261 Building a Successful Model 264 To Learn More 265
- 12. CONTENTS ix 14. ModelingCounts and Correlated Data 267 Counts 268 Binomial Outcomes 268 Common Sources of Error 269 Panel Data 270 Fixed- and Random-Effects Models 270 Population-Averaged Generalized Estimating Equation Models (GEEs) 271 Subject-Specific or Population-Averaged? 272 Variance Estimation 272 Quick Reference for Popular Panel Estimators 273 To Learn More 275 15. Validation 277 Objectives 277 Methods of Validation 278 Measures of Predictive Success 283 To Learn More 285 Glossary 287 Bibliography 291 Author Index 319 Subject Index 329
- 14. PREFACE xi ONE OFTHE VERY FIRST TIMES DR. GOOD served as a statistical consultant, he was asked to analyze the occurrence rate of leukemia cases in Hiroshima, Japan following World War II. On August 7, 1945 this city was the target site of the first atomic bomb dropped by the United States. Was the high incidence of leukemia cases among survivors the result of exposure to radiation from the atomic bomb? Was there a relationship between the number of leukemia cases and the number of survivors at certain distances from the atomic bomb’s epicenter? To assist in the analysis, Dr. Good had an electric (not an electronic) calculator, reams of paper on which to write down intermediate results, and a prepublication copy of Scheffe’s Analysis of Variance. The work took several months and the results were somewhat inconclusive, mainly because he could never seem to get the same answer twice—a consequence of errors in transcription rather than the absence of any actual relationship between radiation and leukemia. Today, of course, we have high-speed computers and prepackaged statistical routines to perform the necessary calculations. Yet, statistical software will no more make one a statistician than a scalpel will turn one into a neurosurgeon. Allowing these tools to do our thinking is a sure recipe for disaster. Pressed by management or the need for funding, too many research workers have no choice but to go forward with data analysis despite having insufficient statistical training. Alas, though a semester or two of undergraduate statistics may develop familiarity with the names of some statistical methods, it is not enough to be aware of all the circumstances under which these methods may be applicable. Preface
- 15. xii PREFACE The purposeof the present text is to provide a mathematically rigorous but readily understandable foundation for statistical procedures. Here are such basic concepts in statistics as null and alternative hypotheses, p-value, significance level, and power. Assisted by reprints from the statistical literature, we reexamine sample selection, linear regression, the analysis of variance, maximum likelihood, Bayes’ Theorem, meta-analysis and the bootstrap. New to this edition are sections on fraud and on the potential sources of error to be found in epidemiological and case-control studies. Examples of good and bad statistical methodology are drawn from agronomy, astronomy, bacteriology, chemistry, criminology, data mining, epidemiology, hydrology, immunology, law, medical devices, medicine, neurology, observational studies, oncology, pricing, quality control, seismology, sociology, time series, and toxicology. More good news: Dr. Good’s articles on women sports have appeared in the San Francisco Examiner, Sports Now, and Volleyball Monthly; 22 short stories of his are in print; and you can find his 21 novels on Amazon and zanybooks.com. So, if you can read the sports page, you’ll find this text easy to read and to follow. Lest the statisticians among you believe this book is too introductory, we point out the existence of hundreds of citations in statistical literature calling for the comprehensive treatment we have provided. Regardless of past training or current specialization, this book will serve as a useful reference; you will find applications for the information contained herein whether you are a practicing statistician or a well-trained scientist who just happens to apply statistics in the pursuit of other science. The primary objective of the opening chapter is to describe the main sources of error and provide a preliminary prescription for avoiding them. The hypothesis formulation—data gathering—hypothesis testing and estimation—cycle is introduced, and the rationale for gathering additional data before attempting to test after-the-fact hypotheses detailed. A rewritten Chapter 2 places our work in the context of decision theory. We emphasize the importance of providing an interpretation of each and every potential outcome in advance data collection. A much expanded Chapter 3 focuses on study design and data collection, as failure at the planning stage can render all further efforts valueless. The work of Berger and his colleagues on selection bias is given particular emphasis. Chapter 4 on data quality assessment reminds us that just as 95% of research efforts are devoted to data collection, 95% of the time remaining should be spent on ensuring that the data collected warrant analysis.
- 16. PREFACE xiii Desirable featuresof point and interval estimates are detailed in Chapter 5 along with procedures for deriving estimates in a variety of practical situations. This chapter also serves to debunk several myths surrounding estimation procedures. Chapter 6 reexamines the assumptions underlying testing hypotheses and presents the correct techniques for analyzing binomial trials, counts, categorical data, continuous measurements, and time-to-event data. We review the impacts of violations of assumptions, and detail the procedures to follow when making two- and k-sample comparisons. Chapter 7 is devoted to the analysis of nonrandom data (cohort and case-control studies), plus discussions of the value and limitations of Bayes’ theorem, meta-analysis, and the bootstrap and permutation tests, and contains essential tips on getting the most from these methods. A much expanded Chapter 8 lists the essentials of any report that will utilize statistics, debunks the myth of the “standard” error, and describes the value and limitations of p-values and confidence intervals for reporting results. Practical significance is distinguished from statistical significance and induction is distinguished from deduction. Chapter 9 covers much the same material but from the viewpoint of the reader rather than the writer. Of particular importance are sections on interpreting computer output and detecting fraud. Twelve rules for more effective graphic presentations are given in Chapter 10 along with numerous examples of the right and wrong ways to maintain reader interest while communicating essential statistical information. Chapters 11 through 15 are devoted to model building and to the assumptions and limitations of a multitude of regression methods and data mining techniques. A distinction is drawn between goodness of fit and prediction, and the importance of model validation is emphasized. Finally, for the further convenience of readers, we provide a glossary grouped by related but contrasting terms, an annotated bibliography, and subject and author indexes. Our thanks go to William Anderson, Leonardo Auslender, Vance Berger, Peter Bruce, Bernard Choi, Tony DuSoir, Cliff Lunneborg, Mona Hardin, Gunter Hartel, Fortunato Pesarin, Henrik Schmiediche, Marjorie Stinespring, and Peter A. Wright for their critical reviews of portions of this text. Doug Altman, Mark Hearnden, Elaine Hand, and David Parkhurst gave us a running start with their bibliographies. Brian Cade, David Rhodes, and the late Cliff Lunneborg helped us complete the
- 17. xiv PREFACE second edition.Terry Therneau and Roswitha Blasche helped us complete the third edition. We hope you soon put this text to practical use. Phillip Good drgood@statcourse.com Huntington Beach, CA James Hardin jhardin@sc.edu Columbia, SC May 2012
- 18.
- 19. CHAPTER 1 SOURCESOF ERROR 3 Don’t think—use the computer. Dyke (tongue in cheek) [1997]. We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not been carried out in close collaboration with professional statisticians, the panel remarked in its conclusions. From the report of an independent panel looking into “Climategate.”1 STATISTICAL PROCEDURES FOR HYPOTHESIS TESTING, ESTIMATION, AND MODEL building are only a part of the decision- making process. They should never be quoted as the sole basis for making a decision (yes, even those procedures that are based on a solid deductive mathematical foundation). As philosophers have known for centuries, extrapolation from a sample or samples to a larger, incompletely examined population must entail a leap of faith. The sources of error in applying statistical procedures are legion and include all of the following: 1. a) Replying on erroneous reports to help formulate hypotheses (see Chapter 9) b) Failing to express qualitative hypotheses in quantitative form (see Chapter 2) c) Using the same set of data both to formulate hypotheses and to test them (see Chapter 2) Chapter 1 Sources of Error Common Errors in Statistics (and How to Avoid Them), Fourth Edition. Phillip I. Good and James W. Hardin. © 2012 John Wiley & Sons, Inc. Published 2012 by John Wiley & Sons, Inc. 1 This is from an inquiry at the University of East Anglia headed by Lord Oxburgh. The inquiry was the result of emails from climate scientists being released to the public.
- 20. 4 PART IFOUNDATIONS 2. a) Taking samples from the wrong population or failing to specify in advance the population(s) about which inferences are to be made (see Chapter 3) b) Failing to draw samples that are random and representative (see Chapter 3) 3. Measuring the wrong variables or failing to measure what you intended to measure (see Chapter 4) 4. Using inappropriate or inefficient statistical methods. Examples include using a two-tailed test when a one-tailed test is appropriate and using an omnibus test against a specific alternative (see Chapters 5 and 6). 5. a) Failing to understand that p-values are functions of the observations and will vary in magnitude from sample to sample (see Chapter 6) b) Using statistical software without verifying that its current defaults are appropriate for your application (see Chapter 6) 6. Failing to adequately communicate your findings (see Chapters 8 and 10) 7. a) Extrapolating models outside the range of the observations (see Chapter 11) b) Failure to correct for confounding variables (see Chapter 13) c) Use the same data to select variables for inclusion in a model and to assess their significance (see Chapter 13) d) Failing to validate models (see Chapter 15) But perhaps the most serious source of error lies in letting statistical procedures make decisions for you. In this chapter, as throughout this text, we offer first a preventive prescription, followed by a list of common errors. If these prescriptions are followed carefully, you will be guided to the correct, proper, and effective use of statistics and avoid the pitfalls. PRESCRIPTION Statistical methods used for experimental design and analysis should be viewed in their rightful role as merely a part, albeit an essential part, of the decision-making procedure. Here is a partial prescription for the error-free application of statistics. 1. Set forth your objectives and your research intentions before you conduct a laboratory experiment, a clinical trial, or survey, or analyze an existing set of data. 2. Define the population about which you will make inferences from the data you gather.
- 21. CHAPTER 1 SOURCESOF ERROR 5 3. a) Recognize that the phenomena you are investigating may have stochastic or chaotic components. b) List all possible sources of variation. Control them or measure them to avoid their being confounded with relationships among those items that are of primary interest. 4. Formulate your hypotheses and all of the associated alternatives. (See Chapter 2.) List possible experimental findings along with the conclusions you would draw and the actions you would take if this or another result should prove to be the case. Do all of these things before you complete a single data collection form, and before you turn on your computer. 5. Describe in detail how you intend to draw a representative sample from the population. (See Chapter 3.) 6. Use estimators that are impartial, consistent, efficient, robust, and minimum loss. (See Chapter 5.) To improve results, focus on sufficient statistics, pivotal statistics, and admissible statistics, and use interval estimates. (See Chapters 5 and 6.) 7. Know the assumptions that underlie the tests you use. Use those tests that require the minimum of assumptions and are most powerful against the alternatives of interest. (See Chapter 6.) 8. Incorporate in your reports the complete details of how the sample was drawn and describe the population from which it was drawn. If data are missing or the sampling plan was not followed, explain why and list all differences between data that were present in the sample and data that were missing or excluded. (See Chapter 8.) FUNDAMENTAL CONCEPTS Three concepts are fundamental to the design of experiments and surveys: variation, population, and sample. A thorough understanding of these concepts will prevent many errors in the collection and interpretation of data. If there were no variation, if every observation were predictable, a mere repetition of what had gone before, there would be no need for statistics. Variation Variation is inherent in virtually all our observations. We would not expect outcomes of two consecutive spins of a roulette wheel to be identical. One result might be red, the other black. The outcome varies from spin to spin. There are gamblers who watch and record the spins of a single roulette wheel hour after hour hoping to discern a pattern. A roulette wheel is, after all, a mechanical device and perhaps a pattern will emerge. But even those observers do not anticipate finding a pattern that is 100% predetermined. The outcomes are just too variable.
- 22. 6 PART IFOUNDATIONS Anyone who spends time in a schoolroom, as a parent or as a child, can see the vast differences among individuals. This one is tall, that one short, though all are the same age. Half an aspirin and Dr. Good’s headache is gone, but his wife requires four times that dosage. There is variability even among observations on deterministic formula- satisfying phenomena such as the position of a planet in space or the volume of gas at a given temperature and pressure. Position and volume satisfy Kepler’s Laws and Boyle’s Law, respectively (the latter over a limited range), but the observations we collect will depend upon the measuring instrument (which may be affected by the surrounding environment) and the observer. Cut a length of string and measure it three times. Do you record the same length each time? In designing an experiment or survey we must always consider the possibility of errors arising from the measuring instrument and from the observer. It is one of the wonders of science that Kepler was able to formulate his laws at all given the relatively crude instruments at his disposal. Deterministic, Stochastic, and Chaotic Phenomena A phenomenon is said to be deterministic if given sufficient information regarding its origins, we can successfully make predictions regarding its future behavior. But we do not always have all the necessary information. Planetary motion falls into the deterministic category once one makes adjustments for all gravitational influences, the other planets as well as the sun. Nineteenth century physicists held steadfast to the belief that all atomic phenomena could be explained in deterministic fashion. Slowly, it became evident that at the subatomic level many phenomena were inherently stochastic in nature, that is, one could only specify a probability distribution of possible outcomes, rather than fix on any particular outcome as certain. Strangely, twenty-first century astrophysicists continue to reason in terms of deterministic models. They add parameter after parameter to the lambda cold-dark-matter model hoping to improve the goodness of fit of this model to astronomical observations. Yet, if the universe we observe is only one of many possible realizations of a stochastic process, goodness of fit offers absolutely no guarantee of the model’s applicability. (See, for example, Good, 2012.) Chaotic phenomena differ from the strictly deterministic in that they are strongly dependent upon initial conditions. A random perturbation from an unexpected source (the proverbial butterfly’s wing) can result in an
- 23. CHAPTER 1 SOURCESOF ERROR 7 unexpected outcome. The growth of cell populations has been described in both deterministic (differential equations) and stochastic terms (birth and death process), but a chaotic model (difference-lag equations) is more accurate. Population The population(s) of interest must be clearly defined before we begin to gather data. From time to time, someone will ask us how to generate confidence intervals (see Chapter 8) for the statistics arising from a total census of a population. Our answer is no, we cannot help. Population statistics (mean, median, and thirtieth percentile) are not estimates. They are fixed values and will be known with 100% accuracy if two criteria are fulfilled: 1. Every member of the population is observed. 2. All the observations are recorded correctly. Confidence intervals would be appropriate if the first criterion is violated, for then we are looking at a sample, not a population. And if the second criterion is violated, then we might want to talk about the confidence we have in our measurements. Debates about the accuracy of the 2000 United States Census arose from doubts about the fulfillment of these criteria.2 “You didn’t count the homeless,” was one challenge. “You didn’t verify the answers,” was another. Whether we collect data for a sample or an entire population, both these challenges or their equivalents can and should be made. Kepler’s “laws” of planetary movement are not testable by statistical means when applied to the original planets (Jupiter, Mars, Mercury, and Venus) for which they were formulated. But when we make statements such as “Planets that revolve around Alpha Centauri will also follow Kepler’s Laws,” then we begin to view our original population, the planets of our sun, as a sample of all possible planets in all possible solar systems. A major problem with many studies is that the population of interest is not adequately defined before the sample is drawn. Do not make this mistake. A second major problem is that the sample proves to have been drawn from a different population than was originally envisioned. We consider these issues in the next section and again in Chapters 2, 6, and 7. 2 City of New York v. Department of Commerce, 822 F. Supp. 906 (E.D.N.Y, 1993). The arguments of four statistical experts who testified in the case may be found in Volume 34 of Jurimetrics, 1993, 64–115.
- 24. 8 PART IFOUNDATIONS Sample A sample is any (proper) subset of a population. Small samples may give a distorted view of the population. For example, if a minority group comprises 10% or less of a population, a jury of 12 persons selected at random from that population fails to contain any members of that minority at least 28% of the time. As a sample grows larger, or as we combine more clusters within a single sample, the sample will grow to more closely resemble the population from which it is drawn. How large a sample must be to obtain a sufficient degree of closeness will depend upon the manner in which the sample is chosen from the population. Are the elements of the sample drawn at random, so that each unit in the population has an equal probability of being selected? Are the elements of the sample drawn independently of one another? If either of these criteria is not satisfied, then even a very large sample may bear little or no relation to the population from which it was drawn. An obvious example is the use of recruits from a Marine boot camp as representatives of the population as a whole or even as representatives of all Marines. In fact, any group or cluster of individuals who live, work, study, or pray together may fail to be representative for any or all of the following reasons (Cummings and Koepsell, 2002): 1. Shared exposure to the same physical or social environment; 2. Self selection in belonging to the group; 3. Sharing of behaviors, ideas, or diseases among members of the group. A sample consisting of the first few animals to be removed from a cage will not satisfy these criteria either, because, depending on how we grab, we are more likely to select more active or more passive animals. Activity tends to be associated with higher levels of corticosteroids, and corticosteroids are associated with virtually every body function. Sample bias is a danger in every research field. For example, Bothun [1998] documents the many factors that can bias sample selection in astronomical research. To prevent sample bias in your studies, before you begin determine all the factors that can affect the study outcome (gender and lifestyle, for example). Subdivide the population into strata (males, females, city dwellers, farmers) and then draw separate samples from each stratum. Ideally, you would assign a random number to each member of the stratum and let a computer’s random number generator determine which members are to be included in the sample.
- 25. CHAPTER 1 SOURCESOF ERROR 9 SURVEYS AND LONG-TERM STUDIES Being selected at random does not mean that an individual will be willing to participate in a public opinion poll or some other survey. But if survey results are to be representative of the population at large, then pollsters must find some way to interview nonresponders as well. This difficulty is exacerbated in long-term studies, as subjects fail to return for follow-up appointments and move without leaving a forwarding address. Again, if the sample results are to be representative, some way must be found to report on subsamples of the nonresponders and the dropouts. AD-HOC, POST-HOC HYPOTHESES Formulate and write down your hypotheses before you examine the data. Patterns in data can suggest, but cannot confirm, hypotheses unless these hypotheses were formulated before the data were collected. Everywhere we look, there are patterns. In fact, the harder we look the more patterns we see. Three rock stars die in a given year. Fold the United States twenty-dollar bill in just the right way and not only the Pentagon but the Twin Towers in flames are revealed.3 It is natural for us to want to attribute some underlying cause to these patterns, but those who have studied the laws of probability tell us that more often than not patterns are simply the result of random events. Put another way, finding at least one cluster of events in time or in space has a greater probability than finding no clusters at all (equally spaced events). How can we determine whether an observed association represents an underlying cause-and-effect relationship or is merely the result of chance? The answer lies in our research protocol. When we set out to test a specific hypothesis, the probability of a specific event is predetermined. But when we uncover an apparent association, one that may well have arisen purely by chance, we cannot be sure of the association’s validity until we conduct a second set of controlled trials. In the International Study of Infarct Survival [1988], patients born under the Gemini or Libra astrological birth signs did not survive as long when their treatment included aspirin. By contrast, aspirin offered apparent beneficial effects (longer survival time) to study participants from all other astrological birth signs. Szydloa et al. [2010] report similar spurious correlations when hypothesis are formulated with the data in hand. 3 A website with pictures is located at http://www.foldmoney.com/.
- 26. 10 PART IFOUNDATIONS Except for those who guide their lives by the stars, there is no hidden meaning or conspiracy in this result. When we describe a test as significant at the 5% or one-in-20 level, we mean that one in 20 times we will get a significant result even though the hypothesis is true. That is, when we test to see if there are any differences in the baseline values of the control and treatment groups, if we have made 20 different measurements, we can expect to see at least one statistically significant difference; in fact, we will see this result almost two-thirds of the time. This difference will not represent a flaw in our design but simply chance at work. To avoid this undesirable result—that is, to avoid attributing statistical significance to an insignificant random event, a so-called Type I error—we must distinguish between the hypotheses with which we began the study and those which came to mind afterward. We must accept or reject our initial hypotheses at the original significance level while demanding additional corroborating evidence for those exceptional results (such as a dependence of an outcome on astrological sign) that are uncovered for the first time during the trials. No reputable scientist would ever report results before successfully reproducing the experimental findings twice, once in the original laboratory and once in that of a colleague.4 The latter experiment can be particularly telling, as all too often some overlooked factor not controlled in the experiment—such as the quality of the laboratory water—proves responsible for the results observed initially. It is better to be found wrong in private, than in public. The only remedy is to attempt to replicate the findings with different sets of subjects, replicate, then replicate again. Persi Diaconis [1978] spent some years investigating paranormal phenomena. His scientific inquiries included investigating the powers linked to Uri Geller, the man who claimed he could bend spoons with his mind. Diaconis was not surprised to find that the hidden “powers” of Geller were more or less those of the average nightclub magician, down to and including forcing a card and taking advantage of ad-hoc, post-hoc hypotheses (Figure 1.1). When three buses show up at your stop simultaneously, or three rock stars die in the same year, or a stand of cherry trees is found amid a forest of oaks, a good statistician remembers the Poisson distribution. This distribution applies to relatively rare events that occur independently of one another (see Figure 1.2). The calculations performed by Siméon- 4 Remember “cold fusion”? In 1989, two University of Utah professors told the newspapers they could fuse deuterium molecules in the laboratory, solving the world’s energy problems for years to come. Alas, neither those professors nor anyone else could replicate their findings, though true believers abound (see http://www.ncas.org/erab/intro.htm).
- 27. CHAPTER 1 SOURCESOF ERROR 11 FIGURE 1.1. Photo of Geller. (Reprinted from German Language Wikipedia.) FIGURE 1.2. Frequency plot of the number of deaths in the Prussian army as a result of being kicked by a horse (there are 200 total observations). 109 65 22 3 2 3 4 5 or more 1 1 0 0 0 20 40 60 80 100 120 Number of deaths Frequency
- 28. 12 PART IFOUNDATIONS Denis Poisson reveal that if there is an average of one event per interval (in time or in space), whereas more than a third of the intervals will be empty, at least a quarter of the intervals are likely to include multiple events. Anyone who has played poker will concede that one out of every two hands contains “something” interesting. Do not allow naturally occurring results to fool you nor lead you to fool others by shouting, “Isn’t this incredible?” The purpose of a recent set of clinical trials was to see if blood flow and distribution in the lower leg could be improved by carrying out a simple surgical procedure prior to the administration of standard prescription medicine. The results were disappointing on the whole, but one of the marketing representatives noted that the long-term prognosis was excellent when a marked increase in blood flow was observed just after surgery. She suggested we calculate a p-value5 for a comparison of patients with an improved blood flow after surgery versus patients who had taken the prescription medicine alone. Such a p-value is meaningless. Only one of the two samples of patients in question had been taken at random from the population (those patients who received the prescription medicine alone). The other sample (those patients who had increased blood flow following surgery) was determined after the fact. To extrapolate results from the samples in hand to a larger TABLE 1.1. Probability of finding something interesting in a five-card hand Hand Probability Straight flush 0.0000 4-of-a-kind 0.0002 Full house 0.0014 Flush 0.0020 Straight 0.0039 Three of a kind 0.0211 Two pairs 0.0475 Pair 0.4226 Total 0.4988 5 A p-value is the probability under the primary hypothesis of observing the set of observations we have in hand. We can calculate a p-value once we make a series of assumptions about how the data were gathered. These days, statistical software does the calculations, but it’s still up to us to validate the assumptions.
- 29. CHAPTER 1 SOURCESOF ERROR 13 population, the samples must be taken at random from, and be representative of, that population. The preliminary findings clearly called for an examination of surgical procedures and of patient characteristics that might help forecast successful surgery. But the generation of a p-value and the drawing of any final conclusions had to wait for clinical trials specifically designed for that purpose. This does not mean that one should not report anomalies and other unexpected findings. Rather, one should not attempt to provide p-values or confidence intervals in support of them. Successful researchers engage in a cycle of theorizing and experimentation so that the results of one experiment become the basis for the hypotheses tested in the next. A related, extremely common error whose correction we discuss at length in Chapters 13 and 15 is to use the same data to select variables for inclusion in a model and to assess their significance. Successful model builders develop their frameworks in a series of stages, validating each model against a second independent dataset before drawing conclusions. One reason why many statistical models are incomplete is that they do not specify the sources of randomness generating variability among agents, i.e., they do not specify why otherwise observationally identical people make different choices and have different outcomes given the same choice.—James J. Heckman TO LEARN MORE On the necessity for improvements in the use of statistics in research publications, see Altman [1982, 1991, 1994, 2000, 2002]; Cooper and Rosenthal [1980]; Dar, Serlin, and Omer [1994]; Gardner and Bond [1990]; George [1985]; Glantz [1980]; Goodman, Altman, and George [1998]; MacArthur and Jackson [1984]; Morris [1988]; Strasak et al. [2007]; Thorn et al. [1985]; and Tyson et al. [1983]. Brockman and Chowdhury [1997] discuss the costly errors that can result from treating chaotic phenomena as stochastic.
- 30. CHAPTER 2 HYPOTHESES:THE WHY OF YOUR RESEARCH 15 All who drink of this treatment recover in a short time, Except those whom it does not help, who all die, It is obvious therefore, that it only fails in incurable cases. —Galen (129–199) IN THIS CHAPTER, AIMED AT BOTH RESEARCHERS WHO will analyze their own data as well as those researchers who will rely on others to assist them in the analysis, we review how to formulate a hypothesis that is testable by statistical means, the appropriate use of the null hypothesis, Neyman–Pearson theory, the two types of error, and the more general theory of decisions and losses. PRESCRIPTION Statistical methods used for experimental design and analysis should be viewed in their rightful role as merely a part, albeit an essential part, of the decision-making procedure: 1. Set forth your objectives and the use you plan to make of your research before you conduct a laboratory experiment, a clinical trial, a survey, or analyze an existing set of data. 2. Formulate your hypothesis and all of the associated alternatives. List possible experimental findings along with the conclusions you would draw and the actions you would take if this or another result should prove to be the case. Do all of these things before you complete a single data collection form, and before you turn on your computer. Chapter 2 Hypotheses: The Why of Your Research Common Errors in Statistics (and How to Avoid Them), Fourth Edition. Phillip I. Good and James W. Hardin. © 2012 John Wiley & Sons, Inc. Published 2012 by John Wiley & Sons, Inc.
- 31. 16 PART IFOUNDATIONS WHAT IS A HYPOTHESIS? A well-formulated hypothesis will be both quantifiable and testable, that is, involve measurable quantities or refer to items that may be assigned to mutually exclusive categories. It will specify the population to which the hypothesis will apply. A well-formulated statistical hypothesis takes one of two forms: 1. Some measurable characteristic of a defined population takes one of a specific set of values. 2. Some measurable characteristic takes different values in different defined populations, the difference(s) taking a specific pattern or a specific set of values. Examples of well-formed statistical hypotheses include the following: • For males over 40 suffering from chronic hypertension, a 100 mg daily dose of this new drug will lower diastolic blood pressure an average of 10 mm Hg. • For males over 40 suffering from chronic hypertension, a daily dose of 100 mg of this new drug will lower diastolic blood pressure an average of 10 mm Hg more than an equivalent dose of metoprolol. • Given less than 2 hours per day of sunlight, applying from 1 to 10 lbs of 23-2-4 fertilizer per 1000 square feet will have no effect on the growth of fescues and Bermuda grasses. “All redheads are passionate” is not a well-formed statistical hypothesis, not merely because “passionate” is ill defined, but because the word “all” suggests there is no variability. The latter problem can be solved by quantifying the term “all” to, let’s say, 80%. If we specify “passionate” in quantitative terms to mean “has an orgasm more than 95% of the time consensual sex is performed,” then the hypothesis “80% of redheads have an orgasm more than 95% of the time consensual sex is performed” becomes testable. Note that defining “passionate” to mean “has an orgasm every time consensual sex is performed” would not be provable as it too is a statement of the “all-or-none” variety. The same is true for a hypothesis such as “has an orgasm none of the times consensual sex is performed.” Similarly, qualitative assertions of the form “not all,” or “some” are not statistical in nature because these terms leave much room for subjective interpretation. How many do we mean by “some”? Five out of 100? Ten out of 100? The statements, “Doris J. is passionate,” or “Both Good brothers are 5′10″ tall″ is equally not statistical in nature as they concern specific
- 32. CHAPTER 2 HYPOTHESES:THE WHY OF YOUR RESEARCH 17 individuals rather than populations [Hagood, 1941]. Finally, note that until someone other than Thurber succeeds in locating unicorns, the hypothesis, “80% of unicorns are white” is not testable. Formulate your hypotheses so they are quantifiable, testable, and statistical in nature. HOW PRECISE MUST A HYPOTHESIS BE? The chief executive of a drug company may well express a desire to test whether “our antihypertensive drug can beat the competition.” The researcher, having done done preliminary reading of the literature, might want to test a preliminary hypothesis on the order of “For males over 40 suffering from chronic hypertension, there is a daily dose of our new drug that will lower diastolic blood pressure an average of 20 mm Hg.” But this hypothesis is imprecise. What if the necessary dose of the new drug required taking a tablet every hour? Or caused liver malfunction? Or even death? First, the researcher would need to conduct a set of clinical trials to determine the maximum tolerable dose (MTD). Subsequently, she could test the precise hypothesis, “a daily dose of one-third to one-fourth the MTD of our new drug will lower diastolic blood pressure an average of 20 mm Hg in males over 40 suffering from chronic hypertension.” In a series of articles by Horwitz et al. [1998], a physician and his colleagues strongly criticize the statistical community for denying them (or so they perceive) the right to provide a statistical analysis for subgroups not contemplated in the original study protocol. For example, suppose that in a study of the health of Marine recruits, we notice that not one of the dozen or so women who received a vaccine contracted pneumonia. Are we free to provide a p-value for this result? Statisticians Smith and Egger [1998] argue against hypothesis tests of subgroups chosen after the fact, suggesting that the results are often likely to be explained by the “play of chance.” Altman [1998; pp. 301–303], another statistician, concurs noting that, “. . . the observed treatment effect is expected to vary across subgroups of the data . . . simply through chance variation,” and that “doctors seem able to find a biologically plausible explanation for any finding.” This leads Horwitz et al. to the incorrect conclusion that Altman proposes that we “dispense with clinical biology (biologic evidence and pathophysiologic reasoning) as a basis for forming subgroups.” Neither Altman nor any other statistician would quarrel with Horwitz et al.’s assertion that physicians must investigate “how do we [physicians] do our best for a particular patient.”
- 33. 18 PART IFOUNDATIONS Scientists can and should be encouraged to make subgroup analyses. Physicians and engineers should be encouraged to make decisions based upon them. Few would deny that in an emergency, coming up with workable, fast-acting solutions without complete information is better than finding the best possible solution.1 But, by the same token, statisticians should not be pressured to give their imprimatur to what, in statistical terms, is clearly an improper procedure, nor should statisticians mislabel suboptimal procedures as the best that can be done.2 We concur with Anscombe [1963] who writes, “. . . the concept of error probabilities of the first and second kinds . . . has no direct relevance to experimentation. . . . The formation of opinions, decisions concerning further experimentation, and other required actions, are not dictated . . . by the formal analysis of the experiment, but call for judgment and imagination. . . . It is unwise for the experimenter to view himself seriously as a decision-maker. . . . The experimenter pays the piper and calls the tune he likes best; but the music is broadcast so that others might listen. . . . ” A Bill of Rights for Subgroup Analysis • Scientists can and should be encouraged to make subgroup analyses. • Physicians and engineers should be encouraged to make decisions utilizing the findings of such analyses. • Statisticians and other data analysts can and should rightly refuse to give their imprimatur to related tests of significance. FOUND DATA p-values should not be computed for hypotheses based on “found data” as of necessity all hypotheses related to found data are after the fact. This rule does not apply if the observer first divides the data into sections. One part is studied and conclusions drawn; then the resultant hypotheses are tested on the remaining sections. Even then, the tests are valid only if the found data can be shown to be representative of the population at large. 1 Chiles [2001; p. 61]. 2 One is reminded of the Dean, several of them in fact, who asked me to alter my grades. “But that is something you can do as easily as I.” “Why Dr. Good, I would never dream of overruling one of my instructors.” See also Murder at Oklahoma by J. M. Bickham.
- 34. CHAPTER 2 HYPOTHESES:THE WHY OF YOUR RESEARCH 19 NULL OR NIL HYPOTHESIS A major research failing seems to be the exploration of uninteresting or even trivial questions. . . . In the 347 sampled articles in Ecology containing null hypotheses tests, we found few examples of null hypotheses that seemed biologically plausible.— Anderson, Burnham, and Thompson [2000]. We do not perform an experiment to find out if two varieties of wheat or two drugs are equal. We know in advance, without spending a dollar on an experiment, that they are not equal.— Deming [1975]. Test only relevant null hypotheses. The null hypothesis has taken on an almost mythic role in contemporary statistics. Obsession with the null (more accurately spelled and pronounced nil), has been allowed to shape the direction of our research. We have let the tool use us instead of us using the tool.3 Virtually any quantifiable hypothesis can be converted into null form. There is no excuse and no need to be content with a meaningless nil. For example, suppose we want to test that a given treatment will decrease the need for bed rest by at least three days. Previous trials have convinced us that the treatment will reduce the need for bed rest to some degree, so merely testing that the treatment has a positive effect would yield no new information. Instead, we would subtract three from each observation and then test the nil hypothesis that the mean value is zero. We often will want to test that an effect is inconsequential, not zero but close to it, smaller than d, say, where d is the smallest biological, medical, physical, or socially relevant effect in our area of research. Again, we would subtract d from each observation before proceeding to test a null hypothesis. The quote from Deming above is not quite correct as often we will wish to demonstrate that two drugs or two methods yield equivalent results. As shown in Chapter 5, we may test for equivalence using confidence intervals; a null hypothesis is not involved To test that “80% of redheads are passionate,” we have two choices depending on how “passion” is measured. If “passion” is an all-or-none phenomena, then we can forget about trying to formulate a null hypothesis and instead test the binomial hypothesis that the probability p 3 See, for example, Hertwig and Todd [2000].
- 35. 20 PART IFOUNDATIONS that a redhead is passionate is 80%. If “passion” can be measured on a seven-point scale and we define “passionate” as “passion” greater than or equal to 5, then our hypothesis becomes “the 20th percentile of redhead passion exceeds 5.” As in the first example above, we could convert this to a null hypothesis by subtracting five from each observation. But the effort is unnecessary as this problem, too, reduces to a test of a binomial parameter. NEYMAN–PEARSON THEORY Formulate your alternative hypotheses at the same time you set forth the hypothesis that is of chief concern to you. When the objective of our investigations is to arrive at some sort of conclusion, then we need not only have a single primary hypothesis in mind but one or more potential alternative hypotheses. The cornerstone of modern hypothesis testing is the Neyman–Pearson lemma. To get a feeling for the working of this mathematical principle, suppose we are testing a new vaccine by administering it to half of our test subjects and giving a supposedly harmless placebo to each of the remainder. We proceed to follow these subjects over some fixed period and note which subjects, if any, contract the disease that the new vaccine is said to offer protection against. We know in advance that the vaccine is unlikely to offer complete protection; indeed, some individuals may actually come down with the disease as a result of taking the vaccine. Many factors over which we have no control, such as the weather, may result in none of the subjects, even those who received only placebo, contracting the disease during the study period. All sorts of outcomes are possible. The tests are being conducted in accordance with regulatory agency guidelines. Our primary hypothesis H is that the new vaccine can cut the number of infected individuals in half. From the regulatory agency’s perspective, the alternative hypothesis A1 is that the new vaccine offers no protection or, A2, no more protection than is provided by the best existing vaccine. Our task before the start of the experiment is to decide which outcomes will rule in favor of the alternative hypothesis A1 (or A2) and which in favor of the primary hypothesis H. Note that neither a null nor a nil hypothesis is yet under consideration. Because of the variation inherent in the disease process, each and every one of the possible outcomes could occur regardless of which of the hypotheses is true. Of course, some outcomes are more likely if A1 is true, for example, 50 cases of pneumonia in the placebo group and 48 in the
- 36. CHAPTER 2 HYPOTHESES:THE WHY OF YOUR RESEARCH 21 vaccine group, and others are more likely if the primary hypothesis is true, for example, 38 cases of pneumonia in the placebo group and 20 in the vaccine group. Following Neyman and Pearson, we order each of the possible outcomes in accordance with the ratio of its probability or likelihood when the primary hypothesis is true to its probability when the alternative hypothesis is true.4 When this likelihood ratio is large, we shall say the outcome rules in favor of the alternative hypothesis. Working downward from the outcomes with the highest values, we continue to add outcomes to the rejection region of the test—so-called because these are the outcomes for which we would reject the primary hypothesis—until the total probability of the rejection region under the primary hypothesis is equal to some predesignated significance level.5 In the following example, we would reject the primary hypothesis at the 10% level only if the test subject really liked a product. Really Hate Dislike Indifferent Like Really Like Primary Hypothesis 10% 20% 40% 20% 10% Alternate Hypothesis 5% 10% 30% 30% 25% Likelihood Ratio 1/2 1/2 3/4 3/2 5/2 To see that we have done the best we can do, suppose we replace one of the outcomes we assigned to the rejection region with one we did not. The probability that this new outcome would occur if the primary hypothesis is true must be less than or equal to the probability that the outcome it replaced would occur if the primary hypothesis is true. Otherwise, we would exceed the significance level. Because of how we assigned outcome to the rejection region, the likelihood ratio of the new outcome is smaller than the likelihood ratio of the old outcome. Thus, the probability the new outcome would occur if the alternative hypothesis is true must be less than or equal to the probability that the outcome it replaced would occur if the alternative 4 When there are more than two hypotheses, the rejection region of the best statistical test (and the associated power and significance level) will be based upon the primary and alternative hypotheses that are the most difficult to distinguish from one another. 5 For convenience in calculating a rejection region, the primary and alternate hypotheses may be interchanged. Thus, the statistician who subsequently performs an analysis of the vaccine data may refer to testing the nil hypothesis A1 against the alternative H.
- 37. 22 PART IFOUNDATIONS hypothesis is true. That is, by swapping outcomes we have reduced the power of our test. By following the method of Neyman and Pearson and maximizing the likelihood ratio, we obtain the most powerful test at a given significance level. To take advantage of Neyman and Pearson’s finding, we need to have an alternative hypothesis or alternatives firmly in mind when we set up a test. Too often in published research, such alternative hypotheses remain unspecified or, worse, are specified only after the data are in hand. We must specify our alternatives before we commence an analysis, preferably at the same time we design our study. Are our alternatives one-sided or two-sided? If we are comparing several populations at the same time, are their means ordered or unordered? The form of the alternative will determine the statistical procedures we use and the significance levels we obtain. Decide beforehand whether you wish to test against a one-sided or a two- sided alternative. One-sided or Two-sided Suppose on examining the cancer registry in a hospital, we uncover the following data that we put in the form of a 2 × 2 contingency table: Survived Died Total Men 9 1 10 Women 4 10 14 Total 13 11 24 The 9 denotes the number of males who survived, the 1 denotes the number of males who died, and so forth. The four marginal totals or marginals are 10, 14, 13, and 11. The total number of men in the study is 10, whereas 14 denotes the total number of women, and so forth. The marginals in this table are fixed because, indisputably, there are 11 dead bodies among the 24 persons in the study and 14 women. Suppose that before completing the table, we lost the subject IDs so that we could no longer identify which subject belonged in which category. Imagine you are given two sets of 24 labels. The first set has 14 labels with the word “woman” and 10 labels with the word “man.” The second set of labels has 11 labels with the word “dead” and 12 labels with the word “alive.” Under the null hypothesis, you are allowed to distribute the labels to subjects independently of one another. One label from each of the two sets per subject, please.
- 38. CHAPTER 2 HYPOTHESES:THE WHY OF YOUR RESEARCH 23 There are a total of 24 10 24 10 14 1 961 256 ⎛ ⎝ ⎜ ⎞ ⎠ ⎟ = = ! ( ! !) , , / ways you could hand out the labels; Table 2.1 illustrates two possible configurations. 14 10 10 1 10 010 ⎛ ⎝ ⎜ ⎞ ⎠ ⎟ ⎛ ⎝ ⎜ ⎞ ⎠ ⎟ = , of the assignments result in tables that are as extreme as our original table (that is, in which 90% of the men survive) and 14 11 10 0 364 ⎛ ⎝ ⎜ ⎞ ⎠ ⎟ ⎛ ⎝ ⎜ ⎞ ⎠ ⎟ = in tables that are more extreme (100% of the men survive). This is a very small fraction of the total, (10,010 + 364)/ (1,961,256) = 0.529%, so we conclude that a difference in survival rates of the two sexes as extreme as the difference we observed in our original table is very unlikely to have occurred by chance alone. We reject the hypothesis that the survival rates for the two sexes are the same and accept the alternative hypothesis that, in this instance at least, males are more likely to profit from treatment. In the preceding example, we tested the hypothesis that survival rates do not depend on sex against the alternative that men diagnosed with cancer are likely to live longer than women similarly diagnosed. We rejected the null hypothesis because only a small fraction of the possible tables were as extreme as the one we observed initially. This is an example of a one-tailed test. But is it the correct test? Is this really the alternative hypothesis we would have proposed if we had not already seen the data? Wouldn’t we have been just as likely to reject the null hypothesis that men and women profit the same from treatment if we had observed a table of the following form? Survived Died Total Men 0 10 10 Women 13 1 14 Total 13 11 24 TABLE 2.1. In terms of the relative survival rates of the two sexes, the first of these tables is more extreme than our original table. The second is less extreme. Survived Died Total Men 10 0 10 Women 3 11 14 Total 13 11 24 Survived Died Total Men 8 2 10 Women 5 9 14 Total 13 11 24
- 39. 24 PART IFOUNDATIONS Of course, we would! In determining the significance level in the present example, we must add together the total number of tables that lie in either of the two extremes or tails of the permutation distribution. The critical values and significance levels are quite different for one- tailed and two-tailed tests and, all too often, the wrong test has been employed in published work. McKinney et al. [1989] reviewed some 70-plus articles that appeared in six medical journals. In over half of these articles, Fisher’s exact test was applied improperly. Either a one-tailed test had been used when a two-tailed test was called for or the authors of the paper simply had not bothered to state which test they had used. Of course, unless you are submitting the results of your analysis to a regulatory agency, no one will know whether you originally intended a one-tailed test or a two-tailed test and subsequently changed your mind. No one will know whether your hypothesis was conceived before you started or only after you had examined the data. All you have to do is lie. Just recognize that if you test an after-the-fact hypothesis without identifying it as such, you are guilty of scientific fraud. When you design an experiment, decide at the same time whether you wish to test your hypothesis against a two-sided or a one-sided alternative. A two-sided alternative dictates a two-tailed test; a one-sided alternative dictates a one-tailed test. As an example, suppose we decide to do a follow-on study of the cancer registry to confirm our original finding that men diagnosed as having tumors live significantly longer than women similarly diagnosed. In this follow-on study, we have a one-sided alternative. Thus, we would analyze the results using a one-tailed test rather than the two-tailed test we applied in the original study. Determine beforehand whether your alternative hypotheses are ordered or unordered. Ordered or Unordered Alternative Hypotheses? When testing qualities (number of germinating plants, crop weight, etc.) from k samples of plants taken from soils of different composition, it is often routine to use the F-ratio of the analysis of variance. For contingency tables, many routinely use the chi-square test to determine if the differences among samples are significant. But the F-ratio and the chi-square are what are termed omnibus tests, designed to be sensitive to all possible alternatives. As such, they are not particularly sensitive to ordered alternatives such “as more fertilizer equals more growth” or “more aspirin equals faster relief of headache.” Tests for such ordered responses at k distinct treatment levels should properly use the Pitman
- 40. CHAPTER 2 HYPOTHESES:THE WHY OF YOUR RESEARCH 25 correlation described by Frank, Trzos, and Good [1978] when the data are measured on a metric scale (e.g., weight of the crop). Tests for ordered responses in 2 × C contingency tables (e.g., number of germinating plants) should use the trend test described by Berger, Permutt, and Ivanova [1998]. We revisit this topic in more detail in the next chapter. DEDUCTION AND INDUCTION When we determine a p-value as we did in the example above, we apply a set of algebraic methods and deductive logic to deduce the correct value. The deductive process is used to determine the appropriate size of resistor to use in an electric circuit, to determine the date of the next eclipse of the moon, and to establish the identity of the criminal (perhaps from the fact the dog did not bark on the night of the crime). Find the formula, plug in the values, turn the crank and out pops the result (or it does for Sherlock Holmes,6 at least). When we assert that for a given population a percentage of samples will have a specific composition, this also is a deduction. But when we make an inductive generalization about a population based upon our analysis of a sample, we are on shakier ground. It is one thing to assert that if an observation comes from a normal distribution with mean zero, the probability is one-half that it is positive. It is quite another if, on observing that half the observations in the sample are positive, we assert that half of all the possible observations that might be drawn from that population will be positive also. Newton’s Law of Gravitation provided an almost exact fit (apart from measurement error) to observed astronomical data for several centuries; consequently, there was general agreement that Newton’s generalization from observation was an accurate description of the real world. Later, as improvements in astronomical measuring instruments extended the range of the observable universe, scientists realized that Newton’s Law was only a generalization and not a property of the universe at all. Einstein’s Theory of Relativity gives a much closer fit to the data, a fit that has not been contradicted by any observations in the century since its formulation. But this still does not mean that relativity provides us with a complete, correct, and comprehensive view of the universe. In our research efforts, the only statements we can make with God-like certainty are of the form “our conclusions fit the data.” The true nature of the real world is unknowable. We can speculate, but never conclude. 6 See “Silver Blaze” by A. Conan-Doyle, Strand Magazine, December 1892.
- 41. 26 PART IFOUNDATIONS LOSSES In our first advanced course in statistics, we read in the first chapter of Lehmann [1986] that the “optimal” statistical procedure would depend on the losses associated with the various possible decisions. But on day one of our venture into the real world of practical applications, we were taught to ignore this principle. At that time, the only computationally feasible statistical procedures were based on losses that were proportional to the square of the difference between estimated and actual values. No matter that the losses really might be proportional to the absolute value of those differences, or the cube, or the maximum over a certain range. Our options were limited by our ability to compute. Computer technology has made a series of major advances in the past half century. What forty years ago required days or weeks to calculate takes only milliseconds today. We can now pay serious attention to this long-neglected facet of decision theory: the losses associated with the varying types of decision. Suppose we are investigating a new drug: We gather data, perform a statistical analysis, and draw a conclusion. If chance alone is at work yielding exceptional values and we opt in favor of the new drug, we have made an error. We also make an error if we decide there is no difference and the new drug really is better. These decisions and the effects of making them are summarized in Table 2.2. We distinguish the two types of error because they have quite different implications, as described in Table 2.2. As a second example, Fears, Tarone, and Chu [1977] use permutation methods to assess several standard screens for carcinogenicity. As shown in Table 2.3 their Type I error, a false positive, consists of labeling a relatively innocuous compound as carcinogenic. Such an action means economic loss for the manufacturer and the denial to the public of the compound’s benefits. Neither consequence is desirable. But a false negative, a Type II error, is much TABLE 2.2. Decision making under uncertainty The Facts Our Decision No Difference No Difference Drug is Better Type I error: Correct Manufacturer wastes money developing ineffective drug. Drug is Better Type II error: Manufacturer misses opportunity for profit. Correct Public denied access to effective treatment.
- 42. CHAPTER 2 HYPOTHESES:THE WHY OF YOUR RESEARCH 27 worse as it would mean exposing a large number of people to a potentially lethal compound. What losses are associated with the decisions you will have to make? Specify them now before you begin. DECISIONS The primary hypothesis/alternative hypothesis duality is inadequate in most real-life situations. Consider the pressing problems of global warming and depletion of the ozone layer. We could collect and analyze yet another set of data and then, just as is done today, make one of three possible decisions: reduce emissions, leave emission standards alone, or sit on our hands and wait for more data to come in. Each decision has consequences, as shown in Table 2.4. As noted at the beginning of this chapter, it is essential that we specify in advance the actions to be taken for each potential result. Always suspect are after-the-fact rationales that enable us to persist in a pattern of conduct despite evidence to the contrary. If no possible outcome of a study will be sufficient to change our mind, then we ought not undertake such a study in the first place. Every research study involves multiple issues. Not only might we want to know whether a measurable, biologically (or medically, physically, or TABLE 2.3. Decision making under uncertainty The Facts Fears et al.’s Decision Not a Carcinogen Compound a Carcinogen Not a Carcinogen Type I error: Manufacturer misses opportunity for profit. Public denied access to effective treatment. Carcinogen Type II error: Patients die; families suffer; Manufacturer sued. TABLE 2.4. Results of a presidential decision under different underlying facts about the cause of hypothesized global warming President’s Decision on Emissions The Facts Reduce Emissions Gather More Data Change Unnecessary Emissions responsible Global warming slows Decline in quality of life (irreversible?) Decline in quality of life Emissions have no effect Economy disrupted Sampling costs
- 43. 28 PART IFOUNDATIONS sociologically) significant effect takes place, but what the size of the effect is and the extent to which the effect varies from instance to instance. We would also want to know what factors, if any, will modify the size of the effect or its duration. We may not be able to address all these issues with a single dataset. A preliminary experiment might tell us something about the possible existence of an effect, along with rough estimates of its size and variability. Hopefully, we glean enough information to come up with doses, environmental conditions, and sample sizes to apply in collecting and evaluating the next dataset. A list of possible decisions after the initial experiment includes “abandon this line of research,” “modify the environment and gather more data,” and “perform a large, tightly controlled, expensive set of trials.” Associated with each decision is a set of potential gains and losses. Common sense dictates we construct a table similar to Table 2.2 or 2.3 before we launch a study. For example, in clinical trials of a drug we might begin with some animal experiments, then progress to Phase I clinical trials in which, with the emphasis on safety, we look for the maximum tolerable dose. Phase I trials generally involve only a small number of subjects and a one-time or short-term intervention. An extended period of several months may be used for follow-up purposes. If no adverse effects are observed, we might decide to pursue a Phase II set of trials in the clinic, in which our objective is to determine the minimum effective dose. Obviously, if the minimum effective dose is greater than the maximum tolerable dose, or if some dangerous side effects are observed that we did not observe in the first set of trials, we will abandon the drug and go on to some other research project. But if the signs are favorable, then and only then will we go to a set of Phase III trials involving a large number of subjects observed over an extended time period. Then, and only then, will we hope to get the answers to all our research questions. Before you begin, list all the consequences of a study and all the actions you might take. Persist only if you can add to existing knowledge. TO LEARN MORE For more thorough accounts of decision theory, the interested reader is directed to Berger [1986], Blyth [1970], Cox [1958], DeGroot [1970], and Lehmann [1986]. For an applied perspective, see Clemen [1991], Berry [1995], and Sox, Blatt, Higgins, and Marton [1988]. Over 300 references warning of the misuse of null hypothesis testing can be accessed online at http://www.cnr.colostate.edu/∼anderson/ thompson1.html. Alas, the majority of these warnings are ill informed,
- 44. CHAPTER 2 HYPOTHESES:THE WHY OF YOUR RESEARCH 29 stressing errors that will not arise if you proceed as we recommend and place the emphasis on the why, not the what, of statistical procedures. Use statistics as a guide to decision making rather than a mandate. Neyman and Pearson [1933] first formulated the problem of hypothesis testing in terms of two types of error. Extensions and analyses of their approach are given by Lehmann [1986] and Mayo [1996]. Their approach has not gone unchallenged, as seen in Berger [2003], Berger and Berry [1988], Berger and Selke [1987], Berkson [1942], Morrison and Henkel [1970], Savage [1972], Schmidt [1996], Seidenfeld [1979], and Sterne, Smith, and Cox [2001]. Hunter and Schmidt [1997] list and dismiss many of their objections. Guthery, Lusk, and Peterson [2001] and Rozeboom [1960] are among those who have written about the inadequacy of the null hypothesis. For more guidelines on formulating meaningful primary hypotheses, see Selike, Bayarri, and Berger [2001]. Clarity in hypothesis formulation is essential; ambiguity can only yield controversy; see, for example, Kaplan [2001]. Venn [1888] and Reichenbach [1949] are among those who have attempted to construct a mathematical bridge between what we observe and the reality that underlies our observations. Such efforts to the contrary, extrapolation from the sample to the population is not merely a matter of applying Sherlock Holmes-like deductive logic but entails a leap of faith. A careful reading of Locke [1700], Berkeley [1710], Hume [1748], and Lonergan [1992] is an essential prerequisite to the application of statistics. See also Buchanan-Wollaston [1935], Cohen [1990], Copas [1997], and Lindley [2000].
- 45. CHAPTER 3 COLLECTINGDATA 31 GIGO: Garbage in; Garbage out. “Fancy statistical methods will not rescue garbage data.” Course notes of Raymond J. Carroll [2001]. THE VAST MAJORITY OF ERRORS IN STATISTICS (and, not incidentally, in most human endeavors) arise from a reluctance (or even an inability) to plan. Some demon (or demonic manager) seems to be urging us to cross the street before we have had the opportunity to look both ways. Even on those rare occasions when we do design an experiment, we seem more obsessed with the mechanics than with the underlying concepts. In this chapter, we review the fundamental concepts of experimental design, the choice of primary and secondary variables, the selection of measurement devices, the determination of sample size, the assumptions that underlie most statistical procedures along with the precautions necessary to ensure they are satisfied and that the data you collect will be representative of the population as a whole. We do not intend to replace a text on experiment or survey design, but to supplement one, providing examples and solutions that are often neglected in courses on the subject. PREPARATION The first step in data collection is to have a clear, preferably written statement of your objectives. In accordance with Chapter 1, you will have defined the population or populations from which you intend to sample and have identified the characteristics of these populations you wish to investigate. Chapter 3 Collecting Data Common Errors in Statistics (and How to Avoid Them), Fourth Edition. Phillip I. Good and James W. Hardin. © 2012 John Wiley & Sons, Inc. Published 2012 by John Wiley & Sons, Inc.
- 46. 32 PART IFOUNDATIONS You developed one or more well-formulated hypotheses (the topic of Chapter 2) and have some idea of the risks you will incur should your analysis of the collected data prove to be erroneous. You will need to decide what you wish to observe and measure, and how you will go about observing it. We refer here not only to the primary variables or endpoints, but to the secondary variables or cofactors that may influence the former’s values. Indeed, it is essential that you be aware of all potential sources of variation. Good practice is to draft the analysis section of your final report based on the conclusions you would like to make. What information do you need to justify these conclusions? All such information must be collected. The next section is devoted to the choice of response variables and measuring devices, followed by sections on determining sample size and preventive steps to ensure your samples will be analyzable by statistical methods. RESPONSE VARIABLES Know What You Want to Measure If you do not collect the values of cofactors, you will be unable to account for them later. As whiplash injuries are a common consequence of rear-end collisions, there is an extensive literature on the subject. Any physician will tell you that the extent and duration of such injuries will depend upon the sex, age, and physical condition of the injured individual as well as any prior injuries the individual may have suffered. Yet we found article after article that failed to account for these factors; for example, Krafft, Kullgren, Ydenius, and Tingvall [2002], Kumar, Ferrari, and Narayan [2005], and Tencer, Sohail, and Kevin [2001] did not report the sex, age, or prior injuries of their test subjects. Will you measure an endpoint such as death or a surrogate such as the presence of HIV antibodies? A good response variable takes values over a sufficiently large range so that they discriminate well [Bishop and Talbot, 2001]. The regression slope describing the change in systolic blood pressure (in mm Hg) per 100 mg of calcium intake is strongly influenced by the approach used for assessing the amount of calcium consumed (Cappuccio et al., 1995). The association is small and only marginally significant with diet histories (slope −0.01 (−0.003 to −0.016)) but large and highly significant when food frequency questionnaires are used (slope −0.15 (−0.11 to −0.19). With studies using 24 hour recall, an intermediate result emerges (slope −0.06 (−0.09 to −0.03). Diet histories assess patterns of
- 47. CHAPTER 3 COLLECTINGDATA 33 usual intake over long periods of time and require an extensive interview with a nutritionist, whereas 24-hour recall and food frequency questionnaires are simpler methods that reflect current consumption (Block, 1982). Before we initiate data collection, we must have a firm idea of what we will measure and how we will measure it. A good response variable • Is easy to record—imagine weighing a live pig. • Can be measured objectively on a generally accepted scale. • Is measured in appropriate units. • Takes values over a sufficiently large range that discriminates well. • Is well defined. A patient is not “cured” but may be “discharged from hospital” or “symptom-free for a predefined period.” • Has constant variance over the range used in the experiment (Bishop and Talbot, 2001). Collect exact values whenever possible. A second fundamental principle is also applicable to both experiments and surveys: Collect exact values whenever possible. Worry about grouping them in intervals or discrete categories later. A long-term study of buying patterns in New South Wales illustrates some of the problems caused by grouping prematurely. At the beginning of the study, the decision was made to group the incomes of survey subjects into categories—under $20,000, $20,000 to $30,000, and so forth. Six years of steady inflation later and the organizers of the study realized that all the categories had to be adjusted. An income of $21,000 at the start of the study would only purchase $18,000 worth of goods and housing at the end (see Figure 3.1). The problem was that those surveyed toward the end had filled out forms with exactly the same income categories. Had income been tabulated to the nearest dollar, it would have been easy to correct for increases in the cost of living and convert all responses to the same scale. But the study designers had not considered these issues. A precise and costly survey had been reduced to mere guesswork. You can always group your results (and modify your groupings) after a study is completed. If after-the-fact grouping is a possibility, your design should state how the grouping will be determined; otherwise there will be the suspicion that you chose the grouping to obtain desired results. Experiments Measuring devices differ widely both in what they measure and the precision with which they measure it. As noted in the next section of this chapter, the greater the precision with which measurements are made, the
- 48. 34 PART IFOUNDATIONS smaller the sample size required to reduce both Type I and Type II errors below specific levels. All measuring devices have both linear and nonlinear ranges; the sensitivity, accuracy, and precision of the device are all suspect for both very small and very large values. Your measuring device ought be linear over the entire range of interest. Before you rush out and purchase the most expensive and precise measuring instruments on the market, consider that the total cost C of an experimental procedure is S + nc, where n is the sample size and c is the cost per unit sampled. The startup cost S includes the cost of the measuring device. c is made up of the cost of supplies and personnel costs. The latter includes not only the time spent on individual measurements but in preparing and calibrating the instrument for use. Less obvious factors in the selection of a measuring instrument include impact on the subject, reliability (personnel costs continue even when an instrument is down), and reusability in future trials. For example, one of the advantages of the latest technology for blood analysis is that less blood needs to be drawn from patients. Less blood means happier subjects mean fewer withdrawals and a smaller initial sample size. FIGURE 3.1. Equivalent purchasing powers over time using Consumer Price Index calculations. Each year shows the cost of the equivalent goods/services. 1970 1975 1980 1985 1990 1995 2000 2005 $0.00 $0.50 $1.00 $1.50 $2.00 $2.50 $3.00 $3.50 $4.00 $4.50 $5.00 $5.50 $1.00 $1.39 $2.12 $2.77 $3.37 $3.93 $4.44 $5.02 Year
- 49. Another Random ScribdDocument with Unrelated Content
- 50. Used with therifle, this grenade has a maximum range of 400 meters. At the extremity opposite the grenade, the copper rod ends in a copper stem about 3 centimeters in length, movable about the axis of the rod. This stem is covered with a copper sleeve of slight thickness, which is attached to it only at the extremity fastened to the rod. The diameter of the exterior of the sleeve must be such that it can be pushed into the gun barrel without pressure. To fire the grenade, a blank cartridge is placed in the chamber of the rifle; the quantity of powder left in the cartridge is regulated according to the distance at which the missile is to be thrown. At the moment of firing, the explosive gases penetrate between the sleeve and the stem and jam the sleeve against the grooves of the barrel. The sleeve and the stem, which is attached to it, take a movement of rotation in the grooves of the barrel, which insures the direction of the missile and the maximum efficiency of the explosive gases of the cartridge. The bracelet grenade and the German grenade just described have to be made in a factory. Attempts have been made to construct similar missiles with the explosives which are at hand at the front, cheddite and melinite. Several kinds have been made: a primed cartridge and a primed hand petard, fitted on a wooden paddle, a preserved meat tin can filled with explosive, etc. The Germans have hand petards similar to those of the Allies but with different explosives. These missiles are primed by a detonator and a slow match and can be thrown about 30 meters. The discharge takes place either automatically or by tinder. They are made on the spot and very rapidly. The assaulting troops carry them in baskets or strung on a circle of wire carried on the shoulder.
- 51. Plate 41d.—Throwing handgrenades. Grenades and petards constitute a terrible weapon. These projectiles exert considerable moral effect owing to the violence of their explosion and the awful wounds they occasion, and they make it possible to reach the enemy at points where it is impossible to use the rifle and bayonet.
- 52. Emanation CHAPTER IX Gas Warfare Germanyfirst made use of poisonous and asphyxiating gases on the field of battle. It has become an accepted element in the present war. Every soldier should, therefore, have a knowledge of the various ways in which gas is employed in the attack, as well as the measures to be taken to counteract its effect in the defense. The two methods of disseminating the gas over the battlefield are by emanation and grenades charged with it. This method has for its object to create a poisonous or irritant atmosphere. This is accomplished by means of the arsenic and phosphorous gas being forced through tubes in the direction of the enemy or by means of liquefied chlorine, bromide, phosgene and sulphuretted hydrogen gas stored in cylinders under high pressure. To be successful the gas attack must be attended by the following conditions. 1. The weather must be comparatively calm with a wind blowing in the direction of the enemy at about 5 miles an hour. If the wind is too strong the gas will be carried over the enemy's trenches so rapidly that it will not settle in them. If the wind is too light the gas will be carried up into the air and disseminate or may even be blown back into our own trenches, in which case chloride of lime scattered about freely will disperse them. 2. There must be no rain, for that would quickly disseminate the gas and negative the effect. 3. The attack must come as a surprise. If the elements of surprise are missing and the enemy has time to take protective measures,
- 53. Shell and Grenade Method Defense theeffect is lost. If the surprise is complete, the enemy trenches should be emptied very quickly. 4. The gas used must be heavier than the air, so that it will sift into the enemy's trenches as it passes them. It is impracticable to decide upon any definite hour for launching the gas attack. Everything depends upon the direction and velocity of the wind. If an hour has been tentatively designated and the wind changes, the attack will have to be postponed. When an assault follows the gas attack the men should wear the smoke helmets for at least 30 minutes after the dissemination has ceased; in fact they must not be removed until the order to do so is given by the officer commanding the attack. You will appreciate that the enemy's machine gunners may have better protection than the men in the bays of the trenches. In this method the gas dissemination is effected by means of shells or bombs being fired into the enemy's trenches containing the desired substances which are released and give off irritant fumes on explosion. The grenades used weigh about 1 pound. They are similar in appearance to the ordinary tin can grenade. Their effect in a trench will continue for 20 to 30 minutes. In the attack a large number should be concentrated in a particular area to produce a large volume of gas. They are thrown by hand, trench mortar or catapult. Surprise must be guarded against in every possible way. The direction of the wind must be continually watched, and when its velocity and direction are specially favorable the protective measures must be kept ready for instant use and special observers posted. Previous to an attack the enemy may remain comparatively quiet for several days. Noises like the moving of sheet iron may be heard. Preparations may be observed along the position. When the attack starts a hissing noise is heard; this latter is one of the indications that may be evident at night.
- 54. Helmets Dummy Helmets Each manon duty in the trenches is provided with two smoke helmets, specially devised and constructed so as to absorb the gas and neutralize its effect, and which if properly cared for and used will provide complete protection from any substance likely to be used by the enemy. They are fitted with a valve tube through which to breathe and with goggles to see through. There are certain rules prescribed for their care and use. 1. They must not be removed from the protective covering except for actual use against an attack. 2. When the helmet has been used once it should be replaced by a new one. Dummy gas helmets will be provided in each organization by which the men may be practiced in putting them on. The men must be thoroughly drilled in the methods to be employed. The following directions accompany the helmets issued to the British Army. When our helmets are issued it is probable that each will be accompanied by a complete set of rules for its use and full instructions for the method of getting into it and for its care and preservation.
- 55. Description Plate 41e.—Gas helmet. Directionfor Use and Care of Tube Helmets These helmets are the same as the smoke helmet already issued, except that stronger chemicals are added and a tube valve provided through which to breathe out. The tube valve makes the helmet cooler and saves
- 56. Directions for Use Directions for Careof Tube Helmet Improvised Methods chemicals from being affected by the breath. The wearer cannot breathe in through the tube valve; this is intended for breathing out only. Remove paper cap from mouthpiece of tube valve. Remove service cap. Pull helmet over head. Adjust so that goggles are over eyes. Tuck in skirt of helmet under coat collar and button coat so as to close in skirt of helmet. Hold the tube lightly in lips or teeth like stem of pipe, so as to be able to breathe in past it and out through it. Breathe in through mouth and nose, using the air inside the helmet. Breathe out through tube only. 1. Do not remove the helmet from its waterproof case except to use for protection against gas. 2. Never use your tube helmet for practice or drill. Special helmets are kept in each company for instruction only. Should the goggles become misty during use they can be cleared by rubbing them gently against the forehead. When lacrimatory gases are used goggles affording mechanical protection may be worn, as these gases are not likely to irritate the lungs, though they sometimes produce sickness. If a soldier does not possess one of the official pattern respirators, the following measures will be found useful: 1. Wet and wring out any woolen article, such as a stocking or muffler, so as to form a thick pad large enough to cover the nose and mouth, and press firmly over both. 2. Place in a scarf, stocking or handkerchief, a pad of about three handfuls of earth, preferably damp, and tie it firmly over the mouth and nose.
- 57. Knapsack Sprayers 3. A wetcloth pulled down over the eyes will be found useful as additional protection, especially against certain gases other than chlorine or when the gas is too strong for the ordinary respirator. 4. A stocking, wetted with water and soda solution or tea, folded into eight folds and firmly held or tied over the nose. 5. A sock folded fourfold similarly wetted and held or tied. If the sock or comforter has been soaked in soda solution it will still act efficiently when dry, though, if possible, it should be moist. The spare tapes from puttees may be used for tying on the sock. 6. Any loose fabric, such as a sock, sandbag, woolen scarf or comforter, soaked in urine, then wrung out to allow of free breathing and tied tightly over the nose and mouth. In the absence of any other cloths, the flannel waistbands issued for winter use could be used for this purpose. Knapsack sprayers are issued for use to clear gases out of the trenches after the cloud has blown over. A man with the sprayer on his back (and wearing his smoke helmet) slowly traverses the trench, working the spray. If this is not done the heavy poisonous gas may linger in the trench for days and be a source of great danger. If supports or reinforcements enter a trench charged with gas, they should be preceded by a man using a sprayer. Sprayers are charged with sodium thiosulphate—more commonly known as "hypo"—6 pounds being dissolved in a bucket of water and a handful of ordinary washing soda added. Garden syringes and buckets may be used if sprayers are not available, but these are not so effective. Sprayers should be charged before they are taken up to the trenches, and should be kept ready for immediate use.
- 58. Every officer defendinga trench against an enemy gas attack should endeavor to collect information whenever possible, to be sent to headquarters through the usual channels. Particularly valuable is the capture of apparatus used by the enemy either for disseminating gas or for protection against it. If a shell attack is made, unexploded shells or portions of them should be sent through to headquarters at once. The time of day, duration of attack, color, taste or smell of gas used, effect on the eyes, breathing, and all other symptoms should be noted. New gases may be used at any time, and speedy information greatly forwards the adoption of preventive measures.
- 59. Preparations for Entering Trenches Inspection of Section CHAPTERX Service in the Trenches Preparing to enter upon a period of service in the trenches the company commander makes a complete inspection of the company which includes: 1. Inspection of rifles and ammunition. 2. Inspection of equipment, contents of packs, intrenching tools, field glasses, wire cutters, first-aid packets, emergency rations, gas helmets, identification tags, canteens, clothing, etc. 3. Canteens to be filled with water. 4. Test bayonets, fix and unfix. 5. Have company fill magazines. The company commander precedes the company into the trenches and makes a tour and inspection of the section assigned, which includes: 1. Layout of the trenches: fire trench, supervision trench, communication trenches, machine-gun positions, snipers' positions, listening and observation trenches, dugouts, latrines, etc. 2. Locate telephones, reserve ammunition and munitions depots, water supply, gas alarms, tools and any trench accessories and utilities that may be included in the section. 3. Get any information of the enemy that may be of value from the outgoing company commander.
- 60. Tactical Disposition The company commanderwill then make his tactical dispositions. In occupying the trenches a certain section of the line is assigned to each company. This section contains so many bays of the trench. The following dispositions are suggested as meeting the requirements under our organization: 1. The company is organized into four platoons of four squads each. 2. The section of the line assigned to the company contains eight bays. 3. Support No. 1 consists of the first and second platoons. 4. Support No. 2 consists of the third and fourth platoons. 5. From Support No. 1: Two squads of the first platoon occupy bays 1 and 2; two squads of the second platoon occupy bays 3 and 4. 6. From Support No. 2: Two squads of the third platoon occupy bays 5 and 6; two squads of the fourth platoon occupy bays 7 and 8. Plate 41f. Tactical dispositions One company of infantry.
- 61. Going into the Trenches Informationto be Obtained 7. Each squad establishes a double sentinel post in the bay assigned to it and the remaining members go into the squad shelters just in rear of the bays. This gives three reliefs for a double sentinel post and allows one extra man to be utilized as "runner," etc. 8. The remainder of the company is established in the company dugouts. 9. Depending upon the length of the tours of duty of the company in the first line trenches, the squads are changed according to a system that will have to be varied to suit the occasion, the squads in support taking their place in the fire trench and those in the fire trench returning to the support. Platoons enter by not more than two squads at one time, thus minimizing the danger from shell fire. The platoon commander will explain to his squad leaders the extent of trench to be taken over and the action to be taken in case they are caught under shell fire or rapid fire while going up to the trenches. A second in command in each squad will be designated, so that if casualties occur among the squad leaders the relief will proceed as previously arranged. The operation will proceed in silence. Rifles must be carried so that they do not show over the parapet. On reaching the fire trench the men of the first relief are posted to relieve the old detail and each man finds out any points that may be useful from his predecessor on that post. The platoon commander confers with the commander of the outgoing party and secures all the information possible about the position which includes: 1. Behavior of enemy during period preceding relief, and any point in their line requiring special information, e. g., enemy may have cut wire as though preparing to attack.
- 62. Routine 2. Machine-gun emplacementmay be suspected at some particular point. 3. Anything ascertained by patrols about ground between firing lines, thus avoiding unnecessary reconnaissance. 4. Any standing arrangements for patrols at night, including point at which wire can best be passed, ground to be patrolled, or place where they can lie under cover. 5. Any parts of trench from which it is not safe to fire. Such positions are apt to occur in winding trenches, and are not always recognizable in the dark. 6. Special features of trench, recent improvements, work not completed, dangerous points (on which machine guns are trained at night), useful loopholes for observation. 7. Places from which food and water can be safely obtained. 8. Amount of ammunition, number of picks, shovels and empty sandbags in that section of the line. Information on these points cannot always be given properly by word of mouth. Written notes and plans should therefore be handed over to a platoon commander taking over for the first time. Every man is required to see that he has a good firing position for all directions. Section commanders must satisfy themselves that men have done this, and report. The whole line "Stands to Arms" during the hour before dawn. After dark, unless the moon is bright, rifles should be left in firing position on the parapet. All men not on sentry should keep rifles, with bayonets fixed, in the trench. 1. Double sentinel posts are established in each bay. They are on post one hour at a time.
- 63. 2. When theenemy's trench mortar detachments are active, special sentinels will be posted to give notice of coming bombs. 3. Every man in the platoon is to know: (a) The location of the platoon reserve ammunition and munitions. (b) The location of latrines. (c) The topography of the trenches in the platoon section and the adjoining sections, including the approaches. The location of the accessory defenses, listening and observation posts, machine-gun positions, snipers' positions, trench mortar positions, etc. (d) The tactical disposition in the sector and the general disposition of the company. (e) The location of loopholes. (f) The places of especial danger in order that he may stay away from them. 4. Rifles are inspected twice daily. Every precaution is taken to keep the rifle and ammunition free from mud. 5. There is a gas helmet parade daily. 6. Accurate sketches are made of the trench and any addition or alteration entered on them.
- 64. Plate 42. Organization offire of Observers in the Combat Trench 7. Loopholes are inspected at dusk. 8. Wire entanglements are inspected and repaired under cover of darkness. 9. A log of events hour by hour should be kept which shows every item of enemy activity and the measures taken during the tour in the trenches. This will be a valuable reference when turning the trench over and will make a record of the habits of the enemy that may be most valuable as a guide for making plans to circumvent him. 10. The police and sanitation of the trenches will be carefully looked after. 11. Platoon commanders may divide the tour of supervision of the platoon sector with the squad leaders. 12. The whole company stands to arms during the hour before dawn.
- 65. Observation Observation of theenemy's line should be continuous. The observation and firing system will be arranged so that all parts of the enemy's line will be under observation and fire at all times. Plate 42 shows the arrangement in general. The appliances for carrying it out are shown in Plates 43, 44 and 45. Plate 43. The observation is conducted through a small loophole made by a stick through the parapet or an iron tube run through and directed
- 66. toward the pointto be observed. To conceal the exit a few tufts of earth and grass are placed there in an irregular manner. Steel loopholes may also be employed for observation and firing purposes. They may be arranged a yard or two apart, so that one man observing through one can direct his mate using the rifle at the other so that he may bring fire to bear upon any member of the enemy's force that exposes himself at the point under observation. The loopholes, both observation and firing, are arranged slantwise in the parapet so that the observer does not look straight to his front nor does the firer fire in that direction. Plate 44.—Observation loophole and rifle firing rack. An aiming rack constructed so as to resist the recoil of the rifle and not derange its aim on firing may be arranged near the observation loophole. When the enemy exposes himself all that is
- 67. Field Glasses necessary isa press on the trigger and the bullet goes straight to its mark. Such an aiming rack may be easily constructed, as shown in Plate 44. Observation of the enemy trenches may also be effected by use of the periscope or, in the absence of one of these, by a looking-glass in a slanted position fastened to a stick planted at the rear wall of the trench and protruding over the parapet, to reflect his trenches. (See Plate 45.) The enemy's trench usually appears completely deserted, but on observing it through field glasses you are astonished by the details revealed. You will see, from time to time, the eye of the enemy observer who shows himself at the loophole, or any other activity that is capable of being observed from the outside. The observer watching through the field glasses will soon become so familiar with the appearance of the opposing trenches that he will be able to detect immediately any alteration in the obstacles, or changes that may be made, such as the establishment of new listening or observation posts, new sap heads, machine-gun emplacements, etc. Observers are charged especially with detecting the location of machine-gun emplacements. The examination should be so complete and detailed as to prevent their existence without their location being accurately known.
- 68.
- 69. Plate 45a.—Trench showingwire overhead cover and wire trapdoor obstacle. Machicoulis gallery in background. Any observations of enemy activities, of any nature whatever, are reported immediately so that they may be passed on to the commander whose unit is manning the trenches directly opposite the same. Loopholes should be screened at the rear by a sandbag split and hung over them. They should be carefully concealed to prevent their
- 70. Snipers location being discoveredby the enemy. There must be no alteration in the parapet where they are located. The enemy's sojourn in the trenches should be made as disagreeable to him as possible. He must be kept continually on the alert. Our operations must be made a constant menace to him. It is in this way that casualties are effected and he is gradually worn out. One of the best methods of accomplishing all of the above is the employment of snipers, who are specially selected and trained in this branch of trench warfare. The snipers are on duty all day, but they have their nights in bed. They conduct their operations in pairs and are given a definite post to occupy and in exceptional cases may be given a roving commission. The advantage of having the same men regularly on the same post is that they learn thoroughly the appearance of every square foot of the ground included in their area of observation and are able immediately to note any change that may take place. They soon learn where to look for the enemy and in fact learn the habits, etc., of the enemy occupying their sphere of observation. The sniper must be an expert in: 1. The construction of loopholes by day and by night. 2. The use of telescopic sights, field glasses, periscopes and all optical contrivances designed for observation purposes. 3. The selection of good positions for sniping. 4. Judging distances and estimating or measuring ranges. 5. Rifle firing. He should be an expert rifleman in order that full advantage may be taken of the opportunities to inflict losses on the enemy. 6. In trench warfare each pair of snipers will be required to report each evening to the company commander the result of their day's operations.
- 71. What to FireAt When the enemy makes his attack you will generally fire at those who appear in the sector that has been allotted to you to cover. You may, however, abandon your target on your own initiative under the following circumstances and fire: 1. On officers and noncommissioned officers. These can be recognized by their gestures. They are generally in the center of groups and get up and start first. They should be disabled, as this is the surest way of breaking up the attack. 2. At a group on the move. Fire should be concentrated on an advancing group. The time when the group is preparing to start its rush may be indicated by rifles being raised and the movements that take place along the line. After a rush has started, look out for the late comers trying to rejoin their comrades. They make good targets. 3. When the enemy attempts to build up his line to the front by a process of infiltration. That is, by having single men crawl from one point to the other, each man should be fired on during his advance. 4. Fire will be immediately concentrated on any machine gun that comes into action. With the German gun prolonged firing heats the water in the jacket to the boiling point and puffs of steam are given off. Do not be deceived into thinking that this necessarily gives away the position of the gun, for this steam has been piped to a distant place and allowed to escape so as to draw fire that otherwise might be directed on the real position of the gun. 5. On signallers or runners. These are carrying information that will probably be of benefit to the enemy's commander. You will appreciate the necessity of preventing this. 6. On an enemy showing a flank. No opportunity must be lost to fire upon an enemy that exposes his flank. The fire of a single rifleman down the flank may cause a whole line to retreat.
- 72. Use of Rifle Grenades Riflegrenades are capable of causing more losses to the enemy than bombardment. The rifle grenade arrives at its destination unexpectedly without any noise; it explodes before one has even time to get out of the way. As it does not arrive at fixed hours like the bombardment, the enemy cannot continually avoid it by taking refuge in his dugouts and shelters; when he is moving about a trench which is subject to rifle grenading he must be continually on the alert. This perpetual menace, hour in and hour out, day in and day out, renders his sojourn in the trenches extremely disagreeable. Before rifle grenades are thrown careful observation of the opposing trench must have been made to determine the point where the grenade is likely to do the greatest damage. Rifles are placed in the aiming racks and the grenades fired from time to time, day and night, at moments when it seems propitious. In this way a sentinel may be taken by surprise; a noncommissioned officer or officer may be caught unawares. It should be remembered that we will probably be able to throw twenty grenades to the enemy's one. Advantage should always be taken of this munitions superiority. Every man of the enemy we can put out of action is one less to kill us in the advance which will eventually come. Sometimes the enemy will try to reply. Here is where our munitions superiority comes in again. We can fairly shower him with grenades and make him take to his shelters. It may be advisable to execute a sudden burst of grenade fire. This is started by a volley and followed by fire at will. When the artillery has destroyed parts of the enemy's trenches or makes breaches in his obstacles by day he will endeavor to repair them at night. He may be considerably annoyed and losses inflicted upon him by a well-directed shower of rifle grenades arriving at points where his working parties are located. To make this effective the rifle racks should be placed in position and secured during the
- 73. Shelling How to Protect Yourselffrom Shells During the Combat day after trial shots have demonstrated conclusively the direction and angle for them. You will be impressed by the shells, especially the big ones. The din and blast of the explosions are, to say the least, terrifying. But you will soon come to know that the shell often makes more noise than it does harm and that, after a terrific bombardment, by no means is everybody destroyed. The big shell, which is so appalling, is only really dangerous if it falls on the place where a man is standing, because the splinters rise in the air. Fall down flat when the shell bursts. Even if you are quite close, there is comparatively little risk. Get up immediately after the explosion, especially if you are 200 to 300 yards away from the place where it burst. The splinters do not fall for some time after the explosion. The steel helmets and the infantry pack will furnish considerable protection from shrapnel fragments and balls. The safest place to avoid the enemy's shell fire when the attack has been launched is close up to the enemy's position, where the artillery fire has to cease for fear of placing shells indiscriminately in his own troops and ours. Some men, completely distracted, lie down with their face to the ground. They will be crushed where they lie. Artillery fire, when it is violent, tends to throw the ranks into confusion and disorder. You have only ears for the roar of the approaching shell. You slow down and attempt to seek cover where there is no cover. The unit breaks up, and runs wild or stops altogether. Disorder and confusion means massacre. March strictly in place. To the front is your safest haven of refuge. Get hold of the frightened ones and keep them in place. You will need them to help you when you reach the goal.
- 74. In the Trenches Scoutingand Patrolling Dugouts with strong overhead cover are provided for your protection when not actually required to man the trench. In some places it may be possible to dig shelter caves and shore up the roofs. To the Reader: You will find a wealth of information on the methods to be employed by scouts and patrols in a little book similar to this one in size, entitled "Scouting and Patrolling," by the author of this volume. Published and for sale by the United States Infantry Association, Washington, D. C. Price 50 cents, by mail, postpaid. Get your copy now and prepare yourself for these important duties. Scouting and patrolling to the front is of greatest importance. It is kept up both day and night. The units occupying the first line send out patrols whenever necessary. They are frequently able to obtain valuable information and at the same time serve to counteract the enemy's efforts in this direction. The patrols generally consist of a junior officer or noncommissioned officer and from four to six selected men. Their operations are conducted in accordance with the situation and the mission they are sent on. Hand grenades are frequently carried for both offensive and defensive operations. Grenade patrols always carry them. The operations of patrols may include: 1. Reconnaissance of sectors of the enemy's position with a view to determining his dispositions and arrangement of obstacles. 2. Making sketches of positions. 3. Capturing prisoners. 4. Opposing enemy patrols. 5. Harassing the enemy.
- 75. Care of Arms Whenthe patrol goes out every man in the sector of the firing line must be informed of such fact and the possibility of its returning through his post. It is not sufficient to simply notify the men on post at the time the patrol goes out, as a man cannot always be trusted to pass the information on to his relief. Word should be quietly taken along the line by the noncommissioned officer in charge of the relief in person. When the patrol is out, special instructions have to be given with respect to firing. To cease firing altogether is very undesirable. It arouses the enemy's suspicions. A few trustworthy riflemen are directed to fire high at intervals. No lights are sent up while the patrol is out. If the patrol is to remain stationary, similar to the outguard of an outpost, communication may be maintained by means of a string, spelling out the messages by Morse code, two jerks meaning a dash and one jerk meaning a dot. Where night patrols have to remain out under trying conditions special dugouts should be reserved where they can rest upon their return. The infantryman's rifle is his best friend. The personal care that he gives to it is indicative of his soldierness and discipline. Your rifle must be kept in prime condition, otherwise it may fail you at a critical moment. A canvas breech cover that will protect the bolt and magazine mechanism will be found a great advantage when the rifle is not in use. Care must be taken to exclude mud and dirt from the bolt mechanisms. Do not put mud- covered cartridges into the magazine. Wipe them off first. Arrange a proper receptacle near your post for the storage of your reserve ammunition. Be careful that you do not clog the muzzle of the rifle with mud and dirt. If fired in this condition it will ruin the rifle. Be careful not to clog up the sight cover with mud. Oil the rifle frequently with good sperm oil. Half of the oilers in the squad should be filled with oil and the other half with Hoppe's No. 9 Powder Solvent.
- 76. Care of Trenches Latrines Maps Rifles mustbe carefully inspected daily by platoon commanders and the men required to work on them during the periods off post. Repairs will have to be made daily. The widening of trenches in the making of repairs should be strictly forbidden. Under no circumstances must they be altered in any manner except on the order of the company commander. Platoon commanders will go over every part of the trench several times daily with the squad leaders of the various sections and decide upon the repairs and improvements to be made. A complete and thorough police will be made prior to being relieved. All refuse will be removed. Fired cartridges will be disposed of, as they might get imbedded in the trench floor and hinder subsequent digging. Each squad leader will be held strictly responsible for the state of police of the section of trench occupied by his squad. Latrines are located at convenient points in the trenches. For the men on duty in the first line they are generally dug to the flank of a connecting trench and connected therewith by a passageway. Their location is plainly marked. The rules of sanitation are even more strictly observed in trenches than they are in soldier camps. The trenches and passageways must not under any circumstances be defiled. Latrines should be kept clean and sanitary. They will be carefully protected from flies. The free use of chloride of lime daily is an absolute necessity. A complete detailed plan of our own trenches and as much as is known of those of the enemy opposite should be made, and be available for study and to refer to in making reports. Every bay of the trench should be numbered, every traverse lettered. All junction points of fire and communicating trenches, all dugouts, all posts, mortar positions, machine-gun positions, observation posts, and any points that it may be necessary to refer to in reports should be designated by numbers.
- 77. Frost Bite; Chilled Feet Prevention Thecauses are: 1. Prolonged standing in cold water or liquid mud. 2. Tight boots and leggings, that interfere with the blood circulation. 1. Before going into the trenches wash the feet and legs and dry them thoroughly. The British Army has an issue of an anti-freeze mixture which will probably be issued to our troops also. The feet and legs should be rubbed with it. Put on perfectly dry socks. An extra pair of dry socks should be carried. 2. During the period of service in the trenches the feet should be treated in this manner from time to time. 3. When the feet are cold, hot water will not be used for washing nor will they be held close to a fire. 4. Rubber boots must be worn only in the trenches. On no account must they be worn while on reserve. Trench Soldiers' Creed To be of the greatest effectiveness in the trench every soldier, personally and collectively, must be able to adopt the following creed and live up to it: 1. We are here for two purposes, to do as much damage as possible to the enemy and to hold our section of the line against all attacks. We are doing everything in our power to accomplish these missions. We realize that every man of the enemy confronting us that is now placed hors de combat will be there ready to shoot us down when the assault takes place. We realize also that if the enemy makes a lodgment on our section of the line that it
- 78. endangers others anda costly counter attack may be necessary. We will hold on. 2. With the means at hand and those we are able to devise we will make the enemy's stay in his trenches as uncomfortable and disagreeable as possible. All of our utilities are being utilized to the fullest extent and our various detachments are organized and their tactical operations are conducted with this object in view. 3. We have done everything possible to strengthen our line. 4. If, despite all the precautions we can take and the hardest fight we are able to make, the enemy succeeds in effecting a lodgment on our section of the line, we will meet him with the bayonet and fight to the last drop of our blood. 5. We are all familiar with the tactical dispositions in our section of the line. Those of us on the flanks connect up with the platoons to our right and left. We know the route to company and battalion headquarters and know where the nearest support is located. We know the position of our machine guns and the sector they cover. We are in constant communication with the observing posts that cover our front, and our observing posts covering the other platoons are in constant communication with them. 6. We know the firing position assigned to us and are familiar with the use to be made of the accessories furnished us. We can fire over the parapet at the foot of our wire entanglements to repel night attack. 7. We will at all times be careful about needlessly exposing ourselves. We appreciate the fact that it is absolutely stupid to get killed or wounded in the trench through negligence. By so doing one has served no purpose and a soldier cannot be replaced. Our leaders have warned us of the especially dangerous places. We know where they are and avoid them except when our presence there is necessary as a matter of duty.
- 79. 8. The sectionsof the enemy's line that we are to cover with our fire have all been pointed out and each of us is familiar with same. We have located the enemy's loopholes and are doing our best to keep them under fire. 9. We know our way and move noiselessly about the trenches. When we enter and leave it is with absolute silence. 10. We are doing our utmost to collect information about the enemy, his defenses, his activity, his movements, and especially his night operations. All of this information we transmit immediately to the platoon leader. 11. We know the best way to get over our parapet to reach the enemy. 12. Our appliances for protection from gas attacks are complete and ready for instant use. We have our helmets on our persons ready to put on. We are familiar with their use and have confidence in their effectiveness. We will wait for the signal to don our gas helmets (signal is usually made by beating a gong, and care must be taken to follow exactly the directions for putting on the gas helmets; carelessness may mean your disablement). 13. Our trenches are drained and every precaution is being taken to keep the drains and sump holes in condition to perform their functions. 14. We have rendered the parapets and shelters throughout our sector bullet-proof, and effective measures are being taken to prevent them from caving in. 15. We are keeping our trenches sanitary and clean; our reserve munitions are carefully stored in their proper places ready for instant use. Refuse is always placed in receptacles when it can be carried away. We do not under any circumstances litter up our trench floor. Our empty shells are collected and sent to the rear.
- 80. Welcome to OurBookstore - The Ultimate Destination for Book Lovers Are you passionate about books and eager to explore new worlds of knowledge? At our website, we offer a vast collection of books that cater to every interest and age group. From classic literature to specialized publications, self-help books, and children’s stories, we have it all! Each book is a gateway to new adventures, helping you expand your knowledge and nourish your soul Experience Convenient and Enjoyable Book Shopping Our website is more than just an online bookstore—it’s a bridge connecting readers to the timeless values of culture and wisdom. With a sleek and user-friendly interface and a smart search system, you can find your favorite books quickly and easily. Enjoy special promotions, fast home delivery, and a seamless shopping experience that saves you time and enhances your love for reading. Let us accompany you on the journey of exploring knowledge and personal growth! ebookgate.com