9. Chapter 1
“I have been impressed with the urgency of doing. Knowing is not enough; we
must apply. Being willing is not enough; we must do.”
-Leonardo da Vinci
10. Balka ISE 1.10 1.1. INTRODUCTION 2
When first encountering the study of statistics, students often have a preconceived—
and incorrect—notion of what the field of statistics is all about. Some people
think that statisticians are able to quote all sorts of unusual statistics, such as
32% of undergraduate students report patterns of harmful drinking behaviour,
or that 55% of undergraduates do not understand what the field of statistics is
all about. But the field of statistics has little to do with quoting obscure percent-
ages or other numerical summaries. In statistics, we often use data to answer
• Is a newly developed drug more effective than one currently in use?
• Is there a still a sex effect in salaries? After accounting for other relevant
variables, is there a difference in salaries between men and women? Can
we estimate the size of this effect for different occupations?
• Can post-menopausal women lower their heart attack risk by undergoing
hormone replacement therapy?
To answer these types of questions, we will first need to find or collect appropriate
data. We must be careful in the planning and data collection process, as unfortu-
nately sometimes the data a researcher collects is not appropriate for answering
the questions of interest. Once appropriate data has been collected, we summa-
rize and illustrate it with plots and numerical summaries. Then—ideally—we use
the data in the most effective way possible to answer our question or questions
Before we move on to answering these types of questions using statistical in-
ference techniques, we will first explore the basics of descriptive statistics.
1.2 Descriptive Statistics
In descriptive statistics, plots and numerical summaries are used to describe a
Example 1.1 Consider a data set representing final grades in a large introduc-
tory statistics course. We may wish to illustrate the grades using a histogram or
a boxplot,1 as in Figure 1.1.
You may not have encountered boxplots before. Boxplots will be discussed in detail in Sec-
tion 3.4. In their simplest form, they are plots of the five-number summary: minimum, 25th
percentile, median, 75th percentile, and the maximum. Extreme values are plotted in indi-
vidually. They are most useful for comparing two or more distributions.
11. Balka ISE 1.10 1.3. INFERENTIAL STATISTICS 3
20 40 60 80 100
(a) Histogram of final grades.
(b) Boxplot of final grades.
Figure 1.1: Boxplot and histogram of final grades in an introductory statistics
Plots like these can give an effective visual summary of the data. But we are
also interested in numerical summary statistics, such as the mean, median, and
variance. We will investigate descriptive statistics in greater detail in Chapter 3.
But the main purpose of this text is to introduce statistical inference concepts
1.3 Inferential Statistics
The most interesting statistical techniques involve investigating the relationship
between variables. Let’s look at a few examples of the types of problems we will
Example 1.2 Do traffic police officers in Cairo have higher levels of lead in their
blood than that of other police officers? A study2 investigated this question by
drawing random samples of 126 Cairo traffic officers and 50 officers from the
suburbs. Lead levels in the blood (µg/dL) were measured.
The boxplots in Figure 1.2 illustrate the data. Boxplots are very useful for
comparing the distributions of two or more groups.
The boxplots show a difference in the distributions—it appears as though the
distribution of lead in the blood of Cairo traffic officers is shifted higher than that
of officers in the suburbs. In other words, it appears as though the traffic officers
have a higher mean blood lead level. The summary statistics are illustrated in
Kamal, A., Eldamaty, S., and Faris, R. (1991). Blood level of Cairo traffic policemen. Science
of the Total Environment, 105:165–170. The data used in this text is simulated data based on
the summary statistics from that study.
The standard deviation is a measure of the variability of the data. We will discuss it in detail
in Section 3.3.3.
12. Balka ISE 1.10 1.3. INFERENTIAL STATISTICS 4
Cairo Traffic Suburbs
Figure 1.2: Lead levels in the blood of Egyptian police officers.
Number of observations 126 50
Mean 29.2 18.2
Standard deviation 7.5 5.8
Table 1.1: Summary statistics for the blood lead level data.
In this scenario there are two main points of interest:
1. Estimating the difference in mean blood lead levels between the two groups.
2. Testing if the observed difference in blood lead levels is statistically signifi-
cant. (A statistically significant difference means it would be very unlikely
to observe a difference of that size, if in reality the groups had the same
true mean, thus giving strong evidence the observed effect is a real one.
We will discuss this notion in much greater detail later.)
Later in this text we will learn about confidence intervals and hypothesis
tests—statistical inference techniques that will allow us to properly address these
points of interest.
Example 1.3 Can self-control be restored during intoxication? Researchers in-
vestigated this in an experiment with 44 male undergraduate student volunteers.4
The males were randomly assigned to one of 4 treatment groups (11 to each
1. An alcohol group, receiving drinks containing a total of 0.62 mg/kg alcohol.
2. A group receiving drinks with the same alcohol content as Group A, but
also containing 4.4 mg/kg of caffeine. (Group AC)
Grattan-Miscio, K. and Vogel-Sprott, M. (2005). Alcohol, intentional control, and inappropri-
ate behavior: Regulation by caffeine or an incentive. Experimental and Clinical Psychophar-
13. Balka ISE 1.10 1.3. INFERENTIAL STATISTICS 5
3. A group receiving drinks with the same alcohol content as Group A, but
also receiving a monetary reward for success on the task. (Group AR)
4. A group told they would receive alcoholic drinks, but instead given a
placebo (a drink containing a few drops of alcohol on the surface, and
misted to give a strong alcoholic scent). (Group P)
After consuming the drinks and resting for a few minutes, the participants car-
ried out a word stem completion task involving “controlled (effortful) memory
processes”. Figure 1.3 shows the boxplots for the four treatment groups. Higher
scores are indicative of greater self-control.
A AC AR P
Figure 1.3: Boxplots of word stem completion task scores.
The plot seems to show a systematic difference between the groups in their task
scores. Are these differences statistically significant? How unlikely is it to see
differences of this size, if in reality there isn’t a real effect of the different treat-
ments? Does this data give evidence that self-control can in fact be restored by
the use of caffeine or a monetary reward? In Chapter 14 we will use a statistical
inference technique called ANOVA to help answer these questions.
Example 1.4 A study5 investigated a possible relationship between eggshell
thickness and environmental contaminants in brown pelican eggs. It was sus-
pected that higher levels of contaminants would result in thinner eggshells. This
study looked at the relationship of several environmental contaminants on the
thickness of shells. One contaminant was DDT, measured in parts per million
of the yolk lipid. Figure 1.4 shows a scatterplot of shell thickness vs. DDT in a
sample of 65 brown pelican eggs from Anacapa Island, California.
There appears to be a decreasing relationship. Does this data give strong ev-
Risebrough, R. (1972). Effects of environmental pollutants upon animals other than man. In
Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability.
14. Balka ISE 1.10 1.3. INFERENTIAL STATISTICS 6
0 500 1000 1500 2000 2500 3000
Figure 1.4: Shell thickness (mm) vs. DDT (ppm) for 65 brown pelican eggs.
idence of a relationship between DDT contamination and eggshell thickness?
Can we use the relationship to help predict eggshell thickness for a given level
of DDT contamination? In Chapter 15 we will use a statistical method called
linear regression analysis to help answer these questions.
In the world of statistics—and our world at large for that matter—we rarely if
ever know anything with certainty. Our statements and conclusions will involve
a measure of the reliability of our estimates. We will make statements like we
can be 95% confident that the true difference in means lies between 4 and 10, or,
the probability of seeing the observed difference, if in reality the new drug has no
effect, is less than 0.001. So probability plays an important role in statistics, and
we will study the basics of probability in Chapter 4. In our study of probability,
keep in mind that we have an end in mind—the use of probability to quantify
our uncertainty when attempting to answer questions of interest.
17. Balka ISE 1.10 2.1. INTRODUCTION 9
In this chapter we will investigate a few important concepts in data collection.
Some of the concepts introduced in this chapter will be encountered throughout
the remainder of this text.
2.2 Populations and Samples, Parameters and Statis-
In 1994 the Center for Science in the Public Interest (CSPI) investigated nutri-
tional characteristics of movie theatre popcorn, and found that the popcorn was
often loaded with saturated fats. The study received a lot of media attention,
and even motivated some theatre chains to change how they made popcorn.
In 2009, the CSPI revisited movie theatre popcorn.1 They found that in addi-
tion to the poor stated nutritional characteristics, in reality the popcorn served
at movie theatres was nutritionally worse than the theatre companies claimed.
What was served in theatres contained more calories, fat, and saturated fat than
was stated by the companies.
Suppose that you decide to follow up on this study in at your local theatre.
Suppose that your local movie theatre claims that a large untopped bag of their
popcorn contains 920 calories on average. You spend a little money to get a sam-
ple of 20 large bags of popcorn, have them analyzed, and find that they contain
1210 calories on average. Does this provide strong evidence that the theatre’s
claimed average of 920 calories false? What values are reasonable estimates of
the true mean calorie content? These questions cannot be answered with the
information we have at this point, but with a little more information and a few
essential tools of statistical inference, we will learn how to go about answering
To speak the language of statistical inference, we will need a few definitions:
• Individuals or units or cases are the objects on which a measurement is
taken. In this example, the bags of popcorn are the units.
• The population is the set of all individuals or units of interest to an
investigator. In this example the population is all bags of popcorn of this
type. A population may be finite (all 25 students in a specific kindergarten
18. Balka ISE 1.10 2.3. TYPES OF SAMPLING 10
class, for example) or infinite (all bags of popcorn that could be produced
by a certain process, for example).2
• A parameter is a numerical characteristic of a population. Examples:
– The mean calorie content of large bags of popcorn made by a movie
theatre’s production process.
– The mean weight of 40 year-old Canadian males.
– The proportion of adults in the United States that approve of the way
the president is handling his job.
In practice it is usually impractical or impossible to take measurements
on the entire population, and thus we typically do not know the values
of parameters. Much of this text is concerned with methods of parameter
• A sample is a subset of individuals or units selected from the population.
The 20 bags of popcorn that were selected is a sample.
• A statistic is a numerical characteristic of a sample. The sample mean of
the 20 bags (1210 calories) is a statistic.
In statistical inference we attempt to make meaningful statements about popu-
lation parameters based on sample statistics. For example, do we have strong
evidence that a food item’s average calorie content is greater than what the
A natural and important question that arises is: How do we go about obtaining a
proper sample? This is often not an easy question to answer, but there are some
important considerations to take into account. Some of these considerations will
be discussed in the following sections.
2.3 Types of Sampling
Suppose a right-wing radio station in the United States asks their listeners to
call in with their opinions on how the president is handling his job. Eighteen
people call in, with sixteen saying they disapprove.
Sometimes the term population is used to represent the set of individuals or objects, and
sometimes it is used to represent the set of numerical measurements on these individuals or
objects. For example, if we are interested in the weights of 25 children in a specific kindergarten
class, then the term population might be used to represent the 25 children, or it might be
used to represent the 25 weight measurements on these children. In practice, this difference
is unlikely to cause any confusion.
19. Balka ISE 1.10 2.3. TYPES OF SAMPLING 11
How informative is this type of sample? If we are hoping to say something about
the opinion of Americans in general, it is not very informative. This sample is
hopelessly biased (not representative of the population of interest). We might say
that the sampled population is different from the target population. (Listeners
of a right-wing radio station are not representative of Americans as a whole.)
What if we consider the population of interest to be listeners of this right-wing
radio station? (Which would be the case if we only wish to make statements
about listeners of this station.) There is still a major issue: the obtained sample
is a voluntary response sample (listeners chose whether to respond or not—
they self-selected to be part of the sample). Voluntary response samples tend to
be strongly biased; individuals are more likely to respond if they feel very strongly
about the issue and less likely to respond if they are indifferent. Student course
evaluations are another example of a voluntary response sample, and individuals
that love or hate a professor may be more likely to fill in a course evaluation
than those who think the professor was merely average.
Self-selection often results in a hopelessly biased sample. It is one potential
source of bias in a sample, but there are many other potential sources. For
example, an investigator may consciously or subconsciously select a sample that
is likely to support their hypothesis. This is most definitely something an honest
statistician tries to avoid.
So how do we avoid bias in our sample? We avoid bias by randomly selecting
members of the population for our sample. Using a proper method of random
sampling ensures that the sample does not have any systematic bias. There are
many types of random sampling, each with its pros and cons. One of the simplest
and most important random sampling methods is simple random sampling. In
the statistical inference procedures we encounter later in this text, oftentimes the
procedures will be appropriate only if the sample is a simple random sample
(SRS) from the population of interest. Simple random sampling is discussed in
the next section.
2.3.1 Simple Random Sampling
Simple random sampling is a method used to draw an unbiased sample from
a population. The precise definition of a simple random sample depends on
whether we are sampling from a finite or infinite population. But there is one
constant: no member of the population is more or less likely to be contained in
the sample than any other member.
In a simple random sample of size n from a finite population, each possible sample
of size n has the same chance of being selected. An implication of this is that
20. Balka ISE 1.10 2.3. TYPES OF SAMPLING 12
every member of the population has the same chance of being selected. Simple
random samples are typically carried out without replacement (a member of the
population cannot appear in the sample more than once).
Example 2.1 Let’s look at a simple fictitious example to illustrate simple ran-
dom sampling. Suppose we have a population of 8 goldfish:
Goldfish number 1 2 3 4 5 6 7 8
Goldfish name Mary Hal Eddy Tony Tom Pete Twiddy Jemmy
With such a small population size we may be able to take a measurement on every
member. (We could find the mean weight of the population, say, since it would
not be overly difficult or time consuming to weigh all 8 goldfish.) But most often
the population size is too large to measure all members. And sometimes even for
small populations it is not possible to observe and measure every member. In
this example suppose the desired measurement requires an expensive necropsy,
and we have only enough resources to perform three necropsies. We cannot afford
to measure every member of the population, and to avoid bias we may choose
to draw a simple random sample of 3 goldfish. How do we go about drawing a
simple random sample of 3 goldfish from this population? In the modern era, we
simply get software to draw the simple random sample for us.34
The software may randomly pick Mary, Tony, and Jemmy. Or Pete, Twiddy,
and Mary, or any other combination of 3 names. Since we are simple random
sampling, any combination of 3 names has the same chance of being picked. If
we were to list all possible samples of size 3, we would see that there are 56
possible samples. Any group of 3 names will occur with probability 1
56, and any
individual goldfish will appear in the sample with probability 3
Many statistical inference techniques assume that the data is the result of a sim-
ple random sample from the population of interest. But at times it is simply
not possible to carry out this sampling method. (For example, in practice we
couldn’t possibly get a simple random sample of adult dolphins in the Atlantic
ocean.) So at times we may not be able to properly randomize, and we may
simply try to minimize obvious biases and hope for the best. But this is danger-
ous, as any time we do not carry out a proper randomization method, bias may
be introduced into the sample and the sample may not be representative of the
For example, in the statistical software R, if the names are saved in an object called gold-
fish.names, the command sample(goldfish.names,3) would draw a simple random sample
of 3 names.
In the olden days, we often used a random number table to help us pick the sample.
Although random number tab