This document outlines basic probability concepts, including definitions of probability, views of probability (objective and subjective), and elementary properties. It discusses calculating probabilities of events from data in tables, including unconditional/marginal probabilities, conditional probabilities, and joint probabilities. Rules of probability are presented, including the multiplicative rule that the joint probability of two events is equal to the product of the marginal probability of one event and the conditional probability of the other event given the first event. Examples are provided to illustrate key concepts.
4. Introduction of probability
Probability theory – foundation for statistical inference
eg. 50-50 chance of surviving an operation
95% certain that he has a stomach cancer
Nine out of ten patients take drugs regularly
Probability - expressed in terms of percentage (generally) -
expressed in terms of fractions (mathematically)
Probability of occurrence – between zero and one
5. What is probability?
A number that represents the chance that a particular event will occur
for a random variable.
Eg: Odds of winning a lottery, chance of rolling a seven when rolling two
dice, percent chance of rain in a forecast
The frequentist definition of probability used in statistics
Betty R. Kirkwood, Jonathan A.C. Sterne, “Essential Medical statistics”, 2nd edition
6. ■ This states that the probability of the occurrence of a particular event equals the
proportion of times that the event would (or does) occur in a large number of similar
repeated trials.
7. ■ Random variable – numerical quantity that takes on different values
depending on chance
■ Population – the set of all possible outcomes for a random variable (
only hypothetical population, not a population of people)
■ An event – an outcome or set of outcomes for a random variable
■ Probability – the proportion of times an event is expected to occur in the
long run. Probabilities are always numbers between 0 and 1
corresponding to always.
B.Burt Gerstman,”Basic Biostatistics,2nd edition
8. Types of RandomVariables
Discrete random variables
no: of Leukemia cases in geographic region
no: of smokers in a simple random sample of size n
Continuous random variables
Amount of time it take to complete a task
average weight in simple random sample of newborn
the height of individual
9. Probability of an event
The probability of an event is viewed as a numerical measure
of the chance that the event will occur.
Event: An outcome of an experiment or survey.
Eg: rolling a die and turning up six dots
Elementary event: An outcome that satisfies only one
criterion.
Eg: A red card from a standard deck of cards
Joint event: An outcome that satisfies two or more criteria
Eg: A red ace from a standard deck of cards
10. Three basic event operations
The complement of an event A, denoted by Ā is the set of
all elementary outcomes that are not in A.The occurrence of
Ā means that A does not occur.
The union of two events A and B, denoted by A U B, is the
set of all elementary outcomes that are in A, B, or both.The
occurrence of A U B means that either A or B or both occur.
Richard A. Johnson, Gouri K. Bhattacharyya, “Statistics: Principles and methods”, 6th edition
11. Three basic event operations
The intersection of two events A and B, denoted by A ∩ B,
is the set of all elementary outcomes that are in A and B.The
occurrence of A ∩ B means that both A and B occur.
Note:The operations of union and intersection can be extended
to more than two events.
Richard A. Johnson, Gouri K. Bhattacharyya, “Statistics: Principles and methods”, 6th edition
12. Two views of Probability
(1) Objective probability
a. Classical or a priori probability
b. Relative frequency or a posteriori probability
(2) Subjective probability
13. Three views of Probability
Three views of probability:
(1) the subjective-personalistic view,
(2) the classical, or logical view, and
(3) the empirical relative-frequency view
Roger E. Kirk, “Statistics:An introduction”, 5th edition
14. Objective probability
a. Classical or a priori probability
A fair six-sided die – Number one --- 1/6
A well-shuffled playing cards – Heart – 13/52
■ If an event can occur in N mutually exclusive and equally
likely ways, and if m of these possess a trait, E, the
probability of the occurrence of E is equal to m / N.
P (E) = m /N
15. Objective probability
b. Relative frequency or a posteriori probability
depends on repeatability of some process/ability to count
number of repetitions and number of times that the event
of interest occurs
■ If some process is repeated a large number of times, n,
and if some resulting event with the characteristic E
occurs m times, the relative frequency of occurrence of E,
m /n , will be approximately equal to the probability of E
P (E) = m /n [ m /n is an estimate of P (E) ]
16. Subjective Probability
Personalistic or subjective concept of probability
An event that can occur once eg.The probability that a cure for cancer
will be discovered within the next 10 years.
Some statisticians do not accept it.
17. Subjective Probability
Subjective definition, where the size of the probability simply represents one’s degree
of belief in the occurrence of an event, or in an hypothesis.
This definition corresponds more closely with everyday usage and is the foundation of
the Bayesian approach to statistics.
In this approach, the investigator assigns a prior probability to the event (or
hypothesis) under investigation.
The study is then carried out, the data collected and the probability modified in the
light of the results obtained, using Bayes’ rule.
The revised probability is called the posterior probability.
Betty R. Kirkwood, Jonathan A.C. Sterne, “Essential Medical statistics”, 2nd edition
18. Elementary properties of probability
Three properties:
(1) Given some process (or experiment) with n mutually
exclusive outcomes (called events), E1, E2, …., En, the
probability of any event Ei is assigned a nonnegative number.
ie. P(Ei) ≥ 0 (Two mutually exclusive outcomes –Two not
occurring at the same time)
Mutually exclusive: the probability of both events A and B occurring
is 0.This means that the two events cannot occur at the same time.
Eg: On a single roll of a die, you cannot get a die that has a face
with three dots and also have four dots because such elementary
events are mutually exclusive.
19. Elementary properties of probability
Three properties:
Mutually exclusive (Incompatible) event
Two events A and B are called incompatible or mutually
exclusive if their intersection AB is empty.
20. Elementary properties of probability
Three properties:
(2)The sum of the probabilities of mutually exclusive outcomes
is equal to 1
P(E1) + P(E2) + … + P(En) = 1
(Property of exhaustiveness –→ all possible events)
Collectively exhaustive event: A set of events that includes all
the possible events.
Eg: Heads and tails in the toss of a coin, male and female, all
six faces of a die.
21. Elementary properties of probability
Three properties:
(3) Consider any two mutually exclusive events, Ei and Ej.
The probability of the occurrence of either Ei or Ej is equal
to the sum of their individual properties
P(Ei + Ej) = P(Ei) + P(Ej)
22. Elementary properties of probability
Three properties:
1. The probability of an event must be between 0 and 1.The
smallest possible probability value is 0.You cannot have a
negative probability. The largest possible probability value is
1.0.You cannot have a probability greater than 1.0.
2. If events in a set are mutually exclusive and collectively
exhaustive, the sum of their probabilities must add up to 1.0.
3. If two events A and B are mutually exclusive, the probability
of either event A or event B occurring is the sum of their
separate probabilities.
Betty R. Kirkwood, Jonathan A.C. Sterne, “Even you can learn statistics”, 2nd edition
23. Calculating the probability of an event
Table 1. Frequency of family history of mood disorder by age group
among bipolar subjects
The probability that a person randomly selected from total population
will be 18 year or younger = ?
P (E) = 141 / 318 = 0.4434
24. Calculating the probability of an event
Unconditional or Marginal Probability
One of marginal total was used in numerator and
The size of total group serve as the denominator
No conditions were imposed to restrict the size of the
denominator
Conditional Probability
When probabilities are calculated with a subset of the total
group as the denominator, the result is a conditional
probability
25. Calculating the probability of an event
Conditional Probability
The probability that a person randomly selected from those
18 yr or younger will be the one without family history of
mood disorder=?
P (A | E) = 28 / 141 = 0.1986
P (A | E) is read as “ probability of A given E”
26. Calculating the probability of an event
Joint Probability
When a person selected possesses two characteristics at
same time, the probability is called ‘joint probability’
The probability that a person randomly selected from total
population will be early (E) and will be the one without
family history of mood disorder (A) = ?
P (E ∩ A) = 28 / 318 = 0.0881
(Symbol ∩ is read ‘intersection’ or ‘and’)
27. Rules of probability
1. Multiplicative rule
Joint probability can be calculated as the product of
appropriate marginal probability and appropriate conditional
probability
This relationship is known as multiplication rule of
probability
P (A ∩ B) = P (B) P (A | B), if P (B) ≠ 0
P (A ∩ B) = P (A) P (B | A), if P (A) ≠ 0
[ Note: P (B), P (A) are marginal probabilities ]
29. Rules of probability
1. Multiplicative rule
Marginal probability P (E) = 141 / 318 = 0.4434
Conditional probability P (A | E) = 28 / 141 = 0.1986
Joint probability P (E ∩ A) = 28 / 318 = 0.0881
Joint probability = Marginal probability × Conditional probability
P (E ∩ A) = P (E) P (A | E)
= (141 / 318) (28 / 141)
= 0.0881
30. Rules of probability
1. Multiplicative rule
Joint probability = Marginal probability × Conditional probability
P (E ∩ A) = P (E) P (A | E)
Conditional probability = Joint probability/Marginal probability
P (A | E) = P (E ∩ A) / P (E)
Conditional probability of A given E is equal to the probability of E ∩ A
divided by the probability of E, provided the probability of E is not zero
Definition
Conditional probability of A given B is equal to the probability of A ∩ B
divided by the probability of B, provided the probability of B is not zero
P (A | B) = P (A ∩ B) / P (B), P (B) ≠ 0
31. Rules of probability
1. Multiplicative rule
Conditional probability = Joint probability/Marginal probability
P (A | E) = P (E ∩ A) / P (E)
= (28/318) / (141/318)
= 0.1986
Conditional probability P (A | E) = 28 / 141
= 0.1986
32. Rules of probability
2. Additive rule
The probability of the occurrence of either one or the other of
two mutually exclusive events is equal to the sum of their
individual probabilities (Third property of probability)
The probability that a person will be early age (E) or later age
(L) = ?
P(E U L) = P (E) + P (L)
= 141/318 + 177/318
= 1
[Symbol ‘U’ is read as ‘union’ or ‘or’]
33. Rules of probability
2. Additive rule
Given two events A and B, the probability that event A, or event B, or
both occur is equal to the probability that event A occurs, plus the
probability that event B occurs, minus the probability that the events
occur simultaneously (ie not mutually exclusive)
P(A U B) = P (A) + P (B) - P (A ∩ B)
The probability that a person will be an early age (E) or no family h/o (A)
or both = ?
P(E U A) = P (E) + P (A) - P (E ∩ A)
= 141/318 + 63/318 – 28/318
= 0.5534 (duplication/overlapping is adjusted)
34. Rules of probability
Two rules underlying the calculation of all probabilities
1. the multiplicative rule for the probability of the occurrence of both of two events, A
and B, and;
2. the additive rule for the occurrence of at least one of event A or event B.This is
equivalent to the occurrence of either event A or event B (or both).
Betty R. Kirkwood, Jonathan A.C. Sterne, “Essential Medical statistics”, 2nd edition
35. Independent events
Occurrence of event B has no effect on the probability of
event A (i.eThe probability of event A is the same regardless
of whether or not B occurs)
i.e. P(A | B) = P(A), P(B | A) = P(B), P(A ∩ B) = P(A) (B)
(if P (A) ≠ 0, P (B) ≠ 0 )
(Note:The terms ‘independent’ and ‘mutually exclusive’ do not mean
the same thing)
If A an B are independent and event A occurs, the occurrence of B is
not affected.
If A and B are mutually exclusive, however, and event A occurs,
event B cannot occur.
36. Example of independent events
Girl (B-) Boy (B) Total
Eyeglasses wearing (E) 24 16 40
No eyeglasses wearing (E-
)
36 24 60
Total 60 40 100
The probability that a person randomly selected wears eyeglasses = ?
P(E) = 40/100 = 0.4
I,eWearing eyeglasses is not concerned
with gender)
37. Complementary Events
Given some variable that can be broken down into m categories
designated by A1, A2, …Ai...Am and another jointly occurring
variable that is broken down to n categories designated by B1,
B2…Bj...Bn, the marginal probability of Ai, P(Ai) , is equal to the sum
of the joint probabilities of Ai with all the categories of B.
P(Ai) = ΣP(Ai ∩ Bj ), for all value of j
Example: P (B) = 1- P (B-)
B and B- are complementary events
Event B and its complement event B- are mutually exclusive
(third property of probability)
39. BAYES’THEOREM
Thomas Bayes, an English Clergyman (1702-1761)
Estimates of the predictive value positive and predictive
value negative of a test → based on test’s sensitivity,
specificity and probability of the relevant disease in general
population
Bayes’ rule for relating conditional probabilities:
40. BAYES’THEOREM
Suppose that we know that 10% of young girls in India are malnourished,
and that 5% are anaemic, and that we are interested in the relationship
between the two. Suppose that we also know that 50% of anaemic girls are
also alnourished. This means that the two conditions are not independent,
since if they were then only 10% (not 50%) of anaemic girls would also be
malnourished, the same proportion as the population as a whole. However,
we don’t know the relationship the other way round, that is what
percentage of malnourished girls are also anaemic. We can use Bayes’ rule
to deduce this.
41. BAYES’THEOREM
Probability (malnourished) = 0.1
Probability (anaemic) = 0.05
Probability (malnourished given anaemic) = 0.5
Using Bayes rule gives:
Prob (anaemic given malnourished)
We can thus conclude that 25%, or one quarter, of malnourished girls
are also anaemic.
42. Sensitivity
The sensitivity of a test (or symptom) is the probability of a positive test result (or
presence of the symptom) given the presence of the disease.
43. Specificity
The specificity of a test (or symptom) is the probability of a negative test result (or
absence of the symptom) given the absence of the disease.
44. Predictive value positive
The predictive value positive of a screening test (or symptom) is the probability that
a subject has the disease given that the subject has a positive screening test result (or
has the symptom)
45. Predictive value negative
The predictive value negative of a screening test (or symptom) is the probability that
a subject does not have the disease given that the subject has a negative screening
test result (or does not have the symptom)
46. BAYES’THEOREM
Predictive value positive of a screening test (or symptom)
Predictive value negative of a screening test (or symptom)
47. Example
Sensitivity of the test
P(T/D) = 436 / 450 = 0.9689
Specificity of the test
P(T-/D-) = 495 / 500 = 0.99
48. Example
Sensitivity of the test
P(T/D) = 436 / 450 = 0.9689
Specificity of the test
P(T-/D-) = 495 / 500 = 0.99
Predictive value positive of the test
If P(D) = 0.113 (11.3% of the U.S population aged 65 and over have
Alzheimer’s disease)
49. Example
Predictive value positive of the test
If P(D) = 0.113 (11.3% of the U.S population aged 65 and over haveAlzheimer’s
disease)
Predictive value of positive test is very high.
*The predictive value positive of the test depends on the rate of the disease
in the relevant population in general
52. Normal Distribution
Also known as Gaussian distribution [Carl Friedrich Gauss (1777-1855)]
Normal density
53. Characteristics of Normal Distribution
1. It is symmetrical about the mean, µ .The curve on either side
of µ is mirror image of the other side
2.The mean, the median and mode are all equal.
54. Characteristics of Normal Distribution
3. The total area under the curve above the x axis is one square
unit. Normal distribution is probability distribution. 50% of
the area is to the right of a perpendicular erected at the
mean and 50% is to the left.
55. Characteristics of Normal Distribution
4. 1 SD from the mean in both directions, the area is 68%.
For 2 SD and 3 SD, areas are 95% and 99.7% respectively
56. Characteristics of Normal Distribution
5.The normal distribution is determined by µ and σ
Different values of µ cause the distribution graph shift along
the x axis
Therefore, µ is often referred to as a location parameter
57. Characteristics of Normal Distribution
5.The normal distribution is determined by µ and σ
Different values of σ cause the degree of flatness or
peakedness of distribution graphs
σ is often referred to as a shape parameter