Probability

1. BASIC PROBABILITY CONCEPTSBASIC PROBABILITY CONCEPTS
Dr Htin Zaw SoeDr Htin Zaw Soe
MBBS, DFT, MMedSc (P & TM), PhD, DipMedEdMBBS, DFT, MMedSc (P & TM), PhD, DipMedEd
Associate ProfessorAssociate Professor
Department of BiostatisticsDepartment of Biostatistics
University of Public HealthUniversity of Public Health

2.  Probability theory – foundation for statistical inferenceProbability theory – foundation for statistical inference
eg. 50-50 chance of surviving an operationeg. 50-50 chance of surviving an operation
95% certain that he has a stomach cancer95% certain that he has a stomach cancer
Nine out of ten patients take drugs regularlyNine out of ten patients take drugs regularly
Probability - expressed in terms of percentage (generally)Probability - expressed in terms of percentage (generally)
- expressed in terms of fractions (mathematically)- expressed in terms of fractions (mathematically)
Probability of occurrence – between zero and oneProbability of occurrence – between zero and one

3.  I. Two views of probabilityI. Two views of probability
(1) Objective(1) Objective
(2) Subjective(2) Subjective
 (1) Objective probability(1) Objective probability
a. Classical or a priori probabilitya. Classical or a priori probability
b. Relative frequency or a posteriori probabilityb. Relative frequency or a posteriori probability
 Classical or a priori probabilityClassical or a priori probability
A fair six-sided die –A fair six-sided die – Number oneNumber one --- 1/6--- 1/6
A well-shuffled playing cards –A well-shuffled playing cards – HeartHeart – 13/52– 13/52
Definition: If an event can occur inDefinition: If an event can occur in NN mutually exclusive andmutually exclusive and
equally likely ways, and ifequally likely ways, and if mm of these possess a trait,of these possess a trait, EE, the, the
probability of the occurrence ofprobability of the occurrence of EE is equal tois equal to m / N.m / N.
P (E) = m /NP (E) = m /N

4.  Relative frequency or a posteriori probabilityRelative frequency or a posteriori probability
Depends onDepends on
Repeatability of some process/ability to count number ofRepeatability of some process/ability to count number of
repetitions and number of times that the event of interest occursrepetitions and number of times that the event of interest occurs
-- Probability of occurrence of event-- Probability of occurrence of event EE as follows:as follows:
 Definition: If some process is repeated a large number of times,Definition: If some process is repeated a large number of times,
nn, and if some resulting event with the characteristic, and if some resulting event with the characteristic EE occurs moccurs m
times, the relative frequency of occurrence oftimes, the relative frequency of occurrence of E, m /nE, m /n , will be, will be
approximately equal to the probability ofapproximately equal to the probability of EE
P (E) = m /n [ m /n is an estimate of P (E) ]P (E) = m /n [ m /n is an estimate of P (E) ]

5.  (2) Subjective probability(2) Subjective probability
 Personalistic or subjective concept of probabilityPersonalistic or subjective concept of probability
An event that can occur once eg. The probability that a cureAn event that can occur once eg. The probability that a cure
for cancer will be discovered within the next 10 years.for cancer will be discovered within the next 10 years.
Some statisticians do not accept it.Some statisticians do not accept it.

6.  II. Elementary properties of probabilityII. Elementary properties of probability
(Russian mathematician, AN Kolmogorov)(Russian mathematician, AN Kolmogorov)
Three properties:Three properties:
(1) Given some process (or experiment) with(1) Given some process (or experiment) with nn mutually exclusivemutually exclusive
outcomes (called events),outcomes (called events), EE11, E, E22, …., E, …., Enn, the probability of any, the probability of any
eventevent EEii is assigned a nonnegative number.is assigned a nonnegative number.
ie.ie. P(EP(Eii)) ≥ 0≥ 0
(Two mutually exclusive outcomes – Two not occurring at the(Two mutually exclusive outcomes – Two not occurring at the
same time)same time)

7. (2) The sum of the probabilities of mutually exclusive outcomes(2) The sum of the probabilities of mutually exclusive outcomes
is equal to 1is equal to 1
P(EP(E11) + P(E) + P(E22) + … + P(E) + … + P(Enn) = 1) = 1
(Property of(Property of eexhaustiveness –→ all possible events)xhaustiveness –→ all possible events)
 (3) Consider any two mutually exclusive events,(3) Consider any two mutually exclusive events, EEii andand EEjj..
The probability of the occurrence of eitherThe probability of the occurrence of either EEii oror EEjj is equalis equal
to the sum of their individual propertiesto the sum of their individual properties
P(EP(Eii + E+ Ejj) = P(E) = P(Eii) + P(E) + P(Ejj))

8.  III. Calculating the probability of an eventIII. Calculating the probability of an event
 Table 1. Frequency of family history of mood disorder by ageTable 1. Frequency of family history of mood disorder by age
group among bipolar subjectsgroup among bipolar subjects
Family h/o EarlyFamily h/o Early ≤≤ 18 (E) Later > 18 (L) Total18 (E) Later > 18 (L) Total
Negative (A) 28 35 63Negative (A) 28 35 63
Bipolar (B) 19 38 57Bipolar (B) 19 38 57
Unipolar (C) 41 44 85Unipolar (C) 41 44 85
Unipolar & bipolar (D) 53 60 113Unipolar & bipolar (D) 53 60 113
TotalTotal 141 177 318141 177 318
Q 1: What is the probability that a person randomly selected fromQ 1: What is the probability that a person randomly selected from
total population will be 18 year or younger?total population will be 18 year or younger?

9. Answer 1: P (E) = 141 / 318 = 0.4434Answer 1: P (E) = 141 / 318 = 0.4434
Unconditional probability or marginal probabilityUnconditional probability or marginal probability

10. Family h/o Early ≤ 18 (E) Later > 18 (L) TotalFamily h/o Early ≤ 18 (E) Later > 18 (L) Total
Negative (A) 28 35 63Negative (A) 28 35 63
Bipolar (B) 19 38 57Bipolar (B) 19 38 57
Unipolar (C) 41 44 85Unipolar (C) 41 44 85
Unipolar & bipolar 53 60 113Unipolar & bipolar 53 60 113
TotalTotal 141 177 318141 177 318
Conditional probabilityConditional probability: When probabilities are calculated with a: When probabilities are calculated with a
subset of the total group as the denominator, the result is asubset of the total group as the denominator, the result is a
conditional probabilityconditional probability
Q 2: What is the probability that a person randomly selected fromQ 2: What is the probability that a person randomly selected from
those 18 yr or younger will be the one without family history ofthose 18 yr or younger will be the one without family history of
mood disorder?mood disorder?

11. Answer 2:Answer 2: P (AP (A || E)E) = 28 / 141 = 0.1986= 28 / 141 = 0.1986
( vertical line read ‘given’)( vertical line read ‘given’)

12. Family h/o Early ≤ 18 (E) Later > 18 (L) TotalFamily h/o Early ≤ 18 (E) Later > 18 (L) Total
Negative (A) 28 35 63Negative (A) 28 35 63
Bipolar (B) 19 38 57Bipolar (B) 19 38 57
Unipolar (C) 41 44 85Unipolar (C) 41 44 85
Unipolar & bipolar 53 60 113Unipolar & bipolar 53 60 113
TotalTotal 141 177 318141 177 318
Joint probabilityJoint probability: When a person selected possesses two: When a person selected possesses two
characteristics at same time, the probability is called ‘jointcharacteristics at same time, the probability is called ‘joint
probability’probability’
Q3. What is the probability that a person randomly selected fromQ3. What is the probability that a person randomly selected from
total population will be earlytotal population will be early (E)(E) andand will be the one withoutwill be the one without
family history of mood disorderfamily history of mood disorder (A)(A)??

13. Answer 3:Answer 3: P (EP (E ∩∩ A)A) = 28 / 318 = 0.0881= 28 / 318 = 0.0881
(Symbol(Symbol ∩∩ is read ‘intersection’ or ‘and’)is read ‘intersection’ or ‘and’)

14.  The multiplication ruleThe multiplication rule
- Joint probability can be calculated as the- Joint probability can be calculated as the productproduct ofof
appropriate marginal probability and appropriate conditionalappropriate marginal probability and appropriate conditional
probabilityprobability
- This relationship is known as- This relationship is known as multiplication rule of probabilitymultiplication rule of probability
P (AP (A ∩ B∩ B) = P (B) P (A) = P (B) P (A || B), if P (B)B), if P (B) ≠ 0≠ 0
P (AP (A ∩ B∩ B) = P (A) P (B) = P (A) P (B || A), if P (A)A), if P (A) ≠ 0≠ 0
[ Note:[ Note: P (P (BB), P (A)), P (A) are marginal probabilities ]are marginal probabilities ]

15. Family h/o EarlyFamily h/o Early ≤≤ 18 (E) Later > 18 (L) Total18 (E) Later > 18 (L) Total
Negative (A) 28 35 63Negative (A) 28 35 63
Bipolar (B) 19 38 57Bipolar (B) 19 38 57
Unipolar (C) 41 44 85Unipolar (C) 41 44 85
Unipolar & bipolar 53 60 113Unipolar & bipolar 53 60 113
TotalTotal 141 177 318141 177 318
Marginal prob…… P (E) = 141 / 318 = 0.4434Marginal prob…… P (E) = 141 / 318 = 0.4434
Conditional prob.. P (AConditional prob.. P (A || E)E) = 28 / 141 = 0.1986= 28 / 141 = 0.1986
Joint prob……..Joint prob…….. P (EP (E ∩∩ A)A) = 28 / 318 == 28 / 318 = 0.08810.0881
Joint prob = Marginal prob × Conditional ProbJoint prob = Marginal prob × Conditional Prob
P (EP (E ∩∩ A) =A) = P (E)P (E) P (AP (A || E)E)
= (= (141 / 318) (28 / 141)141 / 318) (28 / 141)
== 0.08810.0881

16.  P (EP (E ∩∩ A) =A) = P (E)P (E) P (AP (A || E)E)
 P (AP (A || E) = P (EE) = P (E ∩∩ A) /A) / P (E)P (E)
 Conditional probability ofConditional probability of AA givengiven EE is equal to the probability ofis equal to the probability of
EE ∩ A∩ A divided bydivided by the probability ofthe probability of EE, provided the probability of, provided the probability of
EE is not zerois not zero
ThereforeTherefore
Definition :Definition : Conditional probabilityConditional probability ofof AA givengiven BB is equal to theis equal to the
probability ofprobability of AA ∩ B∩ B divided bydivided by the probability ofthe probability of BB, provided the, provided the
probability ofprobability of BB is not zerois not zero
P (AP (A || B) = P (AB) = P (A ∩ B∩ B) /) / P (P (BB),), P (B)P (B) ≠ 0≠ 0

17. Family h/o Early= 18 (E) Later > 18 (L)Family h/o Early= 18 (E) Later > 18 (L) TotalTotal
Negative (A) 28 35 63Negative (A) 28 35 63
Bipolar (B) 19 38 57Bipolar (B) 19 38 57
Unipolar (C) 41 44 85Unipolar (C) 41 44 85
Unipolar & bipolar 53 60 113Unipolar & bipolar 53 60 113
TotalTotal 141 177 318141 177 318
P (AP (A || E) = P (EE) = P (E ∩∩ A) /A) / P (E)P (E)
= (28/318) / (141/318)= (28/318) / (141/318)
= 0.1987= 0.1987
The same as in Direct Method:The same as in Direct Method: P (AP (A || E) = 28/141E) = 28/141
= 0.1987= 0.1987

18.  The Addition Rule:The Addition Rule:
 The probability of the occurrence of either oneThe probability of the occurrence of either one oror the other ofthe other of
twotwo mutually exclusivemutually exclusive events is equal to the sum of theirevents is equal to the sum of their
individual probabilities (individual probabilities (Third property of probabilityThird property of probability))
Family h/o Early= 18 (E) Later > 18 (L)Family h/o Early= 18 (E) Later > 18 (L) TotalTotal
Negative (A) 28 35 63Negative (A) 28 35 63
Bipolar (B) 19 38 57Bipolar (B) 19 38 57
Unipolar (C) 41 44 85Unipolar (C) 41 44 85
Unipolar & bipolar 53 60 113Unipolar & bipolar 53 60 113
TotalTotal 141 177 318141 177 318
Q 4: What is the probability that a person will be early age (E) orQ 4: What is the probability that a person will be early age (E) or
later age (L)?later age (L)?

19. Answer4:Answer4: P(EP(E UU L) = P (E) + P (L)L) = P (E) + P (L)
= 141/318 + 177/318= 141/318 + 177/318
= 1= 1
[Symbol ‘U’ read ‘union’ or ‘or’][Symbol ‘U’ read ‘union’ or ‘or’]

20.  Definition :Definition :
Given two eventsGiven two events AA andand BB, the probability that event, the probability that event AA, or event, or event
BB, or, or both occurboth occur is equal to the probability that eventis equal to the probability that event AA occurs,occurs,
plus the probability that eventplus the probability that event BB occurs, minus the probabilityoccurs, minus the probability
that the events occurthat the events occur simultaneouslysimultaneously
(ie(ie not mutually exclusivenot mutually exclusive))
P(AP(A UU B) = P (A) + P (B) -B) = P (A) + P (B) - P (AP (A ∩∩ B)B)
Family h/o Early= 18 (E) Later > 18 (L)Family h/o Early= 18 (E) Later > 18 (L) TotalTotal
Negative (A) 28 35 63Negative (A) 28 35 63
Bipolar (B) 19 38 57Bipolar (B) 19 38 57
Unipolar (C) 41 44 85Unipolar (C) 41 44 85
Unipolar & bipolar 53 60 113Unipolar & bipolar 53 60 113
TotalTotal 141 177 318141 177 318
Q 5: what is the probability that a person will be an early age (E)Q 5: what is the probability that a person will be an early age (E)
or no family h/o (A) or bothor no family h/o (A) or both

21. Answer5:Answer5: P(EP(E UU A) = P (E) + P (A) -A) = P (E) + P (A) - P (EP (E ∩ A∩ A))
= 141/318 + 63/318 – 28/318= 141/318 + 63/318 – 28/318
= 0.5534= 0.5534
(duplication/overlapping is adjusted)(duplication/overlapping is adjusted)

22.  Independent events:Independent events:
 Occurrence of event B has no effect on the probability of eventOccurrence of event B has no effect on the probability of event
AA
(ie The probability of event A is the same regardless of whether(ie The probability of event A is the same regardless of whether
or not B occurs)or not B occurs)
ie.ie. P(AP(A || B) = P(A), P(BB) = P(A), P(B || A) = P(B), P(AA) = P(B), P(A ∩∩ B) = P(A) (B)B) = P(A) (B)
(if(if P (A)P (A) ≠ 0,≠ 0, P (B)P (B) ≠ 0 )≠ 0 )
((NoteNote: The terms ‘independent’ and ‘mutually exclusive’ do not: The terms ‘independent’ and ‘mutually exclusive’ do not
mean the same thing)mean the same thing)

23.  Example of Independent events:Example of Independent events:
Girl (B)Girl (B) Boy (B)Boy (B)
Eyeglasses wearing (E) 24 16 40Eyeglasses wearing (E) 24 16 40
No Eyeglasses wearing (E) 36 24 60No Eyeglasses wearing (E) 36 24 60
Total 60 40 100Total 60 40 100
Q 6: What is the probability that a person randomly selectedQ 6: What is the probability that a person randomly selected
wears eyeglasses?wears eyeglasses?

24. Answer6:Answer6: P(E)P(E) = 40/100 = 0.4= 40/100 = 0.4

25. Girl (B)Girl (B) Boy (B)Boy (B)
Eyeglasses wearing (Eyeglasses wearing (EE) 24 16 40) 24 16 40
No Eyeglasses wearing (E) 36 24 60No Eyeglasses wearing (E) 36 24 60
Total 60 40 100Total 60 40 100
P(E)P(E) = 40/100 == 40/100 = 0.40.4
P (EP (E || BB) = P(E) = P(E ∩∩ BB) / P() / P(BB)) = (16/100) / (40/100)= (16/100) / (40/100)
== 0.40.4
P (EP (E || BB) = P(E) = P(E ∩∩ BB) / P() / P(BB)) = (24/100) / (60/100)= (24/100) / (60/100)
== 0.40.4
(ie Wearing eyeglasses is not concerned with gender)(ie Wearing eyeglasses is not concerned with gender)

26.  Complementary EventsComplementary Events
Girl (B)Girl (B) Boy (B)Boy (B)
Eyeglasses wearing (Eyeglasses wearing (EE) 24 16 40) 24 16 40
No Eyeglasses wearing 36 24 60No Eyeglasses wearing 36 24 60
Total 60 40 100Total 60 40 100
P (B) = 1 – P(B)P (B) = 1 – P(B)
BB andand B are complementary eventsB are complementary events
Event BEvent B and its complement eand its complement eventvent BB are mutually exclusiveare mutually exclusive
 (Third property of probability)(Third property of probability)

27.  Marginal probability
 Definition: Given some variable that can be broken down into m
categories designated by A1, A2, …Ai...Am and another jointly
occurring variable that is broken down to n categories
designated by B1, B2…Bj...Bn, the marginal probability of Ai,
P(Ai) , is equal to the sum of the joint probabilities of Ai with all
the categories of B.
 P(Ai) = ΣP(Ai ∩ Bj ), for all value of j
Family h/o Early ≤ 18 (E) Later > 18 (L)Family h/o Early ≤ 18 (E) Later > 18 (L) TotalTotal
Negative (A) 28 35 63Negative (A) 28 35 63
Bipolar (B) 19 38 57Bipolar (B) 19 38 57
Unipolar (C) 41 44 85Unipolar (C) 41 44 85
Unipolar & bipolar 53 60 113Unipolar & bipolar 53 60 113
TotalTotal 141 177 318141 177 318

28.  IV. Bayes’s Theorem, Screening Tests, Sensitivity, specificity,
and Predictive value Positive and Negative
 Definition:
1. A false positive results when a test indicates a positive status
when the true status is negative
2. A false negative results when a test indicates a negative
status when the true status is positive
Disease
Test result Present (D) Absent (D) Total
Positive (T) a b a + b
Negative (T) c d c+ d
Total a + c b + d

29. Disease
Test result Present (D) Absent (D) Total
Positive (T) a b a + b
Negative (T) c d c+ d
Total a + c b + d
The sensitivity of a test (or symptom) is the probability of a
positive test result (or presence of symptom) given the
presence of the disease
P(T| D) = a/ a + c
The specificity of a test (or symptom) is the probability of a
negative test result (or absence of symptom) given the absence
of the disease
P(T | D) = d/ b + d

30.  The predictive value positive of a screening test (or symptom) is
the probability that a subject has the disease given that the
subject has a positive screening test result (or has the
symptom)
 P(D | T)
 The predictive value negative of a screening test (or symptom)
is the probability that a subject does not have the disease given
that the subject has a negative screening test result (or does
not have the symptom)
 P(D | T)

31.  Bayes’s Theorem
 Thomas Bayes, an English Clergyman (1702-1761)
 Based on test’s sensitivity, specificity and probability of the
relevant disease in general population
 P(D | T) = P(T | D) P(D) / P(T | D) P(D) + P(T | D) P(D)
 P(D |T)= P(T | D) P(D) / P(T |D) P(D) + P(T |D) P(D)
Alzheimer’s disease Total
Test Result Yes (D) No (D)
Positive (T) 436 5 441
Negative (T) 14 495 509
Total 450 500 950

32. Alzheimer’s disease Total
Test Result Yes (D) No (D)
Positive (T) 436 5 441
Negative (T) 14 495 509
Total 450 500 950
 P(T | D) = a/ a + c = 436 / 450 = 0.9689
 P(T | D) = d/ b + d = 495 / 500 = 495 / 500 = 0.99
 P(D | T)
 P(D | T)

33. P(D | T) = P(T | D) P(D) / P(T | D) P(D) + P(T | D) P(D)
= (0.9689) P(D) / (0.9689) P(D) + (0.1) P (D)
P(T | D) = 436/450 = 0.9689
P(T | D) = 5/500 = 0.01
If P(D) = 0.113
P(D | T)?
P(D | T) = P(T | D) P(D) / P(T | D) P(D) + P(T | D) P(D)
= (0.9689) P(D) / (0.9689) P(D) + (0.1) P (D)
= (0.9689) (0.113) / (0.9689) (0.113) + (0.01) (1- 0.113)
= 0.93
Predictive value of the test is very high.

34. THE END