Probability theory isthe branch
of mathematics concerned with
uncertainty which provides the
foundation for statistical
inference, risk analysis, and
stochastic modelling.
It provides a rigorous framework
for understanding the likelihood
of different outcomes from
random processes.
What is a Probability Theory?
3.
Key Concepts inProbability Theory:
Experiment
An action or process that can lead to
different outcomes, and whose exact
outcome cannot be predicted with
certainty beforehand. Examples include
flipping a coin, rolling a die, or drawing a
card from a deck.
Examples include flipping a coin, rolling
a die, or drawing a card from a deck.
4.
Key Concepts inProbability Theory:
Sample Space (S)
The set of all possible outcomes
of a random experiment.
Examples:
▪ Coin flip: Ω={Heads, Tails}
▪ Rolling a single die:
Ω={1,2,3,4,5,6}
▪ Flipping two coins:
Ω={HH, HT, TH, TT}
5.
Key Concepts inProbability Theory:
Event (A)
Any subset of the sample space. An
event is a specific outcome or a
collection of outcomes that you are
interested in.
Examples (based on rolling a single die):
Event A: Rolling an even number. A = {2,4,6}
Event B: Rolling a number less than 3. B = {1,2}
Event C: Rolling a 7. C={} (This is an impossible
event)
6.
Key Concepts inProbability Theory:
Probability (P(A))
A numerical measure of the
likelihood of an event
occurring. It is a real number
between 0 and 1, inclusive.
Examples:
▪ P(A)= 0 means the event is
impossible.
▪ P(A)= 1 means the event is certain.
▪ The closer P(A) is to 1, the more
likely the event is to occur.
7.
Classical Probability:
Classical Probability,also known as A Priori Probability or Theoretical
Probability, is a method of calculating probability based on the
assumption that all possible outcomes of an event are equally likely to
occur.
Main Interpretations of Probability
The formula for Classical Probability is:
Where:
▪ P(A) is the probability of event A occurring.
▪ "Number of favorable outcomes for event A" refers to the count of
outcomes where the specific event A happens.
▪ "Total number of possible outcomes in the sample space" refers to the
total count of all equally likely outcomes that could occur in the
experiment.
8.
Main Interpretations ofProbability
Examples:
▪ Tossing a fair coin: The
probability of getting a head is
1/2 (1 favorable outcome out
of 2 total outcomes).
▪ Rolling a fair six-sided die: The
probability of rolling a 3 is 1/6
(1 favorable outcome out of 6
total outcomes).
▪ Drawing a specific card from a
standard deck: The probability
of drawing the Ace of Spades
is 1/52 (1 favorable outcome
out of 52 total outcomes).
9.
Empirical Probability:
Empirical probability,also known as relative frequency probability or
experimental probability, is based on observations from actual
experiments or historical data. It is used to estimate the likelihood of an
event occurring based on how often it has occurred in the past.
Main Interpretations of Probability
The formula for Empirical Probability is:
Where:
▪ P(E) represents the empirical probability of event E.
▪ "Number of times event E occurs" is the frequency of the event you are
interested in.
▪ "Total number of trials" is the total number of times the experiment
was conducted, or observations were made.
10.
Main Interpretations ofProbability
Example 1:
Suppose you flip a coin 100 times and it lands on heads 53 times.
▪ Event E: Getting a head.
▪ Number of times event E occurs: 53
▪ Total number of trials: 100
So, the empirical probability of getting a head in this experiment is 0.53 or 53%.
Example 2:
A quality control inspector checked 500 items from a production line and
found 15 defective items.
▪ Event E: An item is defective.
▪ Number of times event E occurs: 15
▪ Total number of trials: 500
So, empirical probability of a randomly selected item being defective is 0.03 or 3%.
11.
Subjective Probability:
Subjective probabilityis unique because it does not rely on a strict
mathematical formula or objective data. Instead, it is based on an
individual's personal judgment, beliefs, experiences, and intuition about
the likelihood of an event occurring.
Main Interpretations of Probability
Key Characteristics of Subjective Probability:
▪ Personal and Individual: It varies from person to person. Two individuals
might assign different subjective probabilities to the same event.
▪ Based on Judgment: It's rooted in one's perception, experience, and
sometimes even "gut feeling.
▪ "No Formal Calculation: There's no mathematical equation you can plug
numbers into to arrive at a subjective probability.
▪ Influenced by Bias: Personal biases, optimism, pessimism, and recent
experiences can heavily influence subjective probabilities.
▪ Adaptable: It can change as new information or experiences are gained.
12.
Main Interpretations ofProbability
Example 1:
Predicting the outcome of a sports game: A passionate fan might assign a 90%
chance of their favorite team winning, even if statistical analyses suggest a much
lower probability. Their belief is driven by loyalty, past wins they witnessed, or a
feeling about the team's current form.
Example 2:
A person deciding if it will rain: You might look at the sky, feel a certain humidity in
the air, and based on your past experiences with the weather, decide there's a
40% chance of rain, even if the weather forecast gives a different percentage.
Example 3:
A doctor's estimate of a patient's recovery: A doctor might give a patient a 60%
chance of a full recovery from a rare illness, based on their experience with similar
cases, their medical knowledge, and their assessment of the patient's overall
health, even if there isn't extensive statistical data for that specific condition.
13.
Axioms of Probability
TheAxioms of Probability are fundamental
rules that form the bedrock of probability
theory. They were formally introduced by
the Russian mathematician Andrey
Kolmogorov in 1933. These axioms define
what a probability measure is and ensure
its consistency and coherence.
Kolmogorov’s Three Axioms of Probability:
▪ Axiom 1: Non-Negativity
▪ Axiom 2: Unit Measure (or Normalization)
▪ Axiom 3: Additivity (or Sigma Additivity)
14.
Axioms of Probability
Axiom1: Non-Negativity
This axiom simply states that probabilities cannot be negative. A probability
of 0 means the event is impossible, while a probability greater than 0 means
the event has some chance of occurring. It is intuitive: you cannot have a
"negative chance" of something happening.
Statement: For any event A, the probability of A is a non-negative real number.
Mathematical Notation: P(A) ≥ 0
Examples:
▪ When flipping a fair coin, the probability of getting heads, P(Heads),
must be ≥0. It is 0.5.
▪ The probability of rolling a 7 on a standard six-sided die, P(7), is 0,
which is ≥ 0.
15.
Axioms of Probability
Axiom2: Unit Measure
This axiom means that something from the set of all possible outcomes
must occur. If you perform an experiment, one of the possible results will
happen, so the probability of the entire sample space is 1 (or 100%).
Statement: The probability of the entire sample space S is 1.
Mathematical Notation: P(S)=1
Examples:
▪ When rolling a standard six-sided die, the sample space is S={1,2,3,4,5,6}.
The probability of rolling any of these numbers is P(S)=1. You are guaranteed
to roll a number between 1 and 6.
▪ When flipping a coin, the sample space is S={Heads, Tails}. The probability of
getting either heads or tails is P(S)=P(Heads)+P(Tails)=0.5+0.5=1.
16.
Axioms of Probability
Axiom3: Additivity
If events cannot happen together, the probability of at least one of them
happening is simply the sum of their individual probabilities. There is no
overlap to account for.
Statement: For any sequence of mutually exclusive (or disjoint) events A 1 ,A 2
,A 3 ,…, the probability of their union is the sum of their individual probabilities.
Mathematical Notation: If A i ∩A j =∅ for all i=j (meaning they are mutually
exclusive), then: P(A 1 ∪A 2 ∪A 3 ∪…)=P(A 1 )+P(A 2 )+P(A 3 )+…
Examples:
▪ Let A 1 be the event of rolling an even number: A 1 ={2,4,6}. P(A 1 )=3/6=0.5.
▪ Let A 2 be the event of rolling an odd number: A 2 ={1,3,5}. P(A 2 )=3/6=0.5.
▪ The probability of rolling an even or an odd number is
P(A 1 ∪A 2 )=P(A 1 )+P(A 2 )=0.5+0.5=1.
17.
Complement Rule
The complementof an event E, denoted as E c or Eˉ , is the
event that E does not occur.
Key Rules and Theorems
Example:
If the probability of rain is 0.3, the probability of no rain is 1 − 0.3 = 0.7.
18.
Key Rules andTheorems
Addition Rule
For Mutually Exclusive Events: If events A and B cannot occur
simultaneously (A ∩ B = ∅), then:
Example: The probability of rolling a 2 or a 4 on a single die:
19.
Key Rules andTheorems
Addition Rule
For Non-Mutually Exclusive Events: If events A and B can
occur simultaneously (A∩B=∅), then:
20.
Key Rules andTheorems
Conditional Probability
The probability of an event A occurring, given that another
event B has already occurred. It is denoted as P(A∣B) (read as
"the probability of A given B").
Example: What is the probability of drawing a King, given that the card
drawn is a Face Card (Jack, Queen, King)?Let A = Drawing a King, B =
Drawing a Face Card.
21.
Key Rules andTheorems
Multiplication Rule
The multiplication rule of probability theory is a fundamental
concept used to calculate the probability of two or more
events occurring together. It differentiates based on whether
the events are independent or dependent.
General Multiplication Rule
(for Dependent Events)
This is the most general form
and applies when the
occurrence of one event
affects the probability of the
other event.
22.
Key Rules andTheorems
Example:
Imagine you have a bag with 5 red marbles and 3 blue marbles. You draw two
marbles without replacement. What is the probability that both marbles are red?
▪ Let A be the event of drawing a red marble on the first draw. P(A)=5/8 (5 red
marbles out of 8 total)
▪ Let B be the event of drawing a red marble on the second draw. Since you
didn't replace the first marble, there are now only 4 red marbles left and 7
total marbles. P(B∣A)=4/7 (probability of drawing another red, given the first
was red)
23.
Key Rules andTheorems
Special Multiplication Rule (for Independent Events)
This is a simplified version of the general rule and applies when the
occurrence of one event does not affect the probability of the other
event.
Explanation:
If two events are independent, the probability of both happening is
simply the product of their individual probabilities.
24.
Key Rules andTheorems
Example:
What is the probability of flipping a coin and getting heads, AND rolling
a standard six-sided die and getting a 4?
▪ Let A be the event of getting heads on a coin flip. P(A)=1/2
▪ Let B be the event of rolling a 4 on a die. P(B)=1/6
