QT1 - 04 - Probability

Probability Theory Q U A N T T E C H I N T E U Q I A S E V I T 1 0 S

Contents

Probability

Classical

Relative

Subjective

Marginal Probability

Joint Probability

Exclusive Events

Independent Events

Dependent Events

Bayes Theorem

Probability : Basic Terminology

Event

One or more of the possible outcomes of doing something

Toss a coin

Getting head

Getting tail

Person enters a mall

Person is male

Person is a student

Pick a carton of candies at a store

Candy is Amul

Candy is spoilt

Experiment

An activity that produces or causes an “event” is referred to as an “experiment”

Tossing a coin is an experiment.

Tossing three coins in a row is an experiment

Choosing a person at the entrance of a mall door and asking questions is an experiment

Pick a carton of candies at a store

Probability : Basic Terminology

Mutually Exclusive Events

Only one of the events can take place at a time

Either Head or Tail

Either Man or Woman

Non Exclusive Events

Both or more can take place

Person is male

Person is student

Collectively Exhaustive List

List of events that between them consider all cases

Toss of coin : Head or Tail – nothing else is possible

Choice of cold drink

Coke

Pepsi

Anything else

Classical Probability

Probability of an event is defined as

[ number of outcomes where event occurs ]

[ total number of possible outcomes ]

Experiment of tossing a single coin

P(Head) = 1 / 2 = 0.5

Experiment of rolling a dice

P(getting 6) = 1 / 6 = 0.167

Experiment of choosing a person at the mall

P(Person is male ) = 0.5 [ unless we know more ]

P(Person is a student ) = not known as yet

Problem with Classical Probability

State Space All Possible Outcomes Event A : Head Event B : Tail Event A : Identical Reading Event B : Different Reading Event A : Three Heads Event B : Two Heads Event C : Anything Else One Coin Tossed Two Coins Tossed Three Coins Tossed H T HH HT TH TT HHH HHT HTT THH HTH THT TTH TTT

Probability is NOT certainty ! H T T T T T H T T H P(H) = 0.5 Probability of getting H = 0.5 H H T T T H H H T H T T T T T H H T T T T H H H T H H T H T H H H H H T H H H T 3/10 3/10 6/10 9/20 2/10 11/30 6/10 17/40 8/10 25/50 T T H H T T H T H T 4/10 29/60 H T H H T T H H H T 6/10 499/1000 M a n y m o r e t r I a l s

Simple Example

Classical Approach

I have three friends Ram, Gopal, Bharat each of whom is equally likely to visit me for lunch

What is the probability that today it will be Ram ?

Total 3 outcomes possible : R G B

One is the outcome in question : R

P(R) = 1/3

In the past one month

R has visited 20 times

G has visited 5 times

B has visited 5 times

What is the probability that today it will be Ram ?

Is it P(R) = 1/3

Relative Frequency

Probability calculated from statistical data on the relative frequency of certain occurrences

Observed relative frequency of an event in a very large number of trials

Proportion of times that an event occurs in the long run when conditions are stable

Carbon monoxide emissions

Challenges with relative probability

Number of events considered

Suppose you know of three of your friends who have had jaundice this year

Can you conclude that the probability of getting jaundice is higher this year than in the last year ?

Long run stability

Suppose Tata Motors brings a new model of Indica in the market and in the first three months 10% cars have a clutch failure.

Can you conclude that the probability of a clutch failure in Indica cars is 0.1 ?

If you toss a coin

Two times you may get two heads

100 times, you would get between 45 – 55 heads

Two Questions

A yoga club consists of 10 members of whom

2 Doctors, 3 Engineers, 3 CA and 2 Other

4 women, 6 men

7 married, 3 single

If we were to choose a club secretary by lottery, what is the probability that the secretary is a

Doctor ?

Woman ?

Single person ?

The following table shows the frequency of marks in a quiz

If I were to ask a random student what is your score, what is the probability that it is in the range of

2 – 3 ?

6 – 7 ?

Points to note

Numerical value of a probability of any event is between 0 and 1

0 : if it is impossible, e.g. The probability of teacher being an alien from outer space

1 : if is totally certain, e.g. The probability of teacher being a human being

Sum of the probabilities of events that must be 1, provided the events are

Mutually exclusive

Part of a collectively exhaustive list

Something must happen !!

Subjective Probability

Based on beliefs, not hard facts ..

Because hard facts are simply not available and you cannot sit and do nothing because you do not have facts

Widely used in cases where the event in question occurs very rarely

To be replaced with statistical models if possible

Consider

Selection of a candidate for a job

Sanction of a loan

Earlier these were all subjective but now with more data being collected

Strategic Decisions are still taken on the basis of subjective probability !!

Relative Probability

Marginal Probability [unconditional probability]

Probability of an event is written as

P(A) = n where

A is the event that secretary is

Doctor

Engineer

CA

Other

n is the value of the probability 0 <= n <= 1

Marginal or Unconditional Probabilities are as follows :

P(Doctor) = 0.2

P(Engineer) = 0.3

P(CA) = 0.3

P(Other) = 0.2

yoga club of 10

4 women, 6 men

7 married, 3 single

Venn Diagram / Exclusive Events

P(Doctor) = 0.2

2

2 + 3 + 3 + 2

P(Engineer) = 0.3

3

2 + 3 + 3 + 2

P(Doctor OR Engineer)

2+3

2 + 3 + 3 + 2

P(A or B) = P(A) + P(B)

Provided the events are exclusive

Doctor 2 CA 3 Engineer 3 Other 2

Venn Diagram / Non-Exclusive Events

P(Engineer) = 0.3

3

2 + 3 + 3 + 2

P(Married) = 0.7

7

7+3

P(Engineer OR Married)

= P(Eng) + P(Mar) – P(Eng AND Mar)

P(A or B) = P(A) + P(B) - P( AB )

Provided the events are non-exclusive

Single 3 Married 7 Doctor 2 CA 3 Engineer 3 Other 2

Double Counting of Non Exclusive Events

P(Engineer OR Married )

= P (Single Engineer) + P(Married Engineer) + P(Married NonEngineer)

= (3 – x)/10 + x/10 + (7 -x)/10

= 3/10 + 7/10 – x/10

= P(Engineer) + P(Married) – P(Married AND Engineer)

Since Profession and Marital Status are NOT EXCLUSIVE

All Engineer = 3 All Married = 7 Single Engineer = 3 - x Married, non Engg = 7 - x Married, Engg = x Engineers, married = x

Non Exclusivity : Question

Probability of Either of Two Events A, B =

P( A OR B )

= P(A) + P(B)

If A and B are exclusive events

= P(A) + P(B) – P(A AND B)

If A and B are not exclusive

= P(A) + P(B) - P(AB)

A Quality Control inspector checks cartons of breakfast cereals for

Correctness of weight

Damage to cartons

Historical data shows that

1% cartons are underweight

0.5% cartons are damaged

0.5% cartons have both flaws

If he inspects 1000 cartons how many would he expected to reject

Where are we ? 1

Marginal Probability P(A)

Probability of A or B

Exclusive Events P(AorB) = P(A) + P(B)

Non Exclusive Events P(AorB) = P(A)+P(B)-P(AB)

Joint Probability of Two Events

What are event pairs ?

Person is DOCTOR, Person is MARRIED

Carton is DAMAGED, Carton is UNDERWEIGHT

Major Question

Are the two events in the pair dependent or independent

Consider the following

Person is DOCTOR, Person is MARRIED

Two events seem to be independent

Person is DOCTOR, Person has PAN number

Two events seem to be dependent

Whether Independent or Not Independent is to be determined OUTSIDE Probability theory

Back to the Yoga Club

A yoga club consists of 10 members of whom

4 women, 6 men

7 married, 3 single

If we were to choose a club secretary by lottery, what is the probability that the secretary is a

Doctor AND Woman

Man AND Single

Engineer AND Married

Events are deemed to be independent

Profession, Gender and Marital Status are independent of each other

We are interested in

P(Doc Wom)

P(Man Sing)

P(Engg Marr)

Joint Probability under Statistical Independence

Probability of two or more independent events happening together or in succession is the product of their individual marginal probabilities

P(AB) = P(A) x P(B)

Consider the Yoga Club

A = person is DOCTOR, B = person is WOMAN

P(A) = 0.2 P(W) = 0.4

P(DW) = 0.2 x 0.4 = 0.08

=> that out of hundred possible outcomes, there will be 8 where the secretary will be a WOMAN DOCTOR

Is this correct ? Let us check

Classical Definition of Probability When we choose a secretary, there are 100 possible outcomes .. But only 8 of these outcomes will give us a WOMAN DOCTOR

Points to note

Can be extended to more than two events

P(ABC) = P(A) x P(B) x P(C)

P(Married Woman Doctor)

= P(Married) x P(Woman) x P(Doctor)

= 0.7 x 0.4 x 0.2 = 0.056

= 56 out of 1000 possible outcomes

If we consider all possible outcomes ...

P(M-W-D) + P(S-W-D) + P(M-M-D) + P(S-M-D) +

P(M-W-E) + P(S-W-E) + P(M-M-E) + P(S-M-E) +

P(M-W-C) + P(S-W-C) + P(M-M-C) + P(S-M-C) +

P(M-W-O) + P(S-W-O) + P(M-M-O) + P(S-M-O)

= 1.000 !

Points to note

This result holds only if the outcomes are independent of each other ..

Will not hold if we have a situation where

ALL (or MOST) of the DOCTORS are WOMAN

ALL (or MOST) of the ENGINEERS are MEN

We do not know what fraction of any Profession is

Man or Woman

Married or Single

Which means that even if we are told that a DOCTOR has been selected as secretary our estimate of the probability of P(W) is still 0.4

Conditional Probability under Statistical Independence

The probability of Event B given that Event A has happened = P(B/A)

Given that the secretary is DOCTOR, what is the probability that secretary is WOMAN

P(WOMAN / DOCTOR)

In the case of INDEPENDENT events

P(B/A) = P(B)

In the case of the Yoga Club

P(WOMAN / DOCTOR ) = P(WOMAN) = 0.4

Mathematical Definition of Independence

Where are we ? 2

Two Events under Statistical Independence

Joint probability P(AB) = P(A) x P(B)

Conditional Probability P(B/A) = P(B)

When are things independent ?

Is gender independent of profession ?

Most police / fire brigade personnel are men

Most kindergarten teachers are women

Is the probability of having a PAN card independent of profession ?

Are rickshaw pullers likely to have PAN cards

Is the probability of defective products independent of factory ( or machine) where it is produced ?

Old plants, worn out machinery likely to cause more defects

Is the probability of loan default independent of gender ?

Ask Mohd Younus !!

Conditional Probability under Statistical Dependence

Another yoga club has the following kinds of members

3 employed men

1 employed women

2 student men

4 student women

Once again we select one person at random to work as the secretary of the club

Suppose that we know that the secretary is an employed person ( not a student)

What is the probability that the secretary is a

Man ?

Woman ?

We want to know

P( M / E ) or

P( W / E )

Conditional Probability that secretary is Man given that he is employed

Person is named 1 to 10

Selection of person n is event n

Since all persons are equally likely

P(any specific individual) = 1/10

= 0.10

We first use Classical Probability to list out all possibilities

P( Man / Employed ) = 3 / 4 = 0.75

¾ the of all employed members are men

P( Woman / Employed ) = 1 / 4 = 0.25

P( M / E ) + P( W / E ) = 1.00

Employed Student 3 Employed man 1 Employed woman 2 Student man 4 Student Woman Employed 3 Employed man 1 Employed woman

Rule for Conditional Probability under Statistical Dependence

P(Man/Employed)

P(Man AND Employed)

P(Employed)

3/10

4/10

0.75

similarly

P(Woman/Employed) = 0.25

= = =

Joint & Conditional Probability under Statistical Dependence

P( Man / Employed)

P ( Man AND Employed)

P ( Employed)

P( A / B)

P ( A B )

P ( B )

= = Conditional Probability Joint Probability Marginal or Unconditional Probability

Conditional Probabilities from Joint Probabilities

P(E/M) = P(E M) / P(M) = [3/10] / [5/10] = 3/5

P(S/M) = P(S M) / P(M) = [2/10] / [5/10] = 2/5

P(E/W) = P(E W) / P(W) = [1 / 10] / [ 5 / 10] = 1 / 5

P(S/W) = P(S W) / P(W) = [4/10] / [5/10] = 4/5

P(M/E) = P(E M) / P(E) = [3 / 10] / [ 4 / 10 ] = 3/4

P(W/E) = P(W E) / P(E) = [1/10] / [4/10] = 1/4

P(M/S) = P(M S) / P(S) = [2 / 10 ] / [ 6 / 10 ] = 2 / 6

P(W/S) = P( W S) / P(S) = [4/10] / [6/10] = 4/6

Conditional => Joint Probability under Statistical Dependence

P(Man/Employed)

P(Man AND Employed)

P(Employed)

P(A / B)

P(A B)

P(B)

P(Man and Employed)

= P(Man / Emp) x P(Emp)

P (A B ) = P (A / B ) x P(B)

= =

Joint Probabilities from Conditional Probabilities

P(E M) = P(E/M) x P(M) = 3/5 x 5/10 = 3/10

= P(M/E) x P(E) = 3 / 4 x 4/10 = 3/10

P(E W) = P(E/W) x P(W) = 1/5 x 5/10 = 1/10

P(S M) = P(S/M) x P(M) = 2/5 x 5/10 = 2/10

P(S W) = P(S/W) x P(W) = 4/5 x 5/10 = 4/10

Probabilities under Statistical Dependence Conditional Probability P(B/A) = P(B A) / P(A) P(A/B) = P(B A) / P(B) Joint Probability P(B A) = P(B / A) x P(A) = P(A / B) X P(B)

Where are we ? 3

Two Events under Statistical Dependence

Conditional Probability P(B/A) = P(AB) / P(A)

Joint Probability P(AB) = P(B/A) x P(A)

Marginal Probabilities under Statistical Dependence

Marginal Properties under statistical dependence are computed by simply summing up the probabilities of all the joint events where the simple event occurs

P(A) = P(AB) + P(AC)

P(Man) = P(Man Employed) + P(Man Student)

P(Student) = P(Man Student) + P(Woman Student)

Marginal Probabilities under Statistical Dependence

P(M) = P( E M) + P(S M) = 0.3 + 0.2 = 0.5

P(W) = P(E W) + P(S W) = 0.1 + 0.4 = 0.5

P(E) = P(E M) + P(E W) = 0.3 + 0.1 = 0.4

P(S) = P(S M) + P(S W) = 0.2 + 0.4 = 0.6

Where are we ? 4

Two Events under Statistical Dependence

Conditional Probability P(B/A) = P(AB) / P(A)

Joint Probability P(AB) = P(B/A) x P(A)

Multiple Events under Statistical Dependence

Marginal Probality P(A) = P(AB) + P(AC) + P(AD) + .....

Bayes Theorem revising prior estimates of probability

Central Theorem that connects

Joint Probability

Conditional Probability

Marginal Probability

Using the basic formula

P(B A) = P(B/A) x P(A)

P(B A) = P(A/B) x P(B)

P(B/A) x P(A) = P(A/B) x P(B)

Its value lies in being able to calculate P(B/A) from the value of P(A/B)

Brand Conscious Customer

Suppose there are two types of customers

Type A : Brand Conscious; Type B : Brand Indifferent

Demographic information tells us that 20% customers are Type A and rest 80% is Type B

P(A) = 0.2 ; P(B) = 0.8

P(A) + P(B) = 1 since events are mutually exclusive

Probability of sale of Designer Shirt is

60% if the customer is Brand Conscious

10% if the customer is Brand Indifferent

P(S/A) = 0.6, P(S/B) = 0.1

A person walks in and purchases a Designer Shirt

What is the probability that he is Brand Conscious

Calculate P(A/S)

Brand Conscious Customer : 1 Demographics tells us that P(A) = 0.2; P(B) = 0.8 We know the conditional probabilities P(S/A) = 0.6, P(S/B) = 0.1 Joint Probability of Sale and given type of customer hence P(SA) = P(S/A)xP(A) Marginal Probability of Sale is sum of the two joint probabilities P(S) = P(SA) + P(SB)

Brand Conscious Customer : 2