Upcoming SlideShare
×

Topic 6 stat probability theory

904 views

Published on

Published in: Technology, Education
1 Like
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
Your message goes here
• Be the first to comment

Views
Total views
904
On SlideShare
0
From Embeds
0
Number of Embeds
23
Actions
Shares
0
38
0
Likes
1
Embeds 0
No embeds

No notes for slide

Topic 6 stat probability theory

1. 1. Some Basic Probability Concepts
2. 2. Some Basic Probability ConceptsExperiments, Outcomes and Random Variables• An experiment is the process by which an observation is made.• Sample Space: ‘set of all possible well distinguished outcomes of an experiment’ and is usually denoted by the letter ‘S’.• For example, Tossing a coin: S= {H, T}, Tossing a die: S = {1,2,3,4,56}• Sample Point: ‘each outcome in a sample space’• Event: ‘Subset of the sample space’• A random variable is ‘a real valued function defined on the sample space’.• A random variable is a variable whose value is unknown until it is observed. The value of a random variable results from an experiment; it is not perfectly predictable. Some Basic Probability Concepts 2
3. 3. Some Basic Probability Concepts• A discrete random variable can take only a finite number of values that can be counted by using the positive integers.• A continuous random variable can take any real value (not just whole numbers) in an interval on the real number line• A continuous random variable can take any real value (not just whole numbers) in an interval on the real number line. Some Basic Probability Concepts 3
4. 4. The Probability Distribution of a Random Variable• The term Probability is used to give a quantitative measure to the uncertainty associated with outcomes of a random experiment.• Probability: The Classical Definition• In a random experiment, if there are ‘n’ equally likely and mutually exclusive outcomes, of which ‘f’ are favorable to an event ‘A’, then the probability of occurrence of event A, denoted by P(A), is given by the ratio, f/n.• The frequency approach: ‘the limit of relative frequency as the number of observations approached infinity’. Some Basic Probability Concepts 4
5. 5. Some Basic Postulates• Postulate 1: The probability of an event is a nonnegative real number; that is, 0  P (Ai)  1 for each subset Si of S;• Postulate 2: P(S) = 1• Postulate 3: If S1, S2, S3,…Sn are mutually exclusive events defined on the sample space S, then P(S1U S2U S3… U Sn) = P(S1) + P(S2) P(S3)+…+P(Sn)• An Illustration:• Suppose we have information about the population in Comilla . We are interested in two characteristics only, Sex (M or F) and economic status (Poor or Non poor). The two characteristics are not mutually exclusive.• S = { (M & P), (F & P), (M & NP), (F &NP)} Some Basic Probability Concepts 5
6. 6. If the population is finite, then the distribution is Economic Status Totals Poor Non poor Male  β α+βSex Female γ δ γ+δ α +β +γ +δ = Total α+γ +δ N Some Basic Probability Concepts 6
7. 7. In terms of probabilities, the distribution would look like Economic Status Poor Non poor Male P(M∩P) P(M∩NP) P(M) Sex Female P(F∩P) P(F∩NP) P(F) P(poor) P(Non poor) 1 Some Basic Probability Concepts 7
8. 8. • The probabilities pertaining to intersection of sets are called joint probabilities. For instance, P (Male ∩ Poor) is the probability that a person selected at random in Comilla will be both male and poor, i.e., has two joint characteristics.• The probabilities that appear in the last row and in the last column of the table are called marginal probabilities. P (M) gives the probability of drawing a male regardless of his economic status.• It may be noted that marginal probabilities are equal to the corresponding joint probabilities. Some Basic Probability Concepts 8
9. 9. • What is the probability that a person of given sex is poor, or that a person of given economic status is a male (female)? Such probabilities are called conditional probabilities. For instance, P(Poor/Male) means that we have a male and we want to find out the probability that he is poor, which is given by    P( M  P) P( P / M )         P( M )    Some Basic Probability Concepts 9
10. 10. • When the values of a discrete random variable are listed with their chances of occurring, the resulting table of outcomes is called a probability function.• For a discrete random variable X the value of the probability function f(x) is the probability that the random variable X takes the value x, f(x) =P(X=x).• Therefore, 0  f(xi)  1 and, if X takes n values x1, .., xn, then.• For the continuous random variable Y the probability density function f(y) can be represented by an equation, which can be described graphically by a curve. For continuous random variables the area under the probability density function corresponds to probability. Some Basic Probability Concepts 10
11. 11. Probability function & Its Advantages• Consider the experiment of tossing two six-sided dice. Define the random variable as the sum total of dots observed. Its values range from 2, 3.. to 12. The sample space will consist of all possible permutations of the two sets of numbers from 1 to 6. In sum, there will be 36 permutations. Some Basic Probability Concepts 11
12. 12. The resulting probability distribution will be as follows:X Elements of sample F(x) space2 11 1/363 12, 21 2/364 13, 31, 22 3/365 14, 41, 23, 32 4/366 15, 51, 24, 42, 33 5/367 16, 61, 25, 52, 34, 43 6/368 26, 62, 35, 53, 44 5/369 36, 63, 45, 54 4/3610 46, 64, 55 3/3611 56, 65 2/3612 66 1/36 1 6 x 7 f ( x)  36 Some Basic Probability Concepts 12
13. 13. Expected Values Involving a Single Random Variable• The Rules of Summation• If X takes n values x1, ..., xn then their sum is n  x  x  x  x i 1 i 1 2 n• If a is a constant, then n  a  na i 1• If a is a constant then n n  ax i 1 i a  xi i 1 Some Basic Probability Concepts 13
14. 14. • If X and Y are two variables, then n n n  ( x  y )  x   y i 1 i i i 1 i i 1 i• If X and Y are two variables, then n n n  (ax  by )  a x  b y i 1 i i i 1 i i 1 i• The arithmetic mean (average) of n values of X is n  x x  x   x i x i 1  1 2 n• Also, n n n (x  x )  0 i 1 i Some Basic Probability Concepts 14
15. 15. • We often use an abbreviated form of the summation notation. For example, if f(x) is a function of the values of X, n  f (x )  f (x )  f (x )   f (x ) i 1 i 1 2 n =  f ( xi ) ("Sum over all values of the index i") i   f ( x) ("Sum over all possible values of X ") x• Several summation signs can be used in one expression. Suppose the variable Y takes n values and X takes m values, and let f(x, y) =x+y. Then the double summation of this function is Some Basic Probability Concepts 15
16. 16. m n m n  f ( x , y )   ( x  y ) i 1 j 1 i j i 1 j 1 i j• To evaluate such expressions work from the innermost sum outward. First set i=1 and sum over all values of j, and so on.• To illustrate, let m = 2 and n = 3. Then  f  x , y     f  x , y   f  x , y   f  x , y  2 3 2 i j i 1 i  2 i 3 i 1 j 1 i 1  f  x1 , y1   f  x1 , y2   f  x1 , y3   f  x2 , y1   f  x2 , y2   f  x2 , y3  Some Basic Probability Concepts 16
17. 17. • The order of summation does not matter, so m n n m  f ( x , y )   f ( x , y ) i 1 j 1 i j j 1 i 1 i j Some Basic Probability Concepts 17
18. 18. The Mean of a Random Variable• The expected value of a random variable X is the average value of the random variable in an infinite number of repetitions of the experiment (repeated samples); it is denoted E[X].• If X is a discrete random variable which can take the values x1, x2,…,xn with probability density values f(x1), f(x2),…, f(xn), the expected value of X is E[ X ]  x1 f ( x1 )  x2 f ( x2 )    xn f ( xn ) n   xi f ( xi ) i 1   xf ( x) x Some Basic Probability Concepts 18
19. 19. Expectation of a Function of a Random Variable• If X is a discrete random variable and g(X) is a function of it, then E[ g ( X )]   g ( x) f ( x) x• However, in general, if X is a discrete random variable and g(X) = g1(X) + g2(X), where g1(X) and g2(X) are functions of X, then E[ g ( X )]  [ g1 ( x)  g 2 ( x)] f ( x) x   g1 ( x) f ( x)   g 2 ( x) f ( x) x x  E[ g1 ( x)]  E[ g 2 ( x)] Some Basic Probability Concepts 19
20. 20. • The expected value of a sum of functions of random variables, or the expected value of a sum of random variables, is always the sum of the expected values.• If c is a constant, E[c]  c• If c is a constant and X is a random variable, then E[cX ]  cE[ X ]• If a and c are constants then E[a  cX ]  a  cE[ X ] Some Basic Probability Concepts 20
21. 21. The Variance of a Random Variable var( X )  2  E[ g ( X )]  E  X  E ( X )  E[ X 2 ]  [ E ( X )]2 2• Let a and c be constants, and let Z = a + cX. Then Z is a random variable and its variance is var(a  cX )  E[(a  cX )  E (a  cX )]2  c 2 var( X ) Some Basic Probability Concepts 21
22. 22. A Recap• Probability: Basic Concepts• Classical & Frequency approaches• Some Basic Postulates• Some Examples• Probability function & its advantages• Mathematical expectation Some Basic Probability Concepts 22
23. 23. Using Joint Probability Functions• Marginal Probability Functions• If X and Y are two discrete random variables then f ( x )   f ( x, y ) for each value X can take y f ( y )   f ( x, y ) for each value Y can take x• Conditional Probability Functions f ( x, y ) f ( x | y )  P[ X  x | Y  y ]  f ( y) Some Basic Probability Concepts 23
24. 24. Independent Random Variables• If X and Y are independent random variables, then f ( x, y)  f ( x) f ( y) for each and every pair of values of x and y. The converse is also true.• If X1, …, Xn are statistically independent the joint probability function can be factored and written as f ( x1 , x2 ,, xn )  f1 ( x1 )  f 2 ( x2 )   f n ( xn ) Some Basic Probability Concepts 24
25. 25. • If X and Y are independent random variables, then the conditional probability function of X given that Y=y is f ( x, y ) f ( x ) f ( y ) f ( x | y)    f ( x) f ( y) f ( y) for each and every pair of values x and y. The converse is also true. Some Basic Probability Concepts 25
26. 26. The Expected Value of a Function of Several Random Variables: Covariance and Correlation• If X and Y are random variables, then their covariance is cov( X ,Y )  E[( X  E[ X ])(Y  E[Y ])]• If X and Y are discrete random variables, f(x,y) is their joint probability function, and g(X,Y) is a function of them, then E[ g ( X , Y )]   g ( x, y ) f ( x, y ) x y Some Basic Probability Concepts 26
27. 27. • If X and Y are discrete random variables and f(x,y) is their joint probability function, then cov( X , Y )  E[( X  E[ X ])(Y  E[Y ])]  [ x  E ( X )][ y  E (Y )] f ( x, y ) x y• If X and Y are random variables then their correlation is cov( X ,Y ) = var( X ) var(Y ) Some Basic Probability Concepts 27
28. 28. • The Mean of a Weighted Sum of Random Variables E[aX  bY ]  aE( X )  bE(Y )• If X and Y are random variables, then E  X  Y   E  X   E Y  Some Basic Probability Concepts 28
29. 29. The Variance of a Weighted Sum of Random Variables• If X, Y, and Z are random variables and a, b, and c are constants, then var  aX  bY  cZ   a 2 var  X   b 2 var Y   c 2 var  Z   2ab cov  X ,Y   2ac cov  X , Z   2bc cov Y , Z • If X, Y, and Z are independent, or uncorrelated, random variables, then the covariance terms are zero and: var  aX  bY  cZ   a2 var  X   b2 var Y   c2 var  Z  Some Basic Probability Concepts 29
30. 30. • If X, Y, and Z are independent, or uncorrelated, random variables, and if a = b = c = 1, then var  X  Y  Z   var  X   var Y   var  Z  Some Basic Probability Concepts 30
31. 31. Theoretical Derivation of Sampling Distribution of Estimators & Test Statistics:• Binomial Distribution:• Comilla Story: Picking a BPL Person• Let p be the proportion of BPL population in Comilla and q be the proportion of APL population.• Let n denote the sample size.• Let be the proportion of BPL in the sample.• Let X denote the number of poor in the sample.• If the person picked up happens to be poor, the experiment is a success and its probability is p. Otherwise, it is a failure with a probability given by q, that is, (1-p). Some Basic Probability Concepts 31
32. 32. • Let us define the sampling distributions of X ˆ and for samples of various sizes. Since p = (X/n) or X = n , by the different results that we have learnt so far, we can determine the distribution of , if we know that of X and vice versa. Some Basic Probability Concepts 32
33. 33. Sampling Distribution for n = 1Number of Poor Probability: f(x) x f(x) x2 f(x) 0 P(F) = q 0 0 1 P(S) = p p p Sum p+q=1 p p Some Basic Probability Concepts 33
34. 34. • Mean and variance of X: E( X )  x i i f ( xi )  p Var( X )  E ( X 2 )  [ E ( X )]2  x i 2 i f ( xi )  [ xi f ( xi )]2 i  p  p2  p(1  p)  pq• Mean and variance of p : ˆ E( p) – E(X/n) = E(X) = p ˆ ^ Var( p )=Var(X/n)=Var(X)=pq Some Basic Probability Concepts 34
35. 35. Sampling Distribution for n = 2Number of Poor Probability: f(x) xf(x) x2 f(x) 0 P(F)P(F) = q2 0 0 P(F)P(S)+P(S)P(F) 1 2pq 2pq = 2pq 2 P(S)P(S) = p2 2p2 4p2 Sum (p+q)2 = 1 2p(p+q) 2p(2p+q) Some Basic Probability Concepts 35
36. 36. • Mean and variance of X: E( X )  x i i f ( xi )  2 p( p  q) 2 p Var( X )  E ( X 2 )  [ E ( X )]2  x i 2 i f ( xi )  [ xi f ( xi )]2 i  2 p (q  2 p)  (2 p) 2  2 pq  4 p  4 p 2  2 pq 2• Mean and variance of p: ˆ ^ E( p)  E(X/n)  E(X/2)  p ^ Var (p)  Var(X/2)  (1/4)Var(X  (pq/2) ) Some Basic Probability Concepts 36
37. 37. Sampling Distribution for n = 3Number of Probability: xf(x) x2 f(x) Poor f(x) 0 P(F) = q3 0 0 pqq+ qpq+ 1 3p2q 3q2p qqp= 3pq2 ppq+ pqp+ 2 6pq2 12qp2 qpp=3p2q 3 ppp = p3 3p3 9p2 Sum (p+q)3 = 1 Some Basic Probability Concepts 37
38. 38. • Mean and variance of X: E( X )  x i f (xi )  3q 2 p  6qp 2  3 p 3  3 p(q 2  2 pq  p 2 )  3 p( p  q) 2  3 p i• Mean and variance of Some Basic Probability Concepts 38
39. 39. • In general, we have: Some Basic Probability Concepts 39
40. 40. • That is, the probability of getting x poor people in a sample size of ‘n’ is• Properties: Some Basic Probability Concepts 40
41. 41. • E( ) = p, that is, unbiased estimator. ^ p• ,that is, the distribution gets concentrated as sample size increases. This property together ^ with (i) implies p is a consistent estimator. The dispersion of the sampling distribution decreases in inverse proportion to the square root of sample size. That is, if sample size increases k times, then the std. deviation of the sampling distribution decreases k times. Some Basic Probability Concepts 41
42. 42. ^• The sampling distribution of p is most dispersed when the population parameter p is equal to ½ and is least dispersed when p is 0 or 1. Some Basic Probability Concepts 42
43. 43. • The asymmetry (skewness) of the sampling ^ distribution of p decreases in inverse proportion to the square root of sample size (since ))))))))))))))))).• It is least skewed when p = ½ and is most skewed when p is 0 or 1. Some Basic Probability Concepts 43
44. 44. The Normal DistributionProperties:•The distribution is continuous and symmetricaround its mean μ. This implies: (i) mean = median =mode; and (ii) the mean divides the area under thenormal curve into exact halves.•The range of the distribution extends from -∞ to +∞. In other words, the distribution is unbounded. Some Basic Probability Concepts 44
45. 45. • The curve attains maximum height at x = μ; the points of inflection occur at x = μ σ(which means the standard deviation measures the distance from the center of the distribution to a point of inflection).• Normal distribution is fully specified by two parameters, mean (μ) and variance (σ2). If we know these two parameters, we know all there is to know about it.• If X, Y,…, Z are normally and independently distributed random variables and a,b,…,c are constants, then the linear combination aX+bY+…+cZ is also normally distributed. Some Basic Probability Concepts 45
46. 46. How to calculate probabilities for a normal random variable?• From tabulated results• Different normal distributions lead to different probabilities due to differences in mean and variance. For the same reason, if we know the area under one specific normal curve, the area under any other normal curve can be computed by accounting for the differences in mean and variance.• One specific distribution for which areas have been tabulated is a normal distribution with mean μ = 0 and variance σ2 = 1, called the standard normal distribution (also called unit normal distribution).• Given that (i) X is normally distributed with mean μ and variance σ2; and (ii) the areas under the standard normal curve, how to determine the probability that x lies in some interval, say, (x1 and x2) ? Some Basic Probability Concepts 46
47. 47. • Let Z denote a normally distributed variable with mean zero and variance equal to unity. That is,• P(x1 < x < x2) = probability that X will lie between x1 and x2(x1 < x2); and P(z1< z <z2) = probability that Z will lie between z1 and z2 (z1 < z2) .• Since X is normally distributed, a linear function of X will also be normal. Some Basic Probability Concepts 47
48. 48. • Let it be denoted by aX + b, where a and b are constants.• Choose a and b such that (aX+b) is a standard normal variable. That is, Some Basic Probability Concepts 48
49. 49. • Solving for a and b , we get• Thus, we have aX+b = =Z• In other words, any variable with mean μ and variance σ2 can be transformed into a standard normal variable by expressing it as a deviation from its mean and dividing by σ. Some Basic Probability Concepts 49
50. 50. • Consider P(x1 < x < x2) where x1 < x2. (X  )• From  = Z, we get X = Z+. Hence, we can write x1 = z1 +  and x2 = z2 + • Now, P(x1 < x < x2) = P(z1 +  < Z +  <z2 +  ) = P(z1 < Z < z2) (X   ) (X  )• where z1 =  and z2 =  Some Basic Probability Concepts 50
51. 51. Thank You