The Quest What can we say about this black box? E.g., What is the probability that it generates a number 5 12 3 9 28 bigger than 5?Observations
DistributionsWhat if we had many many observations? Value Frequency -1 0.3 0 0.2 Sum of 1 0.1 frequencies This table is the 2 0.1 is 1 distribution 3 0.1 associated with 4 0.1 this black box 5 0.1
Distributions Graphically -1 0 1 2 3 4 5 Area under the curve is 1
The ChallengeWe do not have many many observations So we cannot infer the distribution from the observations What can we do then?
What can we do with few observations?Assume distribution is known E.g., Normal, Binomial (from prior knowledge or etc other means) I.e., model approximately using a canonical distribution But the parameters are not known Can these parameters be determined from the observations?
Why Canonical Distributions Value Frequency -1 0.3 0 0.2 1 0.1 Too verbose a description for the 2 0.1 distribution 3 0.1 4 0.1 5 0.1 Can the entire distribution be described (even approximately) by just a few parameters, while modeling the data accurately
Example: Binomial Distribution A coin that yields 1 with Observations probability p and 0 with probability 1- p, tossed n 1 0 1 1 1 …. times, independently Value Frequency Number of 1’s? 0 1 Distribution, 2 μ=np,σ2=np(1-p) n-1Can one determine p from the (few) n observations?
Other Canonical Distributions Normal μ, σ2 Poisson μ =r,σ2=r Negative Binomial μ =rp/(1-p), σ2= rp/(1-p)2 What arethese? Later talk Gamma μ=kθ, σ2 =kθ2
Back to the QuestWe have few observations Assume these are from a known distribution family But with unknown parameters How do we determine the parameters? How do we determine μ, σ2?
Estimating Mean μ, σ2Estimate for the mean; a good estimate??
μ, σ2 What is the mean and varianceNormal!! For of this distribution? modest n.
A Mind Reading Game• Your friend chooses a number (one of 1,3,5) in his/her mind – Call this i• He/She then rolls a 6-faced die 30 times, privately – For each roll, he/she declares Heads if the number on the die is <=i, and Tails otherwise• Your goal is to guess i solely from this sequence of n Heads and Tails.• Can you read your friend’s mind?