Upcoming SlideShare
×

# 21 Ml

668 views

Published on

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

### 21 Ml

1. 1. Stat310 Maximum likelihood Hadley Wickham Sunday, 11 April 2010
2. 2. 1. Assessment 2. Feedback 3. Joint pdf 4. Maximum likelihood Sunday, 11 April 2010
3. 3. Assessment • All grading now 100% up to date (as far as I know) • Overall grade to date in owlspace (but doesn’t account for dropping lowest homework) • Quizzes were going to be worth 10%, change to 5%? Sunday, 11 April 2010
4. 4. So far • 2 / 2 tests * 10% = 20% • 7 / 10 homeworks * 40% = 28% • 3 / 5 quizzes * 5% = 3% • Total: 51% of grade Sunday, 11 April 2010
5. 5. To come • 1 ﬁnal * 30% = 30% • 3 / 10 homeworks * 40% = 12% • 2 / 5 quizzes * 5% = 2% • 5% TBA • Total: 49% of grade Sunday, 11 April 2010
6. 6. Test • Bad news: It was harder • Good news: I’ve ﬁgured out why, so it won’t happen on the ﬁnal Sunday, 11 April 2010
7. 7. 14 12 10 8 count 6 4 2 0 0.0 0.2 0.4 0.6 0.8 1.0 T2 Sunday, 11 April 2010
8. 8. 1.0 Better ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.6 ● ● ● ● ● ● ● ● ● ● ● ● ● Test 2 ● ● ● ● ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● ● ● 0.2 0.0 Worse 0.0 0.2 0.4 0.6 0.8 1.0 Test 1 Sunday, 11 April 2010
9. 9. 15 10 count 5 0 10 20 30 40 50 Overall Sunday, 11 April 2010
10. 10. These are minimums 15 described in the syllabus 10 count 5 F C B A 0 10 20 30 40 50 Overall Sunday, 11 April 2010
11. 11. Options • Do nothing. • Add 3 points on to test. Distribute 5% evenly across all assessment. • 1 hour take home exam worth 5%. 2-3 problems from the book. • 1 extra homework worth 5%. 4-5 problems from the book. Sunday, 11 April 2010
12. 12. Homeworks • Due Thursday in class • Out of the goodness of my heart I have been accepting late homeworks • But it is getting excessive - I shouldn’t have to deal with 15 late homeworks a week • Please turn in on time or I will start enforcing the late homework penalty. Sunday, 11 April 2010
13. 13. Feedback Sunday, 11 April 2010
14. 14. Feedback about me Doing well: Lectures/teaching (13), engaging/ interesting lectures (11), website (10), examples (10), homeworks (8), help sessions (6), pace (4), funny (3), being awesome (2) Needs improvement: test too hard (too many to count), hard to study from ppt (7), more activities (5), less mistakes (5), too fast (4), homework session should be a tutorial (3) Sunday, 11 April 2010
15. 15. Changes My notes are scattered between slides, the board and my voice. Your notes should not be! Will continue to try and ﬁnd interesting examples and activities. For ﬁnal review session, will have voting system and I’ll re-cover popular topics on the board. Sunday, 11 April 2010
16. 16. You Doing well Needs improvement Sunday, 11 April 2010
17. 17. You Marijuana? Doing well Needs improvement Sunday, 11 April 2010
18. 18. You Doing well Probably read Needs ahead, but improvement who does that anyways Sunday, 11 April 2010
19. 19. You I’m enjoying Doing well the weather Needs improvement Sunday, 11 April 2010
20. 20. You Doing well Needs my grade improvement Sunday, 11 April 2010
21. 21. Why do we care about random variables? Sunday, 11 April 2010
22. 22. Experiments If we capture all the relevant information about an experiment, we can repeat virtually (either mathematically or computationally). This is usually easier and cheaper than doing the real experiment! The mathematical abstraction we use to do this is the random variable. Sunday, 11 April 2010
23. 23. So The purpose of a random variable is to describe (or at least approximate) the behaviour of an experiment. So: X ~ SomeDist(some params) means we have a single experiment whose behaviour is deﬁned. Sunday, 11 April 2010
24. 24. Replications X1 ~ SomeDist(some params) X2 ~ SomeDist(some params) Means we repeat the experiment twice - it’s the same distribution, which implies that the experiment is repeated under identical conditions. f(x1, x2) is the bivariate pdf which allows us to ﬁgure out the probability of any event involving the two replicates Sunday, 11 April 2010
25. 25. Replicates Xi ~ SomeDist(some params) i = 1, 2, ..., n Means we repeat the experiment n times. f(x1, x2, ..., xn) is the joint pdf which allows us to ﬁgure out the probability of any event involving the n replicates Sunday, 11 April 2010
26. 26. Maximum likelihood Sunday, 11 April 2010
27. 27. Your turn On Tuesday I was dismayed to ﬁnd that if Xi ~ Binomial(n, p) then an estimator for p n is i Xi /n 2 In fact, this estimator is basically correct, but there is a problem with my notation. Can you spot where I went wrong? (everything you need is on this slide) Sunday, 11 April 2010
28. 28. Formal deﬁnition The maximum likelihood estimator is a value of the parameter that maximises the likelihood function with respect to the parameter. ˆM L = max l(θ; x1 , x2 , . . . , xn ) θ θ∈Θ Sunday, 11 April 2010
29. 29. Steps Write out likelihood (=joint pdf) Write out log-likelihood (Discard constants) Find maximum: Differentiate and set to 0 (Check second derivative is negatice) (Check end points) Sunday, 11 April 2010
30. 30. Maximum • Derivative zero • Derivative undeﬁned • At boundary points Sunday, 11 April 2010
31. 31. Your turn Xi ~ Poisson(λ) i = 1,..., n Use maximum likelihood to ﬁnd an estimator for λ Sunday, 11 April 2010
32. 32. Invariance principle One neat property of maximum likelihood estimators is invariance Sunday, 11 April 2010
33. 33. What else? MLEs are: Unbiased Minimum variance Have asymptotically normal distribution! ˆM L ) = −1 V ar(θ δ2 E δθ2 l(X|θ) Sunday, 11 April 2010
34. 34. But That math is too hard for this course :( So we need some other ways to work out how much error our estimators have. Sunday, 11 April 2010
35. 35. Your turn What is the variance of ˆM L ? λ Sunday, 11 April 2010
36. 36. Your turn I repeated an experiment deﬁned by Poisson(λ) 10 times, and recorded the following results: 6 11 10 6 12 7 8 5 7 10 What is the MLE of λ? What is the standard deviation of our estimate? Sunday, 11 April 2010
37. 37. Answer Mean = 8.2 SD = 0.90 Can you create an interval around the estimate that ensures that the true value will be inside it 95% of the time? (Use clt) Sunday, 11 April 2010
38. 38. Reading 6.1, 6.1.1 Sunday, 11 April 2010