21 Ml
Upcoming SlideShare
Loading in...5
×
 

21 Ml

on

  • 901 views

 

Statistics

Views

Total Views
901
Views on SlideShare
900
Embed Views
1

Actions

Likes
0
Downloads
5
Comments
0

1 Embed 1

http://www.slideshare.net 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

21 Ml 21 Ml Presentation Transcript

  • Stat310 Maximum likelihood Hadley Wickham Sunday, 11 April 2010
  • 1. Assessment 2. Feedback 3. Joint pdf 4. Maximum likelihood Sunday, 11 April 2010
  • Assessment • All grading now 100% up to date (as far as I know) • Overall grade to date in owlspace (but doesn’t account for dropping lowest homework) • Quizzes were going to be worth 10%, change to 5%? Sunday, 11 April 2010
  • So far • 2 / 2 tests * 10% = 20% • 7 / 10 homeworks * 40% = 28% • 3 / 5 quizzes * 5% = 3% • Total: 51% of grade Sunday, 11 April 2010
  • To come • 1 final * 30% = 30% • 3 / 10 homeworks * 40% = 12% • 2 / 5 quizzes * 5% = 2% • 5% TBA • Total: 49% of grade Sunday, 11 April 2010
  • Test • Bad news: It was harder • Good news: I’ve figured out why, so it won’t happen on the final Sunday, 11 April 2010
  • 14 12 10 8 count 6 4 2 0 0.0 0.2 0.4 0.6 0.8 1.0 T2 Sunday, 11 April 2010
  • 1.0 Better ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.6 ● ● ● ● ● ● ● ● ● ● ● ● ● Test 2 ● ● ● ● ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● ● ● 0.2 0.0 Worse 0.0 0.2 0.4 0.6 0.8 1.0 Test 1 Sunday, 11 April 2010
  • 15 10 count 5 0 10 20 30 40 50 Overall Sunday, 11 April 2010
  • These are minimums 15 described in the syllabus 10 count 5 F C B A 0 10 20 30 40 50 Overall Sunday, 11 April 2010
  • Options • Do nothing. • Add 3 points on to test. Distribute 5% evenly across all assessment. • 1 hour take home exam worth 5%. 2-3 problems from the book. • 1 extra homework worth 5%. 4-5 problems from the book. Sunday, 11 April 2010
  • Homeworks • Due Thursday in class • Out of the goodness of my heart I have been accepting late homeworks • But it is getting excessive - I shouldn’t have to deal with 15 late homeworks a week • Please turn in on time or I will start enforcing the late homework penalty. Sunday, 11 April 2010
  • Feedback Sunday, 11 April 2010
  • Feedback about me Doing well: Lectures/teaching (13), engaging/ interesting lectures (11), website (10), examples (10), homeworks (8), help sessions (6), pace (4), funny (3), being awesome (2) Needs improvement: test too hard (too many to count), hard to study from ppt (7), more activities (5), less mistakes (5), too fast (4), homework session should be a tutorial (3) Sunday, 11 April 2010
  • Changes My notes are scattered between slides, the board and my voice. Your notes should not be! Will continue to try and find interesting examples and activities. For final review session, will have voting system and I’ll re-cover popular topics on the board. Sunday, 11 April 2010
  • You Doing well Needs improvement Sunday, 11 April 2010
  • You Marijuana? Doing well Needs improvement Sunday, 11 April 2010
  • You Doing well Probably read Needs ahead, but improvement who does that anyways Sunday, 11 April 2010
  • You I’m enjoying Doing well the weather Needs improvement Sunday, 11 April 2010
  • You Doing well Needs my grade improvement Sunday, 11 April 2010
  • Why do we care about random variables? Sunday, 11 April 2010
  • Experiments If we capture all the relevant information about an experiment, we can repeat virtually (either mathematically or computationally). This is usually easier and cheaper than doing the real experiment! The mathematical abstraction we use to do this is the random variable. Sunday, 11 April 2010
  • So The purpose of a random variable is to describe (or at least approximate) the behaviour of an experiment. So: X ~ SomeDist(some params) means we have a single experiment whose behaviour is defined. Sunday, 11 April 2010
  • Replications X1 ~ SomeDist(some params) X2 ~ SomeDist(some params) Means we repeat the experiment twice - it’s the same distribution, which implies that the experiment is repeated under identical conditions. f(x1, x2) is the bivariate pdf which allows us to figure out the probability of any event involving the two replicates Sunday, 11 April 2010
  • Replicates Xi ~ SomeDist(some params) i = 1, 2, ..., n Means we repeat the experiment n times. f(x1, x2, ..., xn) is the joint pdf which allows us to figure out the probability of any event involving the n replicates Sunday, 11 April 2010
  • Maximum likelihood Sunday, 11 April 2010
  • Your turn On Tuesday I was dismayed to find that if Xi ~ Binomial(n, p) then an estimator for p ￿n is i Xi /n 2 In fact, this estimator is basically correct, but there is a problem with my notation. Can you spot where I went wrong? (everything you need is on this slide) Sunday, 11 April 2010
  • Formal definition The maximum likelihood estimator is a value of the parameter that maximises the likelihood function with respect to the parameter. ˆM L = max l(θ; x1 , x2 , . . . , xn ) θ θ∈Θ Sunday, 11 April 2010
  • Steps Write out likelihood (=joint pdf) Write out log-likelihood (Discard constants) Find maximum: Differentiate and set to 0 (Check second derivative is negatice) (Check end points) Sunday, 11 April 2010
  • Maximum • Derivative zero • Derivative undefined • At boundary points Sunday, 11 April 2010
  • Your turn Xi ~ Poisson(λ) i = 1,..., n Use maximum likelihood to find an estimator for λ Sunday, 11 April 2010
  • Invariance principle One neat property of maximum likelihood estimators is invariance Sunday, 11 April 2010
  • What else? MLEs are: Unbiased Minimum variance Have asymptotically normal distribution! ˆM L ) = −1 V ar(θ δ2 E δθ2 l(X|θ) Sunday, 11 April 2010
  • But That math is too hard for this course :( So we need some other ways to work out how much error our estimators have. Sunday, 11 April 2010
  • Your turn What is the variance of ˆM L ? λ Sunday, 11 April 2010
  • Your turn I repeated an experiment defined by Poisson(λ) 10 times, and recorded the following results: 6 11 10 6 12 7 8 5 7 10 What is the MLE of λ? What is the standard deviation of our estimate? Sunday, 11 April 2010
  • Answer Mean = 8.2 SD = 0.90 Can you create an interval around the estimate that ensures that the true value will be inside it 95% of the time? (Use clt) Sunday, 11 April 2010
  • Reading 6.1, 6.1.1 Sunday, 11 April 2010