Successfully reported this slideshow.
Upcoming SlideShare
×

Jefferys Berger 1992

689 views

Published on

Jefferys Berger 1992

Published in: Education
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Jefferys Berger 1992

1. 1. Ockham’s Razor and Bayesian Analysis Maryam Zolghadr Carlo Alberto College Turin, April 21st, 2015 Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
2. 2. Ockham’s razor principle “Pluralitas non est poneda sine neccessitate.” — William Ockham It can be translated as Plurality must not be posited without necessity. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
3. 3. Various interpretations of O.R. principle “Entities should not be multiplied without necessity.” “It is vain to do more what can be done with less.” “An explanation of the facts should be no more complicated than necessary.” “Among competing hypothesis, favor the simplest one.” Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
4. 4. Connection between O.R. and Bayesian Analysis O.R. enjoins us to favor the simplest hypothesis that is consistent with the data, but determining which hypothesis is simplest is often no simple matter. Bayesian analysis can offer concrete help in judging the degree to which a simpler model is to be preferred. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
5. 5. Galileo’s Problem: Law of accelerated motion The law describing the motion of falling bodies proposed by Galileo and familiar to students of physics can be expressed as a quadratic equation: s = a + ut + 1 2 gt2 where a, u and g are adjustable parameters that can be assigned arbitrary values in order to ﬁr the empirical data. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
6. 6. Galileo’s Problem: Ballistic trajectory is Parabola Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
7. 7. Galileo’s Problem: Question Why is it, that the quadratic law is the choice of physics everywhere? Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
8. 8. Galileo’s Problem: Prior Probability Galileo and a modern student of physics would favor the quadratic law since it is simpler, whereas higher-degree polynomials are unnecessarily complicated. Jeffreys suggested that the reason for favoring the simpler law is that it has a higher prior probability. In other words, it is consider the likelier explanation at the outset of the experiment, before any measurements have been made. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
9. 9. Galileo’s Problem: Jeffreys and Dorothy Wrinch’s suggestion for a measure of simplicity that depends on prior probabilities For laws that can be expressed as differential equations, they suggested a straightforward algorithm for counting parameters. Having sorted all possible laws according to this criterion, one can try the simpler laws ﬁrst, only moving on to more complicated laws as the simple ones prove inadequate to represent the data. Thus the ordering of hypothesis provides a kind of rationalized O.R. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
10. 10. Problem with Jeffrey’s appeal to prior probability Deﬁning the simplest law as the one with the fewest adjustable parameters is a useful strategy, but it cannot be extended to yield a clear, universal rule for assigning prior probabilities. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
11. 11. Galileo’s Problem: Jeffreys suggestion for a measure of simplicity that does not depend on prior probabilities If a law has any adjustable parameters, then it will be signiﬁcantly preferred to the simpler law only if its prediction are considerably more accurate. If the prediction of the two models are roughly equivalent, the simpler law can have greater posterior probability. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
12. 12. To Catch a cheat: Objective quantiﬁcation of O.R. by Berger and Jeffreys (1992) Suppose a friend who has a reputation as a prankster offers to ﬂip a coin to decide who will perform a little chore: Heads he wins, tails he loses. HHH : The hypothesis that the coin has two heads (unfair coin). HHT : The hypothesis that the coin is fair. The hypothesis HHH that the coin has two heads is a simpler one than the hypothesis HHT that the coin is fair. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
13. 13. To Catch a cheat: Objective quantiﬁcation of O.R. by Berger and Jeffreys (1992) Before the coin is ﬂipped, you might believe that the hypothesis HHH and HHT are equally likely. After series of coin tosses, if heads appears invariably in a long series of tosses, the hypothesis of two headed coin becomes more attractive. The fair-coin hypothesis is consistent with every possible observation. However, the two-heads hypothesis would be falsiﬁed by a single appearance of tails. Since the two-heads hypothesis makes such sharp predictions, it is given greater (belief) credit when those predictions come to pass. Back to Galielo’s problem Maryam Zolghadr Ockham’s Razor and Bayesian Analysis
14. 14. Conclusion The key idea links Bayesian analysis to O.R. is a notion of simplicity in a hypothesis. We have discussed two ways in which Ockham’s razor can be interpreted in Bayesian terms. 1 By choosing the prior probabilities of hypotheses, one can quantify the scientiﬁc judgment that simpler hypotheses are more likely to be correct. 2 Bayesian analysis also shows that a hypothesis with fewer adjustable parameters automatically has an enhanced posterior probability, because the predictions it makes are sharp about what data will be observed, and it is more readily falsiﬁed by arbitrary data. Both of these ideas are in agreement with the intuitive notion of what makes a scientiﬁc theory powerful and believable. Maryam Zolghadr Ockham’s Razor and Bayesian Analysis