Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Jeffreys' and BDeu Priors for Model Selection
WITMSE 2016
Helsinki, Finland, September 20
Joe Suzuki
(prof-joe)
Joe Suzuki...
Goal and Contributions
[Goal]
Compare for model selection
• BDeu (Bayesian Dirichlet equivalent uniform)
• Jeffreys prior ...
Road Map
1. Bayesian Dirichlet Scores
2. BDeu and Jeffreys Scores
3. A Found Property and its Proof
4. Main Theorem
5. Reg...
Assign a Prob. to each Seq.
Express a Prob. by the product of Cond. Probs.
Simultaneous Probs.
Cond. Probs.
BDeu and Jeffreys’ Prior
Example 1 : Bayesian Network Structure Learning (BNSL)
Example 2: Independence Testing
A Motivating Example
A Found Property
Sketch of J(n)>0 for BDeu
Sketch of J(n)≦0 for Jeffreys’
An Intuitive Reasoning
Main Theorem
Examples
more likely
unlikely
Regularity in Model Selection
Fitness + Simplicity → optimal
(-1) x Likelihood + Penalty Term → min
Newton’s
Law of
Motion...
BDeu violates regularity in model selection
Z XZ X
Y
Y X
B&B for efficient BNSL (Depth First Search)
Those bounds utilize regularity
Campos and Ji 2011 figured out one (=nice)
but the bound is not efficient (experiments).
D...
Bayes Prior
Based on his/her Belief:
Nobody should reject it from a general point of view.
BDeu violates regularity
contra...
Summary
The prior behind BDeu might have been based on a wrong belief
That contradicts regularity in model selection
Futur...
Jeffreys' and BDeu Priors for Model Selection
Upcoming SlideShare
Loading in …5
×

Jeffreys' and BDeu Priors for Model Selection

347 views

Published on

The Ninth Workshop on Information Theoretic Methods in Science and Engineering (WITMSE), Helsinki, Finland, on September 19–21, 2016.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Jeffreys' and BDeu Priors for Model Selection

  1. 1. Jeffreys' and BDeu Priors for Model Selection WITMSE 2016 Helsinki, Finland, September 20 Joe Suzuki (prof-joe) Joe Suzuki (Osaka Univ., Japan)
  2. 2. Goal and Contributions [Goal] Compare for model selection • BDeu (Bayesian Dirichlet equivalent uniform) • Jeffreys prior (T-K estimator) [Contribution] Mathematically Proves
  3. 3. Road Map 1. Bayesian Dirichlet Scores 2. BDeu and Jeffreys Scores 3. A Found Property and its Proof 4. Main Theorem 5. Regularity in Model Selection 6. Summary
  4. 4. Assign a Prob. to each Seq.
  5. 5. Express a Prob. by the product of Cond. Probs.
  6. 6. Simultaneous Probs.
  7. 7. Cond. Probs.
  8. 8. BDeu and Jeffreys’ Prior
  9. 9. Example 1 : Bayesian Network Structure Learning (BNSL)
  10. 10. Example 2: Independence Testing
  11. 11. A Motivating Example
  12. 12. A Found Property
  13. 13. Sketch of J(n)>0 for BDeu
  14. 14. Sketch of J(n)≦0 for Jeffreys’
  15. 15. An Intuitive Reasoning
  16. 16. Main Theorem
  17. 17. Examples more likely unlikely
  18. 18. Regularity in Model Selection Fitness + Simplicity → optimal (-1) x Likelihood + Penalty Term → min Newton’s Law of Motion Maxwell Equations If model A is better than model B w.r.t. fitness and simplicity, model A should be chosen (regularity). Information Criteria LASSO
  19. 19. BDeu violates regularity in model selection Z XZ X Y Y X
  20. 20. B&B for efficient BNSL (Depth First Search)
  21. 21. Those bounds utilize regularity Campos and Ji 2011 figured out one (=nice) but the bound is not efficient (experiments). Designing Pruning rules for BDeu is HARDer. because regularity cannot be assumed
  22. 22. Bayes Prior Based on his/her Belief: Nobody should reject it from a general point of view. BDeu violates regularity contradicts with Newton, Maxwell, Information Critreria, LASSO, etc. People might notice that their beliefs have been wrong, after knowing the new result in this paper.
  23. 23. Summary The prior behind BDeu might have been based on a wrong belief That contradicts regularity in model selection Future Work: Consider NML and others in a similar way

×