1. Bayesian methods can be used to approximate the probability of B given A (P(B|A)) by treating it as a function (f(A;w)) of the exemplars A and weights w, where the weights are unknown. 2. A Bayesian model is set up where the model/basis shapes are normally distributed, the confidence weights are independent, and there are precision tuners on the basis and weights. 3. Complexity control through regularization adds a penalty term (E_w) to the standard regression error term (E_d) controlled by a hyperparameter (lambda) to solve overfitting.

1. Bayesian Methods in Machine
Learning
Mohammad Shazri
mshazri@gmail.com
shazri.shahrir@extolcorp.com

2. Sources
• Papers from Michael Tipping
-Bayesian Inference: An introduction to Principles in Machine Learning
• Youtube from mathematicalmonk

3. approximate P(B|A)
• P(B|A)=f(A;w) ….. A is the exemplars and w is
the weights.|”given A, what is B?”.
• B and A usually relates as D= {A_n,B_n} from
n=1 to N.
• Prior statements, possible to over-fit.

4. Initial statement
• Given A , what is the likelihood of B.
• P(B|A)
• Then to approximate P(B|A)

5. Setup and Model
• Setup; it is a supervised learning situation
• (1) Models/Basis* shape are normally
distributed
• (2) confidence weights/tuneVar are
independent.
• (3) there are precision tuners on
basis+weights. Var and alpha
• (4) all are known except weights.

6. Usual vs MAP
• Usual ; over-fitting
• MAP ; solves over-fitting but there is no
interpretation of BR.
______________________________________
• Confidence-Level/Interpretation of the
unknown

7. Complexity Control : Regularization
• E(w)= E_d(w)+lambda*E_w(w)
• E_d is the standard regression
model, liner, sigmoid etc.
• E_w is the penalty
• Lamda is a hyperparameter

8. Error model
-Note there is no x…

9. Log of error model

10. Tune Shape
Confidence on w
+
shape of error contribution

11. Interpretation

12. Lastly
-Correspondence to NN
-How we honestly manage complexity in Extol

13. Questions..
• thanks