SlideShare a Scribd company logo
Dr. Florian Wilhelm
March 13th 2016
PyData Amsterdam
P 1
2
What‘s the best model to
describe our data?
And what does „best“ actually mean?
3
4
5
Simple model
„Generality“
Complex model
„Best Fit“
Occam‘s Razor:
„It is vain to do with more
what can be done with fewer“
6
Simple Model
ℋ1
Complex model
ℋ2
Space of all possible datasets 𝐷
7
Simple Model
ℋ1
Complex model
ℋ2
ℋ1 fits only a small subset of 𝐷 well
8
Simple Model
ℋ1
Complex model
ℋ2
ℋ2 can fit large parts of 𝐷 well
9
Prefer the model with high evidence for a given dataset
Source: D. J. C. MacKay. Bayesian Interpolation. 1992
1. Model fitting: Assume ℋ𝑖 is the right model and fit its parameters 𝒘 with Bayes:
𝑃 𝒘 𝐷, ℋ𝑖 =
𝑃 𝐷 𝒘, ℋ𝑖 𝑃(𝒘|ℋ𝑖)
𝑃(𝐷|ℋ𝑖)
“Business as usual”
2. Model comparison: Compare different models with the help of their evidence
𝑃 𝐷 ℋ𝑖 and model prior 𝑃 ℋ𝑖 :
𝑃 ℋ𝑖 𝐷 ∝ 𝑃 𝐷 ℋ𝑖 𝑃 ℋ𝑖
“Occam‘s razor at work“
10
Marginalize & approximate:
𝑃 𝐷 ℋ𝑖 = 𝑃 𝐷 𝒘, ℋ𝑖 𝑃 𝒘 ℋ𝑖 𝑑𝒘
𝑃 𝐷 ℋ𝑖 ≅ 𝑃 𝐷 𝒘 𝑀𝑃, ℋ𝑖 𝑃 𝒘 𝑀𝑃 ℋ𝑖 ∆𝒘
𝑒𝑣𝑖𝑑𝑒𝑛𝑐𝑒 ≅ 𝑏𝑒𝑠𝑡 𝑓𝑖𝑡 𝑙𝑖𝑘𝑒𝑙𝑖ℎ𝑜𝑜𝑑 × 𝑂𝑐𝑐𝑎𝑚 𝑓𝑎𝑐𝑡𝑜𝑟
11
Occam factor:
∆𝒘
∆ 𝟎 𝒘
Source: D. J. C. MacKay. Bayesian Interpolation. 1992
12
Given:
 Dataset 𝐷 = 𝑥 𝑛, 𝑡 𝑛 with 𝑛 = 1 … 𝑁
 Set of (non-linear) functions Φ = {𝜙ℎ: 𝑥 ⟼ 𝜙(𝑥)} with ℎ = 1 … 𝑀
Assumption:
𝑦 𝒙; 𝒘 =
ℎ=1
𝑀
𝑤ℎ 𝜙ℎ(𝒙) ,
𝑡 𝑛 = 𝑦 𝒙; 𝒘 + 𝜐 𝑛,
where 𝜐 𝑛 is an additive noise with 𝒩 0, 𝛼−1
Task: Find min
𝒘
‖Φ𝒘 − 𝒕‖2
(Ordinary Least Squares)
13
14
Problem:
Having too many features leads to overfitting!
Regularization
Assumption: „Weights are small“
𝑝 𝒘; 𝜆 ~𝒩(0, 𝜆−1 𝕀)
Task: Given 𝛼, 𝜆 find
min
𝒘
𝛼 Φ𝒘 − 𝒕 2 + 𝜆 𝒘 2
15
Consider each 𝛼𝑖, 𝜆𝑖 defining a model ℋ𝑖 𝛼, 𝜆 .
Yes! That means we can use
our Bayesian Interpolation to
find 𝒘, 𝜶, 𝝀 with the highest
evidence!
This is the idea behind BayesianRidge as found in sklearn.linear_model
Consider that each weight has an individual variance, so that
𝑝 𝒘 𝝀 ~𝒩 0, Λ−1 ,
where Λ = diag(𝜆1, … , 𝜆 𝐻), 𝜆ℎ ∈ ℝ+.
Now, our minimization problem is:
min
𝒘
𝛼 Φ𝒘 − 𝒕 2 + 𝒘 𝑡Λ𝒘
16
Pruning: If precision 𝜆ℎ of feature ℎ is high, its weight 𝑤ℎ is very likely to
be close to zero and is therefore pruned.
This is called Sparse Bayesian Learning or Automatic Relevance
Determination. Found as ARDRegression under sklearn.linear_model.
Crossvalidation can be used for the estimation of hyperparmeters but suffers from
the curse of dimensionality (inappropriate for low-statistics).
17
Source: Peter Ellerton, http://pactiss.org/2011/11/02/bayesian-inference-homo-bayesianis/
• Random 100 × 100 design matrix Φ with 100 samples and 100
features
• Weights 𝑤𝑖, 𝑖 ∈ 𝐼 = 1, … , 100 , random subset J ⊂ 𝐼 with 𝐽 = 10, and
𝑤𝑖 =
0, 𝑖 ∈ 𝐼J
𝒩(𝑤𝑖; 0, 1
4), 𝑖 ∈ 𝐽
• Target 𝒕 = Φ𝒘 + 𝝂 with random noise 𝜈𝑖 ∼ 𝒩(0, 1
50)
Task: Reconstruct the weights, especially the 10 non-zero weights!
Source: http://scikit-learn.org/stable/auto_examples/linear_model/plot_ard.html#example-linear-model-plot-ard-py
18
19
20
21
22
We have to determine the parameters 𝑤, 𝜆, 𝛼 for
𝑃 𝒘, 𝝀, 𝛼 𝒕 = 𝑃 𝒘 𝒕, 𝝀, 𝛼 𝑃 𝝀, 𝛼 𝒕
1) Model fitting:
For the first factor, we have 𝑃 𝒘 𝒕, 𝝀, 𝛼 ~𝒩(𝝁, Σ) with
Σ = Λ + 𝛼Φ 𝑇
Φ −1
,
𝝁 = 𝛼ΣΦT 𝐭.
23
2) Model comparison:
For the second factor, we have
𝑃 𝝀, 𝛼 𝒕 ∝ 𝑃 𝒕 𝝀, 𝛼 𝑃 𝝀 𝑃 𝛼 ,
where 𝑃 𝝀 and 𝑃(𝛼) are hyperpriors which we assume uniform.
Using marginalization, we have
𝑃 𝒕 𝝀, 𝛼 = 𝑃 𝒕 𝒘, 𝛼 𝑃 𝒘 𝝀 𝑑𝒘,
i.e. marginal likelihood or the “evidence for the hyperparameter“.
24
Differentiation of the log marginal likelihood with respect to 𝜆𝑖 and 𝛼 as
well as setting these to zero, we get
𝜆𝑖 =
𝛾𝑖
𝜇𝑖
2 ,
𝛼 =
𝑁 − 𝑖 𝛾𝑖
𝒕 − Φ𝝁 2
,
with 𝛾𝑖 = 1 − 𝜆𝑖Σ𝑖𝑖.
These formulae are used to find the maximum points 𝝀 𝑀𝑃 and 𝛼 𝑀𝑃.
25
1. Starting values 𝛼 = 𝜎−2(𝒕), 𝝀 = 𝟏
2. Calculate Σ = Λ + 𝛼Φ 𝑇Φ −1 and 𝒘 = 𝝁 = 𝛼ΣΦT 𝐭
3. Update 𝜆𝑖 =
𝛾 𝑖
𝜇 𝑖
2 and 𝛼 =
𝑁− 𝑖 𝛾 𝑖
𝒕−Φ𝝁 2 where 𝛾𝑖 = 1 − 𝜆𝑖Σ𝑖𝑖
4. Prune 𝜆𝑖 and 𝜙𝑖 if 𝜆𝑖 > 𝜆 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑
5. If not converged go to 2.
Sklearn implementation:
The parameters 𝛼1, 𝛼2 as well as 𝜆1, 𝜆2 are the hyperprior parameters
for 𝛼 and 𝝀 with
𝑃 𝛼 ∼ Γ 𝛼1, 𝛼2
−1
, 𝑃 𝜆𝑖 ∼ Γ 𝜆1, 𝜆2
−1
.
𝐸 Γ 𝛼, 𝛽 =
𝛼
𝛽
and 𝑉 Γ 𝛼, 𝛽 =
𝛼
𝛽2.
26
Given a some new data 𝑥∗, a prediction for 𝑡∗ is made by
𝑃 𝑡∗ 𝒕, 𝝀 𝑀𝑃, 𝛼 𝑀𝑃 = 𝑃 𝑡∗ 𝒘, 𝛼 𝑀𝑃 𝑃 𝒘 𝒕, 𝝀 𝑀𝑃, 𝛼 𝑀𝑃 𝑑𝒘
= 𝒩 𝝁 𝑇 𝜙 𝑥∗ , 𝛼 𝑀𝑃
−1
+ 𝜙 𝑥∗
𝑡Σ𝜙 𝑥∗ .
This is a good approximation of the predictive distribution
𝑃 𝑡∗ 𝒕 = 𝑃 𝑡∗ 𝒘, 𝝀, 𝛼 𝑃 𝒘, 𝝀, 𝛼 𝒕 𝑑𝒘 𝑑𝝀 𝑑α .
27
1. D. J. C. MacKay. Bayesian Interpolation. 1992
(… to understand the overall idea)
2. M. E.Tipping. Sparse Bayesian learning and the RelevanceVector
Machine. June, 2001
(… to understand the ARD algorithm)
3. T. Fletcher. RelevanceVector Machines Explained. October, 2010
(… to understand the ARD algorithm in detail)
4. D.Wipf. A NewView of Automatic Relevance Determination. 2008
(… not as good as the ones above)
Graphs from slides 7 and 9 were taken from [1] and the awesome
tutorials of Scikit-Learn were consulted many times.
28
29

More Related Content

What's hot

はじパタ6章前半
はじパタ6章前半はじパタ6章前半
はじパタ6章前半
T T
 
Visual Explanation of Ridge Regression and LASSO
Visual Explanation of Ridge Regression and LASSOVisual Explanation of Ridge Regression and LASSO
Visual Explanation of Ridge Regression and LASSO
Kazuki Yoshida
 
[DL輪読会]Deep Learning 第16章 深層学習のための構造化確率モデル
[DL輪読会]Deep Learning 第16章 深層学習のための構造化確率モデル[DL輪読会]Deep Learning 第16章 深層学習のための構造化確率モデル
[DL輪読会]Deep Learning 第16章 深層学習のための構造化確率モデル
Deep Learning JP
 
2013.12.26 prml勉強会 線形回帰モデル3.2~3.4
2013.12.26 prml勉強会 線形回帰モデル3.2~3.42013.12.26 prml勉強会 線形回帰モデル3.2~3.4
2013.12.26 prml勉強会 線形回帰モデル3.2~3.4
Takeshi Sakaki
 
VBAで数値計算 10 逆行列と疑似逆行列
VBAで数値計算 10 逆行列と疑似逆行列VBAで数値計算 10 逆行列と疑似逆行列
VBAで数値計算 10 逆行列と疑似逆行列
Katsuhiro Morishita
 
パターン認識 第12章 正則化とパス追跡アルゴリズム
パターン認識 第12章 正則化とパス追跡アルゴリズムパターン認識 第12章 正則化とパス追跡アルゴリズム
パターン認識 第12章 正則化とパス追跡アルゴリズム
Miyoshi Yuya
 
PRML 第4章
PRML 第4章PRML 第4章
PRML 第4章
Akira Miyazawa
 
zkStudyClub: HyperPlonk (Binyi Chen, Benedikt Bünz)
zkStudyClub: HyperPlonk (Binyi Chen, Benedikt Bünz)zkStudyClub: HyperPlonk (Binyi Chen, Benedikt Bünz)
zkStudyClub: HyperPlonk (Binyi Chen, Benedikt Bünz)
Alex Pruden
 
線形代数の視覚的理解のためのノート
線形代数の視覚的理解のためのノート線形代数の視覚的理解のためのノート
線形代数の視覚的理解のためのノート
Kenji Hiranabe
 
PRML第9章「混合モデルとEM」
PRML第9章「混合モデルとEM」PRML第9章「混合モデルとEM」
PRML第9章「混合モデルとEM」
Keisuke Sugawara
 
[DL輪読会]Deep Learning 第11章 実用的な方法論
[DL輪読会]Deep Learning 第11章 実用的な方法論[DL輪読会]Deep Learning 第11章 実用的な方法論
[DL輪読会]Deep Learning 第11章 実用的な方法論
Deep Learning JP
 
StanとRでベイズ統計モデリング 11章 離散値をとるパラメータ
StanとRでベイズ統計モデリング 11章 離散値をとるパラメータStanとRでベイズ統計モデリング 11章 離散値をとるパラメータ
StanとRでベイズ統計モデリング 11章 離散値をとるパラメータ
Miki Katsuragi
 
Approximation algorithms
Approximation  algorithms Approximation  algorithms
Approximation algorithms
Bipesh Raj Subedi
 
PRML輪読#3
PRML輪読#3PRML輪読#3
PRML輪読#3
matsuolab
 
グラフィカル Lasso を用いた異常検知
グラフィカル Lasso を用いた異常検知グラフィカル Lasso を用いた異常検知
グラフィカル Lasso を用いた異常検知
Yuya Takashina
 
データ解析14 ナイーブベイズ
データ解析14 ナイーブベイズデータ解析14 ナイーブベイズ
データ解析14 ナイーブベイズ
Hirotaka Hachiya
 
パターン認識と機械学習 13章 系列データ
パターン認識と機械学習 13章 系列データパターン認識と機械学習 13章 系列データ
パターン認識と機械学習 13章 系列データ
emonosuke
 
Prml3.5 エビデンス近似〜
Prml3.5 エビデンス近似〜Prml3.5 エビデンス近似〜
Prml3.5 エビデンス近似〜
Yuki Matsubara
 
SMO徹底入門 - SVMをちゃんと実装する
SMO徹底入門 - SVMをちゃんと実装するSMO徹底入門 - SVMをちゃんと実装する
SMO徹底入門 - SVMをちゃんと実装する
sleepy_yoshi
 

What's hot (20)

はじパタ6章前半
はじパタ6章前半はじパタ6章前半
はじパタ6章前半
 
Visual Explanation of Ridge Regression and LASSO
Visual Explanation of Ridge Regression and LASSOVisual Explanation of Ridge Regression and LASSO
Visual Explanation of Ridge Regression and LASSO
 
[DL輪読会]Deep Learning 第16章 深層学習のための構造化確率モデル
[DL輪読会]Deep Learning 第16章 深層学習のための構造化確率モデル[DL輪読会]Deep Learning 第16章 深層学習のための構造化確率モデル
[DL輪読会]Deep Learning 第16章 深層学習のための構造化確率モデル
 
2013.12.26 prml勉強会 線形回帰モデル3.2~3.4
2013.12.26 prml勉強会 線形回帰モデル3.2~3.42013.12.26 prml勉強会 線形回帰モデル3.2~3.4
2013.12.26 prml勉強会 線形回帰モデル3.2~3.4
 
VBAで数値計算 10 逆行列と疑似逆行列
VBAで数値計算 10 逆行列と疑似逆行列VBAで数値計算 10 逆行列と疑似逆行列
VBAで数値計算 10 逆行列と疑似逆行列
 
パターン認識 第12章 正則化とパス追跡アルゴリズム
パターン認識 第12章 正則化とパス追跡アルゴリズムパターン認識 第12章 正則化とパス追跡アルゴリズム
パターン認識 第12章 正則化とパス追跡アルゴリズム
 
PRML 第4章
PRML 第4章PRML 第4章
PRML 第4章
 
zkStudyClub: HyperPlonk (Binyi Chen, Benedikt Bünz)
zkStudyClub: HyperPlonk (Binyi Chen, Benedikt Bünz)zkStudyClub: HyperPlonk (Binyi Chen, Benedikt Bünz)
zkStudyClub: HyperPlonk (Binyi Chen, Benedikt Bünz)
 
線形代数の視覚的理解のためのノート
線形代数の視覚的理解のためのノート線形代数の視覚的理解のためのノート
線形代数の視覚的理解のためのノート
 
PRML第9章「混合モデルとEM」
PRML第9章「混合モデルとEM」PRML第9章「混合モデルとEM」
PRML第9章「混合モデルとEM」
 
[DL輪読会]Deep Learning 第11章 実用的な方法論
[DL輪読会]Deep Learning 第11章 実用的な方法論[DL輪読会]Deep Learning 第11章 実用的な方法論
[DL輪読会]Deep Learning 第11章 実用的な方法論
 
StanとRでベイズ統計モデリング 11章 離散値をとるパラメータ
StanとRでベイズ統計モデリング 11章 離散値をとるパラメータStanとRでベイズ統計モデリング 11章 離散値をとるパラメータ
StanとRでベイズ統計モデリング 11章 離散値をとるパラメータ
 
Approximation algorithms
Approximation  algorithms Approximation  algorithms
Approximation algorithms
 
PRML輪読#3
PRML輪読#3PRML輪読#3
PRML輪読#3
 
グラフィカル Lasso を用いた異常検知
グラフィカル Lasso を用いた異常検知グラフィカル Lasso を用いた異常検知
グラフィカル Lasso を用いた異常検知
 
データ解析14 ナイーブベイズ
データ解析14 ナイーブベイズデータ解析14 ナイーブベイズ
データ解析14 ナイーブベイズ
 
パターン認識と機械学習 13章 系列データ
パターン認識と機械学習 13章 系列データパターン認識と機械学習 13章 系列データ
パターン認識と機械学習 13章 系列データ
 
Prml3.5 エビデンス近似〜
Prml3.5 エビデンス近似〜Prml3.5 エビデンス近似〜
Prml3.5 エビデンス近似〜
 
Prml 4.1.1
Prml 4.1.1Prml 4.1.1
Prml 4.1.1
 
SMO徹底入門 - SVMをちゃんと実装する
SMO徹底入門 - SVMをちゃんと実装するSMO徹底入門 - SVMをちゃんと実装する
SMO徹底入門 - SVMをちゃんと実装する
 

Viewers also liked

t4_20110728_IGARSS11_tsutomuy.pdf
t4_20110728_IGARSS11_tsutomuy.pdft4_20110728_IGARSS11_tsutomuy.pdf
t4_20110728_IGARSS11_tsutomuy.pdf
grssieee
 
Prigogine, i. tan solo una_ilusion
Prigogine, i. tan solo una_ilusionPrigogine, i. tan solo una_ilusion
Prigogine, i. tan solo una_ilusion
Joan M Ferrer Díaz
 
Inference for stochastic differential equations via approximate Bayesian comp...
Inference for stochastic differential equations via approximate Bayesian comp...Inference for stochastic differential equations via approximate Bayesian comp...
Inference for stochastic differential equations via approximate Bayesian comp...
Umberto Picchini
 
Hirsch s.w., smale s. differential equations, dynamical systems and linear ...
Hirsch s.w., smale s.   differential equations, dynamical systems and linear ...Hirsch s.w., smale s.   differential equations, dynamical systems and linear ...
Hirsch s.w., smale s. differential equations, dynamical systems and linear ...
yuejia2001
 
Francisco
FranciscoFrancisco
Francisco
Janet Ramírez
 
Libro de Mancil
Libro de MancilLibro de Mancil
Libro de Mancil
Lenore Corella
 
Un Juego Diferencial Estocástico para Reaseguro
Un Juego Diferencial Estocástico para ReaseguroUn Juego Diferencial Estocástico para Reaseguro
Un Juego Diferencial Estocástico para Reaseguro
David Solis
 
Prigogine esayo
Prigogine esayoPrigogine esayo
Prigogine esayo
Julian Gomez Valdez
 
Calculo diferencial e integral2
Calculo diferencial e integral2Calculo diferencial e integral2
Calculo diferencial e integral2
Alfredo Loayza Guzmán
 
MODELACIÓN MATEMÁTICA A TRAVÉS DE LAS ECUACIONES EN DIFERENCIA
MODELACIÓN MATEMÁTICA A TRAVÉS DE LAS ECUACIONES EN DIFERENCIAMODELACIÓN MATEMÁTICA A TRAVÉS DE LAS ECUACIONES EN DIFERENCIA
MODELACIÓN MATEMÁTICA A TRAVÉS DE LAS ECUACIONES EN DIFERENCIA
Nubia Mejía
 
Financial Markets with Stochastic Volatilities - markov modelling
Financial Markets with Stochastic Volatilities - markov modellingFinancial Markets with Stochastic Volatilities - markov modelling
Financial Markets with Stochastic Volatilities - markov modelling
guest8901f4
 
Calculo diferencial e_integral_en_la_vida_cotidiana (2)
Calculo diferencial e_integral_en_la_vida_cotidiana (2)Calculo diferencial e_integral_en_la_vida_cotidiana (2)
Calculo diferencial e_integral_en_la_vida_cotidiana (2)
Hugo Rosales Vera
 
Calculo diferencial e integral
Calculo diferencial e integralCalculo diferencial e integral
Calculo diferencial e integral
ronalrepi1
 
Calculo integral
Calculo integralCalculo integral
Calculo integral
karlaaflores
 
Solucionario de matematicas de g. mancill.
Solucionario de matematicas de g. mancill.Solucionario de matematicas de g. mancill.
Solucionario de matematicas de g. mancill.
C 12
 
Fractales y Teoría del Caos
Fractales y Teoría del CaosFractales y Teoría del Caos
Fractales y Teoría del Caos
Jimmy Campo
 
Ejercicios algebra superior hall y knight
Ejercicios algebra superior hall y knightEjercicios algebra superior hall y knight
Ejercicios algebra superior hall y knight
guido guzman perez
 
Algebra Elemental Moderna
Algebra Elemental ModernaAlgebra Elemental Moderna
Algebra Elemental Moderna
Ricardo Avila Legra
 
Algebra proschle
Algebra  proschleAlgebra  proschle
Algebra proschle
3DSalesianos
 
Algebra arrayan
Algebra arrayanAlgebra arrayan
Algebra arrayan
gomezaa
 

Viewers also liked (20)

t4_20110728_IGARSS11_tsutomuy.pdf
t4_20110728_IGARSS11_tsutomuy.pdft4_20110728_IGARSS11_tsutomuy.pdf
t4_20110728_IGARSS11_tsutomuy.pdf
 
Prigogine, i. tan solo una_ilusion
Prigogine, i. tan solo una_ilusionPrigogine, i. tan solo una_ilusion
Prigogine, i. tan solo una_ilusion
 
Inference for stochastic differential equations via approximate Bayesian comp...
Inference for stochastic differential equations via approximate Bayesian comp...Inference for stochastic differential equations via approximate Bayesian comp...
Inference for stochastic differential equations via approximate Bayesian comp...
 
Hirsch s.w., smale s. differential equations, dynamical systems and linear ...
Hirsch s.w., smale s.   differential equations, dynamical systems and linear ...Hirsch s.w., smale s.   differential equations, dynamical systems and linear ...
Hirsch s.w., smale s. differential equations, dynamical systems and linear ...
 
Francisco
FranciscoFrancisco
Francisco
 
Libro de Mancil
Libro de MancilLibro de Mancil
Libro de Mancil
 
Un Juego Diferencial Estocástico para Reaseguro
Un Juego Diferencial Estocástico para ReaseguroUn Juego Diferencial Estocástico para Reaseguro
Un Juego Diferencial Estocástico para Reaseguro
 
Prigogine esayo
Prigogine esayoPrigogine esayo
Prigogine esayo
 
Calculo diferencial e integral2
Calculo diferencial e integral2Calculo diferencial e integral2
Calculo diferencial e integral2
 
MODELACIÓN MATEMÁTICA A TRAVÉS DE LAS ECUACIONES EN DIFERENCIA
MODELACIÓN MATEMÁTICA A TRAVÉS DE LAS ECUACIONES EN DIFERENCIAMODELACIÓN MATEMÁTICA A TRAVÉS DE LAS ECUACIONES EN DIFERENCIA
MODELACIÓN MATEMÁTICA A TRAVÉS DE LAS ECUACIONES EN DIFERENCIA
 
Financial Markets with Stochastic Volatilities - markov modelling
Financial Markets with Stochastic Volatilities - markov modellingFinancial Markets with Stochastic Volatilities - markov modelling
Financial Markets with Stochastic Volatilities - markov modelling
 
Calculo diferencial e_integral_en_la_vida_cotidiana (2)
Calculo diferencial e_integral_en_la_vida_cotidiana (2)Calculo diferencial e_integral_en_la_vida_cotidiana (2)
Calculo diferencial e_integral_en_la_vida_cotidiana (2)
 
Calculo diferencial e integral
Calculo diferencial e integralCalculo diferencial e integral
Calculo diferencial e integral
 
Calculo integral
Calculo integralCalculo integral
Calculo integral
 
Solucionario de matematicas de g. mancill.
Solucionario de matematicas de g. mancill.Solucionario de matematicas de g. mancill.
Solucionario de matematicas de g. mancill.
 
Fractales y Teoría del Caos
Fractales y Teoría del CaosFractales y Teoría del Caos
Fractales y Teoría del Caos
 
Ejercicios algebra superior hall y knight
Ejercicios algebra superior hall y knightEjercicios algebra superior hall y knight
Ejercicios algebra superior hall y knight
 
Algebra Elemental Moderna
Algebra Elemental ModernaAlgebra Elemental Moderna
Algebra Elemental Moderna
 
Algebra proschle
Algebra  proschleAlgebra  proschle
Algebra proschle
 
Algebra arrayan
Algebra arrayanAlgebra arrayan
Algebra arrayan
 

Similar to Explaining the idea behind automatic relevance determination and bayesian interpolation

Koh_Liang_ICML2017
Koh_Liang_ICML2017Koh_Liang_ICML2017
Koh_Liang_ICML2017
Masa Kato
 
Robot, Learning From Data
Robot, Learning From DataRobot, Learning From Data
Robot, Learning From Data
Sungjoon Choi
 
Generalised Statistical Convergence For Double Sequences
Generalised Statistical Convergence For Double SequencesGeneralised Statistical Convergence For Double Sequences
Generalised Statistical Convergence For Double Sequences
IOSR Journals
 
Boundness of a neural network weights using the notion of a limit of a sequence
Boundness of a neural network weights using the notion of a limit of a sequenceBoundness of a neural network weights using the notion of a limit of a sequence
Boundness of a neural network weights using the notion of a limit of a sequence
IJDKP
 
STLtalk about statistical analysis and its application
STLtalk about statistical analysis and its applicationSTLtalk about statistical analysis and its application
STLtalk about statistical analysis and its application
JulieDash5
 
MM - KBAC: Using mixed models to adjust for population structure in a rare-va...
MM - KBAC: Using mixed models to adjust for population structure in a rare-va...MM - KBAC: Using mixed models to adjust for population structure in a rare-va...
MM - KBAC: Using mixed models to adjust for population structure in a rare-va...
Golden Helix Inc
 
Machine Learning 1
Machine Learning 1Machine Learning 1
Machine Learning 1
cairo university
 
Mncs 16-09-4주-변승규-introduction to the machine learning
Mncs 16-09-4주-변승규-introduction to the machine learningMncs 16-09-4주-변승규-introduction to the machine learning
Mncs 16-09-4주-변승규-introduction to the machine learning
Seung-gyu Byeon
 
Symbolic Computation via Gröbner Basis
Symbolic Computation via Gröbner BasisSymbolic Computation via Gröbner Basis
Symbolic Computation via Gröbner Basis
IJERA Editor
 
Intro to Quant Trading Strategies (Lecture 2 of 10)
Intro to Quant Trading Strategies (Lecture 2 of 10)Intro to Quant Trading Strategies (Lecture 2 of 10)
Intro to Quant Trading Strategies (Lecture 2 of 10)
Adrian Aley
 
A machine learning method for efficient design optimization in nano-optics
A machine learning method for efficient design optimization in nano-optics A machine learning method for efficient design optimization in nano-optics
A machine learning method for efficient design optimization in nano-optics
JCMwave
 
Anti-differentiating Approximation Algorithms: PageRank and MinCut
Anti-differentiating Approximation Algorithms: PageRank and MinCutAnti-differentiating Approximation Algorithms: PageRank and MinCut
Anti-differentiating Approximation Algorithms: PageRank and MinCut
David Gleich
 
Machine learning ppt and presentation code
Machine learning ppt and presentation codeMachine learning ppt and presentation code
Machine learning ppt and presentation code
sharma239172
 
Learning group em - 20171025 - copy
Learning group   em - 20171025 - copyLearning group   em - 20171025 - copy
Learning group em - 20171025 - copy
Shuai Zhang
 
07 Machine Learning - Expectation Maximization
07 Machine Learning - Expectation Maximization07 Machine Learning - Expectation Maximization
07 Machine Learning - Expectation Maximization
Andres Mendez-Vazquez
 
Ck31369376
Ck31369376Ck31369376
Ck31369376
IJMER
 
Kernel Bayes Rule
Kernel Bayes RuleKernel Bayes Rule
Kernel Bayes Rule
Yan Xu
 
PRML Chapter 6
PRML Chapter 6PRML Chapter 6
PRML Chapter 6
Sunwoo Kim
 
A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR...
A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR...A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR...
A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR...
ijscmcj
 
PRML Chapter 8
PRML Chapter 8PRML Chapter 8
PRML Chapter 8
Sunwoo Kim
 

Similar to Explaining the idea behind automatic relevance determination and bayesian interpolation (20)

Koh_Liang_ICML2017
Koh_Liang_ICML2017Koh_Liang_ICML2017
Koh_Liang_ICML2017
 
Robot, Learning From Data
Robot, Learning From DataRobot, Learning From Data
Robot, Learning From Data
 
Generalised Statistical Convergence For Double Sequences
Generalised Statistical Convergence For Double SequencesGeneralised Statistical Convergence For Double Sequences
Generalised Statistical Convergence For Double Sequences
 
Boundness of a neural network weights using the notion of a limit of a sequence
Boundness of a neural network weights using the notion of a limit of a sequenceBoundness of a neural network weights using the notion of a limit of a sequence
Boundness of a neural network weights using the notion of a limit of a sequence
 
STLtalk about statistical analysis and its application
STLtalk about statistical analysis and its applicationSTLtalk about statistical analysis and its application
STLtalk about statistical analysis and its application
 
MM - KBAC: Using mixed models to adjust for population structure in a rare-va...
MM - KBAC: Using mixed models to adjust for population structure in a rare-va...MM - KBAC: Using mixed models to adjust for population structure in a rare-va...
MM - KBAC: Using mixed models to adjust for population structure in a rare-va...
 
Machine Learning 1
Machine Learning 1Machine Learning 1
Machine Learning 1
 
Mncs 16-09-4주-변승규-introduction to the machine learning
Mncs 16-09-4주-변승규-introduction to the machine learningMncs 16-09-4주-변승규-introduction to the machine learning
Mncs 16-09-4주-변승규-introduction to the machine learning
 
Symbolic Computation via Gröbner Basis
Symbolic Computation via Gröbner BasisSymbolic Computation via Gröbner Basis
Symbolic Computation via Gröbner Basis
 
Intro to Quant Trading Strategies (Lecture 2 of 10)
Intro to Quant Trading Strategies (Lecture 2 of 10)Intro to Quant Trading Strategies (Lecture 2 of 10)
Intro to Quant Trading Strategies (Lecture 2 of 10)
 
A machine learning method for efficient design optimization in nano-optics
A machine learning method for efficient design optimization in nano-optics A machine learning method for efficient design optimization in nano-optics
A machine learning method for efficient design optimization in nano-optics
 
Anti-differentiating Approximation Algorithms: PageRank and MinCut
Anti-differentiating Approximation Algorithms: PageRank and MinCutAnti-differentiating Approximation Algorithms: PageRank and MinCut
Anti-differentiating Approximation Algorithms: PageRank and MinCut
 
Machine learning ppt and presentation code
Machine learning ppt and presentation codeMachine learning ppt and presentation code
Machine learning ppt and presentation code
 
Learning group em - 20171025 - copy
Learning group   em - 20171025 - copyLearning group   em - 20171025 - copy
Learning group em - 20171025 - copy
 
07 Machine Learning - Expectation Maximization
07 Machine Learning - Expectation Maximization07 Machine Learning - Expectation Maximization
07 Machine Learning - Expectation Maximization
 
Ck31369376
Ck31369376Ck31369376
Ck31369376
 
Kernel Bayes Rule
Kernel Bayes RuleKernel Bayes Rule
Kernel Bayes Rule
 
PRML Chapter 6
PRML Chapter 6PRML Chapter 6
PRML Chapter 6
 
A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR...
A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR...A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR...
A PROBABILISTIC ALGORITHM OF COMPUTING THE POLYNOMIAL GREATEST COMMON DIVISOR...
 
PRML Chapter 8
PRML Chapter 8PRML Chapter 8
PRML Chapter 8
 

More from Florian Wilhelm

Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
Florian Wilhelm
 
Unlocking the Power of Integer Programming
Unlocking the Power of Integer ProgrammingUnlocking the Power of Integer Programming
Unlocking the Power of Integer Programming
Florian Wilhelm
 
WALD: A Modern & Sustainable Analytics Stack
WALD: A Modern & Sustainable Analytics StackWALD: A Modern & Sustainable Analytics Stack
WALD: A Modern & Sustainable Analytics Stack
Florian Wilhelm
 
Forget about AI and do Mathematical Modelling instead!
Forget about AI and do Mathematical Modelling instead!Forget about AI and do Mathematical Modelling instead!
Forget about AI and do Mathematical Modelling instead!
Florian Wilhelm
 
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
Florian Wilhelm
 
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
Florian Wilhelm
 
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Florian Wilhelm
 
Uncertainty Quantification in AI
Uncertainty Quantification in AIUncertainty Quantification in AI
Uncertainty Quantification in AI
Florian Wilhelm
 
Performance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use casePerformance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use case
Florian Wilhelm
 
Bridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to ProductionBridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to Production
Florian Wilhelm
 
How mobile.de brings Data Science to Production for a Personalized Web Experi...
How mobile.de brings Data Science to Production for a Personalized Web Experi...How mobile.de brings Data Science to Production for a Personalized Web Experi...
How mobile.de brings Data Science to Production for a Personalized Web Experi...
Florian Wilhelm
 
Deep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
Deep Learning-based Recommendations for Germany's Biggest Vehicle MarketplaceDeep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
Deep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
Florian Wilhelm
 
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Florian Wilhelm
 
Declarative Thinking and Programming
Declarative Thinking and ProgrammingDeclarative Thinking and Programming
Declarative Thinking and Programming
Florian Wilhelm
 
Which car fits my life? - PyData Berlin 2017
Which car fits my life? - PyData Berlin 2017Which car fits my life? - PyData Berlin 2017
Which car fits my life? - PyData Berlin 2017
Florian Wilhelm
 
PyData Meetup Berlin 2017-04-19
PyData Meetup Berlin 2017-04-19PyData Meetup Berlin 2017-04-19
PyData Meetup Berlin 2017-04-19
Florian Wilhelm
 

More from Florian Wilhelm (16)

Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Unlocking the Power of Integer Programming
Unlocking the Power of Integer ProgrammingUnlocking the Power of Integer Programming
Unlocking the Power of Integer Programming
 
WALD: A Modern & Sustainable Analytics Stack
WALD: A Modern & Sustainable Analytics StackWALD: A Modern & Sustainable Analytics Stack
WALD: A Modern & Sustainable Analytics Stack
 
Forget about AI and do Mathematical Modelling instead!
Forget about AI and do Mathematical Modelling instead!Forget about AI and do Mathematical Modelling instead!
Forget about AI and do Mathematical Modelling instead!
 
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
 
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
 
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
 
Uncertainty Quantification in AI
Uncertainty Quantification in AIUncertainty Quantification in AI
Uncertainty Quantification in AI
 
Performance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use casePerformance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use case
 
Bridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to ProductionBridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to Production
 
How mobile.de brings Data Science to Production for a Personalized Web Experi...
How mobile.de brings Data Science to Production for a Personalized Web Experi...How mobile.de brings Data Science to Production for a Personalized Web Experi...
How mobile.de brings Data Science to Production for a Personalized Web Experi...
 
Deep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
Deep Learning-based Recommendations for Germany's Biggest Vehicle MarketplaceDeep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
Deep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
 
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
 
Declarative Thinking and Programming
Declarative Thinking and ProgrammingDeclarative Thinking and Programming
Declarative Thinking and Programming
 
Which car fits my life? - PyData Berlin 2017
Which car fits my life? - PyData Berlin 2017Which car fits my life? - PyData Berlin 2017
Which car fits my life? - PyData Berlin 2017
 
PyData Meetup Berlin 2017-04-19
PyData Meetup Berlin 2017-04-19PyData Meetup Berlin 2017-04-19
PyData Meetup Berlin 2017-04-19
 

Recently uploaded

Willis Tower //Sears Tower- Supertall Building .pdf
Willis Tower //Sears Tower- Supertall Building .pdfWillis Tower //Sears Tower- Supertall Building .pdf
Willis Tower //Sears Tower- Supertall Building .pdf
LINAT
 
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy DsouzaOpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata
 
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
saadkhan1485265
 
the potential of the development of the Ford–Fulkerson algorithm to solve the...
the potential of the development of the Ford–Fulkerson algorithm to solve the...the potential of the development of the Ford–Fulkerson algorithm to solve the...
the potential of the development of the Ford–Fulkerson algorithm to solve the...
huseindihon
 
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
tanupasswan6
 
🚂🚘 Premium Girls Call Guwahati 🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
🚂🚘 Premium Girls Call Guwahati  🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...🚂🚘 Premium Girls Call Guwahati  🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
🚂🚘 Premium Girls Call Guwahati 🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
kuldeepsharmaks8120
 
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
birajmohan012
 
Biometric Question Bank 2021 - 1 Soln-1.pdf
Biometric Question Bank 2021 - 1 Soln-1.pdfBiometric Question Bank 2021 - 1 Soln-1.pdf
Biometric Question Bank 2021 - 1 Soln-1.pdf
Joel Ngushwai
 
transgenders community data in india by govt
transgenders community data in india by govttransgenders community data in india by govt
transgenders community data in india by govt
palanisamyiiiier
 
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
45unexpected
 
Potential Uses of the Floyd-Warshall Algorithm as appropriate
Potential Uses of the Floyd-Warshall Algorithm as appropriatePotential Uses of the Floyd-Warshall Algorithm as appropriate
Potential Uses of the Floyd-Warshall Algorithm as appropriate
huseindihon
 
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
fatima shekh$A17
 
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
gargnatasha985
 
Busty Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And...
Busty Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And...Busty Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And...
Busty Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And...
tanupasswan6
 
the unexpected potential of Dijkstra's Algorithm
the unexpected potential of Dijkstra's Algorithmthe unexpected potential of Dijkstra's Algorithm
the unexpected potential of Dijkstra's Algorithm
huseindihon
 
DataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptxDataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptx
Kanchana Weerasinghe
 
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
dizzycaye
 
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
kinni singh$A17
 
Data Preprocessing Cheatsheet for learners
Data Preprocessing Cheatsheet for learnersData Preprocessing Cheatsheet for learners
Data Preprocessing Cheatsheet for learners
mohamed Ibrahim
 
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdfWhy_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Alexander Teggin
 

Recently uploaded (20)

Willis Tower //Sears Tower- Supertall Building .pdf
Willis Tower //Sears Tower- Supertall Building .pdfWillis Tower //Sears Tower- Supertall Building .pdf
Willis Tower //Sears Tower- Supertall Building .pdf
 
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy DsouzaOpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
OpenMetadata Spotlight - OpenMetadata @ Aspire by Vinol Joy Dsouza
 
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
High Girls Call Nagpur 000XX00000 Provide Best And Top Girl Service And No1 i...
 
the potential of the development of the Ford–Fulkerson algorithm to solve the...
the potential of the development of the Ford–Fulkerson algorithm to solve the...the potential of the development of the Ford–Fulkerson algorithm to solve the...
the potential of the development of the Ford–Fulkerson algorithm to solve the...
 
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
 
🚂🚘 Premium Girls Call Guwahati 🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
🚂🚘 Premium Girls Call Guwahati  🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...🚂🚘 Premium Girls Call Guwahati  🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
🚂🚘 Premium Girls Call Guwahati 🛵🚡000XX00000 💃 Choose Best And Top Girl Servi...
 
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
Beautiful Girls Call Pune 000XX00000 Provide Best And Top Girl Service And No...
 
Biometric Question Bank 2021 - 1 Soln-1.pdf
Biometric Question Bank 2021 - 1 Soln-1.pdfBiometric Question Bank 2021 - 1 Soln-1.pdf
Biometric Question Bank 2021 - 1 Soln-1.pdf
 
transgenders community data in india by govt
transgenders community data in india by govttransgenders community data in india by govt
transgenders community data in india by govt
 
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
 
Potential Uses of the Floyd-Warshall Algorithm as appropriate
Potential Uses of the Floyd-Warshall Algorithm as appropriatePotential Uses of the Floyd-Warshall Algorithm as appropriate
Potential Uses of the Floyd-Warshall Algorithm as appropriate
 
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
BDSM Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And ...
 
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Vadodara 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Busty Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And...
Busty Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And...Busty Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And...
Busty Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And...
 
the unexpected potential of Dijkstra's Algorithm
the unexpected potential of Dijkstra's Algorithmthe unexpected potential of Dijkstra's Algorithm
the unexpected potential of Dijkstra's Algorithm
 
DataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptxDataScienceConcept_Kanchana_Weerasinghe.pptx
DataScienceConcept_Kanchana_Weerasinghe.pptx
 
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
Female Service Girls Call Navi Mumbai 9930245274 Provide Best And Top Girl Se...
 
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
New Girls Call Noida 9873940964 Unlimited Short Providing Girls Service Avail...
 
Data Preprocessing Cheatsheet for learners
Data Preprocessing Cheatsheet for learnersData Preprocessing Cheatsheet for learners
Data Preprocessing Cheatsheet for learners
 
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdfWhy_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
Why_are_we_hypnotizing_ourselves-_ATeggin-1.pdf
 

Explaining the idea behind automatic relevance determination and bayesian interpolation

  • 1. Dr. Florian Wilhelm March 13th 2016 PyData Amsterdam P 1
  • 2. 2 What‘s the best model to describe our data? And what does „best“ actually mean?
  • 3. 3
  • 4. 4
  • 5. 5 Simple model „Generality“ Complex model „Best Fit“ Occam‘s Razor: „It is vain to do with more what can be done with fewer“
  • 6. 6 Simple Model ℋ1 Complex model ℋ2 Space of all possible datasets 𝐷
  • 7. 7 Simple Model ℋ1 Complex model ℋ2 ℋ1 fits only a small subset of 𝐷 well
  • 8. 8 Simple Model ℋ1 Complex model ℋ2 ℋ2 can fit large parts of 𝐷 well
  • 9. 9 Prefer the model with high evidence for a given dataset Source: D. J. C. MacKay. Bayesian Interpolation. 1992
  • 10. 1. Model fitting: Assume ℋ𝑖 is the right model and fit its parameters 𝒘 with Bayes: 𝑃 𝒘 𝐷, ℋ𝑖 = 𝑃 𝐷 𝒘, ℋ𝑖 𝑃(𝒘|ℋ𝑖) 𝑃(𝐷|ℋ𝑖) “Business as usual” 2. Model comparison: Compare different models with the help of their evidence 𝑃 𝐷 ℋ𝑖 and model prior 𝑃 ℋ𝑖 : 𝑃 ℋ𝑖 𝐷 ∝ 𝑃 𝐷 ℋ𝑖 𝑃 ℋ𝑖 “Occam‘s razor at work“ 10
  • 11. Marginalize & approximate: 𝑃 𝐷 ℋ𝑖 = 𝑃 𝐷 𝒘, ℋ𝑖 𝑃 𝒘 ℋ𝑖 𝑑𝒘 𝑃 𝐷 ℋ𝑖 ≅ 𝑃 𝐷 𝒘 𝑀𝑃, ℋ𝑖 𝑃 𝒘 𝑀𝑃 ℋ𝑖 ∆𝒘 𝑒𝑣𝑖𝑑𝑒𝑛𝑐𝑒 ≅ 𝑏𝑒𝑠𝑡 𝑓𝑖𝑡 𝑙𝑖𝑘𝑒𝑙𝑖ℎ𝑜𝑜𝑑 × 𝑂𝑐𝑐𝑎𝑚 𝑓𝑎𝑐𝑡𝑜𝑟 11 Occam factor: ∆𝒘 ∆ 𝟎 𝒘 Source: D. J. C. MacKay. Bayesian Interpolation. 1992
  • 12. 12
  • 13. Given:  Dataset 𝐷 = 𝑥 𝑛, 𝑡 𝑛 with 𝑛 = 1 … 𝑁  Set of (non-linear) functions Φ = {𝜙ℎ: 𝑥 ⟼ 𝜙(𝑥)} with ℎ = 1 … 𝑀 Assumption: 𝑦 𝒙; 𝒘 = ℎ=1 𝑀 𝑤ℎ 𝜙ℎ(𝒙) , 𝑡 𝑛 = 𝑦 𝒙; 𝒘 + 𝜐 𝑛, where 𝜐 𝑛 is an additive noise with 𝒩 0, 𝛼−1 Task: Find min 𝒘 ‖Φ𝒘 − 𝒕‖2 (Ordinary Least Squares) 13
  • 14. 14 Problem: Having too many features leads to overfitting! Regularization Assumption: „Weights are small“ 𝑝 𝒘; 𝜆 ~𝒩(0, 𝜆−1 𝕀) Task: Given 𝛼, 𝜆 find min 𝒘 𝛼 Φ𝒘 − 𝒕 2 + 𝜆 𝒘 2
  • 15. 15 Consider each 𝛼𝑖, 𝜆𝑖 defining a model ℋ𝑖 𝛼, 𝜆 . Yes! That means we can use our Bayesian Interpolation to find 𝒘, 𝜶, 𝝀 with the highest evidence! This is the idea behind BayesianRidge as found in sklearn.linear_model
  • 16. Consider that each weight has an individual variance, so that 𝑝 𝒘 𝝀 ~𝒩 0, Λ−1 , where Λ = diag(𝜆1, … , 𝜆 𝐻), 𝜆ℎ ∈ ℝ+. Now, our minimization problem is: min 𝒘 𝛼 Φ𝒘 − 𝒕 2 + 𝒘 𝑡Λ𝒘 16 Pruning: If precision 𝜆ℎ of feature ℎ is high, its weight 𝑤ℎ is very likely to be close to zero and is therefore pruned. This is called Sparse Bayesian Learning or Automatic Relevance Determination. Found as ARDRegression under sklearn.linear_model.
  • 17. Crossvalidation can be used for the estimation of hyperparmeters but suffers from the curse of dimensionality (inappropriate for low-statistics). 17 Source: Peter Ellerton, http://pactiss.org/2011/11/02/bayesian-inference-homo-bayesianis/
  • 18. • Random 100 × 100 design matrix Φ with 100 samples and 100 features • Weights 𝑤𝑖, 𝑖 ∈ 𝐼 = 1, … , 100 , random subset J ⊂ 𝐼 with 𝐽 = 10, and 𝑤𝑖 = 0, 𝑖 ∈ 𝐼J 𝒩(𝑤𝑖; 0, 1 4), 𝑖 ∈ 𝐽 • Target 𝒕 = Φ𝒘 + 𝝂 with random noise 𝜈𝑖 ∼ 𝒩(0, 1 50) Task: Reconstruct the weights, especially the 10 non-zero weights! Source: http://scikit-learn.org/stable/auto_examples/linear_model/plot_ard.html#example-linear-model-plot-ard-py 18
  • 19. 19
  • 20. 20
  • 21. 21
  • 22. 22
  • 23. We have to determine the parameters 𝑤, 𝜆, 𝛼 for 𝑃 𝒘, 𝝀, 𝛼 𝒕 = 𝑃 𝒘 𝒕, 𝝀, 𝛼 𝑃 𝝀, 𝛼 𝒕 1) Model fitting: For the first factor, we have 𝑃 𝒘 𝒕, 𝝀, 𝛼 ~𝒩(𝝁, Σ) with Σ = Λ + 𝛼Φ 𝑇 Φ −1 , 𝝁 = 𝛼ΣΦT 𝐭. 23
  • 24. 2) Model comparison: For the second factor, we have 𝑃 𝝀, 𝛼 𝒕 ∝ 𝑃 𝒕 𝝀, 𝛼 𝑃 𝝀 𝑃 𝛼 , where 𝑃 𝝀 and 𝑃(𝛼) are hyperpriors which we assume uniform. Using marginalization, we have 𝑃 𝒕 𝝀, 𝛼 = 𝑃 𝒕 𝒘, 𝛼 𝑃 𝒘 𝝀 𝑑𝒘, i.e. marginal likelihood or the “evidence for the hyperparameter“. 24
  • 25. Differentiation of the log marginal likelihood with respect to 𝜆𝑖 and 𝛼 as well as setting these to zero, we get 𝜆𝑖 = 𝛾𝑖 𝜇𝑖 2 , 𝛼 = 𝑁 − 𝑖 𝛾𝑖 𝒕 − Φ𝝁 2 , with 𝛾𝑖 = 1 − 𝜆𝑖Σ𝑖𝑖. These formulae are used to find the maximum points 𝝀 𝑀𝑃 and 𝛼 𝑀𝑃. 25
  • 26. 1. Starting values 𝛼 = 𝜎−2(𝒕), 𝝀 = 𝟏 2. Calculate Σ = Λ + 𝛼Φ 𝑇Φ −1 and 𝒘 = 𝝁 = 𝛼ΣΦT 𝐭 3. Update 𝜆𝑖 = 𝛾 𝑖 𝜇 𝑖 2 and 𝛼 = 𝑁− 𝑖 𝛾 𝑖 𝒕−Φ𝝁 2 where 𝛾𝑖 = 1 − 𝜆𝑖Σ𝑖𝑖 4. Prune 𝜆𝑖 and 𝜙𝑖 if 𝜆𝑖 > 𝜆 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 5. If not converged go to 2. Sklearn implementation: The parameters 𝛼1, 𝛼2 as well as 𝜆1, 𝜆2 are the hyperprior parameters for 𝛼 and 𝝀 with 𝑃 𝛼 ∼ Γ 𝛼1, 𝛼2 −1 , 𝑃 𝜆𝑖 ∼ Γ 𝜆1, 𝜆2 −1 . 𝐸 Γ 𝛼, 𝛽 = 𝛼 𝛽 and 𝑉 Γ 𝛼, 𝛽 = 𝛼 𝛽2. 26
  • 27. Given a some new data 𝑥∗, a prediction for 𝑡∗ is made by 𝑃 𝑡∗ 𝒕, 𝝀 𝑀𝑃, 𝛼 𝑀𝑃 = 𝑃 𝑡∗ 𝒘, 𝛼 𝑀𝑃 𝑃 𝒘 𝒕, 𝝀 𝑀𝑃, 𝛼 𝑀𝑃 𝑑𝒘 = 𝒩 𝝁 𝑇 𝜙 𝑥∗ , 𝛼 𝑀𝑃 −1 + 𝜙 𝑥∗ 𝑡Σ𝜙 𝑥∗ . This is a good approximation of the predictive distribution 𝑃 𝑡∗ 𝒕 = 𝑃 𝑡∗ 𝒘, 𝝀, 𝛼 𝑃 𝒘, 𝝀, 𝛼 𝒕 𝑑𝒘 𝑑𝝀 𝑑α . 27
  • 28. 1. D. J. C. MacKay. Bayesian Interpolation. 1992 (… to understand the overall idea) 2. M. E.Tipping. Sparse Bayesian learning and the RelevanceVector Machine. June, 2001 (… to understand the ARD algorithm) 3. T. Fletcher. RelevanceVector Machines Explained. October, 2010 (… to understand the ARD algorithm in detail) 4. D.Wipf. A NewView of Automatic Relevance Determination. 2008 (… not as good as the ones above) Graphs from slides 7 and 9 were taken from [1] and the awesome tutorials of Scikit-Learn were consulted many times. 28
  • 29. 29