TO
Uploaded bytmtm otm
257 views

PRML 条件付き混合モデル 14.5

This document discusses conditional mixture models, including mixtures of linear regression models, mixtures of logistic models, and mixtures of experts models. It provides details on learning the parameters of these models using the EM algorithm. Mixtures of experts models use a gating network to determine which expert network is responsible for different regions of the input space. Hierarchical mixtures of experts extend this idea by incorporating multiple levels of gating networks.

Embed presentation

Download to read offline
PRML 14.5 5560 ota
Conditional Mixture Models p14.5.1 Mixtures of linear regression models p14.5.2 Mixtures of logistic models p14.5.3 Mixtures of experts
Conditional Mixture Models
Decision trees Positive interpretability Negative hard assignment Splits: axis-aligned of the input space. Regression: piecewise-constant predictions discontinuities at the split boundaries. <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
Conditional Mixture Models Positive soft probabilistic splits of the input space Splits : functions of all of the input variables Negative no interpretability <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Fig.14.9 Model A <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Model B Model
Mixture of Experts Model (MoE) <latexit sha1_base64="(null)">(null)</latexit> n a fully probabilistic tree-based model Hierarchical Mixture of Experts (HME) <latexit sha1_base64="(null)">(null)</latexit> Ex)
Mixtures of linear regression models
Gaussian Mixture Models n Clustering (unsupervised Learning) Fig. 9.5 <latexit sha1_base64="(null)">(null)</latexit> Observed data If k = 3 k <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
Mixtures of linear regression models <latexit sha1_base64="(null)">(null)</latexit> n A general of switching regression , 9 3 79 7 97 3 9 8 7 70 7 A7 1 7 3 3 7 )93 . 7 ) 1 ( Linear regression
Learning the parameters EM algorithm <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Parameters Data set <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
E step responsibilities initialized <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> The expectation of the complete-data log likelihood
M step Fixed <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Maximize the function about the constraint 1) <latexit sha1_base64="(null)">(null)</latexit> the mixing coefficients Using a Lagrange multiplier method,
M step 2) <latexit sha1_base64="(null)">(null)</latexit> the parameter of the k-th linear regression model <latexit sha1_base64="(null)">(null)</latexit> Linear regression model (3.12) <latexit sha1_base64="(null)">(null)</latexit> weighted least squares problem If k = 3 k <latexit sha1_base64="(null)">(null)</latexit> n = 1 <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
M step <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> the parameter of the k-th linear regression model <latexit sha1_base64="(null)">(null)</latexit> matrix notation <latexit sha1_base64="(null)">(null)</latexit> n are learned the data of the high responsibility .<latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
M step a precision parameter<latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
Example Fig.14.8 <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
The predictive conditional density <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> No independent of the input
Mixtures of logistic models
Mixtures of logistic models <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> logistic regression Model
Learning the parameters <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> EM algorithm Parameters data set <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> the complete-data log likelihood <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> }
E step <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
M step <latexit sha1_base64="(null)">(null)</latexit> Fixed <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Maximize the function about <latexit sha1_base64="(null)">(null)</latexit> the mixing coefficients }
M step <latexit sha1_base64="(null)">(null)</latexit> the parameter of the k-th logistic regression model does not have a closed-form solution.<latexit sha1_base64="(null)">(null)</latexit> (IRLS) algorithm <latexit sha1_base64="(null)">(null)</latexit> n Need the gradient and Hessian <latexit sha1_base64="(null)">(null)</latexit>
IRLS algorithm <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Gradient Hessian <latexit sha1_base64="(null)">(null)</latexit> While not convergence do <latexit sha1_base64="(null)">(null)</latexit>
A mixture of logistic regression models true probability of the class label single logistic regression mixture of two logistic regression <latexit sha1_base64="(null)">(null)</latexit> Model A Model B
Mixtures of Experts
Mixtures of experts <latexit sha1_base64="(null)">(null)</latexit> Gating function: determine which expert are dominant in which region. be represented by a linear softmax or sigmoid. n a mixture of experts model Expert: can model in different regions of input space. predict in their own region. <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
Gating function <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> softmaxsigmoid <latexit sha1_base64="(null)">(null)</latexit>
MoE Linear regression model <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> sigmoid <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> MoE <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Omg!
Hierarchical Mixture of Experts <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> HME Negative l large number of parameters =>Bayesian HMoE (2003)
5.6 mixture density network <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> output <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> n share the hidden units of the neural network. n the splits of the input space are relaxed, can be nolinear! <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
Bayesian Hierarchical Mixtures of Experts Bishop, C. M. and M. Svense ́n (2003). Bayesian hierarchical mixtures of experts. In U. Kjaerulff and C. Meek (Eds.), Proceedings Nineteenth Conference on Uncertainty in Artificial Intelligence, pp. 57–64. Morgan Kaufmann. Application: the kinematics of robot arms inverse problem two pattern Input: parameters and angles of the robot arm. Output : the end effector of the robot arm .
Bayesian Hierarchical Mixtures of Experts <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Expert <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Graphical Model branch point
Probabilistic tree-based model <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> : The total number of experts <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
HMoE Model <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> branch pointExpert <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Data set Likelihood
Joint probability <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> ⇨ Variational Inference
fin.

More Related Content

PDF
TOC 7 | CFG in Chomsky Normal Form
byMohammad Imam Hossain
 
PDF
K MAP SIMPLIFICATION.pdf
byMeenakshiKS22BCE1551
 
PPT
Sequntial logic design
byPavan Mukku
 
PDF
Bayesian networks in AI
byByoung-Hee Kim
 
PPTX
Image Classification And Support Vector Machine
byShao-Chuan Wang
 
PPTX
Flip flop
byMoni Adhikary
 
PPTX
Learning sets of Rules by Dr.C.R.Dhivyaa Kongu Engineering College
byDhivyaa C.R
 
PDF
Flip flop
byJAGMIT Jamkhandi
 
TOC 7 | CFG in Chomsky Normal Form
byMohammad Imam Hossain
 
K MAP SIMPLIFICATION.pdf
byMeenakshiKS22BCE1551
 
Sequntial logic design
byPavan Mukku
 
Bayesian networks in AI
byByoung-Hee Kim
 
Image Classification And Support Vector Machine
byShao-Chuan Wang
 
Flip flop
byMoni Adhikary
 
Learning sets of Rules by Dr.C.R.Dhivyaa Kongu Engineering College
byDhivyaa C.R
 
Flip flop
byJAGMIT Jamkhandi
 

What's hot

PDF
bv_cvxslides (1).pdf
bySantiagoGarridoBulln
 
PPT
Direct method for soliton solution
byMOHANRAJ PHYSICS
 
PPT
adder and subtractor
byUnsa Shakir
 
PPT
Chap14 multiple regression model building
byUni Azza Aunillah
 
PDF
Numerical analysis multivariable unconstrained
bySHAMJITH KM
 
PPTX
Evaluate and quantify the drift of a measuring
byJean-Michel POU
 
PPTX
Presentation on Numerical Method (Trapezoidal Method)
bySyed Ahmed Zaki
 
PDF
Numerical
by1821986
 
PPTX
BE PPT (FLIP FLOPS)
byDHANESHRKNAIR01
 
PPTX
Genetic algorithm artificial intelligence presentation
byTauhidul Khandaker
 
PPT
fuzzy_measures.ppt
byJanosBotzheim
 
PDF
Gan 발표자료
by종현 최
 
PDF
Nfa to-dfa
byrsivashankari
 
PDF
Faster R-CNN - PR012
byJinwon Lee
 
PDF
Introduction to Limited Dependent variable
byAshok Dsouza
 
PPTX
Unit 1 rules of inference
byraksharao
 
PPT
Fuzzy logic-introduction
byWBUTTUTORIALS
 
PPTX
Signal flow graph
bySudhakar Shastri
 
PPTX
Introduction to random forest and gradient boosting methods a lecture
byShreyas S K
 
PPTX
Deep Reinforcement Learning
byUsman Qayyum
 
bv_cvxslides (1).pdf
bySantiagoGarridoBulln
 
Direct method for soliton solution
byMOHANRAJ PHYSICS
 
adder and subtractor
byUnsa Shakir
 
Chap14 multiple regression model building
byUni Azza Aunillah
 
Numerical analysis multivariable unconstrained
bySHAMJITH KM
 
Evaluate and quantify the drift of a measuring
byJean-Michel POU
 
Presentation on Numerical Method (Trapezoidal Method)
bySyed Ahmed Zaki
 
Numerical
by1821986
 
BE PPT (FLIP FLOPS)
byDHANESHRKNAIR01
 
Genetic algorithm artificial intelligence presentation
byTauhidul Khandaker
 
fuzzy_measures.ppt
byJanosBotzheim
 
Gan 발표자료
by종현 최
 
Nfa to-dfa
byrsivashankari
 
Faster R-CNN - PR012
byJinwon Lee
 
Introduction to Limited Dependent variable
byAshok Dsouza
 
Unit 1 rules of inference
byraksharao
 
Fuzzy logic-introduction
byWBUTTUTORIALS
 
Signal flow graph
bySudhakar Shastri
 
Introduction to random forest and gradient boosting methods a lecture
byShreyas S K
 
Deep Reinforcement Learning
byUsman Qayyum
 

Similar to PRML 条件付き混合モデル 14.5

PPTX
ML_in_QM_JC_02-10-18
bySuzanne Wallace
 
PDF
Data Science Cheatsheet.pdf
byqawali1
 
PPTX
machinelearningpresentationon machine .pptx
byFaizRahman56
 
PPTX
Deep Learning: Introduction & Chapter 5 Machine Learning Basics
byJason Tsai
 
PDF
Cheatsheet supervised-learning
bySteve Nouri
 
PDF
Classical Machine Learning Overview
byBob Marcus
 
PDF
PPT_GATE_ML_updatedduhkdhdihdiuduhid.pdf
bymodiyampurushothamre
 
PDF
Scaling Multinomial Logistic Regression via Hybrid Parallelism
byParameswaran Raman
 
PDF
bayes_machine_learning_book for data scientist
byaddamare554
 
PDF
Lecture 5 - Linear Regression Linear Regression
byviyah59114
 
PPTX
LOGISTIC REGRESSION in Mechine Learning.pptx
bykavyayalapalli
 
PPTX
Introduction to Machine Learning unit2 pdf
bygetintovishnu
 
PDF
Machine Learning Final Summary Cheat Sheet
byKristiyana2
 
PPTX
Conditional mixture model and its application for regression model
byLoc Nguyen
 
PDF
Tips for data science competitions
byOwen Zhang
 
PPTX
Modern classification techniques
bymark_landry
 
PDF
Linear models for classification
bySung Yub Kim
 
PDF
Fault Detection and Classification for Robotic Test-bench
byKürşat İNCE
 
PDF
MLSD18 Evaluations
byBigML, Inc
 
PDF
super-cheatsheet-artificial-intelligence.pdf
byssuser089265
 
ML_in_QM_JC_02-10-18
bySuzanne Wallace
 
Data Science Cheatsheet.pdf
byqawali1
 
machinelearningpresentationon machine .pptx
byFaizRahman56
 
Deep Learning: Introduction & Chapter 5 Machine Learning Basics
byJason Tsai
 
Cheatsheet supervised-learning
bySteve Nouri
 
Classical Machine Learning Overview
byBob Marcus
 
PPT_GATE_ML_updatedduhkdhdihdiuduhid.pdf
bymodiyampurushothamre
 
Scaling Multinomial Logistic Regression via Hybrid Parallelism
byParameswaran Raman
 
bayes_machine_learning_book for data scientist
byaddamare554
 
Lecture 5 - Linear Regression Linear Regression
byviyah59114
 
LOGISTIC REGRESSION in Mechine Learning.pptx
bykavyayalapalli
 
Introduction to Machine Learning unit2 pdf
bygetintovishnu
 
Machine Learning Final Summary Cheat Sheet
byKristiyana2
 
Conditional mixture model and its application for regression model
byLoc Nguyen
 
Tips for data science competitions
byOwen Zhang
 
Modern classification techniques
bymark_landry
 
Linear models for classification
bySung Yub Kim
 
Fault Detection and Classification for Robotic Test-bench
byKürşat İNCE
 
MLSD18 Evaluations
byBigML, Inc
 
super-cheatsheet-artificial-intelligence.pdf
byssuser089265
 

More from tmtm otm

PDF
テーブル・テキスト・画像の反実仮想説明
bytmtm otm
 
PDF
自然言語処理における深層学習を用いた予測の不確実性 - Predictive Uncertainty in NLP -
bytmtm otm
 
PDF
予測の不確かさのユーザー調査
bytmtm otm
 
PDF
深層学習の不確実性 - Uncertainty in Deep Neural Networks -
bytmtm otm
 
PDF
[論文紹介] 機械学習システムの安全性における未解決な問題
bytmtm otm
 
PDF
ICML 2021 Workshop 深層学習の不確実性について
bytmtm otm
 
PDF
Bayesian Neural Networks : Survey
bytmtm otm
 
PDF
PRML学習者から入る深層生成モデル入門
bytmtm otm
 
PDF
PRML ベイズロジスティック回帰 4.5 4.5.2
bytmtm otm
 
PDF
PRML 多項式曲線フィッティング 1.1
bytmtm otm
 
PDF
PRML 2.3.7 2.3.9
bytmtm otm
 
PDF
PRML エビデンス近似 3.5 3.6.1
bytmtm otm
 
PDF
PRML カーネルPCA 12.2.3 12.3
bytmtm otm
 
PDF
PRML BNN 5.7 5.7.3
bytmtm otm
 
PDF
PRML RVM 7.2 7.2.3
bytmtm otm
 
PDF
PRML EP法 10.7 10.7.2
bytmtm otm
 
テーブル・テキスト・画像の反実仮想説明
bytmtm otm
 
自然言語処理における深層学習を用いた予測の不確実性 - Predictive Uncertainty in NLP -
bytmtm otm
 
予測の不確かさのユーザー調査
bytmtm otm
 
深層学習の不確実性 - Uncertainty in Deep Neural Networks -
bytmtm otm
 
[論文紹介] 機械学習システムの安全性における未解決な問題
bytmtm otm
 
ICML 2021 Workshop 深層学習の不確実性について
bytmtm otm
 
Bayesian Neural Networks : Survey
bytmtm otm
 
PRML学習者から入る深層生成モデル入門
bytmtm otm
 
PRML ベイズロジスティック回帰 4.5 4.5.2
bytmtm otm
 
PRML 多項式曲線フィッティング 1.1
bytmtm otm
 
PRML 2.3.7 2.3.9
bytmtm otm
 
PRML エビデンス近似 3.5 3.6.1
bytmtm otm
 
PRML カーネルPCA 12.2.3 12.3
bytmtm otm
 
PRML BNN 5.7 5.7.3
bytmtm otm
 
PRML RVM 7.2 7.2.3
bytmtm otm
 
PRML EP法 10.7 10.7.2
bytmtm otm
 

Recently uploaded

PDF
Step by Step Guide to Buying a Old or Aged Verified Paxful Accounts in US.pdf
byTopSelleriTDotCom
 
PDF
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
 
PPTX
Lecture 14- Introduction to Hypothesis Testing.pptx
bygbadamosiayomide
 
PDF
Trustworthy AI : Governance of AI through Ethics
byKarim Baïna
 
PPTX
250106204341-c238c4fadonebeforelastyear.pptx
byAbduRouf3
 
PPTX
GENPHYSICS-2.pptx this about general physics. Grade 11 lesson
byGwynMorata
 
PPTX
inbound4709748695955476507 Deformable bodies.pptx
byaltheabautistam
 
PDF
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
 
PPTX
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
 
PDF
PathRAG rag pipeline PathRAG rag pipeline .pdf
byhotslix023
 
PDF
Machine_Psychology_2050_Strategic_Scenarios.pdf
byKambs Consulting
 
PPTX
Social Media Audit of Starbucks Coffee Company.pptx
bydiviap21
 
DOCX
Data Analyst Training Course | Advanced Data Analytics Certification | GVT Ac...
byGVT Academy
 
PPTX
Organizational Structure and Design Topic 4
bydavenieralayaay
 
PPTX
The Singularity Sprint (2026): Neuralink + the AI Arms Race — Ridwan Sassman
byridwansassman
 
PDF
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
 
PPT
data collection tool ppt.ppttttdataadfttg
byMuhammadodil
 
PDF
LECTURE - Overcoming the AI Failure Rate - AIG, DQM.pdf
bySami Laine
 
PPTX
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
 
PDF
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
 
Step by Step Guide to Buying a Old or Aged Verified Paxful Accounts in US.pdf
byTopSelleriTDotCom
 
Columbus Data and Analytics Wednesday - People Analytics 101: Making Sense of...
byTim Wilson
 
Lecture 14- Introduction to Hypothesis Testing.pptx
bygbadamosiayomide
 
Trustworthy AI : Governance of AI through Ethics
byKarim Baïna
 
250106204341-c238c4fadonebeforelastyear.pptx
byAbduRouf3
 
GENPHYSICS-2.pptx this about general physics. Grade 11 lesson
byGwynMorata
 
inbound4709748695955476507 Deformable bodies.pptx
byaltheabautistam
 
Understanding Data Analytics: Concepts, Types, and Use Cases
byThe IoT Academy
 
End to End Supply Chain Management- Analytics Case Study.
byNandhuNandha1
 
PathRAG rag pipeline PathRAG rag pipeline .pdf
byhotslix023
 
Machine_Psychology_2050_Strategic_Scenarios.pdf
byKambs Consulting
 
Social Media Audit of Starbucks Coffee Company.pptx
bydiviap21
 
Data Analyst Training Course | Advanced Data Analytics Certification | GVT Ac...
byGVT Academy
 
Organizational Structure and Design Topic 4
bydavenieralayaay
 
The Singularity Sprint (2026): Neuralink + the AI Arms Race — Ridwan Sassman
byridwansassman
 
Analysis of Union Budget 2026-27: Major Announcements and their Impact
byDeeshiPavecha
 
data collection tool ppt.ppttttdataadfttg
byMuhammadodil
 
LECTURE - Overcoming the AI Failure Rate - AIG, DQM.pdf
bySami Laine
 
Top Institute for Data Analyst Course in Navi Mumbai
byPatil sagar
 
US Digital Fan Engagement Index 2026: Benchmarking Online Demand for Sports T...
byHyperset Group Ltd
 

PRML 条件付き混合モデル 14.5

  • 1.
  • 2.
    Conditional Mixture Models p14.5.1Mixtures of linear regression models p14.5.2 Mixtures of logistic models p14.5.3 Mixtures of experts
  • 3.
  • 4.
    Decision trees Positive interpretability Negative hard assignment Splits:axis-aligned of the input space. Regression: piecewise-constant predictions discontinuities at the split boundaries. <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 5.
    Conditional Mixture Models Positive softprobabilistic splits of the input space Splits : functions of all of the input variables Negative no interpretability <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Fig.14.9 Model A <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Model B Model
  • 6.
    Mixture of ExpertsModel (MoE) <latexit sha1_base64="(null)">(null)</latexit> n a fully probabilistic tree-based model Hierarchical Mixture of Experts (HME) <latexit sha1_base64="(null)">(null)</latexit> Ex)
  • 7.
    Mixtures of linearregression models
  • 8.
    Gaussian Mixture Models nClustering (unsupervised Learning) Fig. 9.5 <latexit sha1_base64="(null)">(null)</latexit> Observed data If k = 3 k <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 9.
    Mixtures of linearregression models <latexit sha1_base64="(null)">(null)</latexit> n A general of switching regression , 9 3 79 7 97 3 9 8 7 70 7 A7 1 7 3 3 7 )93 . 7 ) 1 ( Linear regression
  • 10.
    Learning the parameters EMalgorithm <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Parameters Data set <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 11.
    E step responsibilities initialized <latexitsha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> The expectation of the complete-data log likelihood
  • 12.
    M step Fixed <latexitsha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Maximize the function about the constraint 1) <latexit sha1_base64="(null)">(null)</latexit> the mixing coefficients Using a Lagrange multiplier method,
  • 13.
    M step 2) <latexitsha1_base64="(null)">(null)</latexit> the parameter of the k-th linear regression model <latexit sha1_base64="(null)">(null)</latexit> Linear regression model (3.12) <latexit sha1_base64="(null)">(null)</latexit> weighted least squares problem If k = 3 k <latexit sha1_base64="(null)">(null)</latexit> n = 1 <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 14.
    M step <latexit sha1_base64="(null)">(null)</latexit> <latexitsha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> the parameter of the k-th linear regression model <latexit sha1_base64="(null)">(null)</latexit> matrix notation <latexit sha1_base64="(null)">(null)</latexit> n are learned the data of the high responsibility .<latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 15.
    M step a precisionparameter<latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 16.
    Example Fig.14.8 <latexit sha1_base64="(null)">(null)</latexit> <latexitsha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 17.
    The predictive conditionaldensity <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> No independent of the input
  • 18.
  • 19.
    Mixtures of logisticmodels <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> logistic regression Model
  • 20.
    Learning the parameters <latexitsha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> EM algorithm Parameters data set <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> the complete-data log likelihood <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> }
  • 21.
  • 22.
    M step <latexit sha1_base64="(null)">(null)</latexit> Fixed<latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Maximize the function about <latexit sha1_base64="(null)">(null)</latexit> the mixing coefficients }
  • 23.
    M step <latexit sha1_base64="(null)">(null)</latexit> theparameter of the k-th logistic regression model does not have a closed-form solution.<latexit sha1_base64="(null)">(null)</latexit> (IRLS) algorithm <latexit sha1_base64="(null)">(null)</latexit> n Need the gradient and Hessian <latexit sha1_base64="(null)">(null)</latexit>
  • 24.
    IRLS algorithm <latexit sha1_base64="(null)">(null)</latexit> <latexitsha1_base64="(null)">(null)</latexit> Gradient Hessian <latexit sha1_base64="(null)">(null)</latexit> While not convergence do <latexit sha1_base64="(null)">(null)</latexit>
  • 25.
    A mixture oflogistic regression models true probability of the class label single logistic regression mixture of two logistic regression <latexit sha1_base64="(null)">(null)</latexit> Model A Model B
  • 26.
  • 27.
    Mixtures of experts <latexitsha1_base64="(null)">(null)</latexit> Gating function: determine which expert are dominant in which region. be represented by a linear softmax or sigmoid. n a mixture of experts model Expert: can model in different regions of input space. predict in their own region. <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 28.
    Gating function <latexit sha1_base64="(null)">(null)</latexit> <latexitsha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> softmaxsigmoid <latexit sha1_base64="(null)">(null)</latexit>
  • 29.
    MoE Linear regressionmodel <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> sigmoid <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> MoE <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Omg!
  • 30.
    Hierarchical Mixture ofExperts <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> HME Negative l large number of parameters =>Bayesian HMoE (2003)
  • 31.
    5.6 mixture densitynetwork <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> output <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> n share the hidden units of the neural network. n the splits of the input space are relaxed, can be nolinear! <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 32.
    Bayesian Hierarchical Mixturesof Experts Bishop, C. M. and M. Svense ́n (2003). Bayesian hierarchical mixtures of experts. In U. Kjaerulff and C. Meek (Eds.), Proceedings Nineteenth Conference on Uncertainty in Artificial Intelligence, pp. 57–64. Morgan Kaufmann. Application: the kinematics of robot arms inverse problem two pattern Input: parameters and angles of the robot arm. Output : the end effector of the robot arm .
  • 33.
    Bayesian Hierarchical Mixturesof Experts <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Expert <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Graphical Model branch point
  • 34.
    Probabilistic tree-based model <latexitsha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> : The total number of experts <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit>
  • 35.
    HMoE Model <latexit sha1_base64="(null)">(null)</latexit> <latexitsha1_base64="(null)">(null)</latexit> branch pointExpert <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> Data set Likelihood
  • 36.
    Joint probability <latexit sha1_base64="(null)">(null)</latexit> <latexitsha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> <latexit sha1_base64="(null)">(null)</latexit> ⇨ Variational Inference
  • 37.