PRML 9-9.2.2 クラスタリング (K-means とガウス混合モデル) / Clustering (K-means and Gaussian Mixture Models)

Pattern Recognition and Machine Learning
9 – 9.2.2
新田晃大
関西学院大学理工学部
http://www.akihironitta.com
2018 年 7 月 2 日

Agenda
• 9 混合モデルとEM
• 9.1 K-means クラスタリング
• 9.1.1 画像分割と画像圧縮
• 9.2 混合ガウス分布
• 9.2.1 最尤推定
• 9.2.2 混合ガウス分布の EM アルゴリズム
3/64

潜在変数の嬉しさ
5/64
<latexit sha1_base64="RWYGy3sA8nbCNyvWArL17hTuf9o=">AAACFnicbVDLSsNAFJ3UV62vqEs3g0VoQUsigt0IBTcuK9gHNCFMppN26GQSZiZiDf0KN/6KGxeKuBV3/o3TNIi2Hrhw5px7mXuPHzMqlWV9GYWl5ZXVteJ6aWNza3vH3N1ryygRmLRwxCLR9ZEkjHLSUlQx0o0FQaHPSMcfXU79zi0Rkkb8Ro1j4oZowGlAMVJa8syTuJI6fgDvJlV4AR2ZhF72vp/AH+cY5lLVM8tWzcoAF4mdkzLI0fTMT6cf4SQkXGGGpOzZVqzcFAlFMSOTkpNIEiM8QgPS05SjkEg3zc6awCOt9GEQCV1cwUz9PZGiUMpx6OvOEKmhnPem4n9eL1FB3U0pjxNFOJ59FCQMqghOM4J9KghWbKwJwoLqXSEeIoGw0kmWdAj2/MmLpH1as62afX1WbtTzOIrgAByCCrDBOWiAK9AELYDBA3gCL+DVeDSejTfjfdZaMPKZffAHxsc3jY2dsg==</latexit>
複雑な観測変数の周辺分布扱いやすい z と x の同時分布
<latexit sha1_base64="UHyxPgP06wAGrJmEBbqiuD6twdE=">AAACH3icbVDLSsNAFJ3UV62vqEs3g0VoQUoiot0IBTcuK9gHNKFMJpN26OTBzESsIX/ixl9x40IRcde/cZIG0dYDA4dz7mXOPU7EqJCGMdNKK6tr6xvlzcrW9s7unr5/0BVhzDHp4JCFvO8gQRgNSEdSyUg/4gT5DiM9Z3Kd+b17wgUNgzs5jYjto1FAPYqRVNJQv4hqieV48CGtwyto0UDCH+UU5uQxrVs+kmPuJ25aKEO9ajSMHHCZmAWpggLtof5luSGOfRJIzJAQA9OIpJ0gLilmJK1YsSARwhM0IgNFA+QTYSf5fSk8UYoLvZCrpwLm6u+NBPlCTH1HTWZBxaKXif95g1h6TTuhQRRLEuD5R17MoAxhVhZ0KSdYsqkiCHOqskI8RhxhqSqtqBLMxZOXSfesYRoN8/a82moWdZTBETgGNWCCS9ACN6ANOgCDJ/AC3sC79qy9ah/a53y0pBU7h+APtNk3QxOh5A==</latexit>
連続値の潜在変数は 12 章
観測変数と潜在変数の同時分布を定義すれば，
周辺化により観測変数だけの分布が得られる．
複雑な分布をより単純な分布から構成することが可能
定義!!!!
p(x): 混合ガウス分布
p(x|z): ガウス分布
p(z): カテゴリカル分布
例

混合モデル
• 混合分布は，
• より複雑な確率分布を構成する枠組み
• クラスタリングにも使える
6/64
複雑な分布単純な分布

9 章では
• 9.1: 非確率的アプローチ (K-means)
• 9.2: 確率的アプローチ (Mixture of Gaussians)
• 潜在変数の導入
• EM で尤度最大化
• 9.3: 潜在変数の見方 (K-means と MoG の関連等)
• 9.4: EM アルゴリズムの一般化
7/64

9.1: K-means clustering
クラスタとは
目的関数の定義
目的関数の最小化とその解釈
目的関数の収束性
K-means の高速化
K-means の応用
9/64

クラスタとは
10/64
PRML
「内部のデータ点間の距離が、外部のデータとの距離と比べて小さいデータのグループ」
Cambridge Dictionaryhttps://dictionary.cambridge.org/dictionary/english-japanese/cluster
「a group of similar things that are close together」
似てる? 似てる?
K-means では，
ユークリッド距離 (L2距離) が小さい = 似てる
<latexit sha1_base64="Y5sgR0xFNJUVTaMR8kUtbOomByg=">AAACNHicbVDLSgMxFM34rPVVdekmWAQ3lhkR7KZQcCMWpIJ9QKcOmTTThkkyQ5IRy3Q+yo0f4kYEF4q49RtMHwu1Xgg595x7ufceP2ZUadt+sRYWl5ZXVnNr+fWNza3tws5uU0WJxKSBIxbJto8UYVSQhqaakXYsCeI+Iy0/PB/rrTsiFY3EjR7GpMtRX9CAYqQN5RVql7ACXZVwLxUVJ7tNr7JpFk6yWgalUcLMHaWuH8D7zBPwGBocsZ4acvO5PMm80B3dnniFol2yJwHngTMDRTCLuld4cnsRTjgRGjOkVMexY91NkdQUM5Ll3USRGOEQ9UnHQIE4Ud10cnQGDw3Tg0EkzRMaTtifHSniaryhqeRID9RfbUz+p3USHZS7KRVxoonA00FBwqCO4NhB2KOSYM2GBiAsqdkV4gGSCGvjc96Y4Pw9eR40T0qOXXKuT4vV8syOHNgHB+AIOOAMVMEFqIMGwOABPIM38G49Wq/Wh/U5LV2wZj174FdYX99e26vj</latexit>
目的関数

K-means の目的関数
11/64
<latexit sha1_base64="Pg5Cd9za6PyxAvrggWngpPZ6XGw=">AAACKnicbVDLSsNAFJ3UV42vqEs3g0VwVRIR7EaouBEEqWAf0MQwmU7aoZNJmJmIJeR73PgrbrpQils/xEnahVYvXOZwzrnMvSdIGJXKtmdGZWV1bX2jumlube/s7ln7Bx0ZpwKTNo5ZLHoBkoRRTtqKKkZ6iSAoChjpBuPrQu8+ESFpzB/UJCFehIachhQjpSnfunKzzA1C+Jz73M39jF86+eOd65pFl1rMBnIS6Qe6UZr748I1Lly32qHbt2p23S4L/gXOAtTAolq+NXUHMU4jwhVmSMq+YyfKy5BQFDOSm24qSYLwGA1JX0OOIiK9rDw1hyeaGcAwFrq5giX7cyJDkSzW1c4IqZFc1gryP62fqrDhZZQnqSIczz8KUwZVDIvc4IAKghWbaICwoHpXiEdIIKx0uqYOwVk++S/onNUdu+7cn9eajUUcVXAEjsEpcMAFaIIb0AJtgMELeAPv4MN4NabGzPicWyvGYuYQ/Crj6xs066cO</latexit>
: データ集合
: プロトタイプの集合
: クラスタ数
: データ数
目的: J を最小化するを求める.
<latexit sha1_base64="w7K5tIw4DN33Ftxoj0oOwDyUjVU=">AAACTHicbVBNS8MwGE7nd/2aevQSHIoHGa0IehEELx4VnBOWUdLs7RaWpiVJ1VH6A7148Oav8OJBEcFsK6KbDwQenvd5P/KEqeDaeN6LU5mZnZtfWFxyl1dW19arG5s3OskUgwZLRKJuQ6pBcAkNw42A21QBjUMBzbB/Pqw370BpnshrM0ihHdOu5BFn1FgpqDIV5LJf4FPskhC6XObMTtOFi7G/R2JqeirOeVQQnJMwwg+FtReYcInPg/4BIdbn/fgS0wN1zzUUdZeA7JSzgmrNq3sj4Gnil6SGSlwG1WfSSVgWgzRMUK1bvpeadk6V4UxA4ZJMQ0pZn3ahZamkMeh2PgqjwLtW6eAoUfZJg0fq746cxloP4tA6h2frydpQ/K/Wykx00s65TDMDko0XRZnAJsHDZHGHK2BGDCyhTHF7K2Y9qigzNn/XhuBPfnma3BzWfa/uXx3Vzk7KOBbRNtpB+8hHx+gMXaBL1EAMPaJX9I4+nCfnzfl0vsbWilP2bKE/qMx/AzbOsuA=</latexit>
<latexit sha1_base64="+jVsjLNVzVp+5BgFjX87CvPgej4=">AAACCnicbVDLSsNAFJ34rPUVdelmtAgupCQi2GXBjcsK9gFNCJPJpB06MwkzE6GErN34K25cKOLWL3Dn3zhps9DWA8MczrmXe+8JU0aVdpxva2V1bX1js7ZV397Z3du3Dw57KskkJl2csEQOQqQIo4J0NdWMDFJJEA8Z6YeTm9LvPxCpaCLu9TQlPkcjQWOKkTZSYJ94uQxyMSm84gJ6uRcmLFJTbj7o8SyYeEVgN5ymMwNcJm5FGqBCJ7C/vCjBGSdCY4aUGrpOqv0cSU0xI0XdyxRJEZ6gERkaKhAnys9npxTwzCgRjBNpntBwpv7uyBFX5X6mkiM9VoteKf7nDTMdt/ycijTTROD5oDhjUCewzAVGVBKs2dQQhCU1u0I8RhJhbdKrmxDcxZOXSe+y6TpN9+6q0W5VcdTAMTgF58AF16ANbkEHdAEGj+AZvII368l6sd6tj3npilX1HIE/sD5/ANw9mvQ=</latexit>
https://ejje.weblio.jp/content/prototype
=> 「クラスタの中心を表すもの」

歪み尺度としての目的関数
12/64
歪み度大歪み度小
プロトタイプからの距離の2乗
背景: http://stockarch.com/files/10/01/picnic_distorted.jpg

目的関数の最小化 1/6
13/64
E step と M step を収束するまで繰り返す.
<latexit sha1_base64="Sgdq1hsgB55hSlUfkny7mb/nvlY=">AAACTHicbVDPSyMxGM3U9VfXXasevYQtC14sM0XQiyB4ERdEwarQ1CGTZtowSWZIMmKJ+QO9ePC2f4UXD4osbNrOwW33Qcj73vc+8uUlBWfahOHvoLbwZXFpeWW1/nXt2/f1xsbmlc5LRWiH5DxXNwnWlDNJO4YZTm8KRbFIOL1OsuNx//qOKs1yeWlGBe0JPJAsZQQbL8UNggSTsUVWxVZmDjkHT+EhRLoUXjiM3K09c9Mqm1S/HKysDxYlKbx3sYS70KIhNl7IeV+PhL+QKJ2LM/Rw244bzbAVTgDnSVSRJqhwHjeeUT8npaDSEI617kZhYXoWK8MIp66OSk0LTDI8oF1PJRZU9+wkDAd/eqUP01z5Iw2cqJ8nLBZ6vKJ3CmyGerY3Fv/X65YmPehZJovSUEmmD6UlhyaH42RhnylKDB95golifldIhlhhYnz+dR9CNPvleXLVbkVhK7rYax4dVHGsgG3wA+yACOyDI3ACzkEHEPAIXsAbeA+egtfgI/gztdaCamYL/IPa0l9QU7XO</latexit>
E step
<latexit sha1_base64="b9IZ+uhF6+OgnhiwKZIPcmrdjSc=">AAACWXicbVHLSgMxFM2Mr1pfVZdugkVwY5kRwW4EwY0oiIJVoalDJs3YMElmSDJiiflJF4L4Ky5MHwutXgg595x7b5KTtORMmyj6CMK5+YXFpdpyfWV1bX2jsbl1p4tKEdohBS/UQ4o15UzSjmGG04dSUSxSTu/T/Gyk3z9TpVkhb82wpD2BnyTLGMHGU0mjRILJxCKL0oL39VD4DYkqyZFz8AKeQKQrkVh5ErtHe+UmWT7OLh20aIANVF7PnUOvfkgGX1wi4QG0MxOd8zNfHw+TRjNqReOAf0E8BU0wjeuk8Yb6BakElYZwrHU3jkrTs1gZRjh1dVRpWmKS4yfa9VBiQXXPjp1xcM8zfZgVyi9p4Jj92WGx0KMr+kqBzUDPaiPyP61bmazds0yWlaGSTA7KKg5NAUc2wz5TlBg+9AATxfxdIRlghYnxn1H3JsSzT/4L7g5bcdSKb46ap+2pHTWwA3bBPojBMTgF5+AadAAB7+ArWAgWg88wCGthfVIaBtOebfArwu1vSeq2Zg==</latexit>
M step

14/64
E step
n に関して, 各項は独立.
=> 各項別々にrnkを求めればよい.
<latexit sha1_base64="Z2zWWtt3U12JW9Mt42+/VsXhttc=">AAACxHichVFbSxwxFM5Mbzq9uG0ffQldKoXSZbII+iJYCkUqiAVXhc06ZDKZ3TDJZJqLdInjj/RN+mfMrvuw9YIfhHx855wvJ+fkjeDGpul1FD97/uLlq5XV5PWbt+/WOu8/HBvlNGUDqoTSpzkxTPCaDSy3gp02mhGZC3aSVz9m8ZNzpg1X9ZGdNmwkybjmJafEBinr/PsFd5INiI2Tma92UHvm91uoM4+qFl94nJfwb5sh+A16PCE2CEoUZirDhaVr26zCF2d9jJOvD5n0l036T5ts4PNCWfOY3cGy3cGTdlmnm/bSOeB9ghakCxY4zDpXuFDUSVZbKogxQ5Q2duSJtpwK1ibYGdYQWpExGwZaE8nMyM+X0MLPQSlgqXQ4tYVzdbnCE2lmTYZMSezE3I3NxIdiQ2fL7ZHndeMsq+ntQ6UT0Co42ygsuGbUimkghGoeeoV0QjShNuw9CUNAd798nxz3eyjtod+b3d3txThWwDr4BL4ABLbALtgDh2AAaPQ9GkdN9Cf+GYvYxO42NY4WNR/Bf4gvbwCLOtxR</latexit>

15/64
E step
J のある n に関する項
<latexit sha1_base64="nbeyUVeBa+fZJE/yW5dY/LxZ/TM=">AAAC1HicjZJdixMxFIYz49c6flW99CZYXASxTIrg3iwseCP0Zhdsd6HpDplMZhsmH0OSWS3pgCLe+uO88zf4J8y0A+quFz0Q8vKe85CTnOS14Nal6c8ovnHz1u07e3eTe/cfPHw0ePxkZnVjKJtSLbQ5y4llgis2ddwJdlYbRmQu2Glevevyp5fMWK7VB7eq2UKSC8VLTokLVjb4hW0jM18dovbcT1poMq+qFq89zkv4qc0UfA09XhIXDC0Ku5Jhw7Jp26zC6/Nxcri/YdBuDOoYjJNXW2q8GzXuKXxZaGf3//CT3fjJls8Gw3SUbgJeF6gXQ9DHcTb4gQtNG8mUo4JYO0dp7RaeGMepYG2CG8tqQityweZBKiKZXfjNUFr4IjgFLLUJSzm4cf8mPJG2azJUSuKW9mquM/+XmzeuPFh4rurGMUW3B5WNgE7DbsKw4IZRJ1ZBEGp46BXSJTGEuvAPkvAI6OqVr4vZeITSETp5Mzw66J9jDzwDz8FLgMBbcATeg2MwBTQ6iT5Gn6Mv8Sxex1/jb9vSOOqZp+CfiL//Br1g5EM=</latexit>
どれか1つが 1 (one-hot)
プロトタイプ固定での最適解
<latexit sha1_base64="LgZG+jM8mC/FO5V19mkwJQRgzGk=">AAACcXicbVFba9RAFJ7EW4239fIiohy6KIK6JEWwL0LBFx8ruG1hZw2TycnudOcSZiatS5p3f59v/glf/ANOtkG09cAw33zn/k1RS+F8mv6I4itXr12/sXUzuXX7zt17o/sPDpxpLMcpN9LYo4I5lELj1Asv8ai2yFQh8bBYfej9hydonTD6s1/XOFdsoUUlOPOBykffbN7qVQfvIaEFLoRueajmugQge0EV80urWlF1FFYhhjK7oEro/BjoWUuLCr52uYY3QJfMh7eRpVurcFHVdPkxPfuy8xooDcXSP8WMX6I9FQ4nXUJRl0PDfDROJ+nG4DLIBjAmg+3no++0NLxRqD2XzLlZltZ+3jLrBZcYajcOa8ZXbIGzADVT6ObtRrEOngemhMrYcLSHDft3RsuU6zcJkf3Y7qKvJ//nmzW+2p23QteNR83PG1WNBG+glx9KYZF7uQ6AcSvCrMCXzDLuwyclQYTs4sqXwcHOJEsn2ae3473dQY4t8oRsk5ckI+/IHvlI9smUcPIzehQ9jZ5Fv+LHMcTb56FxNOQ8JP9Y/Oo3UOK73w==</latexit>

16/64
M step
各プロトタイプに関する偏微分を0とおく.
<latexit sha1_base64="LE5vTyGawc0PxAhljBY+43Juco0=">AAACVnicbVHPS8MwGE07dbP+qnr0EhyCHhztENxlMPAiHmSCU2GdJc1SF5akJUnFUfpP6kX/FC9iuhX8+UHIy3vfI19eopRRpT3vzbJrS8sr9caqs7a+sbnlbu/cqCSTmAxwwhJ5FyFFGBVkoKlm5C6VBPGIkdtoelbqt49EKpqIaz1LyYijB0FjipE2VOjyIJYI50GKpKaIwYviC+dBlLCxmnGzBTwrwmnhdJ12oDIe5qLrF/eXMJggDaU5Tgt4aAwxfCrg8T/WI6cLvdBtei1vXvAv8CvQBFX1Q/c5GCc440RozJBSQ99L9SgvB8SMFE6QKZIiPEUPZGigQJyoUT6PpYAHhhnDOJFmCQ3n7HdHjrgqRzSdHOmJ+q2V5H/aMNNxZ5RTkWaaCLy4KM4Y1AksM4ZjKgnWbGYAwpKaWSGeIJOzNj/hmBD830/+C27aLd9r+VcnzV6niqMB9sA+OAQ+OAU9cA76YAAweAHvlm3VrFfrw16264tW26o8u+BH2e4nBl+zyQ==</latexit>
クラスタ割当固定での最適解
<latexit sha1_base64="YSHxOIEX7RVAIrR0Uz/UpYjnh8w=">AAACM3icbVDNS8MwHE39nPVr6tFLcAieRiuCuwgDL+JpgvuAtZQ0TbewJC1JKo7S/8mL/4gHQTwo4tX/wXTbQbf9IOTx3vuRvBemjCrtOG/Wyura+sZmZcve3tnd268eHHZUkklM2jhhieyFSBFGBWlrqhnppZIgHjLSDUfXpd59IFLRRNzrcUp8jgaCxhQjbaigept7YcIiNebm8nhWBCNoX9leLBHOPZXxQEBviDSUQS5GhXHH8LEolkhFUK05dWcycBG4M1ADs2kF1RcvSnDGidCYIaX6rpNqP0dSU8xIYXuZIinCIzQgfQMF4kT5+SRzAU8NE8E4keYIDSfs340ccVWmMk6O9FDNayW5TOtnOm74ORVpponA04fijEGdwLJAGFFJsGZjAxCW1PwV4iEydWlTs21KcOcjL4LOed116u7dRa3ZmNVRAcfgBJwBF1yCJrgBLdAGGDyBV/ABPq1n6936sr6n1hVrtnME/o318wufM6wX</latexit>

17/64
E step
M step
再掲

18/64
E step
M step
=> 解釈へ

最適解の解釈 1/2
19/64
E step
一番近いプロトタイプのクラスタに割り当てる!

最適解の解釈 2/2
20/64
M step
<latexit sha1_base64="XiiIpC8/kVZa9tmA9e1V1cv05OM=">AAACNXicbVDNS8MwHE39nPVr6tFLcAieRiuCuwgDLx48THAfsJaSpukWlqQlScVR+k958f/wpAcPinj1XzDddtBtPwh5vPd+JO+FKaNKO86btbK6tr6xWdmyt3d29/arB4cdlWQSkzZOWCJ7IVKEUUHammpGeqkkiIeMdMPRdal3H4hUNBH3epwSn6OBoDHFSBsqqN7mXpiwSI25uTyeFcEI2le2F0uEc09lPBDQGyINZZCLUWHcMXwsAlEsEYugWnPqzmTgInBnoAZm0wqqL16U4IwToTFDSvVdJ9V+jqSmmJHC9jJFUoRHaED6BgrEifLzSeoCnhomgnEizREaTti/GzniqsxlnBzpoZrXSnKZ1s903PBzKtJME4GnD8UZgzqBZYUwopJgzcYGICyp+SvEQ2QK06Zo25TgzkdeBJ3zuuvU3buLWrMxq6MCjsEJOAMuuARNcANaoA0weAKv4AN8Ws/Wu/VlfU+tK9Zs5wj8G+vnF1qprPg=</latexit>
分母: クラスタ k のデータ数
分子: クラスタ k のデータの総和
各クラスタでデータの平均をとる!

アルゴリズムの収束性
21/64
K-means アルゴリズム
For more details, see MacQueen (1967)
各ステップは J を減少させるので
アルゴリズムの収束は保証されている (Ex. 9.1)． ※ 最小点に収束する保証はなし
再割当が起こらなくなるまで
繰り返す
E step: クラスタへの再割当
M step: クラスタ平均の再計算

1
K-means の学習過程
22/64
initialisation
E step
クラスタ割当更新
M step
セントロイド更新
2 3 4
図 9.1

K-means の高速化
23/64
問題: そのままの実装は遅い
理由: 各プロトタイプと各データ点の間の距離の計算 O(NK)
<latexit sha1_base64="sUFkKCDtL7nA4AVe/5A0Uc48Uxs=">AAACbXicbVFLb9QwEHbCq4RHUxAHHkIjVjwOZZVUSPSCVIkLxyKxbaX1EjnOZNdd24lsp7BKc+MXcuMvcOlfqLONELSMZPnTNzPfjD/ntRTWJcmvILx2/cbNWxu3ozt3793fjLceHNiqMRwnvJKVOcqZRSk0TpxwEo9qg0zlEg/z5cc+f3iCxopKf3GrGmeKzbUoBWfOU1n8w2StXnbwASKa41zolns120UA6SuqmFsY1Yqyo7D0NZSZOVVCZ8dAT1ual/C9yzS8BY8rWdiV8hdVTZcd09OvO9tAqRdK/ghVboHmm7A47iKKuhiGZfEoGSfrgKsgHcCIDLGfxT9pUfFGoXZcMmunaVK7WcuME1yi124s1owv2RynHmqm0M7atVsdvPRMAWVl/NEO1uzfHS1Ttn+Jr+zXtpdzPfm/3LRx5e6sFbpuHGp+MahsJLgKeuuhEAa5kysPGDfC7wp8wQzjzn9Q5E1ILz/5KjjYGafJOP38brS3O9ixQZ6SF+QNScl7skc+kX0yIZz8DuLgcfAkOAsfhc/C5xelYTD0PCT/RPj6HIajuh4=</latexit>
E step
解決策 1: 木構造を利用し，データ構造をあらかじめ計算する方法
解決策 2: 距離の三角不等式を利用して不必要な距離計算を避ける方法
<latexit sha1_base64="GqYYiq3VtreMdlyZdsLt9cw2168=">AAACHXicbZDLSsNAFIYn9VbrLerSzWARhEJJpGCXBTcuK9gLNKFMppN26OTizEQMaV7Eja/ixoUiLtyIb+MkjaKtBwa++f9zmDm/EzIqpGF8aqWV1bX1jfJmZWt7Z3dP3z/oiiDimHRwwALed5AgjPqkI6lkpB9ygjyHkZ4zvcj83i3hggb+tYxDYnto7FOXYiSVNNQb1iyxHBfepbAGc4pTawYtRm7gj6WE2vcts4d61agbecFlMAuogqLaQ/3dGgU48ogvMUNCDEwjlHaCuKSYkbRiRYKECE/RmAwU+sgjwk7y7VJ4opQRdAOuji9hrv6eSJAnROw5qtNDciIWvUz8zxtE0m3aCfXDSBIfzx9yIwZlALOo4IhygiWLFSDMqforxBPEEZYq0IoKwVxceRm6Z3XTqJtXjWqrWcRRBkfgGJwCE5yDFrgEbdABGNyDR/AMXrQH7Ul71d7mrSWtmDkEf0r7+AIP7KE/</latexit>

K-means のオンライン化
24/64
Robbins-Monro法 (2.3.5 節)・・・逐次更新式 <latexit sha1_base64="JzCit9VWdVaEUSHOa3qEY3vg+/E=">AAACrnicjVHbitRAEO3E2zreRn30pXAQVhaHZBHcF2HBFx93wZldnMTQ6anMNNPphO6K7hDm8/wB3/wbK9kI666IBU2dPlWnuy55bbSnKPoZhLdu37l7b+/+6MHDR4+fjJ8+m/uqcQpnqjKVO8+lR6MtzkiTwfPaoSxzg2f55kMXP/uKzuvKfqJtjWkpV1YXWkliKht/T/LKLP22ZNdCUjawgwxa2LD/wj4hvCD2Fr8xs4P3cEUA/1JwVq84gARJ9imWr4nBgvZZJ2mdF0xeDPou+Ab+s5zfjydOr9b0OhtPomnUG9wE8QAmYrCTbPwjWVaqKdGSMtL7RRzVlLbSkVYGd6Ok8VhLtZErXDC0skSftv24d/CKmSUUleNjCXr2qqKVpe9a4MyuS3891pF/iy0aKo7SVtu6IbTq8qOiMUAVdLuDpXaoyGwZSOU01wpqLZ1UxBse8RDi6y3fBPPDaRxN49O3k+OjYRx74oV4KfZFLN6JY/FRnIiZUMFBcBp8DhZhFM7DNMwuU8Ng0DwXf1i4/gXtW8xX</latexit>
<latexit sha1_base64="AvwX9GdMGrsjwaek5UIbFs0Zbrg=">AAACMHicbVBNS8NAEN34WetX1KOXxSJ4sSQi2GPBgx4r2Co0tWy2G1262YTdibSE/iQv/hS9KCji1V/hpObg18DyHu/NsDMvTJW04HnPzszs3PzCYmWpuryyurbubmx2bJIZLto8UYm5DJkVSmrRBglKXKZGsDhU4iIcHhf+xa0wVib6HMap6MXsWstIcgYo9d0TGsQMbsKI5nREJ7SPqBH3aRAmamDHMUKOTVlpDhGvEAMQI0DEJlQmfbfm1b1p0b/EL0mNlNXquw/BIOFZLDRwxazt+l4KvZwZkFyJSTXIrEgZH7Jr0UWqWSxsL58ePKG7qAxolBh8GuhU/T6Rs9gWq2NncZ397RXif143g6jRy6VOMxCaf30UZYpCQov06EAawUGNkTBuJO5K+Q0zjANmXMUQ/N8n/yWdg7rv1f2zw1qzUcZRIdtkh+wRnxyRJjklLdImnNyRR/JCXp1758l5c96/WmeccmaL/Cjn4xNMT6aw</latexit>
<latexit sha1_base64="jycOfqFiGV4U+k+LgSgBQcoTGoI=">AAAB/3icbVDLSsNAFL3xWesrKrhxM1gEVyURwS4LblxWsA9oQ5hMJ+3QySTMTMQSs/BX3LhQxK2/4c6/cdJmoa0Hhjmccy9z5gQJZ0o7zre1srq2vrFZ2apu7+zu7dsHhx0Vp5LQNol5LHsBVpQzQduaaU57iaQ4CjjtBpPrwu/eU6lYLO70NKFehEeChYxgbSTfPkaDCOtxEKIMPaAc+eYWKPftmlN3ZkDLxC1JDUq0fPtrMIxJGlGhCcdK9V0n0V6GpWaE07w6SBVNMJngEe0bKnBElZfN8ufozChDFMbSHKHRTP29keFIqWkUmMkirFr0CvE/r5/qsOFlTCSppoLMHwpTjnSMijLQkElKNJ8agolkJisiYywx0aayqinBXfzyMulc1F2n7t5e1pqNso4KnMApnIMLV9CEG2hBGwg8wjO8wpv1ZL1Y79bHfHTFKneO4A+szx+bWJSA</latexit>
<latexit sha1_base64="KzZNMw+eBco0U/DF5YzLLlSH9UI=">AAACF3icbVDLSsNAFJ34rPUVdelmsAiuQiKCXRbcuKxgH9DEMJlO2qGTBzM3Ygn5Czf+ihsXirjVnX/jpM1CWw8M53Duvcy9J0gFV2Db38bK6tr6xmZtq769s7u3bx4cdlWSSco6NBGJ7AdEMcFj1gEOgvVTyUgUCNYLJldlvXfPpOJJfAvTlHkRGcU85JSAtnzTcoNEDNU00pRjN8pwgX2c44nmO80usAfQrJu0U/hmw7bsGfCycCrRQBXavvnlDhOaRSwGKohSA8dOwcuJBE4FK+puplhK6ISM2EDLmERMefnsrgKfameIw0TqFwOeub8nchKpcnXdGREYq8Vaaf5XG2QQNr2cx2kGLKbzj8JMYEhwGRIecskoiKkWhEqud8V0TCShoKOs6xCcxZOXRffccmzLublotJpVHDV0jE7QGXLQJWqha9RGHUTRI3pGr+jNeDJejHfjY966YlQzR+gPjM8fpdCeRg==</latexit>
<latexit sha1_base64="jycOfqFiGV4U+k+LgSgBQcoTGoI=">AAAB/3icbVDLSsNAFL3xWesrKrhxM1gEVyURwS4LblxWsA9oQ5hMJ+3QySTMTMQSs/BX3LhQxK2/4c6/cdJmoa0Hhjmccy9z5gQJZ0o7zre1srq2vrFZ2apu7+zu7dsHhx0Vp5LQNol5LHsBVpQzQduaaU57iaQ4CjjtBpPrwu/eU6lYLO70NKFehEeChYxgbSTfPkaDCOtxEKIMPaAc+eYWKPftmlN3ZkDLxC1JDUq0fPtrMIxJGlGhCcdK9V0n0V6GpWaE07w6SBVNMJngEe0bKnBElZfN8ufozChDFMbSHKHRTP29keFIqWkUmMkirFr0CvE/r5/qsOFlTCSppoLMHwpTjnSMijLQkElKNJ8agolkJisiYywx0aayqinBXfzyMulc1F2n7t5e1pqNso4KnMApnIMLV9CEG2hBGwg8wjO8wpv1ZL1Y79bHfHTFKneO4A+szx+bWJSA</latexit>
<latexit sha1_base64="KzZNMw+eBco0U/DF5YzLLlSH9UI=">AAACF3icbVDLSsNAFJ34rPUVdelmsAiuQiKCXRbcuKxgH9DEMJlO2qGTBzM3Ygn5Czf+ihsXirjVnX/jpM1CWw8M53Duvcy9J0gFV2Db38bK6tr6xmZtq769s7u3bx4cdlWSSco6NBGJ7AdEMcFj1gEOgvVTyUgUCNYLJldlvXfPpOJJfAvTlHkRGcU85JSAtnzTcoNEDNU00pRjN8pwgX2c44nmO80usAfQrJu0U/hmw7bsGfCycCrRQBXavvnlDhOaRSwGKohSA8dOwcuJBE4FK+puplhK6ISM2EDLmERMefnsrgKfameIw0TqFwOeub8nchKpcnXdGREYq8Vaaf5XG2QQNr2cx2kGLKbzj8JMYEhwGRIecskoiKkWhEqud8V0TCShoKOs6xCcxZOXRffccmzLublotJpVHDV0jE7QGXLQJWqha9RGHUTRI3pGr+jNeDJejHfjY966YlQzR+gPjM8fpdCeRg==</latexit>
<latexit sha1_base64="F7TFBxcJEWM8ODjDQI/K1s0Jp0A=">AAACF3icbVA9SwNBEN2LXzF+RS1tFoNgFe5EMGXAxjKC+YBcDHubSbJkb+/YnVPDkX9h41+xsVDEVjv/jXtJCk18sLzHmxl25gWxFAZd99vJrayurW/kNwtb2zu7e8X9g4aJEs2hziMZ6VbADEihoI4CJbRiDSwMJDSD0WVWb96BNiJSNziOoROygRJ9wRlaq1ss+0Eke2YcWkqpHyZ0Qrs0pSPLt5Z9hAe0rODeOpNuseSW3SnosvDmokTmqHWLX34v4kkICrlkxrQ9N8ZOyjQKLmFS8BMDMeMjNoC2lYqFYDrp9K4JPbFOj/YjbZ9COnV/T6QsNNnqtjNkODSLtcz8r9ZOsF/ppELFCYLis4/6iaQY0Swk2hMaOMqxFYxrYXelfMg042ijLNgQvMWTl0XjrOy5Ze/6vFStzOPIkyNyTE6JRy5IlVyRGqkTTh7JM3klb86T8+K8Ox+z1pwznzkkf+B8/gC2n55R</latexit>
更新
例 <latexit sha1_base64="3Li1vFYmsN2sNLM4/vf1urMgYIk=">AAACL3icbVBNS8NAEN34WetX1aOXxSJ4KokIehEEQTxWsFpoatlsJ7p0swm7E7WE/CMv/hUvIop49V+4aXPQ1oHlPd68YWdekEhh0HXfnJnZufmFxcpSdXlldW29trF5ZeJUc2jxWMa6HTADUihooUAJ7UQDiwIJ18HgtOhf34M2IlaXOEygG7FbJULBGVqpVzvzg1j2zTCykFE/SmlOezSjA4s3Fn2ER7So4MEqOT22HoZ3QWi1x9KraN6r1d2GOyo6TbyS1ElZzV7txe/HPI1AIZfMmI7nJtjNmEbBJeRVPzWQMD5gt9CxVLEITDcb3ZvTXav0aRhr+xTSkfp7ImORKU6yzmJZM9krxP96nRTDo24mVJIiKD7+KEwlxZgW4dG+0MBRDi1hXAu7K+V3TDOONuKqDcGbPHmaXO03PLfhXRzUT47KOCpkm+yQPeKRQ3JCzkmTtAgnT+SFvJMP59l5dT6dr7F1xilntsifcr5/ADk5pqE=</latexit>
<latexit sha1_base64="O/rp68Eyx9Go3G0HKwUa8Rnq0Pc=">AAACaXicfVFNS+RAEO1kXT9G1x1XFNFL4bCwsDAki8t6EQQvHhUcFSbj0OmpaDOdTuiu7DqEgL/Rm3/Ai3/CzkwOfiwWNO/xqh5d/TrOlbQUBA+e/2nu8/zC4lJreeXL6tf22rdzmxVGYE9kKjOXMbeopMYeSVJ4mRvkaazwIh4f1f2Lv2iszPQZTXIcpPxay0QKTk4atu+iOFMjO0kdlBClBVQwhBLGDq8cRoS35FDjP6dUcABB9ze8MMFHLjc1df2cuVJON3Hi9NtmXkM1bHeCbjAteE/ChnRYUyfD9n00ykSRoiahuLX9MMhpUHJDUiisWlFhMedizK+x76jmKdpBOU2qgu9OGUGSGXc0wVR96Sh5aut3ucl6Wfu2V4v/6/ULSvYHpdR5QajF7KKkUEAZ1LHDSBoUpCaOcGGk2xXEDTdckPuclgshfPvk9+T8VzcMuuHpXudwv4ljke2wXfaDhewPO2TH7IT1mGCP3oq34W16T/6av+Vvz0Z9r/Gss1fld54BF32ztg==</latexit>
<latexit sha1_base64="5MshylM9qIza1YMQztGOzRa9R3A=">AAACFXicbVDLSsNAFJ3UV42vqEs3g8XiQkIiiqUgFNy4rGAf0IQwmU7aoZNJmJkIJfQn3Pgrblwo4lZw5984bbPQ1gMXDufcy733hCmjUjnOt1FaWV1b3yhvmlvbO7t71v5BWyaZwKSFE5aIbogkYZSTlqKKkW4qCIpDRjrh6Gbqdx6IkDTh92qcEj9GA04jipHSUmCdeUQhGMAccjiB19Ct1qHnmQuqY19W654XWBXHdmaAy8QtSAUUaAbWl9dPcBYTrjBDUvZcJ1V+joSimJGJ6WWSpAiP0ID0NOUoJtLPZ19N4IlW+jBKhC6u4Ez9PZGjWMpxHOrOGKmhXPSm4n9eL1NRzc8pTzNFOJ4vijIGVQKnEcE+FQQrNtYEYUH1rRAPkUBY6SBNHYK7+PIyaZ/brmO7dxeVRq2IowyOwDE4BS64Ag1wC5qgBTB4BM/gFbwZT8aL8W58zFtLRjFzCP7A+PwB0FqaJw==</latexit>
・・・データ点そのもの
・・・データ点とプロトタイプの中間

K-means の非類似度の一般化
25/64
<latexit sha1_base64="ZdCHy+BxqZfTqqX/CzLLTO43RTw=">AAACb3icbZHNatwwEMdlt03TTZNs00MPKWHoEkggLHYpNJdAoJfQQkmhuwmst0aW5V2xkmykcelifO0D9pZ3yCVvUPnj0CYZEPrPb2ak0SgppLAYBDee/+Tps43nmy8GWy+3d3aHr/amNi8N4xOWy9xcJ9RyKTSfoEDJrwvDqUokv0pWn5r41U9urMj1d1wXfK7oQotMMIoOxcPfVYRCphw+13AGkS1VXOmzsP5Rfa07b9V6X2oDMVSgYQU1RIriklHpwLRxJc/wqKNJ5uAvB7vsGk4gSnKZ2rVym2ORKvtoe5IRiyUex8NRMA5ag4ci7MWI9HYZD/9Eac5KxTUySa2dhUGB84oaFEzyehCVlheUreiCz5zUVHE7r9p51XDoSApZbtzSCC39t6Kiyjb9uszmSfZ+rIGPxWYlZqfzSuiiRK5Zd1FWSsAcmuFDKgxnKNdOUGaE6xXYkhrK0H3RwA0hvP/kh2L6fhwG4/Dbh9H5aT+OTbJP3pEjEpKP5JxckEsyIYzcenvevvfWu/Pf+Ac+dKm+19e8Jv+Zf/wXXrC4bQ==</latexit>
K-means の非類似度 <latexit sha1_base64="aUEObEpnaGg0maaxfBdXbySoe9o=">AAACDXicbVC7TsMwFHXKq5RXgJHFoiCxUCUVEh0rsTAWiT6kJkSO47RWHSeyHUSV9gdY+BUWBhBiZWfjb3DaDNByJMtH59yre+/xE0alsqxvo7Syura+Ud6sbG3v7O6Z+wcdGacCkzaOWSx6PpKEUU7aiipGeokgKPIZ6fqjq9zv3hMhacxv1TghboQGnIYUI6UlzzxxJpnjh/Bh6nF4DjWPWSDHkf6cKJ16I2dyV/fMqlWzZoDLxC5IFRRoeeaXE8Q4jQhXmCEp+7aVKDdDQlHMyLTipJIkCI/QgPQ15Sgi0s1m10zhqVYCGMZCP67gTP3dkaFI5hvqygipoVz0cvE/r5+qsOFmlCepIhzPB4UpgyqGeTQwoIJgxcaaICyo3hXiIRIIKx1gRYdgL568TDr1mm3V7JuLarNRxFEGR+AYnAEbXIImuAYt0AYYPIJn8ArejCfjxXg3PualJaPoOQR/YHz+AH7Pm8o=</latexit>
K-medoids の非類似度 <latexit sha1_base64="OJq4mC9DqSnkyYOxIXi09bPWT1I=">AAACFHicbVDLSsNAFJ3UV62vqEs3g0WoKCURwS4LblxWsA9oQphMJu3QySTMTMQS8hFu/BU3LhRx68Kdf+OkzUJbDwxzOOde7r3HTxiVyrK+jcrK6tr6RnWztrW9s7tn7h/0ZJwKTLo4ZrEY+EgSRjnpKqoYGSSCoMhnpO9Prgu/f0+EpDG/U9OEuBEacRpSjJSWPPPMiZAaY8SyXg4bmeOH8CH3+DnUNGaBnEb6c6I09yannlm3mtYMcJnYJamDEh3P/HKCGKcR4QozJOXQthLlZkgoihnJa04qSYLwBI3IUFOOIiLdbHZUDk+0EsAwFvpxBWfq744MRbJYT1cWJ8hFrxD/84apCltuRnmSKsLxfFCYMqhiWCQEAyoIVmyqCcKC6l0hHiOBsNI51nQI9uLJy6R30bStpn17WW+3yjiq4AgcgwawwRVogxvQAV2AwSN4Bq/gzXgyXox342NeWjHKnkPwB8bnD9OJnqQ=</latexit>
一般化
K-medoids の目的関数
E step
M step
各クラスタ k について，
を O(Nk
2) 回評価する必要がある<latexit sha1_base64="UjMnw7GKxVFiZzCP8Umw6FQpqYg=">AAACDXicbZDLSsNAFIYn9VbrLerSzWAVKkhJRLDLghuXFewFmhAm00k7djIJMxOxhLyAG1/FjQtF3Lp359s4aSNo6w8DH/85hznn92NGpbKsL6O0tLyyulZer2xsbm3vmLt7HRklApM2jlgkej6ShFFO2ooqRnqxICj0Gen648u83r0jQtKI36hJTNwQDTkNKEZKW5555IRIjTBiaSerpY4fwPvMo6fwB29PHKfimVWrbk0FF8EuoAoKtTzz0xlEOAkJV5ghKfu2FSs3RUJRzEhWcRJJYoTHaEj6GjkKiXTT6TUZPNbOAAaR0I8rOHV/T6QolHIS+roz313O13Lzv1o/UUHDTSmPE0U4nn0UJAyqCObRwAEVBCs20YCwoHpXiEdIIKx0gHkI9vzJi9A5q9tW3b4+rzYbRRxlcAAOQQ3Y4AI0wRVogTbA4AE8gRfwajwaz8ab8T5rLRnFzD74I+PjGyS0muU=</latexit>
<latexit sha1_base64="p85Trk09clrLzG30Qd9qmNZ58Fc=">AAACc3icbVFba9RAFJ7EW423rYIvffDQValQl0QE+yIUfPGxgrst7CxhMjnZne5cwsykuoT8AX+eb/4LX3x3sg2irQeG+fi+cz9FLYXzafojim/cvHX7zs7d5N79Bw8fjXYfz5xpLMcpN9LYs4I5lELj1Asv8ay2yFQh8bRYf+j10wu0Thj92W9qXCi21KISnPlA5aNvNm/1uoP3kNACl0K3PGRzXQKQvaSK+ZVVrag6CuvgQ5ldUiV0fg5bjTPZzrqDlhYVfO1yfQgBGlm6jQofVU2Xn786BEpDuvRPOuNXaL8Ih5MuoajLoWQ+GqeTdGtwHWQDGJPBTvLRd1oa3ijUnkvm3DxLa79omfWCSwy5G4c142u2xHmAmil0i3a7sw5eBKaEytjwtIct+3dEy5TrxwiefdvuqtaT/9Pmja+OFq3QdeNR88tCVSPBG+gPAKWwyL3cBMC4FaFX4CtmGffhTElYQnZ15Otg9maSpZPs09vx8dGwjh2yR/bJAcnIO3JMPpITMiWc/IyeRs8iiH7Fe/F+/PzSNY6GmCfkH4tf/wZYHLzO</latexit>
プロトタイプはデータのどれかという制約

ハード割当とソフト割当
26/64
曖昧さが表現できていない
ソフト割当
各データ点が複数のクラスタに
やんわりと割当
ハード割当
各データ点が1つのクラスタだけに割当
半々ぐらい?
<latexit sha1_base64="QcdtIHAmrt9nLv0/UuSW6bCopf0=">AAACFnicbVDLSgMxFM3UVx1fVZdugkXoQB1mimI3QsGNywr2AW0pmTTThiaZIckIdehXuPFX3LhQxK24829MH4vaeiDh5Jx7ubkniBlV2vN+rMza+sbmVnbb3tnd2z/IHR7VVZRITGo4YpFsBkgRRgWpaaoZacaSIB4w0giGNxO/8UCkopG416OYdDjqCxpSjLSRurnzQruPOEeFx24q/LFThAvv0thx7GtY8NzLIjSX083lPdebAq4Sf07yYI5qN/fd7kU44URozJBSLd+LdSdFUlPMyNhuJ4rECA9Rn7QMFYgT1Umna43hmVF6MIykOULDqbrYkSKu1IgHppIjPVDL3kT8z2slOix3UiriRBOBZ4PChEEdwUlGsEclwZqNDEFYUvNXiAdIIqxNkrYJwV9eeZXUS67vuf7dRb5SnseRBSfgFBSAD65ABdyCKqgBDJ7AC3gD79az9Wp9WJ+z0ow17zkGf2B9/QINn5tr</latexit>
<latexit sha1_base64="4ZxCEkXXTI+NS2FhBCzIXonhe5w=">AAACAnicbVDLSsNAFJ3UV42vqCtxM1iEFkpJimA3QsGNywr2AW0Ik+mkHTqZhJmJUENx46+4caGIW7/CnX/jpM1CWw9c7uGce5m5x48Zlcq2v43C2vrG5lZx29zZ3ds/sA6POjJKBCZtHLFI9HwkCaOctBVVjPRiQVDoM9L1J9eZ370nQtKI36lpTNwQjTgNKEZKS551Un7wUu7MqjDr9VnFvIJlpwrtimeV7Jo9B1wlTk5KIEfLs74GwwgnIeEKMyRl37Fj5aZIKIoZmZmDRJIY4Qkakb6mHIVEuun8hBk818oQBpHQxRWcq783UhRKOQ19PRkiNZbLXib+5/UTFTTclPI4UYTjxUNBwqCKYJYHHFJBsGJTTRAWVP8V4jESCCudmqlDcJZPXiWdes2xa87tRanZyOMoglNwBsrAAZegCW5AC7QBBo/gGbyCN+PJeDHejY/FaMHId47BHxifP+8DlH4=</latexit>

Image Segmentation and Compression
27/64

K-means の応用 1
image segmentation:
“適度に同質な外見をもつ複数の領域、物体や物体の部品に対応する複数の領域に
分割すること”
segmentate
28/64

データ圧縮: 可逆と非可逆
compress
0101101010
reconstruct
(可逆)
圧縮率と復元の精巧さの
トレードオフ
可逆データ圧縮 (ZIP, PNG, …)
もとのデータを完全に復元できる圧縮
低い圧縮率
非可逆データ圧縮 (JPEG, MP3, …)
もとのデータを完全に復元できない圧縮
高い圧縮率
29/64
compress
reconstruct
(非可逆)

K-means の応用 2
- 画素数: N
- どのクラスタか: log2K bit
- クラスタ数: K
- 何色か: 24 bit
- 画素数: N
- 何色か: 24 bit
<latexit sha1_base64="ejLnuXJgWGaTSEk4O7iCbD24Crs=">AAAB9XicbVBNSwMxEM36WetX1aOXYBE8SNktBXssePEkFewHdNeSTbNtaJJdklmlLP0fXjwo4tX/4s1/Y9ruQVsfDDzem2FmXpgIbsB1v5219Y3Nre3CTnF3b//gsHR03DZxqilr0VjEuhsSwwRXrAUcBOsmmhEZCtYJx9czv/PItOGxuodJwgJJhopHnBKw0kO1dutfYl/LLOQw7ZfKbsWdA68SLydllKPZL335g5imkimgghjT89wEgoxo4FSwadFPDUsIHZMh61mqiGQmyOZXT/G5VQY4irUtBXiu/p7IiDRmIkPbKQmMzLI3E//zeilE9SDjKkmBKbpYFKUCQ4xnEeAB14yCmFhCqOb2VkxHRBMKNqiiDcFbfnmVtKsVz614d7Vyo57HUUCn6AxdIA9doQa6QU3UQhRp9Ixe0Zvz5Lw4787HonXNyWdO0B84nz9wZ5HK</latexit>
<latexit sha1_base64="9PqvzjBSq0QsASm7LyfzFyj5OPg=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWARBKUkpWCXBTdCQSrYBzQhTKaTdujMJMxMhBLqxl9x40IRt/6FO//GSduFVg9cOJxzL/feEyaMKu04X1ZhZXVtfaO4Wdra3tnds/cPOipOJSZtHLNY9kKkCKOCtDXVjPQSSRAPGemG46vc794TqWgs7vQkIT5HQ0EjipE2UmAfVWvN8xuPxcOg2vQuoCd5FlI9LQV22ak4M8C/xF2QMligFdif3iDGKSdCY4aU6rtOov0MSU0xI9OSlyqSIDxGQ9I3VCBOlJ/NPpjCU6MMYBRLU0LDmfpzIkNcqQkPTSdHeqSWvVz8z+unOqr7GRVJqonA80VRyqCOYR4HHFBJsGYTQxCW1NwK8QhJhLUJLQ/BXX75L+lUK65TcW9r5UZ9EUcRHIMTcAZccAka4Bq0QBtg8ACewAt4tR6tZ+vNep+3FqzFzCH4BevjGxiwlVk=</latexit>
K-means で圧縮
<latexit sha1_base64="84KWs4eBM0oplTSo3ye5N1SRR9w=">AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKYI4BL4IgEcwDkiXMTmaTMfNYZmaFsOQfvHhQxKv/482/cZLsQRMLGoqqbrq7ooQzY33/2yusrW9sbhW3Szu7e/sH5cOjllGpJrRJFFe6E2FDOZO0aZnltJNoikXEaTsaX8/89hPVhin5YCcJDQUeShYzgq2TWrc9ztFdv1zxq/4caJUEOalAjka//NUbKJIKKi3h2Jhu4Cc2zLC2jHA6LfVSQxNMxnhIu45KLKgJs/m1U3TmlAGKlXYlLZqrvycyLIyZiMh1CmxHZtmbif953dTGtTBjMkktlWSxKE45sgrNXkcDpimxfOIIJpq5WxEZYY2JdQGVXAjB8surpHVRDfxqcH9ZqdfyOIpwAqdwDgFcQR1uoAFNIPAIz/AKb57yXrx372PRWvDymWP4A+/zB9Zjjpk=</latexit>
仮定
30/64

K-means の応用 2
1,036,800 bit
ビット数 [bit] 圧縮率 [%]
K=2 43,248 4.2
K=3 86,472 8.3
K=10 173,040 16.7
31/64

K-means の応用 2
1,036,800 bit
ビット数 [bit] 圧縮率 [%]
K=2 43,248 4.2
K=3 86,472 8.3
K=10 173,040 16.7
画
質
と
圧
縮
率
の
ト
レ
ー
ド
オ
フ
32/64

K-means の応用例
高画質低画質
高圧縮率低圧縮率
33/64

9.2 Mixtures of Gaussians
34/64

流れ
• 潜在変数 z の導入
• 事後確率 p(z|x) の解釈
• (サンプリング)
• 最尤推定の問題
• 最尤推定 (EM アルゴリズム)
35/64

思い出(2.3.9): 混合ガウス分布
単一のガウス分布よりも豊か
36/64
<latexit sha1_base64="LRrdYxjIenOQ0f/wuj0c1+sKE+I=">AAAClXicbZFtixMxEMez68Od9al3vvCFbwaLcAdSdkVw3xycD4gg6In27qCpJZtmt6HJ7pLMimXdb+Sn8Z3fxtm2Qu05EPKf38wkmUlaGe0xin4H4bXrN27u7d/q3b5z9979/sHhuS9rJ9VIlqZ0l6nwyuhCjVCjUZeVU8KmRl2ki9dd/OKbcl6XxRdcVmpiRV7oTEuBhKb9nxUcAbcC52kGDXyHFo7hBLivLUwJLMiJCX4l/Z52XukNb9dlUhhyP3SuURnuHvYDeFqamV9a2ohxWxP9e8LTrWz+WedWbAW50/kcjzmKvHk2jJOknfYH0TBaGVwV8UYM2MbOpv1ffFbK2qoCpRHej+OowkkjHGppVNvjtVeVkAuRqzHJQljlJ81qqi08ITKDrHS0CoQV3a5ohPVdX5TZNeF3Yx38X2xcY5ZMGl1UNapCri/KagNYQvdFMNNOSTRLEkI6TW8FORdOSKSP7NEQ4t2Wr4pzmlg0jD89H5wmm3Hss0fsMTtiMXvBTtk7dsZGTAaHQRK8DF6FD8OT8E34dp0aBpuaB+wfCz/+AWhKwJw=</latexit>

グラフィカルモデル
37/64
<latexit sha1_base64="e3Q8UNiawoCtitnMvBUMdpbPN2E=">AAACkXiclVFbSwJBFJ7dbmZllo+9HJJCIWQ3hOzBkHoJejHIC7gis+OsDs5emJkNbfP/9Ht66980XsDSXjow8M13Yc6c40acSWVZX4a5tb2zu5faTx8cHmWOsyenTRnGgtAGCXko2i6WlLOANhRTnLYjQbHvctpyRw8zvfVKhWRh8KImEe36eBAwjxGsNNXLfkRQAMfHauh6kMAYplBMX1bBkbEPPc2stDetTeG3X3NXq+tYhx3nf3EowmYL7+sex+ll81bJmhdsAnsJ8mhZ9V720+mHJPZpoAjHUnZsK1LdBAvFCKfTtBNLGmEywgPa0TDAPpXdZD7RKVxopg9eKPQJFMzZn4kE+1JOfFc7Z43KdW1G/qV1YuVVugkLoljRgCwe8mIOKoTZeqDPBCWKTzTARDDdK5AhFpgovcS0HoK9/uVN0Lwu2VbJfi7na5XlOFLoDJ2jArLRDaqhR1RHDUSMjFE2qsadmTNvzZp5v7CaxjKTQ7/KfPoGG+e+eg==</latexit>

目標
• 離散潜在変数 z を用いて混合ガウス分布を表現すること．
• 具体的に…
• 定義: 周辺分布 p(z)
• 定義: 条件付き分布 p(x|z)
• 同時分布 p(x, z) を得る．
• 周辺分布 p(x) を得る．
38/64

潜在変数 z の導入
39/64
<latexit sha1_base64="Equ/jVNLiK4jskSW96/zDD5etDE=">AAACNHicbZBNSwMxEIaz9avWr6pHL4NFqJeyK4K9CAUvQi8V7Ad0a8mm2TY0uxuSrNAu/VFe/CFeRPCgiFd/g2m7iLYOhLzzzAzJvJ7gTGnbfrEyK6tr6xvZzdzW9s7uXn7/oKGiWBJaJxGPZMvDinIW0rpmmtOWkBQHHqdNb3g1rTfvqVQsCm/1SNBOgPsh8xnB2qBuviqgCG6A9cDzIYExTOAULsEVMupB15ChyRxD74yumtsVLOVzNv7JJt18wS7Zs4Bl4aSigNKodfNPbi8icUBDTThWqu3YQncSLDUjnE5ybqyowGSI+7RtZIgDqjrJbOkJnBjSAz+S5oQaZvT3RIIDpUaBZzqn66nF2hT+V2vH2i93EhaKWNOQzB/yYw46gqmD0GOSEs1HRmAimfkrkAGWmGjjc86Y4CyuvCwaZyXHLjk354VKObUji47QMSoiB12gCrpGNVRHBD2gZ/SG3q1H69X6sD7nrRkrnTlEf8L6+gYzP6UY</latexit>
K 次元確率変数 z (one-hot 表現)
<latexit sha1_base64="2iLi18p6ldEO1bezCu4urtdIMQA=">AAACUnicdVLLSgMxFE3rq9aqVZduLhZL3ZQZEeymUHAjdFPBPqBTSibNtMHMg+ROoQ79RkHc+CFuXKjpQ9BWLyQczrmHm5zEjaTQaFmvqfTG5tb2TmY3u5fbPzjMHx23dBgrxpsslKHquFRzKQLeRIGSdyLFqe9K3nYfbmZ6e8yVFmFwj5OI93w6DIQnGEVD9fMiAkdyD0vwCH1IwIYpVM3uKDEc4QUUq+BE4ltynGzRGQ9C1AatOOv/O+szZz9fsMrWvGAd2EtQIMtq9PPPziBksc8DZJJq3bWtCHsJVSiY5NOsE2seUfZAh7xrYEB9rnvJPJIpnBtmAF6ozAoQ5uxPR0J9rSe+azp9iiO9qs3Iv7RujF6ll4ggipEHbDHIiyVgCLN8YSAUZygnBlCmhDkrsBFVlKF5hawJwV698jpoXZZtq2zfXRVqlWUcGXJKzkiJ2OSa1MgtaZAmYeSJvJEP8pl6Sb2nzS9ZtKZTS88J+VXp3BcLqqxE</latexit>
<latexit sha1_base64="iBDrcWhH8Davo1dem1jYdtj7TSg=">AAACAnicbVDLSgNBEOz1GeNr1ZN4GQyCp7ArgjkGvHiMYB6QLMvspDcZMvtwZlYIS/Dir3jxoIhXv8Kbf+Mk2YMmFjQUVd10dwWp4Eo7zre1srq2vrFZ2ipv7+zu7dsHhy2VZJJhkyUikZ2AKhQ8xqbmWmAnlUijQGA7GF1P/fYDSsWT+E6PU/QiOoh5yBnVRvLtY4f0BN6TXsqJT3IyIpO54Pp2xak6M5Bl4hakAgUavv3V6ycsizDWTFCluq6Tai+nUnMmcFLuZQpTykZ0gF1DYxqh8vLZCxNyZpQ+CRNpKtZkpv6eyGmk1DgKTGdE9VAtelPxP6+b6bDm5TxOM40xmy8KM0F0QqZ5kD6XyLQYG0KZ5OZWwoZUUqZNamUTgrv48jJpXVRdp+reXlbqtSKOEpzAKZyDC1dQhxtoQBMYPMIzvMKb9WS9WO/Wx7x1xSpmjuAPrM8f9iCVLw==</latexit>
<latexit sha1_base64="J1KxBJBfUTR1M7T6dmKt3mh//Iw=">AAACEXicbZBNS8MwGMdTX+d8q3r0EhzCTqMVwV2EgRfBywT3AmstaZZuYUlaklQYpV/Bi1/FiwdFvHrz5rcx3XrQzQdCfvz/z0Py/MOEUaUd59taWV1b39isbFW3d3b39u2Dw66KU4lJB8cslv0QKcKoIB1NNSP9RBLEQ0Z64eSq8HsPRCoaizs9TYjP0UjQiGKkjRTYdU+lHAYwgxN4CV2Yw3vDN+b2ElrqeeEEds1pOLOCy+CWUANltQP7yxvGOOVEaMyQUgPXSbSfIakpZiSveqkiCcITNCIDgwJxovxstlEOT40yhFEszREaztTfExniSk15aDo50mO16BXif94g1VHTz6hIUk0Enj8UpQzqGBbxwCGVBGs2NYCwpOavEI+RRFibEKsmBHdx5WXonjVcp+HentdazTKOCjgGJ6AOXHABWuAatEEHYPAInsEreLOerBfr3fqYt65Y5cwR+FPW5w/lj5k4</latexit>
確率であるための制限
まとめて表す
周辺分布 p(z) を定義．
周辺分布
<latexit sha1_base64="KCJdak4lAmrfKMMNbFBc5ZXWhXM=">AAACfnicbVFNj9MwEHXC1xK+Chy5DFSgBbQlQUj0stJKXDhwWCS6u1JdVY4zaa06dmRPFnWj/gz+GDd+CxfcbJCguyNZfvPezHg8k9daeUrTX1F84+at23f27ib37j94+Gjw+MmJt42TOJFWW3eWC49aGZyQIo1ntUNR5RpP89WnrX56js4ra77RusZZJRZGlUoKCtR88ONivoJDyBL+BUtyarEk4Zz9nrQ8L+FiA4cJ10HZTwB4jgtl2qCL9aaVm0ABpJx3Nz8vLHnovb9stuNfF8XRFH3RhHcdvJ4Phuko7QyugqwHQ9bb8XzwkxdWNhUaklp4P83SmmahKimpMdRtPNZCrsQCpwEaUaGftd34NvAyMAWU1oVjCDr234xWVN6vqzxEVoKWflfbktdp04bK8axVpm4Ijbx8qGw0kIXtLqBQDiXpdQBCOhV6BbkUTkgKG0vCELLdL18FJ+9HWTrKvn4YHo37ceyxZ+wF22cZ+8iO2Gd2zCZMst/R8+hN9DZm8av4IH53GRpHfc5T9p/F4z/62rXS</latexit>
混合係数
<latexit sha1_base64="QA/1R5/s/kOMEnXbGVI8oaf3tVk=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx4rmLbQhrLZTtqlm03Y3Qgl9Dd48aCIV3+QN/+N2zYHbX0w8Hhvhpl5YSq4Nq777ZS2tnd298r7lYPDo+OT6ulZRyeZYuizRCSqF1KNgkv0DTcCe6lCGocCu+H0buF3n1BpnshHM0sxiOlY8ogzaqzkD1I+9IbVmlt3lyCbxCtIDQq0h9WvwShhWYzSMEG17ntuaoKcKsOZwHllkGlMKZvSMfYtlTRGHeTLY+fkyiojEiXKljRkqf6eyGms9SwObWdMzUSvewvxP6+fmagZ5FymmUHJVouiTBCTkMXnZMQVMiNmllCmuL2VsAlVlBmbT8WG4K2/vEk6N3XPrXsPt7VWs4ijDBdwCdfgQQNacA9t8IEBh2d4hTdHOi/Ou/Oxai05xcw5/IHz+QNzpY5n</latexit>
<latexit sha1_base64="6PeNhKT4CN8OG5hJMDa3t3wF5Kw=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKYI8FLx4rmLbQhrLZbtqlm03YnQgl9Dd48aCIV3+QN/+N2zYHbX0w8Hhvhpl5YSqFQdf9dkpb2zu7e+X9ysHh0fFJ9fSsY5JMM+6zRCa6F1LDpVDcR4GS91LNaRxK3g2ndwu/+8S1EYl6xFnKg5iOlYgEo2glf5CKYWNYrbl1dwmySbyC1KBAe1j9GowSlsVcIZPUmL7nphjkVKNgks8rg8zwlLIpHfO+pYrG3AT58tg5ubLKiESJtqWQLNXfEzmNjZnFoe2MKU7MurcQ//P6GUbNIBcqzZArtloUZZJgQhafk5HQnKGcWUKZFvZWwiZUU4Y2n4oNwVt/eZN0GnXPrXsPN7VWs4ijDBdwCdfgwS204B7a4AMDAc/wCm+Ocl6cd+dj1Vpyiplz+APn8wd1KY5o</latexit>
<latexit sha1_base64="HPVsbomKbAqfVn8lckIW0nt5UHo=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUsMeCF48VTFtoQ9lsJ+3SzSbsboQS+hu8eFDEqz/Im//GbZuDtj4YeLw3w8y8MBVcG9f9dkobm1vbO+Xdyt7+weFR9fikrZNMMfRZIhLVDalGwSX6hhuB3VQhjUOBnXByN/c7T6g0T+SjmaYYxHQkecQZNVby+ykfXA+qNbfuLkDWiVeQGhRoDapf/WHCshilYYJq3fPc1AQ5VYYzgbNKP9OYUjahI+xZKmmMOsgXx87IhVWGJEqULWnIQv09kdNY62kc2s6YmrFe9ebif14vM1EjyLlMM4OSLRdFmSAmIfPPyZArZEZMLaFMcXsrYWOqKDM2n4oNwVt9eZ20r+qeW/cebmrNRhFHGc7gHC7Bg1towj20wAcGHJ7hFd4c6bw4787HsrXkFDOn8AfO5w92rY5p</latexit>
山を選ぶ確率

40/64
<latexit sha1_base64="P7htaSilHm1ckhpBYQFCSde1pzk=">AAACn3icbZFdb9MwFIad8DXKxwpcwsWBCmmTUJUgJHaDNMEFSENTEXQbakrlOE5r1Y4j+wTRhfwtfgh3/BtO0gqVbUeK/L7PObZzjtNSK49R9CcIr12/cfPWzu3enbv37u/2Hzw88bZyQo6F1dadpdxLrQo5RoVanpVOcpNqeZou37X50+/SeWWLL7gq5dTweaFyJTgSmvV/lbAHieG4SHOo4Qc08HPLn5PfhzeQlM5mMCOyJBcT/Ub6iNauVnBN9ri1WuZ4xYmp1ZlfGVqIJaYiuj6tgRdb1clnNTd8K5k4NV/gfnfd+T/azPqDaBh1AZdFvBEDtonRrP87yayojCxQaO79JI5KnNbcoRJaNr2k8rLkYsnnckKy4Eb6ad3Nt4HnRDLIraOvQOjo9o6aG982R5VtJ/5iroVX5SYV5gfTWhVlhbIQ64vySgNaaB8LMuWkQL0iwYVT9K8gFtxxgfSkPRpCfLHly+Lk5TCOhvGnV4PDg804dthj9oztsZi9ZofsAxuxMRPBk+BtcBR8DJ+G78PjcLQuDYPNnkfsvwi//gU3gMQm</latexit>
条件付き分布 p(x|z) を定義．
<latexit sha1_base64="Pjao6JWF7nELcQ/ie4pRlnsK6xo=">AAADK3icrVJbaxQxFM6MtzretvroS3CxVJBlpgj2pVDxRViQim5b2CxLJpPZDU0mQ3KmdDvu//HFv+KDPnjBV/+HZ3ZH2LZeXjyQ5OP7zkm+JCcttfIQx1+D8NLlK1evrV2Pbty8dftOZ/3uvreVE3IgrLbuMOVealXIASjQ8rB0kptUy4P06HmjHxxL55Ut3sCslCPDJ4XKleCA1Hg9eFZSpmUOm5QZDtM0pzU9oXP6lp7SMeIE8Q7OzKnJFB5FGzvLRME1qi9R/UM5S63O/MzgghwzFbK/Nny8ks1eq4nhK2J7EmPRBjvOLHhE/zDZ/98m+38z2T9jMhp3unEvXgS9CJIWdEkbe+POR5ZZURlZgNDc+2ESlzCquQMltJxHrPKy5OKIT+QQYcGN9KN68ddz+hCZjObW4SiALtjVipob31wIMxv3/rzWkL/ThhXk26NaFWUFshDLg/JKU7C0aRyaKScF6BkCLpxCr1RMueMCsL2aR0jOX/ki2N/qJXEvefWku7vdPscauU8ekE2SkKdkl7wge2RARPAu+BB8Dr6E78NP4bfw+zI1DNqae+RMhD9+An3u9QE=</latexit>
まとめて表す
<latexit sha1_base64="ml0OX7Q25RsKEjsP4ItxwC/tez4=">AAACF3icbZBLS8NAEMc3Pmt9RT16WSxCvYREBHsRCl48VrAPaErYbDft0s2D3YlYY7+FF7+KFw+KeNWb38ZNm4O2Duzy4z8zzMzfTwRXYNvfxtLyyuraemmjvLm1vbNr7u23VJxKypo0FrHs+EQxwSPWBA6CdRLJSOgL1vZHl3m+fcuk4nF0A+OE9UIyiHjAKQEteaaVYFewAKrYDQkM/QBn+A5P8AO+x55mR/OF/l3JB0M48cyKbdnTwIvgFFBBRTQ888vtxzQNWQRUEKW6jp1ALyMSOBVsUnZTxRJCR2TAuhojEjLVy6Z3TfCxVvo4iKV+EeCp+rsjI6FS49DXlfnyaj6Xi//luikEtV7GoyQFFtHZoCAVGGKcm4T7XDIKYqyBUMn1rpgOiSQUtJVlbYIzf/IitE4tx7ac67NKvVbYUUKH6AhVkYPOUR1doQZqIooe0TN6RW/Gk/FivBsfs9Ilo+g5QH/C+PwB9BGcmQ==</latexit> <latexit sha1_base64="lkGl+2ZNtKtaNqk0MTtl+B+PVmY=">AAACF3icbZDJSgNBEIZ74hbjFvXopTAI8RJmgmAuQsCLxwhmgUwIPZ2epEnPQneNGMe8hRdfxYsHRbzqzbexsxw08YeGj7+q6Krfi6XQaNvfVmZldW19I7uZ29re2d3L7x80dJQoxusskpFqeVRzKUJeR4GSt2LFaeBJ3vSGl5N685YrLaLwBkcx7wS0HwpfMIrG6uZLMbiS+1gEN6A48HxI4Q7G8AD30DVcNnwBDrhK9Ad42s0X7JI9FSyDM4cCmavWzX+5vYglAQ+RSap127Fj7KRUoWCSj3NuonlM2ZD2edtgSAOuO+n0rjGcGKcHfqTMCxGm7u+JlAZajwLPdE6W14u1iflfrZ2gX+mkIowT5CGbfeQnEjCCSUjQE4ozlCMDlClhdgU2oIoyNFHmTAjO4snL0CiXHLvkXJ8VqpV5HFlyRI5JkTjknFTJFamROmHkkTyTV/JmPVkv1rv1MWvNWPOZQ/JH1ucP9aOcmg==</latexit>
<latexit sha1_base64="1otBhJp8gZhe0FuDMhoDSunfljY=">AAACF3icbZDJSgNBEIZ74hbjFvXopTAI8RJmVDAXIeDFYwSzQCaEnk5P0qRnobtGjGPewouv4sWDIl715tvYWQ6a+EPDx19VdNXvxVJotO1vK7O0vLK6ll3PbWxube/kd/fqOkoU4zUWyUg1Paq5FCGvoUDJm7HiNPAkb3iDy3G9ccuVFlF4g8OYtwPaC4UvGEVjdfKlGFzJfSyCG1Dsez6kcAcjeIB76Bg+NXwBDrhK9Pp43MkX7JI9ESyCM4MCmanayX+53YglAQ+RSap1y7FjbKdUoWCSj3JuonlM2YD2eMtgSAOu2+nkrhEcGacLfqTMCxEm7u+JlAZaDwPPdI6X1/O1sflfrZWgX26nIowT5CGbfuQnEjCCcUjQFYozlEMDlClhdgXWp4oyNFHmTAjO/MmLUD8pOXbJuT4rVMqzOLLkgBySInHIOamQK1IlNcLII3kmr+TNerJerHfrY9qasWYz++SPrM8f9zWcmw==</latexit>

41/64
周辺分布 p(x)
<latexit sha1_base64="Wdi2pVsIS3lln0DSt0i1QOMU1Z8=">AAADp3iclVJdb9MwFHUTYCN8dfDIyxUFtEqoStCk7WXSJF7QkGBoa1epLpXjOK1VO4lsZ6IL/Wn7E7zxb3DSAF2KKrhS5JNz7r0+98phJrg2vv+j5bh37t7b2b3vPXj46PGT9t7TgU5zRVmfpiJVw5BoJnjC+oYbwYaZYkSGgl2G83elfnnFlOZpcmEWGRtLMk14zCkxlprstW4y2AcsiZmFMRTwFZbQ9V4fA9a5hIll/mjXVlvC7fyS68Jmj2/NHIy3dvVwptKokuZwDIGlvlj8wZ444zW/4q5//y29qg0lwhIfy1TBYrPNCpa5ZX7Vv1lXzvlUkjURKz6dmW7zysYc28z+m7kCh6mI9ELaw9r7P3eTdsfv+VXAJghq0EF1nE3a33GU0lyyxFBBtB4FfmbGBVGGU8HsQnPNMkLnZMpGFiZEMj0uqne2hFeWiSBOlf0SAxW7XlEQqctBbGbpXDe1kvybNspNfDQueJLlhiV0dVGcCzAplI8WIq4YNWJhAaGKW69AZ0QRauzT9uwSgubIm2Dwthf4veDzQefkqF7HLnqOXqB9FKBDdILeozPUR9R56Zw6586F23U/uQN3uEp1WnXNM3QrXPITlyAVvg==</latexit>
zj = 1 のとき
<latexit sha1_base64="SSQ0MC7SYUvAgno+lDI6sjUpgY8=">AAACW3icbZHfS8MwEMfTTt2cU6fiky/BIUyQ0YrgHge++CQT3Q9Yx0izdIsmbUmu4qj7J33SB/8VMd0mzM2DkO997i65XPxYcA2O82nZuY3NrXxhu7hT2t3bLx8ctnWUKMpaNBKR6vpEM8FD1gIOgnVjxYj0Bev4zzdZvPPClOZR+AiTmPUlGYU84JSAQYOy8mKOBzjFT3iKPUlgTIkw7l3mChZAdU79wMBXA9+w50diqCfSbIZ5MjH094SLpWzvgY8kWQp6io/GcD4oV5yaMzO8LtyFqKCFNQfld28Y0USyEKggWvdcJ4Z+ShRwKti06CWaxYQ+kxHrGRkSyXQ/nc1mis8MGeIgUmaFgGd0uSIlUmePMZlZ53o1lsH/Yr0Egno/5WGcAAvp/KIgERginA0aD7liFMTECEIVN71iOiaKUDDfUTRDcFefvC7alzXXqbn3V5VGfTGOAjpBp6iKXHSNGugWNVELUfSBvq28VbC+7JxdtEvzVNta1ByhP2Yf/wCjgLCj</latexit>
「混合ガウス分布について，陽に潜在変数を含む別な表現を見出した」

事後確率の解釈
42/64
混合要素 k が x の観測を「説明する」度合いを表す負担率として解釈可
<latexit sha1_base64="W0dxln4p3qucIynKX2E1C6NhnR8=">AAAERHicpVPLbtQwFHUTHmV4TWHJ5ooRqEholFRIdINUiQ0SEiqCaSvGw8hxnIwzthNip2JI83Fs+AB2fAEbFiDEFuGk02EebSWEpcjX55xr33NjB5ng2njelzXHvXDx0uX1K62r167fuNneuLWn0yKnrEdTkeYHAdFMcMV6hhvBDrKcERkIth+Mn9b8/iHLNU/VazPJ2ECSWPGIU2IsNNxw3mDBItPFAYu5KongsWJhBTgmUhJoyE34AEMoYQwWz3k8Mg8As3cFP4RsVfEEfDgCLIkZBZHF3s9lte5bGkc5oZY4I/dEO6MXtzo6Q15ZDOtCNmTSUBW8tfFzOy8dlfzbUcty6wc3TmZWcMb/dqjegxJhly/q5ek74yAVoZ5IO0GJZVHN8h/OafErHtu/UK20/1y3J9Uk/1lNcl41yUI1mKlw7vI0eHfY7nhdrxmwGvjToIOmY3fY/ozDlBaSKUMF0brve5kZlCQ3nApWtXChWUbomMSsb0NFJNODsnkEFdyzSAhRmttPGWjQ+YySSF1btMrakV7mavA0rl+YaHtQcpUVhil6fFBUCDAp1C8KQp4zasTEBoTm3NYKdETsxTD23bVsE/xly6vB3lbX97r+y0edne1pO9bRHXQXbSIfPUY76BnaRT1EnY/OV+e788P95H5zf7q/jqXO2jTnNloY7u8/+ntPzQ==</latexit>
x はどの山から出てきたかなぁ

事後確率の解釈
43/64
半々ぐらい?
<latexit sha1_base64="QcdtIHAmrt9nLv0/UuSW6bCopf0=">AAACFnicbVDLSgMxFM3UVx1fVZdugkXoQB1mimI3QsGNywr2AW0pmTTThiaZIckIdehXuPFX3LhQxK24829MH4vaeiDh5Jx7ubkniBlV2vN+rMza+sbmVnbb3tnd2z/IHR7VVZRITGo4YpFsBkgRRgWpaaoZacaSIB4w0giGNxO/8UCkopG416OYdDjqCxpSjLSRurnzQruPOEeFx24q/LFThAvv0thx7GtY8NzLIjSX083lPdebAq4Sf07yYI5qN/fd7kU44URozJBSLd+LdSdFUlPMyNhuJ4rECA9Rn7QMFYgT1Umna43hmVF6MIykOULDqbrYkSKu1IgHppIjPVDL3kT8z2slOix3UiriRBOBZ4PChEEdwUlGsEclwZqNDEFYUvNXiAdIIqxNkrYJwV9eeZXUS67vuf7dRb5SnseRBSfgFBSAD65ABdyCKqgBDJ7AC3gD79az9Wp9WJ+z0ow17zkGf2B9/QINn5tr</latexit>

サンプリング
44/64
<latexit sha1_base64="VB3yvm5HjAt6npfPShUceHUXqmE=">AAACFHicbVDLSgMxFM3UV62vUZdugkWoCGVGBLssdOOygn1AZyiZNNOGJpkhyQh1mI9w46+4caGIWxfu/Bsz7Sy09ULI4Zx7ueeeIGZUacf5tkpr6xubW+Xtys7u3v6BfXjUVVEiMengiEWyHyBFGBWko6lmpB9LgnjASC+YtnK9d0+kopG407OY+ByNBQ0pRtpQQ/si9YIQPmTQU5RDjyM9kTxtIZ3VjBKxkZpx83kxzc6HdtWpO/OCq8AtQBUU1R7aX94owgknQmOGlBq4Tqz9FElNMSNZxUsUiRGeojEZGCgQJ8pP50dl8MwwIxhG0jyh4Zz9PZEirnJzpjN3rZa1nPxPGyQ6bPgpFXGiicCLRWHCoI5gnhAcUUmwZjMDEJbUeIV4giTC2uRYMSG4yyevgu5l3XXq7u1Vtdko4iiDE3AKasAF16AJbkAbdAAGj+AZvII368l6sd6tj0VrySpmjsGfsj5/AEl3nvA=</latexit>
<latexit sha1_base64="srb+AvEf8d+1lVFAP+3vJU0ZLIA=">AAACW3icbVFdaxQxFM1Mq13XqmvFJ1+Ci1BBlpki2Beh0BehIBXdtrCZDplMZjdMMhmSO9Jtmj/pU33wr4iZ7T70wwvhnpxzL7n3pGilsJAkv6N4Y/PR463Bk+HT7WfPX4xe7pxY3RnGp0xLbc4KarkUDZ+CAMnPWsOpKiQ/LerDXj/9yY0VuvkBy5Znis4bUQlGIVD5yDhSVPjCX63ypcfECoVJa3SZu/pz6s/dkSeSVzAjrchroigsGJXuq9+91aplaZcqJKI6n9cfcK+R72KuaLi+J0bMF5Cdu8u89vlonEySVeCHIF2DMVrHcT76RUrNOsUbYJJaO0uTFjJHDQgmuR+SzvKWsprO+SzAhipuM7fyxuN3gSlxpU04DeAVe7vDUWX70UNlv5q9r/Xk/7RZB9V+5kTTdsAbdvNQ1UkMGvdG41IYzkAuA6DMiDArZgtqKIPwHcNgQnp/5YfgZG+SJpP028fxwf7ajgF6g96iXZSiT+gAfUHHaIoYukZ/o61oEP2JN+JhvH1TGkfrnlfoTsSv/wG4Xbep</latexit>
どの山かを選ぶ
選んだ山からサンプリング
伝承サンプリング
<latexit sha1_base64="ml0OX7Q25RsKEjsP4ItxwC/tez4=">AAACF3icbZBLS8NAEMc3Pmt9RT16WSxCvYREBHsRCl48VrAPaErYbDft0s2D3YlYY7+FF7+KFw+KeNWb38ZNm4O2Duzy4z8zzMzfTwRXYNvfxtLyyuraemmjvLm1vbNr7u23VJxKypo0FrHs+EQxwSPWBA6CdRLJSOgL1vZHl3m+fcuk4nF0A+OE9UIyiHjAKQEteaaVYFewAKrYDQkM/QBn+A5P8AO+x55mR/OF/l3JB0M48cyKbdnTwIvgFFBBRTQ888vtxzQNWQRUEKW6jp1ALyMSOBVsUnZTxRJCR2TAuhojEjLVy6Z3TfCxVvo4iKV+EeCp+rsjI6FS49DXlfnyaj6Xi//luikEtV7GoyQFFtHZoCAVGGKcm4T7XDIKYqyBUMn1rpgOiSQUtJVlbYIzf/IitE4tx7ac67NKvVbYUUKH6AhVkYPOUR1doQZqIooe0TN6RW/Gk/FivBsfs9Ilo+g5QH/C+PwB9BGcmQ==</latexit> <latexit sha1_base64="lkGl+2ZNtKtaNqk0MTtl+B+PVmY=">AAACF3icbZDJSgNBEIZ74hbjFvXopTAI8RJmgmAuQsCLxwhmgUwIPZ2epEnPQneNGMe8hRdfxYsHRbzqzbexsxw08YeGj7+q6Krfi6XQaNvfVmZldW19I7uZ29re2d3L7x80dJQoxusskpFqeVRzKUJeR4GSt2LFaeBJ3vSGl5N685YrLaLwBkcx7wS0HwpfMIrG6uZLMbiS+1gEN6A48HxI4Q7G8AD30DVcNnwBDrhK9Ad42s0X7JI9FSyDM4cCmavWzX+5vYglAQ+RSap127Fj7KRUoWCSj3NuonlM2ZD2edtgSAOuO+n0rjGcGKcHfqTMCxGm7u+JlAZajwLPdE6W14u1iflfrZ2gX+mkIowT5CGbfeQnEjCCSUjQE4ozlCMDlClhdgU2oIoyNFHmTAjO4snL0CiXHLvkXJ8VqpV5HFlyRI5JkTjknFTJFamROmHkkTyTV/JmPVkv1rv1MWvNWPOZQ/JH1ucP9aOcmg==</latexit>
<latexit sha1_base64="1otBhJp8gZhe0FuDMhoDSunfljY=">AAACF3icbZDJSgNBEIZ74hbjFvXopTAI8RJmVDAXIeDFYwSzQCaEnk5P0qRnobtGjGPewouv4sWDIl715tvYWQ6a+EPDx19VdNXvxVJotO1vK7O0vLK6ll3PbWxube/kd/fqOkoU4zUWyUg1Paq5FCGvoUDJm7HiNPAkb3iDy3G9ccuVFlF4g8OYtwPaC4UvGEVjdfKlGFzJfSyCG1Dsez6kcAcjeIB76Bg+NXwBDrhK9Pp43MkX7JI9ESyCM4MCmanayX+53YglAQ+RSap1y7FjbKdUoWCSj3JuonlM2YD2eMtgSAOu2+nkrhEcGacLfqTMCxEm7u+JlAZaDwPPdI6X1/O1sflfrZWgX26nIowT5CGbfuQnEjCCcUjQFYozlEMDlClhdgXWp4oyNFHmTAjO/MmLUD8pOXbJuT4rVMqzOLLkgBySInHIOamQK1IlNcLII3kmr+TNerJerHfrY9qasWYz++SPrM8f9zWcmw==</latexit>

9.2.1 Maximum likelihood
45/64

データセットに混合ガウス分布をあてはめる = 最尤推定
混合ガウスモデル
46/64
<latexit sha1_base64="ZpLbbYXAQTdmymYNMJs/XgH78sI=">AAAB/nicbVDJSgNBEK1xjXEbFU9eGoPgKcyIYC5CwIsniWAWyIxDT6cnadKz0N0jhmbAX/HiQRGvfoc3/8bOctDEBwWP96qoqhdmnEnlON/W0vLK6tp6aaO8ubW9s2vv7bdkmgtCmyTlqeiEWFLOEtpUTHHayQTFcchpOxxejf32AxWSpcmdGmXUj3E/YREjWBkpsA89rb0wQo9FwLwi0OzSLe5vArviVJ0J0CJxZ6QCMzQC+8vrpSSPaaIIx1J2XSdTvsZCMcJpUfZySTNMhrhPu4YmOKbS15PzC3RilB6KUmEqUWii/p7QOJZyFIemM8ZqIOe9sfif181VVPM1S7Jc0YRMF0U5RypF4yxQjwlKFB8Zgolg5lZEBlhgokxiZROCO//yImmdVV2n6t6eV+q1WRwlOIJjOAUXLqAO19CAJhDQ8Ayv8GY9WS/Wu/UxbV2yZjMH8AfW5w8FtZV5</latexit>
<latexit sha1_base64="weatXJaPVnAZ+Ry5kquvRN1j/Os=">AAACdHicbVFNa9wwEJXdj6TuR7btoYf2MO0SSC+LXQoJhUKgPfQUUsgmC6vtIsvjXRFZNtI4dDH+Bf13vfVn9NJzZceBNOmA4Om9NzOaUVpp5SiOfwXhnbv37m9tP4gePnr8ZGf09NmpK2srcSpLXdpZKhxqZXBKijTOKouiSDWepeefOv3sAq1TpTmhTYWLQqyMypUU5Knl6EfD0xxmLXyMuMac9iIAnuJKmUZYKzZtI1tPAfS+7+0y+dbwQtDaFnDSAue9yC+yktzV7cp6dN3a1UWTDVUjbtVqTW+jD3AEnFSBDj4vR+N4EvcBt0EygDEb4ng5+smzUtYFGpJaODdP4ooWvgUpqdE3qR1WQp6LFc49NMK3WTT90lrY9UwGeWn9MQQ9ez2jEYVzmyL1zm4Id1PryP9p85ryg0WjTFUTGnnZKK81UAndD0CmLErSGw+EtMq/FeRaWCHJ/1Pkl5DcHPk2OH03SeJJ8vX9+PBgWMc2e8nesD2WsH12yL6wYzZlkv0OXgQQvA7+hK/Ccbh7aQ2DIec5+yfCyV8lnLiP</latexit>
<latexit sha1_base64="h6i9QCZz8+YCN6ItUrUCNBiP64Q=">AAACdXicbVFda9RAFJ3Erxq/Vn0RRLh2tdSXNRHBIggFXwShVOi2xZ11mUxudodOJmHmprCG/AN/nW/+DV98dZKmUFsvDJw559x7595JK60cxfGvILx2/cbNWxu3ozt3791/MHr46NCVtZU4laUu7XEqHGplcEqKNB5XFkWRajxKTz52+tEpWqdKc0DrCueFWBqVKynIU4vRj4anOXxto60PEdeY03YEwFNcKtMIa8W6bWTrKYDe+L1dJN8aXgha2QIOWuC8F/lpVpI7v51b9y5au7posqFqxK1aruhV9B72gJMq0MHnxWgcT+I+4CpIBjBmQ+wvRj95Vsq6QENSC+dmSVzR3LcgJTX6JrXDSsgTscSZh0b4NvOm31oLLz2TQV5afwxBz17MaETh3LpIvbMbwl3WOvJ/2qymfGfeKFPVhEaeNcprDVRC9wWQKYuS9NoDIa3ybwW5ElZI8h8V+SUkl0e+Cg7fTJJ4knx5O97dGdaxwZ6yTbbNEvaO7bJPbJ9NmWS/gyfB82Az+BM+C1+EW2fWMBhyHrN/Inz9F4SfuLY=</latexit>
グラフィカルモデル

最尤推定の問題
47/64
対数尤度関数
<latexit sha1_base64="rD00sbKgold7KLItH+81Wfymn3w=">AAADBnicbZLfi9NAEMc38ddZf/X0UR8Gi3AHUhIRvBfhwBdBkBPtXaFby2a7aZfubsLuRiwxT774r/jigyK++jf45n/jpIm19joQZvKZmXxnNpvkSjofRb+D8MLFS5ev7F3tXLt+4+at7v7tU5cVlosBz1RmhwlzQkkjBl56JYa5FUwnSpwli2d1/uydsE5m5o1f5mKs2czIVHLmEU32g3tUGcjhAKhmfp6kUMIQKvgANMnU1C01OmQ0l0gfblNdNHTdS1/LmWYID+EpUFdomCA1+BIjfIvxS/S1JlUi9bT8V7TYKHpRF6Fkw6tGgDO17sfezZHfI2yEdoy+GvLvl3YOu5axcjb3h62n1aTbi/rRyuB8ELdBj7R2Mun+otOMF1oYzxVzbhRHuR+XzHrJlag6tHAiZ3zBZmKEoWFauHG5+o0VPEAyhTSz+BgPK7rZUTLt6q2wsl7BbedquCs3Knx6NC6lyQsvDG+E0kKBz6C+EzCVVnCvlhgwbiXOCnzOLOMeb04HDyHeXvl8cPqoH0f9+NXj3vFRexx75C65Tw5ITJ6QY/KcnJAB4cHH4HPwNfgWfgq/hN/DH01pGLQ9d8h/Fv78A1o66PA=</latexit>
「特異性の存在に起因する重要な問題」

特異性の問題
48/64
仮定: <latexit sha1_base64="zciiC/jjKj4PpJJNM+lUjxZbOEY=">AAACHHicbVBNS8NAEN34WetX1KOXxSJ4KokK9iIUvHisYD+gKWGz2bRrN7thdyOWkB/ixb/ixYMiXjwI/hs3bQ7a+mCZN29m2JkXJIwq7Tjf1tLyyuraemWjurm1vbNr7+13lEglJm0smJC9ACnCKCdtTTUjvUQSFAeMdIPxVVHv3hOpqOC3epKQQYyGnEYUI20k3z7zAsFCNYlNgBn04hTm0DfszsRLkyM9CiKTP5Q6h7lv15y6MwVcJG5JaqBEy7c/vVDgNCZcY4aU6rtOogcZkppiRvKqlyqSIDxGQ9I3lKOYqEE2PS6Hx0YJYSSkeVzDqfp7IkOxKvY3ncWyar5WiP/V+qmOGoOM8iTVhOPZR1HKoBawcAqGVBKs2cQQhCU1u0I8QhJhbfysGhPc+ZMXSee07jp19+a81myUdlTAITgCJ8AFF6AJrkELtAEGj+AZvII368l6sd6tj1nrklXOHIA/sL5+ABTqn2k=</latexit>
<latexit sha1_base64="9rIHuNY282+Ke2ZQVD03ttTttNQ=">AAACLHicbVBNS8NAEN3Ur1q/oh69LBbBU0mKYC9CoRe9VbQf0MSw2W7apZtN2N0IJeQHefGvCOLBIl79HW7aINr6YJm3b2aYmefHjEplWTOjtLa+sblV3q7s7O7tH5iHR10ZJQKTDo5YJPo+koRRTjqKKkb6sSAo9Bnp+ZNWnu89EiFpxO/VNCZuiEacBhQjpSXPbKWOH7GhnIY6QOeOjkKUQQ+mcAIzeAUdmSs/woOOdR2dEKmxH+jfDcw8s2rVrDngKrELUgUF2p756gwjnISEK8yQlAPbipWbIqEoZiSrOIkkMcITNCIDTTkKiXTT+bEZPNPKEAaR0I8rOFd/d6QolPk5ujJfUi7ncvG/3CBRQcNNKY8TRTheDAoSBlUEc+fgkAqCFZtqgrCgeleIx0ggrLS/FW2CvXzyKunWa7ZVs28vqs1GYUcZnIBTcA5scAma4Bq0QQdg8ARewDuYGc/Gm/FhfC5KS0bRcwz+wPj6BpJnpTw=</latexit>
簡単のため:
<latexit sha1_base64="zciiC/jjKj4PpJJNM+lUjxZbOEY=">AAACHHicbVBNS8NAEN34WetX1KOXxSJ4KokK9iIUvHisYD+gKWGz2bRrN7thdyOWkB/ixb/ixYMiXjwI/hs3bQ7a+mCZN29m2JkXJIwq7Tjf1tLyyuraemWjurm1vbNr7+13lEglJm0smJC9ACnCKCdtTTUjvUQSFAeMdIPxVVHv3hOpqOC3epKQQYyGnEYUI20k3z7zAsFCNYlNgBn04hTm0DfszsRLkyM9CiKTP5Q6h7lv15y6MwVcJG5JaqBEy7c/vVDgNCZcY4aU6rtOogcZkppiRvKqlyqSIDxGQ9I3lKOYqEE2PS6Hx0YJYSSkeVzDqfp7IkOxKvY3ncWyar5WiP/V+qmOGoOM8iTVhOPZR1HKoBawcAqGVBKs2cQQhCU1u0I8QhJhbfysGhPc+ZMXSee07jp19+a81myUdlTAITgCJ8AFF6AJrkELtAEGj+AZvII368l6sd6tj1nrklXOHIA/sL5+ABTqn2k=</latexit>
<latexit sha1_base64="bYftCTwMFI+DqSfO10FcAXiUKK4=">AAACX3icbZHfS8MwEMfT+mvOX1WfxJfgEBRktCLo48AXfREFp8I6S5qlW1jSluQqjrp/0jfBF/8Tr9sQfx2E++Zzl+TuEudKWvD9N8edm19YXKot11dW19Y3vM2tO5sVhos2z1RmHmJmhZKpaIMEJR5yI5iOlbiPh+dV/P5JGCuz9BZGuehq1k9lIjkDRJH3FOYyGtJQMxhwpmhJr+iYhkokcDClcYLwGeELDeNM9exIo0MW6gJphGqI/oiGVvY1+wKP6I+ru74uuax2RvYHcBh5Db/pT4z+FcFMNMjMriPvNexlvNAiBa6YtZ3Az6FbMgOSKzGuh4UVOeND1hcdlCnTwnbLyXzGdB9JjyaZwZUCndDvJ0qmbdUWZlbF2t+xCv4X6xSQnHVLmeYFiJRPH0oKRSGj1bBpTxrBQY1QMG4k1kr5gBnGAb+kjkMIfrf8V9wdNwO/GdycNFpns3HUyC7ZIwckIKekRS7INWkTTt4d11lxVp0Pd8ldd71pquvMzmyTH+bufAJ5MbDo</latexit>
<latexit sha1_base64="j+dTbE3CN7/FnFu7xiGwkRzid/s=">AAACX3icbZHfS8MwEMfT+nv+qvokvgSHoCCjFUEfB77oiyg4FdZZ0izdoklbkutw1P2Tvgm++J943Yb46yDcN5+7JHeXOFfSgu+/Oe7M7Nz8wuJSbXlldW3d29i8tVlhuGjxTGXmPmZWKJmKFkhQ4j43gulYibv46ayK3w2EsTJLb2CYi45mvVQmkjNAFHmDMJfRIw01gz5nipb0ko5oqEQC+xMaJwifEb7QMM5U1w41OmShLpBGqB7RH9LQyp5mX+AB/VF119clF9XOyF4fDiKv7jf8sdG/IpiKOpnaVeS9ht2MF1qkwBWzth34OXRKZkByJUa1sLAiZ/yJ9UQbZcq0sJ1yPJ8R3UPSpUlmcKVAx/T7iZJpW7WFmVWx9nesgv/F2gUkp51SpnkBIuWTh5JCUchoNWzalUZwUEMUjBuJtVLeZ4ZxwC+p4RCC3y3/FbdHjcBvBNfH9ebpdByLZIfskn0SkBPSJOfkirQIJ++O6yw7K86Hu+Cuud4k1XWmZ7bID3O3PwFz47Dl</latexit>
対数尤度関数
xn が要素 j を通して、尤度関数に寄与する項
<latexit sha1_base64="sPzbj+CSKT763GbXCrkdWaOJUXU=">AAACM3icbVDLSgMxFM34rPVVdekmWARBKDMi6FJwI66q2Cp0SsmkmTY2jyG5owxD/8mNP+JCEBeKuPUfTNsBnwcCJ+fcy733RIngFnz/yZuanpmdmy8tlBeXlldWK2vrTatTQ1mDaqHNVUQsE1yxBnAQ7CoxjMhIsMtocDzyL2+YsVyrC8gS1pakp3jMKQEndSqnoeU9STrXODS81wdijL7FPg7Pv36hJNA3Mhd84Ob0te4Of1TvhlzFkHUqVb/mj4H/kqAgVVSg3qk8hF1NU8kUUEGsbQV+Au2cGOBUsGE5TC1LCB2QHms5qohktp2Pbx7ibad0cayNewrwWP3ekRNpbSYjVzla3/72RuJ/XiuF+LCdc5WkwBSdDIpTgUHjUYC4yw2jIDJHCDXc7YppnxhCwcVcdiEEv0/+S5p7tcCvBWf71aPDIo4S2kRbaAcF6AAdoRNURw1E0R16RC/o1bv3nr03731SOuUVPRvoB7yPT8IUrCw=</latexit>
対数尤度関数の最大化問題は不良設定問題
<latexit sha1_base64="rB6cboDVq7ofBIBm+/oygiJxL0M=">AAACq3icdZFRb9MwEMedMNgoAzp45OVEhdRJUJJp0vaCNGk8wMu0iXWt1pTKcS+tmeNEtoNWhXw5PgJvfJtd0mraOjjJun9+97/YPse5ktYFwV/Pf7Tx+Mnm1tPWs+3nL162d15d2KwwAvsiU5kZxtyikhr7TjqFw9wgT2OFg/jquK4PfqKxMtPnbpHjOOUzLRMpuCM0af+OUu7mgiso4QQqiBQmrgsNjROC1wQnlDXlX//h7yGycpbyBvwg8J3yXv23W/vX+svI2dztwieIEsMF0ZBoCV3yRrmE3abvM3xseqs11/0tymrpJd+k3Ql6QRPwUIQr0WGrOJ20/0TTTBQpaicUt3YUBrkbl9w4KRRWraiwmHNxxWc4Iql5inZcNrOu4B2RKSSZoaUdNPRuR8lTaxdpTM768na9VsN/1UaFSw7HpdR54VCL5UZJocBlUD8cTKVB4dSCBBdG0llBzDlNyNHztmgI4fqVH4qLvV4Y9MKz/c7R4WocW+wNe8u6LGQH7Ih9Yaesz4TX9U68gTf0P/jf/Es/Wlp9b9Xzmt0LH28AIHzGQQ==</latexit>

特異性の問題多峰
49/64
<latexit sha1_base64="BQbEqJULUP3tMnmeKMvhGwfP6gs=">AAADNXiclVJNaxQxGM6MX3X96FaPXoKLsouyTKaCvRQKXoRCacFtC5t1yGQzu2GSzJBkxGWcP9VL/4cnPXhQxKt/ocl0Fbe1gi8MPHmfj3nzkZaCGxtFn4Pw2vUbN2+t3e7cuXvv/np348GhKSpN2YgWotDHKTFMcMVGllvBjkvNiEwFO0rzV54/ese04YV6Yxclm0gyUzzjlFjXSjaCXVzqYprUahs1b/dg2X+fqA+4xiVPctw8hw7K6jc0fCaJXw06T7fhqhWbSiZ17he7sPU7K7FzSgTca5LcJw8wbo2CZbbvNWhFg5wGDeCz1h6vUHFLYc1nc9umYMslM/+Miq+Oiv8zavPqqM1fUZ2k24uGUVvwMkBL0APL2k+6H/G0oJVkylJBjBmjqLSTmmjLqWBNB1eGlYTmZMbGDirixpzU7a038InrTGFWaPcpC9vun46aSGMWMnVKP7G5yPnm37hxZbOtSc1VWVmm6PmPskpAW0D/hOCUa0atWDhAqOZuVkjnRBNq3UPzh4AubvkyOIyHKBqigxe9na3lcayBR+Ax6AMEXoId8BrsgxGgwUnwKfgafAtPwy/h9/DHuTQMlp6HYKXCn2fRTgFG</latexit>
<latexit sha1_base64="eh8c+zM5DHI0czLM5Arijw/9IdY=">AAAB8nicbVBNSwMxEJ31s9avqkcvwSJ4KhsR7EUoePFYwX5AuyzZNNuGJpslyYpl6c/w4kERr/4ab/4b03YP2vpg4PHeDDPzolRwY33/21tb39jc2i7tlHf39g8OK0fHbaMyTVmLKqF0NyKGCZ6wluVWsG6qGZGRYJ1ofDvzO49MG66SBztJWSDJMOExp8Q6qdeXWYjRDXoKcVip+jV/DrRKcEGqUKAZVr76A0UzyRJLBTGmh/3UBjnRllPBpuV+ZlhK6JgMWc/RhEhmgnx+8hSdO2WAYqVdJRbN1d8TOZHGTGTkOiWxI7PszcT/vF5m43qQ8yTNLEvoYlGcCWQVmv2PBlwzasXEEUI1d7ciOiKaUOtSKrsQ8PLLq6R9WcN+Dd9fVRv1Io4SnMIZXACGa2jAHTShBRQUPMMrvHnWe/HevY9F65pXzJzAH3ifP7rWkDE=</latexit>
<latexit sha1_base64="cmfK0GdlBz58vyfA3fcJhwOjS78=">AAACHnicbVDLSsNAFJ3UV62vqks3g0VoQUpSFLsRCm5cSQX7gCaGyXTaDplJwsxEWmK/xI2/4saFIoIr/RsnbRe19cCFwzn3cu89XsSoVKb5Y2RWVtfWN7Kbua3tnd29/P5BU4axwKSBQxaKtockYTQgDUUVI+1IEMQ9Rlqef5X6rQciJA2DOzWKiMNRP6A9ipHSkps/T2yO1AAjBm/Grl8cluAlnNeKw0ebx65/Cm1J+xy5/n2l5OYLZtmcAC4Ta0YKYIa6m/+yuyGOOQkUZkjKjmVGykmQUBQzMs7ZsSQRwj7qk46mAeJEOsnkvTE80UoX9kKhK1Bwos5PJIhLOeKe7kzvloteKv7ndWLVqzoJDaJYkQBPF/ViBlUI06xglwqCFRtpgrCg+laIB0ggrHSiOR2CtfjyMmlWypZZtm7PCrXqLI4sOALHoAgscAFq4BrUQQNg8ARewBt4N56NV+PD+Jy2ZozZzCH4A+P7F2dvoVs=</latexit>
<latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit>
<latexit sha1_base64="A04I5Eb8yKzMoGJpxklIYyl10zM=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oWy2m3btZhN2J0II/Q9ePCji1f/jzX/jts1BWx8MPN6bYWZekEhh0HW/ndLG5tb2Tnm3srd/cHhUPT7pmDjVjLdZLGPdC6jhUijeRoGS9xLNaRRI3g2mt3O/+8S1EbF6wCzhfkTHSoSCUbRSZyBUiNmwWnPr7gJknXgFqUGB1rD6NRjFLI24QiapMX3PTdDPqUbBJJ9VBqnhCWVTOuZ9SxWNuPHzxbUzcmGVEQljbUshWai/J3IaGZNFge2MKE7MqjcX//P6KYYNPxcqSZErtlwUppJgTOavk5HQnKHMLKFMC3srYROqKUMbUMWG4K2+vE46V3XPrXv317Vmo4ijDGdwDpfgwQ004Q5a0AYGj/AMr/DmxM6L8+58LFtLTjFzCn/gfP4Avy2PMg==</latexit>
<latexit sha1_base64="SnGAWWDvcKOm5XCgld8TjcUaeUk=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RS2tnd294r7pYPDo+OT8ulZR8epYthmsYhVL6AaBZfYNtwI7CUKaRQI7AbTu4XffUKleSwfzCxBP6JjyUPOqLFSyx2WK27VXYJsEi8nFcjRHJa/BqOYpRFKwwTVuu+5ifEzqgxnAuelQaoxoWxKx9i3VNIItZ8tD52TK6uMSBgrW9KQpfp7IqOR1rMosJ0RNRO97i3E/7x+asK6n3GZpAYlWy0KU0FMTBZfkxFXyIyYWUKZ4vZWwiZUUWZsNiUbgrf+8ibp3FQ9t+q1biuNeh5HES7gEq7Bgxo04B6a0AYGCM/wCm/Oo/PivDsfq9aCk8+cwx84nz92F4yq</latexit>
<latexit sha1_base64="KKVjORscqYSmMgHzjTJVy0AGgx0=">AAAB73icbVBNSwMxEJ31s9avqkcvwSJ4kLJbBXspFLwIglSwH9AuJZtm29AkuyZZoSz9E148KOLVv+PNf2Pa7kFbHww83pthZl4Qc6aN6347K6tr6xubua389s7u3n7h4LCpo0QR2iARj1Q7wJpyJmnDMMNpO1YUi4DTVjC6nvqtJ6o0i+SDGcfUF3ggWcgINlZq31UvztFttdwrFN2SOwNaJl5GipCh3it8dfsRSQSVhnCsdcdzY+OnWBlGOJ3ku4mmMSYjPKAdSyUWVPvp7N4JOrVKH4WRsiUNmqm/J1IstB6LwHYKbIZ60ZuK/3mdxIQVP2UyTgyVZL4oTDgyEZo+j/pMUWL42BJMFLO3IjLEChNjI8rbELzFl5dJs1zy3JJ3f1msVbI4cnAMJ3AGHlxBDW6gDg0gwOEZXuHNeXRenHfnY9664mQzR/AHzucP42yOhA==</latexit>
<latexit sha1_base64="NwBJyUhmHDf58wUm5F7vQUNLdqQ=">AAAB+XicbVDLSsNAFL2pr1pfUZduBotQNyURwS4LblxJBfuANoTJdNIOnUzCzKRYQv/EjQtF3Pon7vwbJ20W2npg4HDOvdwzJ0g4U9pxvq3SxubW9k55t7K3f3B4ZB+fdFScSkLbJOax7AVYUc4EbWumOe0lkuIo4LQbTG5zvzulUrFYPOpZQr0IjwQLGcHaSL5tDyKsxwTz7H7uu7WnS9+uOnVnAbRO3IJUoUDLt78Gw5ikERWacKxU33US7WVYakY4nVcGqaIJJhM8on1DBY6o8rJF8jm6MMoQhbE0T2i0UH9vZDhSahYFZjLPqVa9XPzP66c6bHgZE0mqqSDLQ2HKkY5RXgMaMkmJ5jNDMJHMZEVkjCUm2pRVMSW4q19eJ52ruuvU3YfrarNR1FGGMziHGrhwA024gxa0gcAUnuEV3qzMerHerY/laMkqdk7hD6zPH9YhkxY=</latexit>
<latexit sha1_base64="9/OgcykHFv5ZPpPqd1gPxIumm34=">AAAB+XicbVBNS8NAFHypX7V+RT16WSxCvZSkCPZY8OJJKthWaEPYbDft0s0m7G6KJfSfePGgiFf/iTf/jZs2B20dWBhm3uPNTpBwprTjfFuljc2t7Z3ybmVv/+DwyD4+6ao4lYR2SMxj+RhgRTkTtKOZ5vQxkRRHAae9YHKT+70plYrF4kHPEupFeCRYyAjWRvJtexBhPSaYZ3dzv1F7uvTtqlN3FkDrxC1IFQq0fftrMIxJGlGhCcdK9V0n0V6GpWaE03llkCqaYDLBI9o3VOCIKi9bJJ+jC6MMURhL84RGC/X3RoYjpWZRYCbznGrVy8X/vH6qw6aXMZGkmgqyPBSmHOkY5TWgIZOUaD4zBBPJTFZExlhiok1ZFVOCu/rlddJt1F2n7t5fVVvNoo4ynME51MCFa2jBLbShAwSm8Ayv8GZl1ov1bn0sR0tWsXMKf2B9/gDXqJMX</latexit>
<latexit sha1_base64="0EHMyjGEJshkwquXOlKhz7W/UvA=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaI8FLx4r2g9oQ9lsJ+3SzSbsbsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJzdzvPKLSPJYPZpqgH9GR5CFn1Fjp/mngDcoVt+ouQNaJl5MK5GgOyl/9YczSCKVhgmrd89zE+BlVhjOBs1I/1ZhQNqEj7FkqaYTazxanzsiFVYYkjJUtachC/T2R0UjraRTYzoiasV715uJ/Xi81Yd3PuExSg5ItF4WpICYm87/JkCtkRkwtoUxxeythY6ooMzadkg3BW315nbSvqp5b9e6uK416HkcRzuAcLsGDGjTgFprQAgYjeIZXeHOE8+K8Ox/L1oKTz5zCHzifPwiujZY=</latexit><latexit sha1_base64="MaXZUVBkUhMfeY7HmcXyUcRFwUU=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0mKYI8FLx4r2g9oQ9lsN+3SzSbsTsQS+hO8eFDEq7/Im//GbZuDtj4YeLw3w8y8IJHCoOt+O4WNza3tneJuaW//4PCofHzSNnGqGW+xWMa6G1DDpVC8hQIl7yaa0yiQvBNMbuZ+55FrI2L1gNOE+xEdKREKRtFK90+D2qBccavuAmSdeDmpQI7moPzVH8YsjbhCJqkxPc9N0M+oRsEkn5X6qeEJZRM64j1LFY248bPFqTNyYZUhCWNtSyFZqL8nMhoZM40C2xlRHJtVby7+5/VSDOt+JlSSIldsuShMJcGYzP8mQ6E5Qzm1hDIt7K2EjammDG06JRuCt/ryOmnXqp5b9e6uKo16HkcRzuAcLsGDa2jALTShBQxG8Ayv8OZI58V5dz6WrQUnnzmFP3A+fwAKMo2X</latexit><latexit sha1_base64="YxcEA9GPlX8c2FerastqxXH4G0M=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lUsMeCF48V7Qe0oWy2m3bpZhN2J2IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//ci1EbF6wEnC/YgOlQgFo2il+6f+Zb9ccavuHGSVeDmpQI5Gv/zVG8QsjbhCJqkxXc9N0M+oRsEkn5Z6qeEJZWM65F1LFY248bP5qVNyZpUBCWNtSyGZq78nMhoZM4kC2xlRHJllbyb+53VTDGt+JlSSIldssShMJcGYzP4mA6E5QzmxhDIt7K2EjaimDG06JRuCt/zyKmldVD236t1dVeq1PI4inMApnIMH11CHW2hAExgM4Rle4c2Rzovz7nwsWgtOPnMMf+B8/gALto2Y</latexit>
何かしら有限値を持っておけば，
尤度は無限に大きくなる

特異性の問題単峰
50/64
<latexit sha1_base64="eHXLnTYfswK1fjzfHySsDVHhyoE=">AAAEE3iclZPLbtNAFIanNpcSbiks2RwRgRKJtraFRDeVKrUL2ERFIm2lOLHGk3Ey6viimTFqNPU7sOFV2LAAIbZs2PE2jC80JSmVGMny8Tn/mfnOL0+YcSaV4/xas+wbN2/dXr/Tunvv/oOH7Y1HRzLNBaEDkvJUnIRYUs4SOlBMcXqSCYrjkNPj8HS/rB+/p0KyNHmn5hkdxXiasIgRrEwq2LB6fibSSaCTXbcY634BfozVjGAOGvpgPjmNVLfOhhHosyJI4Bz8MOUTOY/Nywj9ODfSF+BLNo0xjE3Kg2LRBG/KL8GmM9VrPd+FlTMjgYmRuUamoWua/YxBr9roALarzZZV9Vm1oqrSs6yh3ayU2i2016i8wj/Xf3DKETb1pQkMvqmPvYbQ9yvG65j6C6jlURq/rmLVXrHwYaxrygNDWfwDf0F/MdD/DlLyZSpd5QypwtcxlHXYD5KLvYJ2x9lyqgWrgdsEHdSsw6D905+kJI9pogjHUg5dJ1MjjYVihNOi5eeSZpic4ikdmjDBMZUjXf3TBTwzmQlEqTBPoqDKXu7QOJbl0EZZuiGXa2XyqtowV9HOSLMkyxVNSH1QlHMwHpUXBCZMUKL43ASYCGZYgcywcUmZa9QyJrjLI68GR96W62y5b1929nYaO9bRE/QUdZGLXqE99BodogEi1gfrk/XF+mp/tD/b3+zvtdRaa3oeo7+W/eM3oHlNWQ==</latexit>
となる因子:
<latexit sha1_base64="v+BpmqxZdRO5sKgmyrCzFj0y8sU=">AAACA3icbVBNS8NAEN3Ur1q/ot70slgETyUpgr0IBS8eK9hWaGLZbDft0t0k7E6EEgJe/CtePCji1T/hzX/jts1BWx8MPN6bYWZekAiuwXG+rdLK6tr6RnmzsrW9s7tn7x90dJwqyto0FrG6C4hmgkesDRwEu0sUIzIQrBuMr6Z+94EpzePoFiYJ8yUZRjzklICR+vaRFzAg+BJ7oSI0c/PM03woyX0979tVp+bMgJeJW5AqKtDq21/eIKapZBFQQbTuuU4CfkYUcCpYXvFSzRJCx2TIeoZGRDLtZ7MfcnxqlAEOY2UqAjxTf09kRGo9kYHplARGetGbiv95vRTChp/xKEmBRXS+KEwFhhhPA8EDrhgFMTGEUMXNrZiOiAkDTGwVE4K7+PIy6dRrrlNzb86rzUYRRxkdoxN0hlx0gZroGrVQG1H0iJ7RK3qznqwX6936mLeWrGLmEP2B9fkDLg+XMA==</latexit>
<latexit sha1_base64="p/feQ33FdunSz1M0G5HzgNpfjsM=">AAACHXicbVDLSsNAFJ3UV62vqks3g0VwY0lKwW6EQjcuK9gHNDFMJpN26GQSZiZiSfMjbvwVNy4UceFG/BunbRbaemCYwzn3cu89XsyoVKb5bRTW1jc2t4rbpZ3dvf2D8uFRV0aJwKSDIxaJvockYZSTjqKKkX4sCAo9RnreuDXze/dESBrxWzWJiROiIacBxUhpyS3XWy6HV9AOBMKplaW1DNrT1PYC+JBp5wJqHjFfTkL92WGS2dO7mluumFVzDrhKrJxUQI62W/60/QgnIeEKMyTlwDJj5aRIKIoZyUp2IkmM8BgNyUBTjkIinXR+XQbPtOLDIBL6cQXn6u+OFIVytp+uDJEayWVvJv7nDRIVNJyU8jhRhOPFoCBhUEVwFhX0qSBYsYkmCAuqd4V4hHRQSgda0iFYyyevkm6taplV66ZeaTbyOIrgBJyCc2CBS9AE16ANOgCDR/AMXsGb8WS8GO/Gx6K0YOQ9x+APjK8fj/GhmA==</latexit>
<latexit sha1_base64="ka+62yeJlSMii0YUZPGAvHWznu0=">AAACBHicbVDLSsNAFJ3UV62vqMtuBovgqiQi2I1QcOOygn1AE8JkMmmHTmbCzEQsoQs3/oobF4q49SPc+TdO2iy09cAwh3Pu5d57wpRRpR3n26qsrW9sblW3azu7e/sH9uFRT4lMYtLFggk5CJEijHLS1VQzMkglQUnISD+cXBd+/55IRQW/09OU+AkacRpTjLSRAruee2EMH2YBh1fQCwWL1DQxn5dkMLAbTtOZA64StyQNUKIT2F9eJHCWEK4xQ0oNXSfVfo6kppiRWc3LFEkRnqARGRrKUUKUn8+PmMFTo0QwFtI8ruFc/d2Ro0QVu5nKBOmxWvYK8T9vmOm45eeUp5kmHC8GxRmDWsAiERhRSbBmU0MQltTsCvEYSYS1ya1mQnCXT14lvfOm6zTd24tGu1XGUQV1cALOgAsuQRvcgA7oAgwewTN4BW/Wk/VivVsfi9KKVfYcgz+wPn8A926Xng==</latexit>
<latexit sha1_base64="FWrg6teoe/BfuATlhh9rIYofWlg=">AAACB3icbVDLSsNAFJ3UV62vqEtBBovgqiQi2GXBjcsK9gFNCJPJpB06M4kzE7GE7tz4K25cKOLWX3Dn3zhps9DWA8MczrmXe+8JU0aVdpxvq7Kyura+Ud2sbW3v7O7Z+wddlWQSkw5OWCL7IVKEUUE6mmpG+qkkiIeM9MLxVeH37olUNBG3epISn6OhoDHFSBspsI9zL4zhwzQQ0BPkDnphwiI14ebzeAYDu+40nBngMnFLUgcl2oH95UUJzjgRGjOk1MB1Uu3nSGqKGZnWvEyRFOExGpKBoQJxovx8dscUnholgnEizRMaztTfHTniqtjNVHKkR2rRK8T/vEGm46afU5Fmmgg8HxRnDOoEFqHAiEqCNZsYgrCkZleIR0girE10NROCu3jyMumeN1yn4d5c1FvNMo4qOAIn4Ay44BK0wDVogw7A4BE8g1fwZj1ZL9a79TEvrVhlzyH4A+vzB6S6mR8=</latexit>
<latexit sha1_base64="WK8kISS0bz94XuoEu+xWDZv9hhc=">AAAB/HicbVBNS8NAEN3Ur1q/oj16WSyCp5IUwR4LevBYwX5AG8tmu2mXbjZhdyKEEP+KFw+KePWHePPfuG1z0NYHA4/3ZpiZ58eCa3Ccb6u0sbm1vVPereztHxwe2ccnXR0lirIOjUSk+j7RTHDJOsBBsH6sGAl9wXr+7Hru9x6Z0jyS95DGzAvJRPKAUwJGGtnVoc+APGTDQBGa3eRZI89Hds2pOwvgdeIWpIYKtEf213Ac0SRkEqggWg9cJwYvIwo4FSyvDBPNYkJnZMIGhkoSMu1li+NzfG6UMQ4iZUoCXqi/JzISap2GvukMCUz1qjcX//MGCQRNL+MyToBJulwUJAJDhOdJ4DFXjIJIDSFUcXMrplNiYgCTV8WE4K6+vE66jbrr1N27y1qrWcRRRqfoDF0gF12hFrpFbdRBFKXoGb2iN+vJerHerY9la8kqZqroD6zPHyRslQs=</latexit>
<latexit sha1_base64="9qKs70t1fl6+g3K2bVdt3HNpI1g=">AAACGXicbVBNS8NAEN34WetX1KOXxSLUgyURQY+CHjxWsFVoatlsJ+3SzSbsTsQS8je8+Fe8eFDEo578N24/Dn49GHi8N8PMvDCVwqDnfTozs3PzC4ulpfLyyuraurux2TRJpjk0eCITfR0yA1IoaKBACdepBhaHEq7CwenIv7oFbUSiLnGYQjtmPSUiwRlaqeN6QQjIbvIg0oznZ0V+UBQ0gLuUBhIirO6PfXraUYEWvT7uddyKV/PGoH+JPyUVMkW9474H3YRnMSjkkhnT8r0U2znTKLiEohxkBlLGB6wHLUsVi8G08/FnBd21SpdGibalkI7V7xM5i40ZxqHtjBn2zW9vJP7ntTKMjtu5UGmGoPhkUZRJigkdxUS7QgNHObSEcS3srZT3mc0IbZhlG4L/++W/pHlQ872af3FYOTmexlEi22SHVIlPjsgJOSd10iCc3JNH8kxenAfnyXl13iatM850Zov8gPPxBQbyoD0=</latexit>
例: N = 2 のとき
となる因子:
<latexit sha1_base64="ZJAuVrfq552DZ78TSjPfRafi9Mw=">AAACJ3icbVDLSgMxFM34rPVVdekmWARBKDMi2JUU3LhwUcE+oFNKJr3ThmYyQ3JHKaV/48ZfcSOoiC79E9PHog8PBA7nnMvNPUEihUHX/XFWVtfWNzYzW9ntnd29/dzBYdXEqeZQ4bGMdT1gBqRQUEGBEuqJBhYFEmpB72bk1x5BGxGrB+wn0IxYR4lQcIZWauWufSM6EaO+Fp0uMq3jJ+pS/w5CnFH8AHA+c+4LFWK/lcu7BXcMuky8KcmTKcqt3LvfjnkagUIumTENz02wOWAaBZcwzPqpgYTxHutAw1LFIjDNwfjOIT21SpuGsbZPIR2rsxMDFhnTjwKbjBh2zaI3Ev/zGimGxeZAqCRFUHyyKEwlxZiOSqNtoYGj7FvCuBb2r5R3mWYcbbVZW4K3ePIyqV4UPLfg3V/mS8VpHRlyTE7IGfHIFSmRW1ImFcLJM3klH+TTeXHenC/nexJdcaYzR2QOzu8f7OamjQ==</latexit>
とすると
<latexit sha1_base64="NzSr5atU66O/KU0/dQpAL1446Yg=">AAACaHichVFdSxtBFJ3dVqvxo9tWKeLLxSDog2FXCvVFCCQPPio0KmRjmJ3cTQZnP5i5qw1L8D/2rT+gL/4KJ8k+aAz0wtw5nHMuM3MmypU05Pt/HffDx5XVT2vrtY3Nre3P3pev1yYrtMCOyFSmbyNuUMkUOyRJ4W2ukSeRwpvovjXVbx5QG5mlv2icYy/hw1TGUnCyVN97CiMkDndQQhhrLuzehontp7ZP4D8q/s7hCE4qWwuO4fzVSHu5JdRyOCKudfYIft+r+w1/VvAeBBWos6ou+96fcJCJIsGUhOLGdAM/p17JNUmhcFILC4M5F/d8iF0LU56g6ZWzoCZwaJkBxJm2KyWYsa8nSp4YM04i60w4jcyiNiWXad2C4rNeKdO8IEzF/KC4UEAZTFOHgdQoSI0t4EJLe1cQI24TJfs3NRtCsPjk9+D6tBH4jeDqR715VsWxxvbZATtiAfvJmuyCXbIOE+yfs+HsOLvOs+u53929udV1qplv7E25By/zY7B+</latexit>
「単一のガウス分布場合には，
…
尤度は0に収束する」

識別可能性 (identifiability)
51/64
同等な解が K! 個存在すること
<latexit sha1_base64="KselwTM3ph0gVoi2MZ2jwG/f4GA=">AAACinicbZHdatswFMdl76Nt1q3Zdrmbw8KghS3YZawdpVDoLgaD0bGlLURZkBU5EZFsIx2XBi8P01fqXd+mx4kHWboDQv/zOx+SjpLCaI9RdBeEjx4/ebqxudV6tv38xU775atzn5dOqp7MTe4uE+GV0ZnqoUajLgunhE2Mukimp3X84ko5r/PsF84KNbBinOlUS4GEhu2bAnaBW4GTJIUKrmEOe3AM3JcWhgSm5MQEf5P+RjsvdMPnyzIpDLnfa9eoFNeb/QGe5GbkZ5Y2YtyWRP92eL+SzX/qsRUrQe70eIJ7w3Yn6kYLg4cibkSHNXY2bN/yUS5LqzKURnjfj6MCB5VwqKVR8xYvvSqEnIqx6pPMhFV+UC1GOYd3REaQ5o5WhrCgqxWVsL5+DGXWN/frsRr+L9YvMT0cVDorSlSZXB6UlgYwh/pfYKSdkmhmJIR0mu4KciKckEi/16IhxOtPfijO97tx1I1/fOycHDbj2GRv2Fu2y2J2wE7YV3bGekwGG8GH4FNwEG6H++Hn8GiZGgZNzWv2j4Vf7gEaGbyd</latexit>
<latexit sha1_base64="Woim2xASXio0p8QDza5mda7Cb+M=">AAACW3icbZHfS8MwEMfTTt2cU6fiky/BIUyQ0YrgHge++CQT3Q9Yx0izdAtL2pJcxVH3T/qkD/4rYrpNmJsHId/73F1yufix4Boc59Oyc1vbO/nCbnGvtH9wWD46busoUZS1aCQi1fWJZoKHrAUcBOvGihHpC9bxJ3dZvPPClOZR+AzTmPUlGYU84JSAQYOy8mKOBzjFLp5hTxIYUyKM+5C5ggVQXVA/MPDVwDfs+ZEY6qk0m2GeTAz9PeFqJdt74iNJVoKe4qMxXA7KFafmzA1vCncpKmhpzUH53RtGNJEsBCqI1j3XiaGfEgWcCjYreolmMaETMmI9I0Mime6n89nM8IUhQxxEyqwQ8JyuVqRE6uwxJjPrXK/HMvhfrJdAUO+nPIwTYCFdXBQkAkOEs0HjIVeMgpgaQajipldMx0QRCuY7imYI7vqTN0X7uuY6NffxptKoL8dRQGfoHFWRi25RA92jJmohij7Qt5W3CtaXnbOLdmmRalvLmhP0x+zTH337r/g=</latexit>
<latexit sha1_base64="qpXaPFQu3A7Z5XoMUwx5Oh6AgtI=">AAACW3icbZHfS8MwEMfTTt2cU6fiky/BIUyQ0Q7BPQ588Ukmuh+wjpFm6RaWtCW5iqPun/RJH/xXxHSbMKcHId/73F1yufix4Boc58Oyc1vbO/nCbnGvtH9wWD467ugoUZS1aSQi1fOJZoKHrA0cBOvFihHpC9b1p7dZvPvMlOZR+ASzmA0kGYc84JSAQcOy8mKOhzjFdTzHniQwoUQY9z5zBQuguqR+YOCLga/Y8yMx0jNpNsM8mRj6c8LVWrb3yMeSrAU9xccTuByWK07NWRj+K9yVqKCVtYblN28U0USyEKggWvddJ4ZBShRwKti86CWaxYROyZj1jQyJZHqQLmYzxxeGjHAQKbNCwAu6XpESqbPHmMysc70Zy+B/sX4CQWOQ8jBOgIV0eVGQCAwRzgaNR1wxCmJmBKGKm14xnRBFKJjvKJohuJtP/is69Zrr1NyH60qzsRpHAZ2hc1RFLrpBTXSHWqiNKHpHX1beKlifds4u2qVlqm2tak7QL7NPvwGDIa/7</latexit>
<latexit sha1_base64="Woim2xASXio0p8QDza5mda7Cb+M=">AAACW3icbZHfS8MwEMfTTt2cU6fiky/BIUyQ0YrgHge++CQT3Q9Yx0izdAtL2pJcxVH3T/qkD/4rYrpNmJsHId/73F1yufix4Boc59Oyc1vbO/nCbnGvtH9wWD46busoUZS1aCQi1fWJZoKHrAUcBOvGihHpC9bxJ3dZvPPClOZR+AzTmPUlGYU84JSAQYOy8mKOBzjFLp5hTxIYUyKM+5C5ggVQXVA/MPDVwDfs+ZEY6qk0m2GeTAz9PeFqJdt74iNJVoKe4qMxXA7KFafmzA1vCncpKmhpzUH53RtGNJEsBCqI1j3XiaGfEgWcCjYreolmMaETMmI9I0Mime6n89nM8IUhQxxEyqwQ8JyuVqRE6uwxJjPrXK/HMvhfrJdAUO+nPIwTYCFdXBQkAkOEs0HjIVeMgpgaQajipldMx0QRCuY7imYI7vqTN0X7uuY6NffxptKoL8dRQGfoHFWRi25RA92jJmohij7Qt5W3CtaXnbOLdmmRalvLmhP0x+zTH337r/g=</latexit>
<latexit sha1_base64="qpXaPFQu3A7Z5XoMUwx5Oh6AgtI=">AAACW3icbZHfS8MwEMfTTt2cU6fiky/BIUyQ0Q7BPQ588Ukmuh+wjpFm6RaWtCW5iqPun/RJH/xXxHSbMKcHId/73F1yufix4Boc58Oyc1vbO/nCbnGvtH9wWD467ugoUZS1aSQi1fOJZoKHrA0cBOvFihHpC9b1p7dZvPvMlOZR+ASzmA0kGYc84JSAQcOy8mKOhzjFdTzHniQwoUQY9z5zBQuguqR+YOCLga/Y8yMx0jNpNsM8mRj6c8LVWrb3yMeSrAU9xccTuByWK07NWRj+K9yVqKCVtYblN28U0USyEKggWvddJ4ZBShRwKti86CWaxYROyZj1jQyJZHqQLmYzxxeGjHAQKbNCwAu6XpESqbPHmMysc70Zy+B/sX4CQWOQ8jBOgIV0eVGQCAwRzgaNR1wxCmJmBKGKm14xnRBFKJjvKJohuJtP/is69Zrr1NyH60qzsRpHAZ2hc1RFLrpBTXSHWqiNKHpHX1beKlifds4u2qVlqm2tak7QL7NPvwGDIa/7</latexit>
パラメータ値の解釈 => 関係あり
良い密度モデルが欲しいだけ => 関係ない

対数尤度最大化の問題
52/64
混合ガウスモデル
- 特異性の問題
- 識別可能性 (同等な解が K! 存在)
単一ガウスモデル
- 特異性の問題なし
- 識別可能性なし (同等な解なし)

最尤推定の困難さはどこから？
53/64
対数尤度関数
log-sum
=> 微分を 0 としても陽な解が得られない
対数関数がガウス密度に直接作用していない
EM アルゴリズムにより最尤推定を行う

9.2.2 EM for Gaussian mixtures
54/64

EM アルゴリズム
55/64
潜在変数を持つモデルの最尤解を求めるための1つの elegant かつ強力な方法
<latexit sha1_base64="KselwTM3ph0gVoi2MZ2jwG/f4GA=">AAACinicbZHdatswFMdl76Nt1q3Zdrmbw8KghS3YZawdpVDoLgaD0bGlLURZkBU5EZFsIx2XBi8P01fqXd+mx4kHWboDQv/zOx+SjpLCaI9RdBeEjx4/ebqxudV6tv38xU775atzn5dOqp7MTe4uE+GV0ZnqoUajLgunhE2Mukimp3X84ko5r/PsF84KNbBinOlUS4GEhu2bAnaBW4GTJIUKrmEOe3AM3JcWhgSm5MQEf5P+RjsvdMPnyzIpDLnfa9eoFNeb/QGe5GbkZ5Y2YtyWRP92eL+SzX/qsRUrQe70eIJ7w3Yn6kYLg4cibkSHNXY2bN/yUS5LqzKURnjfj6MCB5VwqKVR8xYvvSqEnIqx6pPMhFV+UC1GOYd3REaQ5o5WhrCgqxWVsL5+DGXWN/frsRr+L9YvMT0cVDorSlSZXB6UlgYwh/pfYKSdkmhmJIR0mu4KciKckEi/16IhxOtPfijO97tx1I1/fOycHDbj2GRv2Fu2y2J2wE7YV3bGekwGG8GH4FNwEG6H++Hn8GiZGgZNzWv2j4Vf7gEaGbyd</latexit>
EM アルゴリズムの一般化 = 変分推論法 (変分ベイズ)
EM アルゴリズムを混合ガウス分布の最尤推定に適用 =>

更新式の導出
56/64
尤度関数の最大点において満たされるべき条件
- パラメータについての微分が 0
Ø 平均の微分 = 0
Ø 共分散の微分 = 0
Ø 混合係数の微分 = 0

平均に関する微分
57/64
整理すると，
<latexit sha1_base64="2kG1QYgEPq7VMT2vMeEJrxdnKMI=">AAACcXicbVFNa9wwEJWdtEm3H9m0vZTQMmQppAQWuxSSSyDQS08lhWwSWG8XWSvvipVkI41Dtsb3/r7e+id66R/o2DElXwNCb957w8jPaaGVxyj6HYRr648eb2w+6T199vzFVn/75ZnPSyfkSOQ6dxcp91IrK0eoUMuLwkluUi3P0+XnRj+/lM6r3J7iqpATw+dWZUpwJGra/5mkuZ75laELKkhMCTVMCS3pPoIkc1xQF1NXwdf/Sg2JL03bWrI18vfWQMKcG8Mh0TLDPfjReZqhxKn5Aj/QEo6LNCP+qltmoZ72B9Ewagvug7gDA9bVybT/K5nlojTSotDc+3EcFTipuEMltKx7SellwcWSz+WYoOVG+knVJlbDe2JmkOWOjkVo2ZsTFTe+CYWczWP9Xa0hH9LGJWaHk0rZokRpxfWirNSAOTTxw0w5KVCvCHDhFL0VxIJTyEg/qUchxHc/+T44+ziMo2H87dPg+LCLY5PtsF22x2J2wI7ZF3bCRkywP8Hr4G3wLvgbvgkh3L22hkE384rdqnD/H75EtS8=</latexit>
ただし，
<latexit sha1_base64="4uvQjSxAVyYZUoZD3dEpACygsAs=">AAACL3icbVDLSgMxFM34tr6qLt0Ei6CbMiOC3QiCIK5KBdsKnXHIpJk2NMkMyR2hDv0jN/5KNyKKuPUvTNtZ+LoQcu6555DcE6WCG3DdF2dufmFxaXlltbS2vrG5Vd7eaZkk05Q1aSISfRsRwwRXrAkcBLtNNSMyEqwdDS4m8/Y904Yn6gaGKQsk6Skec0rAUmH5so5DnOMBHuEz7JtMTltlG89SdxbX7e33iJQE+4LFcIgfCs3E5Gve68NRWK64VXda+C/wClBBRTXC8tjvJjSTTAEVxJiO56YQ5EQDp4KNSn5mWErogPRYx0JFJDNBPt13hA8s08Vxou1RgKfsd0dOpDFDGVmlJNA3v2cT8r9ZJ4O4FuRcpRkwRWcPxZnAkOBJeLjLNaMghhYQqrn9K6Z9ogkFG3HJhuD9XvkvaB1XPbfqXZ9UzmtFHCtoD+2jQ+ShU3SOrlADNRFFj2iMXtGb8+Q8O+/Ox0w65xSeXfSjnM8v7iSkLQ==</latexit>
解釈: データ点の負担率で重み付けされた平均
解釈: クラスタ k に割り当てられる点の実効的な数
※平均パラメータに関して解けていない
<latexit sha1_base64="Qb5m1Z1e6+j4RkjyvYnjnWUl/WQ=">AAAEHHicrZNNixMxGMezM76s9WW7evTyYBFa2C0zi+BeCgtePMmKdrfQ1CGTZrppk8wwyYh1nA/ixa/ixYMiXjwIfhuTvtHtLj0ZGObPP/8n+eUhiTPBtQmCvzuef+Pmrdu7d2p3791/sFfff3im0yKnrEtTkea9mGgmuGJdw41gvSxnRMaCnceTF27+/D3LNU/VWzPN2ECSkeIJp8RYK9r3jnCSE1rijOSGE1GtFJQ4TsVQT6X9YVlU0aTCQkEGTcCSmIs4gRJ6UMEnWEsCzjgcXHJKmy9s7mCtDr/hI0ms2ap1AOtCQmRdBR0IrfnO6lf2jws1ZHlsAZktcaCu1G7gwhMXcAtSBzvPC5aYdbwP1pwvvIm5hFqudC3capucjy5My6rSsUbluFphjP8bxngbxngdo4pKPCJSkubHqFSTqlVto3fdPJz1dSvY4db+LPfuQBDVG0E7mA24KsKFaKDFOI3qv/EwpYVkylBBtO6HQWYGpbtnVLCqhgvNMkInZMT6VioimR6Us8tdwVPrDCFJc/spAzN3vaIkUjtmm3TH0ptzzrxurl+Y5HhQcpUVhik63ygpBJgU3EuBIc8ZNWJqBaE5t6xAL4i9g8a+p5ptQrh55Kvi7KgdBu3w9bPGyfGiHbvoMXqCmihEz9EJeolOURdR77P31fvu/fC/+N/8n/6vedTbWdQ8QpeG/+cfOzhKBQ==</latexit>

共分散に関する微分
58/64
<latexit sha1_base64="ccjRCJgOWa2VJtlTr9mM+bERzh0=">AAACfnicbZFRb9MwEMedbEDXsRHY415uq5jGBiVBSOsLUiVeeOy0tatUV5XjOq1V24lsB6nK8jH2xfbGZ+EFp81Q6Tgpyj+//51zvoszwY0Nw1+ev7P74uWrxl5z//XB4Zvg7buBSXNNWZ+mItXDmBgmuGJ9y61gw0wzImPB7uLF98q/+8m04am6tcuMjSWZKZ5wSqxDk+DhDCeaUCgAZ0RbTgSUmx9YEjuPE3zDZ5LAxFkLl1ACFgoyOH/yHR86eg84TsXULKV7rY7hjn7cpjJf07+19fElfGh+gxDjSdAK2+Eq4LmIatFCdfQmwSOepjSXTFkqiDGjKMzsuKguQQUrmzg3LCN0QWZs5KQikplxsRpfCe8dmUKSavcoCyu6WVEQaarmXWbVsdn2Kvg/b5TbpDMuuMpyyxRd/yjJBdgUql3AlGtGrVg6Qajmrlegc+LWYd3Gmm4I0faVn4vBl3YUtqPrr61upx5HAx2jU3SOInSFuugH6qE+oui3d+JdeJc+8s/8T/7ndarv1TVH6J/wO38AJyK8DA==</latexit>
<latexit sha1_base64="SOXYZg25I4iOndqPpRcjMqkOGts=">AAAC9XicrVLNbtQwEHZSfsoW6LYcuViskMqBVYKQ2gtSJS6cUCu6baX1duV4nay1thPZE2CJ9j24cAAhrn0XbrwN4zRAaTkykjXfzDfjmbEnq7TykCQ/onjtxs1bt9fv9Dbu3ru/2d/aPvZl7YQciVKX7jTjXmpl5QgUaHlaOclNpuVJtngZ+JO30nlV2iNYVnJieGFVrgQHdE23og1mOMyznDaUvVGF4XRFp2gsUL+gLHdcoJWi1dDXv5kVZb42rWkxLNBnbQASBTd4C9Myhx36oYsJScypYg5PflF/Cr/valrUTynLSj3zS4MqNGXqSx39xyvOWgbznUF0FIaa9gfJMGmFXgdpBwakk4Np/zublaI20oLQ3PtxmlQwabgDJbRc9VjtZcXFghdyjNByI/2kaX9tRR+jZ0bz0uGxQFvv5YyGGx9mwMjQpr/KBee/uHEN+d6kUbaqQVpxUSivNYWShhWgM+WkAL1EwIVT2CsVc44fDbgoPXyE9OrI18Hxs2GaDNPD54P9ve451slD8ojskJTskn3yihyQERGRiz5Gn6Mv8bv4U/w1/nYRGkddzgPyl8TnPwFeV+Hi</latexit>
整理すると，
単一ガウスの場合とほぼ同じ
<latexit sha1_base64="73VMnj+J3DJ2hPX2b9b0SbTd+g0=">AAAC/XicrVJNb9QwEHVSPsrytaVHLhYrpHJglawq0QtSJS4cALWi21babFeO18latZ3InlRdoqh/hQsHEOLK/+iNf8M4DYK2cGMk28/vjWfs8aSlkg6i6EcQrty4eev26p3e3Xv3Hzzsrz3ad0VluRjzQhX2MGVOKGnEGCQocVhawXSqxEF6/MrrByfCOlmYPViWYqpZbmQmOQOkZmvBeqIZLNKM1jR5L3PNaENnfuNpqxG9fYNUQ1/SJLOMIxHjrqbvcE5cpVtvg7Knj34JSmSwQX/HPu3CGlyf0yQt1NwtNS5tqupfSRMr8wU8++/xji6Je15LgOX1aBiPRs2sP4iGUWv0Oog7MCCd7cz658m84JUWBrhizk3iqIRpzSxIrkTTSyonSsaPWS4mCA3Twk3r9vca+hSZOc0Ki8MAbdk/T9RMO/829PQ3dlc1T/5Nm1SQbU1racoKhOEXibJKUSiobwU6l1ZwUEsEjFuJd6V8wfCPARumh0WIrz75OtjHikXDeHdzsL3VlWOVPCZPyAaJyQuyTV6THTImPPgQfAw+B1/Cs/BT+DX8duEaBt2ZdXLJwu8/AWJC5bs=</latexit>
解釈: 各点の負担率で重み付けされた平均

混合係数に関する微分
59/64
λ を消去し，整理すると，
・・・全データ点に対する負担率の平均
<latexit sha1_base64="dlR9McOw8fGLGWehcy0AhL8i9r0=">AAACu3icbVFNj9MwEHXC11K+Chy5jKiQugKqBCHRy0orcUHisgi6W6nuVo7jpN7aTuQPpCr0T3Lj3zBpAlq6jBT55c2bGftNVivpfJL8iuJbt+/cvXd0f/Dg4aPHT4ZPn527KlguZrxSlZ1nzAkljZh56ZWY11YwnSlxkW0+tvmL78I6WZlvfluLpWalkYXkzCO1Gv6khWUcGqA1s14yBbvrP1Rhq5whSZUo/JgqAzWMgWrm11mB0jnmfgDNKpW7rcZjXy6RfXPI6tCxf2vpV1nqtvkxvP4zqpsD1AUNK9Rs4ARSlFwi/tzeA3t3/A7eYoZaWa79cX+gOFkNR8kk2QfcBGkPRqSPs9aEvOJBC+O5Ys4t0qT2y6a1gCuxG9DgRM34hpVigdAwLdyy2Xu/g1fI5FBUFj/jYc9er2iYdq0FqGzf7Q5zLfm/3CL4YrpspKmDF4Z3g4qgwFfQLhJyaQX3aouAcSvxrsDXDJfpcd0DNCE9fPJNcP5ukiaT9Mv70em0t+OIvCAvyZik5AM5JZ/IGZkRHk2jy6iM1vFJzOOrWHXSOOprnpN/Ig6/AZ2e0Ao=</latexit>
<latexit sha1_base64="L3bp3j7RLTFdDjdfbRGjF8tcdqs=">AAACxnicbVFdi9NAFJ3Er7V+VX305WIRuqglEcF9ERb2ZcGXFe1uoVPDZDpph85MwnwoJQb8jb754H/xpglSu14Ic3LOuXPn3ptXSjqfJL+i+MbNW7fvHN0d3Lv/4OGj4eMnl64MlospL1VpZzlzQkkjpl56JWaVFUznSlzlm7NWv/oqrJOl+ey3lVhotjKykJx5pLLhb1pYxqEGWjHrJVPQ7P/UNC/V0m01HlSHJtugTpUo/JgqAxWMgWrm13mBWTPUvsNexu4mieyrQ1aHjv2bSz/JlWZIHsNLLIANLFlXB6gLGjL0bOA9pGj5gvhD+w68u+MbeI0KtXK19sf9geYkG46SSbILuA7SHoxIHxfZ8CddljxoYTxXzLl5mlR+UbfT4Eo0AxqcqBjfsJWYIzRMC7eod2to4AUySyhKi5/xsGP3M2qmXTsCdLZ9u0OtJf+nzYMvTha1NFXwwvCuUBEU+BLancJSWsG92iJg3Ep8K/A1w7163PwAh5AetnwdXL6ZpMkk/fh2dHrSj+OIPCPPyZik5B05JefkgkwJj84iGdnIxeexiUP8rbPGUZ/zlPwT8Y8/jwTVTg==</latexit>
<latexit sha1_base64="iQku1rRghbPPOPFT853maCe+/T4=">AAACEHicbVDLSsNAFL3xWesr6tLNYBFdlUQEuxEKblxJBfuAJoTJdNIOnTyYmQgl5BPc+CtuXCji1qU7/8ZpGkRbDwyce8693LnHTziTyrK+jKXlldW19cpGdXNre2fX3NvvyDgVhLZJzGPR87GknEW0rZjitJcIikOf064/vpr63XsqJIujOzVJqBviYcQCRrDSkmeeOAlDHsrQGOXoEjmBwERXNz9aXlS5Z9asulUALRK7JDUo0fLMT2cQkzSkkSIcS9m3rUS5GRaKEU7zqpNKmmAyxkPa1zTCIZVuVhyUo2OtDFAQC/0ihQr190SGQyknoa87Q6xGct6biv95/VQFDTdjUZIqGpHZoiDlSMVomg4aMEGJ4hNNMBFM/xWREdaZKJ1hVYdgz5+8SDpndduq27fntWajjKMCh3AEp2DDBTThGlrQBgIP8AQv8Go8Gs/Gm/E+a10yypkD+APj4xtq65mh</latexit>
ラグランジュの未定乗数法

尤度関数の最大点の条件
60/64
<latexit sha1_base64="4uvQjSxAVyYZUoZD3dEpACygsAs=">AAACL3icbVDLSgMxFM34tr6qLt0Ei6CbMiOC3QiCIK5KBdsKnXHIpJk2NMkMyR2hDv0jN/5KNyKKuPUvTNtZ+LoQcu6555DcE6WCG3DdF2dufmFxaXlltbS2vrG5Vd7eaZkk05Q1aSISfRsRwwRXrAkcBLtNNSMyEqwdDS4m8/Y904Yn6gaGKQsk6Skec0rAUmH5so5DnOMBHuEz7JtMTltlG89SdxbX7e33iJQE+4LFcIgfCs3E5Gve68NRWK64VXda+C/wClBBRTXC8tjvJjSTTAEVxJiO56YQ5EQDp4KNSn5mWErogPRYx0JFJDNBPt13hA8s08Vxou1RgKfsd0dOpDFDGVmlJNA3v2cT8r9ZJ4O4FuRcpRkwRWcPxZnAkOBJeLjLNaMghhYQqrn9K6Z9ogkFG3HJhuD9XvkvaB1XPbfqXZ9UzmtFHCtoD+2jQ+ShU3SOrlADNRFFj2iMXtGb8+Q8O+/Ox0w65xSeXfSjnM8v7iSkLQ==</latexit>
平均
共分散
混合係数
<latexit sha1_base64="dnO9jWMCzqYb/6CZKUKJ1sWSQls=">AAADBnicrVLPb9MwFHbCgFF+rIMjHCwqpE5CVTJNYhekSVw4oSHoNqmuohfXSd3aTmQ7iBJy4rJ/ZRcOIMSVv2G3/Tez2wiVDXHiSdH7/H3v2Z+dl5aCGxtFF0F4Y+Pmrdubdzp3791/sNXdfnhkikpTNqSFKPRJCoYJrtjQcivYSakZyFSw43T+yuvHH5g2vFDv7aJkYwm54hmnYB2VbAdPSA5SQv9TUqt5s4NfYpJpoLjGpOQ4cXmOG0wk2CkF4ZZv/FKwzPZXbJo58qMjfa1y+TMmaSEmZiFd8vvIqlX9Ts/Xusg7nktYE4nm+dTuOFQTU8mknjW/bcz+m43Zv2zM1m0k3V40iJaBr4O4BT3UxmHSPSeTglaSKUsFGDOKo9KOa9CWU8GaDqkMK4HOIWcjBxVIZsb18jc2+JljJjgrtPuUxUt2vaMGafx1XKX3bq5qnvybNqpstj+uuSoryxRdHZRVAtsC+5nAE64ZtWLhAFDNnVdMp+CmwLrJ6bhHiK9e+To42h3E0SB+u9c72G+fYxM9Rk9RH8XoBTpAr9EhGiIafAnOgm/B9/A0/Br+CH+uSsOg7XmE/ojw1yXXeetb</latexit>
負担率
クラスタ k に割り当てられる点の実効的な数
パラメータに関して
解けていない
・・・パラメータ (平均，共分散，混合係数) に複雑な形で依存
条
件

尤度関数の最大点の条件
61/64
平均
共分散
混合係数
パラメータに関して
解けていない
条
件
繰り返せばいいんじゃない？
右辺のパラメータは何かしら固定して
左辺を新たなパラメータ値とする

MoG のための EM アルゴリズム
1. 初期化，対数尤度計算
2. E step:
3. M step:
4. 対数尤度計算，収束判定 (E step / 終わり)
62/64
<latexit sha1_base64="yIZiGxpZnMm6TgoS8ct21A96ius=">AAACJXicbVDLSsNAFJ3UV62vqEs3g0VwISURwS5cFNy4rGhroQlhMpm0Q2cyYWYilNCfceOvuHFhEcGVv+KkDaKtB4Y5nHMv994Tpowq7TifVmVldW19o7pZ29re2d2z9w+6SmQSkw4WTMheiBRhNCEdTTUjvVQSxENGHsLRdeE/PBKpqEju9TglPkeDhMYUI22kwL7y8twLBYvUmJvP49kkGHmTM7ig39EBRz+Wl9KCBnbdaTgzwGXilqQOSrQDe+pFAmecJBozpFTfdVLt50hqihmZ1LxMkRThERqQvqEJ4kT5+ezKCTwxSgRjIc1LNJypvztyxFWxrankSA/VoleI/3n9TMdNP6dJmmmS4PmgOGNQC1hEBiMqCdZsbAjCkppdIR4iibA2wdZMCO7iycuke95wnYZ7e1FvNcs4quAIHINT4IJL0AI3oA06AIMn8ALewNR6tl6td+tjXlqxyp5D8AfW1zcvlKbV</latexit>
<latexit sha1_base64="c7xclykypXSCDyG64aespqzIil8=">AAAEDnicxVNNb9QwEHUTPsrytYUjlxELqBxYJQiJXipV4sIJFdFtK623K8frZK21nch2oEuUX8CFv8KFAwhx5cyNf8MkG6HS9gAHxEjRPL83Y4+f4qRQ0vko+rEWhBcuXrq8fqV39dr1Gzf7G7f2XV5aLkY8V7k9TJgTShox8tIrcVhYwXSixEGyeNboB6+FdTI3e35ZiIlmmZGp5MwjNd0I7tEkVzO31JigAqpLqGGKaIH5qGG8OPaYjXiDTA0PtoGmlnGkYlxW8OJXeQ3UlbpdGthu5aO2AIWMac2AKpH6TXjb1TRN1Mps7h/iyczPkxT5424CAzWlvaoT6CuZafafZltJ544Ij+BvLfzH27Zas6fViPYaDX2khfxT70661lo07Q+iYdQGnAVxBwaki91p/zud5bzUwniumHPjOCr8pGLWS65E3aOlEwXjC5aJMULDtHCTqv2da7iPzAzS3OJnPLTsyY6KadfYgpXNLd1prSHP08alT7cmlTRF6YXhq4PSUoHPoXkbMJNWcK+WCBi3EmcFPmfoiccX1EMT4tNXPgv2Hw/jaBi/fDLY2ersWCd3yF2ySWLylOyQ52SXjAgP3gUfgk/B5/B9+DH8En5dlQZrXc9t8luE334Cheo7OQ==</latexit>

K-means と EM アルゴリズム
63/64
E step
<latexit sha1_base64="c7xclykypXSCDyG64aespqzIil8=">AAAEDnicxVNNb9QwEHUTPsrytYUjlxELqBxYJQiJXipV4sIJFdFtK623K8frZK21nch2oEuUX8CFv8KFAwhx5cyNf8MkG6HS9gAHxEjRPL83Y4+f4qRQ0vko+rEWhBcuXrq8fqV39dr1Gzf7G7f2XV5aLkY8V7k9TJgTShox8tIrcVhYwXSixEGyeNboB6+FdTI3e35ZiIlmmZGp5MwjNd0I7tEkVzO31JigAqpLqGGKaIH5qGG8OPaYjXiDTA0PtoGmlnGkYlxW8OJXeQ3UlbpdGthu5aO2AIWMac2AKpH6TXjb1TRN1Mps7h/iyczPkxT5424CAzWlvaoT6CuZafafZltJ544Ij+BvLfzH27Zas6fViPYaDX2khfxT70661lo07Q+iYdQGnAVxBwaki91p/zud5bzUwniumHPjOCr8pGLWS65E3aOlEwXjC5aJMULDtHCTqv2da7iPzAzS3OJnPLTsyY6KadfYgpXNLd1prSHP08alT7cmlTRF6YXhq4PSUoHPoXkbMJNWcK+WCBi3EmcFPmfoiccX1EMT4tNXPgv2Hw/jaBi/fDLY2ersWCd3yF2ySWLylOyQ52SXjAgP3gUfgk/B5/B9+DH8En5dlQZrXc9t8luE334Cheo7OQ==</latexit>
M step
Þ 9.3.2 K-means との関連にて詳しく
K-meansEM
似てる

まとめ
• 9 混合モデルとEM
• 9.1 K-means クラスタリング
• 9.1.1 画像分割と画像圧縮
• 9.2 混合ガウス分布
• 9.2.1 最尤推定
• 9.2.2 混合ガウス分布の EM アルゴリズム
64/64

PRML 9-9.2.2 クラスタリング (K-means とガウス混合モデル) / Clustering (K-means and Gaussian Mixture Models)

More Related Content

Similar to PRML 9-9.2.2 クラスタリング (K-means とガウス混合モデル) / Clustering (K-means and Gaussian Mixture Models)

More from Akihiro Nitta

PRML 9-9.2.2 クラスタリング (K-means とガウス混合モデル) / Clustering (K-means and Gaussian Mixture Models)