* Satoshi Hara and Kohei Hayashi. Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach. AISTATS'18 (to appear).
arXiv ver.: https://arxiv.org/abs/1606.09066#
* GitHub
https://github.com/sato9hara/defragTrees
* Satoshi Hara and Kohei Hayashi. Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach. AISTATS'18 (to appear).
arXiv ver.: https://arxiv.org/abs/1606.09066#
* GitHub
https://github.com/sato9hara/defragTrees
Bayesian Generalization Error and Real Log Canonical Threshold in Non-negativ...Naoki Hayashi
I have talked in the conference Algebraic Statistics 2020.
As a background of our research, I briefly explained singular learning theory which can be interpretable as an intersection between algebraic statistics and statistical learning theory.
The main part of this presentation is introducing our recent studies for parameter region restriction in singular learning theory. I showed the researches about the learning coefficient (real log canonical threshold) of NMF and LDA. NMF and LDA are typical models whose parameter regions are restricted.
2018年3月のニューロコンピューティング研究会にて発表.確率行列分解のBayes汎化誤差に関する理論的な不等式について数値実験を試みたかったが,そもそもBayes推定をすることが困難な問題であった:パラメータが単体(simplex)上に存在するために,事後分布からサンプリングを行うことが難しい.そこで本研究ではハミルトニアンモンテカルロ法という効率的なMCMC法を用いてBayes推定をしてみた.理論値と比較し,確率行列分解に対するハミルトニアンモンテカルロ法の有効性を検証した.in Japanese
This research was published in IEEE SSCI 2017 in Hawaii.
The research goal was constructing learning theory of Non-negative Matrix Factorization and we derived a tighter upper bound of the generalization error than our previous research. Moreover, we carried out numerical experiments and discovered a conjecture that showed the exact value of the generalization error.
43. 5.潜在ディリクレ配分
LDAとは
• LDAの典型例: テキストマイニング
‒ LDAの対象: bag of words
‒ トピック: 各文書が持つ潜在的な単語「生成源」
47
MATH
NAME
…
Riemann,
Lebesgue,
Atiyah,
Hironaka,
… integral,
measure,
distribution,
singularity,
…
document
topic
word
word
61. References
[1] Aoyagi, M & Watanabe, S. Stochastic complexities of reduced rank regression in Bayesian estimation. Neural Netw.
2005;18(7):924–33.
[2] Drton, M & Plummer, M. A Bayesian information criterion for singular models. J R Stat Soc B. 2017;79:323–80 with discussion.
[3] H, N & Watanabe, S. Upper bound of Bayesian generalization error in non-negative matrix factorization. Neurocomputing.
2017;266C(29 November):21–8.
[4] H, N & Watanabe, S. Tighter upper bound of real log canonical threshold of non-negative matrix factorization and its application to
Bayesian inference. In IEEE symposium series on computational intelligence (IEEE SSCI). (2017). (pp. 718–725).
[5] H, N & Watanabe, S. Asymptotic Bayesian generalization error in latent Dirichlet allocation. SN Computer Science. 2020;1(69):1-22.
[6] H, N. Variational approximation error in non-negative matrix factorization. Neural Netw. 2020;126(June):65-75.
[7] H, N. The exact asymptotic form of Bayesian generalization error in latent Dirichlet allocation. https://arxiv.org/abs/2008.01304
[8] Imai, T. Estimating real log canonical threshold. https://arxiv.org/abs/1906.01341
[9] Kohjima M, Matsubayashi T, Sawada H. Multiple data analysis and non-negative matrix/tensor factorization [I]: multiple data
analysis and its advances. IEICE Transaction. 2016:99(6);543-550. In Japanese.
[10] Kohjima M., & Watanabe S. (2017). Phase transition structure of variational bayesian nonnegative matrix factorization. In
International conference on artificial neural networks (ICANN) (2017). (pp. 146–154).
[11] Nagata K, Watanabe S. Asymptotic behavior of exchange ratio in exchange monte carlo method. Neural Netw. 2008;21(7):980–8.
[12] Nakada, R & Imaizumi, M. Adaptive approximation and generalization of deep neural network with Intrinsic dimensionality. JMLR.
2020;21(174):1-38.
[13] Watanabe, S. Algebraic geometrical methods for hierarchical learning machines. Neural Netw. 2001;13(4):1049–60.
[14] Watanabe, S. Mathematical theory of Bayesian statistics. Florida: CR Press. 2018.
[15] Yamazaki, K & Watanabe, S. Singularities in mixture models and upper bounds of stochastic complexity. Neural Netw.
2003;16(7):1029–38.
[16] Zwiernik P. An asymptotic behaviour of the marginal likelihood for general Markov models. J Mach Learn Res.
2011;12(Nov):3283–310.
65