園田翔氏の博士論文を解説しました。
Integral Representation Theory of Deep Neural Networks
深層学習を数学的に定式化して解釈します。
3行でいうと、
ーニューラルネットワーク—(連続化)→双対リッジレット変換
ー双対リッジレット変換=輸送写像
ー輸送写像でNeural Networkを定式化し、解釈する。
目次
ー深層ニューラルネットワークの数学的定式化
ーリッジレット変換について
ー輸送写像について
園田翔氏の博士論文を解説しました。
Integral Representation Theory of Deep Neural Networks
深層学習を数学的に定式化して解釈します。
3行でいうと、
ーニューラルネットワーク—(連続化)→双対リッジレット変換
ー双対リッジレット変換=輸送写像
ー輸送写像でNeural Networkを定式化し、解釈する。
目次
ー深層ニューラルネットワークの数学的定式化
ーリッジレット変換について
ー輸送写像について
【DL輪読会】Efficiently Modeling Long Sequences with Structured State SpacesDeep Learning JP
This document summarizes a research paper on modeling long-range dependencies in sequence data using structured state space models and deep learning. The proposed S4 model (1) derives recurrent and convolutional representations of state space models, (2) improves long-term memory using HiPPO matrices, and (3) efficiently computes state space model convolution kernels. Experiments show S4 outperforms existing methods on various long-range dependency tasks, achieves fast and memory-efficient computation comparable to efficient Transformers, and performs competitively as a general sequence model.
KDD Cup 2021で開催された時系列異常検知コンペ
Multi-dataset Time Series Anomaly Detection (https://compete.hexagon-ml.com/practice/competition/39/) に参加して
5位入賞した解法の紹介と上位解法の整理のための資料です.
9/24のKDD2021参加報告&論文読み会 (https://connpass.com/event/223966/) の発表資料です.
This document discusses methods for identifying the source node of information spread in networks based on the observed spread over time. It begins by introducing epidemic models like SIS and SI for modeling information spread over networks. It then discusses maximum likelihood methods for identifying the source node on regular tree networks based on the observed subgraph. The accuracy of these methods increases with network size and degree. Extensions to other network structures and SIR models are also proposed. Overall, the document reviews mathematical models and algorithms for source identification in networks from limited observations of information spread.
This document discusses information theory and related concepts such as entropy, Kullback-Leibler divergence, mutual information, independent component analysis, clustering algorithms, change point detection, kernel density estimation, and nonparametric regression. It provides mathematical definitions and formulas for these concepts. Figures are included to illustrate clustering and change point detection methods. The document contains information that could be useful for understanding techniques in machine learning, signal processing, and statistics.
【DL輪読会】Efficiently Modeling Long Sequences with Structured State SpacesDeep Learning JP
This document summarizes a research paper on modeling long-range dependencies in sequence data using structured state space models and deep learning. The proposed S4 model (1) derives recurrent and convolutional representations of state space models, (2) improves long-term memory using HiPPO matrices, and (3) efficiently computes state space model convolution kernels. Experiments show S4 outperforms existing methods on various long-range dependency tasks, achieves fast and memory-efficient computation comparable to efficient Transformers, and performs competitively as a general sequence model.
KDD Cup 2021で開催された時系列異常検知コンペ
Multi-dataset Time Series Anomaly Detection (https://compete.hexagon-ml.com/practice/competition/39/) に参加して
5位入賞した解法の紹介と上位解法の整理のための資料です.
9/24のKDD2021参加報告&論文読み会 (https://connpass.com/event/223966/) の発表資料です.
This document discusses methods for identifying the source node of information spread in networks based on the observed spread over time. It begins by introducing epidemic models like SIS and SI for modeling information spread over networks. It then discusses maximum likelihood methods for identifying the source node on regular tree networks based on the observed subgraph. The accuracy of these methods increases with network size and degree. Extensions to other network structures and SIR models are also proposed. Overall, the document reviews mathematical models and algorithms for source identification in networks from limited observations of information spread.
This document discusses information theory and related concepts such as entropy, Kullback-Leibler divergence, mutual information, independent component analysis, clustering algorithms, change point detection, kernel density estimation, and nonparametric regression. It provides mathematical definitions and formulas for these concepts. Figures are included to illustrate clustering and change point detection methods. The document contains information that could be useful for understanding techniques in machine learning, signal processing, and statistics.
This document discusses pattern formation in crowd dynamics. It begins with an introduction to crowd dynamics and then discusses two specific patterns: lane formation and freezing-by-heating transition. Lane formation occurs when pedestrians walking in opposite directions spontaneously form lanes to allow for more efficient movement. Freezing-by-heating transition refers to the phenomenon where increasing noise or energy in a crowd leads to the formation of orderly lanes, rather than disorder. The document explores mathematical modeling of these patterns using particle simulation models.
This document summarizes a presentation on rigorously verifying the accuracy of numerical solutions to semi-linear parabolic partial differential equations using analytic semigroups. It introduces the considered problem of finding the solution to a semi-linear parabolic PDE. It then discusses using a piecewise linear finite element discretization in space and time to obtain an initial numerical solution. The goal is to rigorously enclose the true solution within a radius ρ of this numerical solution in the function space L∞(J;H10(Ω)). Key steps involve using properties of the analytic semigroup generated by the operator A and estimating discretization errors to compute the enclosure radius ρ.
This document presents an overview of optimization algorithms on Riemannian manifolds. It begins by introducing concepts such as vector transport and retraction mappings that are used to generalize algorithms from Euclidean spaces to manifolds. It then summarizes several classical optimization methods including gradient descent, conjugate gradient, and variants of quasi-Newton methods adapted to the Riemannian setting using these geometric concepts. The convergence of the Fletcher-Reeves method is analyzed under standard assumptions on the objective function. Overall, the document provides a conceptual and mathematical foundation for optimization on manifolds.
The SlideShare 101 is a quick start guide if you want to walk through the main features that the platform offers. This will keep getting updated as new features are launched.
The SlideShare 101 replaces the earlier "SlideShare Quick Tour".
* Satoshi Hara and Kohei Hayashi. Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach. AISTATS'18 (to appear).
arXiv ver.: https://arxiv.org/abs/1606.09066#
* GitHub
https://github.com/sato9hara/defragTrees
- Hiroaki Shiokawa's research interests include graph mining, network analysis, and efficient algorithms. He was previously employed at NTT from 2011 to 2015.
- His current research focuses on developing clustering algorithms for large-scale networks and evaluating their performance on real-world network datasets.
- He has published highly cited papers in top data mining and network science conferences such as KDD, CIKM, and WSDM.
50. Approximate Message Passing アルゴリズム
AMP アルゴリズム
x(t+1)
= ηt Φ∗
z(t)
+ x(t)
= η Φ∗
z(t)
+ x(t)
; τ(t)
z(t)
= y − Φx(t)
+
1
α
η′
t−1 Φ∗
z(t−1)
+ x(t−1)
τ(t)
=
τ(t−1)
α
η′
t−1 Φ∗
z(t−1)
+ x(t)
•
η(·, b) =
⎧
⎪⎪⎪⎨
⎪⎪⎪⎩
x − b if b < x
0 if − b ≤ x ≤ b
x + b if x < −b
• α = m/n
• ⟨x⟩ = n−1 n
i=1 xi
50 / 54
51. State Evolution
• ファクターグラフが密なグラフなので,LDPC 符号と同様の
解析はできない.
• スピングラス理論で TAP 方程式を解析するために考えられた
conditioning technique [Bolthausen 2009] を使って解析.
⇒ State Evolution (SE) [Bayati+ 2011]
• SE で AMP の性能を記述できることは以前から実験的に分かっ
ていたが,[Bayati+ 2011] によって理論的な証明が与えられた.
51 / 54
52. State Evolution
State Evolution [Bayati+ 2011]
σ2
t = E ||x(t)
− x||2
2 とおく.Φ の各要素が N(0, 1/m) に従うとき,以
下が成り立つ.
σ2
t+1 =
1
α
E |ηt(X + σtZ) − X|2
ただし,X は観測信号の経験分布に従う確率変数,Z ∼ N(0, 1) で,
期待値は X と Z に関してとる.
• 実際にはもう少し一般的な形で証明されている.
• これにより,平均二乗誤差が 0 に収束するかどうかが分かる.
52 / 54