Hands-on tutorial of deep learning (Keras)Chun-Min Chang
Summary
# Fundamentals of deep learning
--- selection of activation function
--- selection of loss function
--- selection of optimizer
--- effect of learning rate
# How to prevent overfitting
--- Regularization
--- Dropout
--- Early stopping
--- Batch Normalization
C2 discrete time signals and systems in the frequency-domainPei-Che Chang
Discrete-Time Signals and Systems in the Frequency-Domain
Discrete-Time Fourier Transform
time domain convolution theorem
frequency domain convolution theorem
Z transform
Hands-on tutorial of deep learning (Keras)Chun-Min Chang
Summary
# Fundamentals of deep learning
--- selection of activation function
--- selection of loss function
--- selection of optimizer
--- effect of learning rate
# How to prevent overfitting
--- Regularization
--- Dropout
--- Early stopping
--- Batch Normalization
C2 discrete time signals and systems in the frequency-domainPei-Che Chang
Discrete-Time Signals and Systems in the Frequency-Domain
Discrete-Time Fourier Transform
time domain convolution theorem
frequency domain convolution theorem
Z transform
15. 卷積層最佳化原理
梯度下降法應用於卷積層之權重和誤差項
15
卷積神經網路
模型
函式切線斜率(對 偏微分) 函式切線斜率(對 偏微分)
目標函式
iw b
修正方式 修正方式b
ii
i
ii xw
w
F
ww
b
b
F
bb
i
i
ii
x
x
w
y
y
F
w
F
1
ˆ
ˆ
2
2
2
1
ˆ
2
1
,
yybWF
bxwy
i
ii
4
1
ˆ
iw
11
ˆ
ˆ b
y
y
F
b
F
52. 卷積神經網路其他型態-分類最佳化
梯度下降法應用於卷積層之權重和誤差項
52
卷積神經網路模型 函式切線斜率(對 偏微分) 函式切線斜率(對 偏微分)
目標函式
iw b
修正方式 修正方式bˆ
ii
i
ii x
yy
yy
w
w
F
ww
ˆ1ˆ
ˆ
yy
yy
b
b
F
bb
ˆ1ˆ
ˆ
ii
ii
x
yy
yy
x
y
y
y
y
w
y
y
F
w
F
ˆ1ˆ
ˆ
ˆ1
1
ˆ
ˆ
ˆ
bxwy
i
ii
4
1
ˆ
iw
yyyybWF ˆ1ln1ˆln,
yy
yy
y
y
y
y
b
y
y
F
b
F
ˆ1ˆ
ˆ
1
ˆ1
1
ˆ
ˆ
ˆ
62. 卷積神經網路其他型態-動量最佳化
梯度下降法應用於卷積層之權重和誤差項
62
卷積神經網路
模型
函式切線斜率(對 偏微分) 函式切線斜率(對 偏微分)
目標函式
iw b
修正方式 修正方式b
i
tt
w
t
i
t
w
t
i
t
i
t
i
t
w
t
w
xvw
vww
w
F
vv
i
i
ii
1
1
11
1
1
i
i
ii
x
x
w
y
y
F
w
F
1
ˆ
ˆ
2
2
2
1
ˆ
2
1
,
yybWF
bxwy
i
ii
4
1
ˆ
iw
11
ˆ
ˆ b
y
y
F
b
F
tt
b
t
i
t
b
t
i
t
i
t
t
b
t
b
vw
vww
b
F
vv
1
1
11
1
1
第t個時間點之修正方式,主要多參考第t-1時間點之修正值
採用動量Momentum
網路結構不變
核心函式不變
目標函式不變