This document discusses neural networks and deep learning concepts such as artificial neurons, edges, weights, biases, activation functions, backpropagation, optimization algorithms like stochastic gradient descent, and neural network architectures like convolutional neural networks. It provides examples of neural network calculations and discusses tasks like image classification using datasets such as ImageNet and CIFAR-10.

1. 2021 2 26
~

2. 2 Copyright © 2021 Oracle and/or its affiliates.
•
•
• (NN)
•
•
•
• NN
•
•

3. 3 Copyright © 2021 Oracle and/or its affiliates.
•
•
•
• Etc.
?%

4. 4 Copyright © 2021 Oracle and/or its affiliates.
•
•
•
• K-means
• SVM
• etc.
•

5. 5 Copyright © 2021 Oracle and/or its affiliates.
: Wikipedia
https://ja.wikipedia.org/wiki/%E7%A5%9E%E7%B5%8C%E7%B4%B0%E8%83%9E
• ニューロン：脳の神経細胞
• シナプス：ニューロンと他のニューロンとの接合部分
• シナプス結合：ニューロンから別のニューロンへ信号が伝達される状態
• 活性化：ニューロンが受け取った刺激の大きさが特定の大きさを超える
と、次のニューロンへと信号を伝達する。この状態を活性化と呼ぶ。
• 人工ニューロン(ノード)：ニューロンに相当。重みとバイアスを持つ。
• エッジ：シナプス結合に相当。ノード間を接続する。
• 重み、バイアス：シナプス結合の強度。ノード間接続時の信号の大きさの
計算に利用される。
• 活性化関数：ノード間の信号の大きさを計算するための関数。次ノードへ
伝える値を決める関数。

6. 6 Copyright © 2021 Oracle and/or its affiliates.
( )
𝑦 = 𝑓 ෍
𝑖=0
𝑛
𝑤𝑖𝑥𝑖 + 𝑏
𝑥0w0 + 𝑥1 w1 +… +𝑥𝑛𝑤𝑛 + 𝑏 =
b
input x w
n
𝑥0
𝑥1
𝑥2
w0
w1
𝑤2
y
𝑥𝑛
input
output
𝑏
0 ⇒ 1
0 ⇒ 0
• ⇒ 1 ⇒ 0
• ⇒ 1 ⇒ 0
• ⇒ 1 ⇒ 0
input output
• ⇒ 1
• ⇒ 0
→ 1
→ 0
→ 1
→ 2
→ 1
→ 3
→ 1
→ 3
→ 7
1*2 + 0*1 + 1*3 +3 = 8
7 ⇒ 1
7 ⇒ 0
)

7. 7 Copyright © 2021 Oracle and/or its affiliates.
• (
)
•
• (
)
• (
)
•
𝑥0
𝑥1
𝑥2
w0
w1
𝑤2
y
𝑥𝑛
input
output
𝑏
)
65
80
70
A
45
60
98
B
65
55
80
C
65
70
63
1
3
2
40
38
43

8. 8 Copyright © 2021 Oracle and/or its affiliates.
ReLU
Tanh
Sigmoid
Step
• 0
0 0
1
• NN (※
)
• 0 1
• 1
0
• -1 1
• 1
-1
• 0
0 0
• NN

9. 9 Copyright © 2021 Oracle and/or its affiliates.
ReLU
Tanh
Sigmoid
Step
• 0
0 0
1
• NN (※
)
• 0 1
• 1
0
• -1 1
• 1
-1
• 0
0 0
• NN
( ) ( )

10. 10 Copyright © 2021 Oracle and/or its affiliates.
Softmax
•
•
• ( )
(※ Sigmoid )
• 0 1
• 1
•
(※ )

11. 11 Copyright © 2021 Oracle and/or its affiliates.
Softmax
ReLU
ReLU 0.8
Softmax
ReLU
ReLU 0.2
1
0
Cat or Tiger ?
( )
Cat
Tiger
𝑥0
𝑥1
𝑥2
𝑏 𝑏 𝑏
𝑏
𝑏
𝑏
w
w
w
w
w
w
w
w
w
w
w
w
w
w
[Cat, Tiger] = [1, 0]
100 pixel
60
pixel
w ( )
( )
( ) w ( )

12. 12 Copyright © 2021 Oracle and/or its affiliates.
= ( )
= ( ) (w)
Softmax
ReLU
ReLU 0.8
Softmax
ReLU
ReLU 0.2
1
0
Cat or Tiger ?
( )
Cat
Tiger
𝑥0
𝑥1
𝑥2
𝑏 𝑏 𝑏
𝑏
𝑏
𝑏
w
w
w
w
w
w
w
w
w
w
w
w
w
w
[Cat, Tiger] = [1, 0]
100 pixel
60
pixel
( )

13. 13 Copyright © 2021 Oracle and/or its affiliates.
( )
w
( )

14. 14 Copyright © 2021 Oracle and/or its affiliates.
( )
≒(( ) – ( ))2
)
• 100
• A 90
• B 120
)
• ⇒ [Cat, Tiger] = [1, 0]
• A ⇒ [Cat, Tiger] = [0.8, 0.2]
• B ⇒ [Cat, Tiger] = [0.2, 0.8]
A
(1 x log0.8 + 0 x log0.2)
(log0.8)
(0.09)
0.09 ⇒
≒ log(( ) x ( ))
B
(1 x log0.2 + 0 x log0.8)
(log0.2)
(0.69)
0.69 ⇒
A
(100 - 90)2
100 ⇒
B
(100 - 120)2
400 ⇒

15. 15 Copyright © 2021 Oracle and/or its affiliates.
(Stochastic Gradient Descent, SGD)
(w) (w) (w)
( ) )
(w)

16. 16 Copyright © 2021 Oracle and/or its affiliates.
Stochastic Gradient Descent(SGD) w
momentumSGD SGD
AdaGrad SGD
RMSprop AdaGrad
AdaDelta SGD
Adaptive Moment Estimation(Adam) momentumSGD RMSprop
(SDG)

17. 17 Copyright © 2021 Oracle and/or its affiliates.

18. 18 Copyright © 2021 Oracle and/or its affiliates.
CNN ( )
• 28x28 /
• 6
•
0 9 0 9
• 28x28 /
• 1
•

19. 19 Copyright © 2021 Oracle and/or its affiliates.
CNN
784
2
16 10
768
1
16
0
1
2
8
9
8
784 786
10 0 9 10
768 x 1
input
input
input
input

20. 20 Copyright © 2021 Oracle and/or its affiliates.
•
•
•
•
•
•
•
•
W
W W

21. 21 Copyright © 2021 Oracle and/or its affiliates.
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
•
( )
• w (
)
• w

22. 22 Copyright © 2021 Oracle and/or its affiliates.
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w

23. 23 Copyright © 2021 Oracle and/or its affiliates.

24. 24 Copyright © 2021 Oracle and/or its affiliates.

25. 25 Copyright © 2021 Oracle and/or its affiliates.
: http://image-net.org
: https://cifar.ca/
• ImageNet
• 160 x 160 pixel
• 140
•
• CIFAR-10
• 32 x 32 pixel
• 5
•
• MobileNetV2
• 1000
• 10
W
W
W
※MobileNetV2by Google)
NN
?%