Visual AI(시각 인공지능) Lecture 4 : Multiple Layer Perceptron (MLP)

1. Visual AI(시각인공지능)
Lecture 4 : Multiple Layer Perceptron
(MLP)
1

2. Content
Multilayer Perceptron (MLP)
(Commonly known as neural network by lay person)
Error Function (Loss Function) of MLP
2

3. Multilayer Perceptron (1/2)
3

4. Multilayer Perceptron (2/2)
4

5. Multiple Layer Perceptron (MLP)
MLP는 fully connected임
MLP는 classification, regression 모두에 쓰일 수 있음
5

6. MLP Architectures (1/2)
6

7. MLP Architectures (2/2)
히든 레이어의 활성화 함수로 전통적으로 sigmoid를 사용
왜 step function을 사용하지 않는가? 미분을 가능케 하기 위해
7

8. Other Commonly use Activation Functions
8

9. MLP Architectures
Classification 문제에서 출력 뉴런은 아래와 같이 구성 가능
9

10. Example
10

11. Multiple Layer Perceptron (MLP)
11

12. Illustration
링크에서 체험
12

13. Multiple Layer Perceptron (MLP)
13

14. Multiple Layer Perceptron (MLP)
14

15. 15

16. 16

17. Common Error/Loss Functions used in
MLP
17

18. Recall
18

19. Mean Square Error (MSE) for Regression
MSE Loss
ϵ = (d − S(y ))
ϵ = ϵ
Q = # of training data
k
2
1
j=2
∑
p
j
k
j
k 2
Q
1
k=1
∑
Q
k
19

20. Error Function in the Weight Space
Error function epsilon(w)는 못생김
매우 높은 차원
non-linear
global minima는 도달 불가능할 수 있음
local minima는 bad or good
20

21. 21

22. Ugly Error Function
22

23. Cross-Entropy for Classification
23

24. 24

25. Error Functions
학습단계에 있어 에러 함수는 validation/test 셋을 사용하지 않
음
에러 함수는 학습 단계에서의 이정표 역할을 해줌
-> 에러를 줄여 최적의 가중치를 찾기위해
25

26. 다음강의
GD 알고리즘을 배웠는데 하나의 단일 뉴런의 가중치 최적화를
알아보자.
GD를 MLP에는 어떻게 적용할 수 있을까?
26

27. Quiz 3
27

28. Assumption
If we assume f , f , f is linear function.
We can say a^1 = f^1(w^1p+b^1) becomes also linear function
since (w^1p+b^1) is a linear function. after applying f , each data
points just move within linear space.
1 2 3
1
28

29. Proof
Let's replace f^1, f^2, f^3 = p which means it's a identity
function and to make calculation simpler.
a = f (w f (w f (w p + b ) + b ) + b )
a = f (w f (w (w p + b ) + b ) + b ) since f = p
a = f (w f (w w p + w b + b ) + b )
a = f (w (w w p + w b + b ) + b ) since f = p
a = f (w w w p + w w b + w b + b )
a = w w w p + w w b + w b + b since f = p
a = w w w p + w w b + w b + b
3 3 3 2 2 1 1 1 2 3
3 3 3 2 2 1 1 2 3 1
3 3 3 2 2 1 2 1 2 3
3 3 3 2 1 2 1 2 3 2
3 3 3 2 1 3 2 1 3 2 3
3 3 2 1 3 2 1 3 2 3 3
3 3 2 1 3 2 1 3 2 3
29

30. we can say
a = Ap + C where A = w w w , C = w w b + w b + b
As a result, final activation function a ^ 3 is a single linear neuron.
This only makes 1-dimension decision boundary(hyperplane), so it
can't solve complex classification problem(using more than 2-
dimension hyper plane).
So, we should use non-linear activation function in MLP.
3 3 2 1 3 2 1 3 2 3
30