SlideShare a Scribd company logo

AlexNet, VGG, GoogleNet, Resnet

•Download as PPTX, PDF•
•
Jungwon Kim
Jungwon Kim

https://drive.google.com/open?id=1XylJLmwjhIiagep6p-hfQIZEYmMPzlgv74lSv-V8BZw

Technology
1 of 25
AlexNet, VGG, GoogLeNet, ResNet 응답하라AI - Season 4 김정원
AlexNet(2012) - Architecture
AlexNet - 특징 1. 양방향 구조의 Network ○ Hardware 한계 2. ReLU 3. Dropout regularization 4. Data Augmentation
AlexNet - ReLU ● Rectified Linear Unit ● Activation 함수 ○ x = Max(0, x) ● AlexNet 논문에서 처음 선보인 개념
AlexNet - ReLU 무엇을 해결했을까? ● Gradient Vanishing ○ 0.xxx ... ● 느린 처리속도 ※ 주의: Network 구성시 마지막 Layer(보통 FC)는 ReLU가 아닌 Sigmoid/Tanh를 사용
AlexNet - Dropout ● AlexNet에서 사용한 방법 ○ Output에 0.5를 곱해준다. (Multiply) ● 현재 사용되는 방법 ○ 일정 확률로 Output을 0으로 만든다.
AlexNet - Dropout ● AlexNet에서 사용한 방법 ○ Output에 0.5를 곱해준다. (Multiply) ● 현재 사용되는 방법 ○ 일정 확률로 Output을 0으로 만든다.
AlexNet - Data Augmentation ● 한정된 수량의 Dataset을 보유했을때 Data의 수량을 늘려주는 Technique ● Neural Network의 구조로 인해 늘어난 Parameter에 비례하여 Data의 양을 늘릴 필요가 있음
AlexNet - Data Augmentation (Methods) 일반적인 방법 ● Flip (반전) ● Rotation (회전) ● Scale (스케일링, 크기조정) ● Crop (잘라내기) ● Translation (좌표조정) ● (Gaussian) Noise (노이즈 삽입)
AlexNet - Data Augmentation 특수한 방법 ● Constant - Unknown 부분을 특정 색으로 채워넣기 ● Edge - Boundary의 색을 Extend ● Reflect - Boundary를 기준으로 좌표대비 RGB값을 Reflect하여 채워넣기 ● Wrap - Boundary를 기준으로 이미지를 반복 이것 외에도 다양한 방법들이 있음!
AlexNet - Data Augmentation ● 한정된 수량의 Dataset을 보유했을때 Data의 수량을 늘려주는 Technique ● Neural Network의 구조로 인해 늘어난 Parameter에 비례하여 Data의 양을 늘릴 필요가 있음
AlexNet - Data Augmentation ● 한정된 수량의 Dataset을 보유했을때 Data의 수량을 늘려주는 Technique ● Neural Network의 구조로 인해 늘어난 Parameter에 비례하여 Data의 양을 늘릴 필요가 있음
VGG (2014)- Architecture
VGG - Improvements ● 3x3 Conv. Filter의 사용 ○ Parameter 수를 줄여주는 역할 ● 더 많은 Layer의 사용을 보여줌 ○ 그래도 ILSVRC 2013 대회에서 우승하지는 못함
GoogLeNet(2014) - Architecture
GoogLeNet - Improvements ● 실제 성능의 향상 ○ ILSVRC 2014 우승 ○ 실제 사람의 performance에 가까운 성능 (Top 5, 6.67%) ● Fully Connected Layer의 제거 ○ 이름에 ‘LeNet’이 들어가게 된 이유 (Google + LeNet) ○ Parameter 개수 감소 ● Inception Module의 첫 소개 ○ Filter Concatenate ○ (Actual Inception) 1x1 Conv. Filter ■ Parameter 개수 감소
GoogLeNet - Inception Module 참고링크: Google Inception Model. (한국어) - 결국 기본적인 컨셉은 Divide and Conquer
ResNet(2015) - Architecture
ResNet - Improvement ● Even deeper ○ ILSVRC 2015 우승 ○ 152개의 Layer 구조 ■ Residual Learning ● Batch Normalization의 더 다양한 사용법 ● Dropout을 사용하지 않은 구조
ResNet - Residual Learning ● Input을 Activation에 넣는 것이 아닌, 학습 후 Input과의 차이만을 학습 ○ Identity Mapping ● 참조링크 ○ Identity Mappings in Deep Residual Networks (한국어)
ResNet - Residual Learning (Shortcut Connections) 실험 결과는, original 빼고 다 별로였음
ResNet - Residual Learning(Activation Functions)
ResNet - Residual Learning(Activation Functions)
ResNet - Current Problem ● Layer 갯수를 무한정 늘릴 수 있는것은 아님 ○ 아직도 Gradient Vanishing 문제는 존재 ○ 다만 Gradient Vanishing이 생기는 지점을 Delay 했을 뿐
Other Neural Networks ● Xception - Inception + Depthwise Separable Conv. ● MobileNet ● DenseNet ● NAS/ENAS Network - AutoML/AutoKeras ● FlowNet ● Siamese Network ● 등등...

Recommended

Convolutional Neural Network (CNN) - image recognition
Convolutional Neural Network (CNN) - image recognitionConvolutional Neural Network (CNN) - image recognition
Convolutional Neural Network (CNN) - image recognitionYUNG-KUEI CHEN
 
Deep Learning Tutorial
Deep Learning TutorialDeep Learning Tutorial
Deep Learning TutorialAmr Rashed
 
Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual IntroductionLukas Masuch
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksChristian Perone
 
Convolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetConvolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetSungminYou
 
Machine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkMachine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkRichard Kuo
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Muhammad Haroon
 
Convolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsConvolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsKasun Chinthaka Piyarathna
 

Recommended

Convolutional Neural Network (CNN) - image recognition
Convolutional Neural Network (CNN) - image recognitionConvolutional Neural Network (CNN) - image recognition
Convolutional Neural Network (CNN) - image recognitionYUNG-KUEI CHEN
 
Deep Learning Tutorial
Deep Learning TutorialDeep Learning Tutorial
Deep Learning TutorialAmr Rashed
 
Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual IntroductionLukas Masuch
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksChristian Perone
 
Convolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetConvolutional neural network from VGG to DenseNet
Convolutional neural network from VGG to DenseNetSungminYou
 
Machine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkMachine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkRichard Kuo
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Muhammad Haroon
 
Convolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsConvolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsKasun Chinthaka Piyarathna
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networksSi Haem
 
Deep Learning
Deep LearningDeep Learning
Deep LearningShaikh Shahzad
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network Yan Xu
 
Image Classification using deep learning
Image Classification using deep learning Image Classification using deep learning
Image Classification using deep learning Asma-AH
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learningleopauly
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural networkMojammilHusain
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowSri Ambati
 
Deep learning
Deep learningDeep learning
Deep learningPratap Dangeti
 
AlexNet
AlexNetAlexNet
AlexNetBertil Hatt
 
CONVOLUTIONAL NEURAL NETWORK
CONVOLUTIONAL NEURAL NETWORKCONVOLUTIONAL NEURAL NETWORK
CONVOLUTIONAL NEURAL NETWORKMd Rajib Bhuiyan
 
Wrapper feature selection method
Wrapper feature selection methodWrapper feature selection method
Wrapper feature selection methodAmir Razmjou
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNNShuai Zhang
 
Intro to deep learning
Intro to deep learning Intro to deep learning
Intro to deep learning David Voyles
 
Introduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkIntroduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkKnoldus Inc.
 
Convolutional Neural Networks
Convolutional Neural NetworksConvolutional Neural Networks
Convolutional Neural NetworksAshray Bhandare
 
CNN Tutorial
CNN TutorialCNN Tutorial
CNN TutorialSungjoon Choi
 
Introduction to Generative Adversarial Networks (GANs)
Introduction to Generative Adversarial Networks (GANs)Introduction to Generative Adversarial Networks (GANs)
Introduction to Generative Adversarial Networks (GANs)Appsilon Data Science
 
04 Multi-layer Feedforward Networks
04 Multi-layer Feedforward Networks04 Multi-layer Feedforward Networks
04 Multi-layer Feedforward NetworksTamer Ahmed Farrag, PhD
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Gaurav Mittal
 
Activation functions
Activation functionsActivation functions
Activation functionsPRATEEK SAHU
 
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNetjdo
 
밑바닥부터 시작하는딥러닝 8장
밑바닥부터 시작하는딥러닝 8장밑바닥부터 시작하는딥러닝 8장
밑바닥부터 시작하는딥러닝 8장Sunggon Song
 

More Related Content

What's hot

Deep neural networks
Deep neural networksDeep neural networks
Deep neural networksSi Haem
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network Yan Xu
 
Image Classification using deep learning
Image Classification using deep learning Image Classification using deep learning
Image Classification using deep learning Asma-AH
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learningleopauly
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural networkMojammilHusain
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowSri Ambati
 
CONVOLUTIONAL NEURAL NETWORK
CONVOLUTIONAL NEURAL NETWORKCONVOLUTIONAL NEURAL NETWORK
CONVOLUTIONAL NEURAL NETWORKMd Rajib Bhuiyan
 
Wrapper feature selection method
Wrapper feature selection methodWrapper feature selection method
Wrapper feature selection methodAmir Razmjou
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNNShuai Zhang
 
Intro to deep learning
Intro to deep learning Intro to deep learning
Intro to deep learning David Voyles
 
Introduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkIntroduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkKnoldus Inc.
 
Convolutional Neural Networks
Convolutional Neural NetworksConvolutional Neural Networks
Convolutional Neural NetworksAshray Bhandare
 
Introduction to Generative Adversarial Networks (GANs)
Introduction to Generative Adversarial Networks (GANs)Introduction to Generative Adversarial Networks (GANs)
Introduction to Generative Adversarial Networks (GANs)Appsilon Data Science
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Gaurav Mittal
 
Activation functions
Activation functionsActivation functions
Activation functionsPRATEEK SAHU
 

What's hot (20)

Deep neural networks
Deep neural networksDeep neural networks
Deep neural networks
 
Deep Learning
Deep LearningDeep Learning
Deep Learning
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network
 
Image Classification using deep learning
Image Classification using deep learning Image Classification using deep learning
Image Classification using deep learning
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural network
 
Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlow
 
Deep learning
Deep learningDeep learning
Deep learning
 
AlexNet
AlexNetAlexNet
AlexNet
 
CONVOLUTIONAL NEURAL NETWORK
CONVOLUTIONAL NEURAL NETWORKCONVOLUTIONAL NEURAL NETWORK
CONVOLUTIONAL NEURAL NETWORK
 
Wrapper feature selection method
Wrapper feature selection methodWrapper feature selection method
Wrapper feature selection method
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNN
 
Intro to deep learning
Intro to deep learning Intro to deep learning
Intro to deep learning
 
Introduction to Recurrent Neural Network
Introduction to Recurrent Neural NetworkIntroduction to Recurrent Neural Network
Introduction to Recurrent Neural Network
 
Convolutional Neural Networks
Convolutional Neural NetworksConvolutional Neural Networks
Convolutional Neural Networks
 
CNN Tutorial
CNN TutorialCNN Tutorial
CNN Tutorial
 
Introduction to Generative Adversarial Networks (GANs)
Introduction to Generative Adversarial Networks (GANs)Introduction to Generative Adversarial Networks (GANs)
Introduction to Generative Adversarial Networks (GANs)
 
04 Multi-layer Feedforward Networks
04 Multi-layer Feedforward Networks04 Multi-layer Feedforward Networks
04 Multi-layer Feedforward Networks
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
 
Activation functions
Activation functionsActivation functions
Activation functions
 

Similar to AlexNet, VGG, GoogleNet, Resnet

[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNetjdo
 
밑바닥부터 시작하는딥러닝 8장
밑바닥부터 시작하는딥러닝 8장밑바닥부터 시작하는딥러닝 8장
밑바닥부터 시작하는딥러닝 8장Sunggon Song
 
Weakly supervised localization
Weakly supervised localizationWeakly supervised localization
Weakly supervised localizationJungwon Kim
 
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 5 - Others
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 5 - Others [컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 5 - Others
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 5 - Othersjdo
 
History of Vision AI
History of Vision AIHistory of Vision AI
History of Vision AITae Young Lee
 
Image net classification with deep convolutional neural networks
Image net classification with deep convolutional neural networks Image net classification with deep convolutional neural networks
Image net classification with deep convolutional neural networks Korea, Sejong University.
 
진화하는 컴퓨터 하드웨어와 게임 개발 기술의 발전
진화하는 컴퓨터 하드웨어와 게임 개발 기술의 발전진화하는 컴퓨터 하드웨어와 게임 개발 기술의 발전
진화하는 컴퓨터 하드웨어와 게임 개발 기술의 발전Sukwoo Lee
 
[자바카페] 람다 일괄처리 계층
[자바카페] 람다 일괄처리 계층[자바카페] 람다 일괄처리 계층
[자바카페] 람다 일괄처리 계층용호 최
 
"Learning transferable architectures for scalable image recognition" Paper Re...
"Learning transferable architectures for scalable image recognition" Paper Re..."Learning transferable architectures for scalable image recognition" Paper Re...
"Learning transferable architectures for scalable image recognition" Paper Re...LEE HOSEONG
 
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 2 - ZFNet, VGG-16
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 2 - ZFNet, VGG-16 [컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 2 - ZFNet, VGG-16
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 2 - ZFNet, VGG-16jdo
 
BigQuery Data Lake & Airflow Contribution
BigQuery Data Lake & Airflow ContributionBigQuery Data Lake & Airflow Contribution
BigQuery Data Lake & Airflow ContributionJason Kim
 
Image Deep Learning 실무적용
Image Deep Learning 실무적용Image Deep Learning 실무적용
Image Deep Learning 실무적용Youngjae Kim
 
Exploring Deep Learning Acceleration Technology Embedded in LLMs
Exploring Deep Learning Acceleration Technology Embedded in LLMsExploring Deep Learning Acceleration Technology Embedded in LLMs
Exploring Deep Learning Acceleration Technology Embedded in LLMsTae Young Lee
 
Bag of Tricks for Image Classification with Convolutional Neural Networks (C...
Bag of Tricks for Image Classification with Convolutional Neural Networks (C...Bag of Tricks for Image Classification with Convolutional Neural Networks (C...
Bag of Tricks for Image Classification with Convolutional Neural Networks (C...gohyunwoong
 

Similar to AlexNet, VGG, GoogleNet, Resnet (20)

[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 3 - GoogLeNet
 
밑바닥부터 시작하는딥러닝 8장
밑바닥부터 시작하는딥러닝 8장밑바닥부터 시작하는딥러닝 8장
밑바닥부터 시작하는딥러닝 8장
 
Weakly supervised localization
Weakly supervised localizationWeakly supervised localization
Weakly supervised localization
 
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 5 - Others
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 5 - Others [컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 5 - Others
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 5 - Others
 
LeNet & GoogLeNet
LeNet & GoogLeNetLeNet & GoogLeNet
LeNet & GoogLeNet
 
History of Vision AI
History of Vision AIHistory of Vision AI
History of Vision AI
 
Image net classification with deep convolutional neural networks
Image net classification with deep convolutional neural networks Image net classification with deep convolutional neural networks
Image net classification with deep convolutional neural networks
 
진화하는 컴퓨터 하드웨어와 게임 개발 기술의 발전
진화하는 컴퓨터 하드웨어와 게임 개발 기술의 발전진화하는 컴퓨터 하드웨어와 게임 개발 기술의 발전
진화하는 컴퓨터 하드웨어와 게임 개발 기술의 발전
 
[자바카페] 람다 일괄처리 계층
[자바카페] 람다 일괄처리 계층[자바카페] 람다 일괄처리 계층
[자바카페] 람다 일괄처리 계층
 
"Learning transferable architectures for scalable image recognition" Paper Re...
"Learning transferable architectures for scalable image recognition" Paper Re..."Learning transferable architectures for scalable image recognition" Paper Re...
"Learning transferable architectures for scalable image recognition" Paper Re...
 
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 2 - ZFNet, VGG-16
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 2 - ZFNet, VGG-16 [컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 2 - ZFNet, VGG-16
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 2 - ZFNet, VGG-16
 
BigQuery Data Lake & Airflow Contribution
BigQuery Data Lake & Airflow ContributionBigQuery Data Lake & Airflow Contribution
BigQuery Data Lake & Airflow Contribution
 
Image Deep Learning 실무적용
Image Deep Learning 실무적용Image Deep Learning 실무적용
Image Deep Learning 실무적용
 
Net adapt
Net adaptNet adapt
Net adapt
 
AUTOML
AUTOMLAUTOML
AUTOML
 
Automl
AutomlAutoml
Automl
 
GoogLenet
GoogLenetGoogLenet
GoogLenet
 
Exploring Deep Learning Acceleration Technology Embedded in LLMs
Exploring Deep Learning Acceleration Technology Embedded in LLMsExploring Deep Learning Acceleration Technology Embedded in LLMs
Exploring Deep Learning Acceleration Technology Embedded in LLMs
 
Redis
RedisRedis
Redis
 
Bag of Tricks for Image Classification with Convolutional Neural Networks (C...
Bag of Tricks for Image Classification with Convolutional Neural Networks (C...Bag of Tricks for Image Classification with Convolutional Neural Networks (C...
Bag of Tricks for Image Classification with Convolutional Neural Networks (C...
 

AlexNet, VGG, GoogleNet, Resnet

  • 1. AlexNet, VGG, GoogLeNet, ResNet 응답하라AI - Season 4 김정원
  • 3. AlexNet - 특징 1. 양방향 구조의 Network ○ Hardware 한계 2. ReLU 3. Dropout regularization 4. Data Augmentation
  • 4. AlexNet - ReLU ● Rectified Linear Unit ● Activation 함수 ○ x = Max(0, x) ● AlexNet 논문에서 처음 선보인 개념
  • 5. AlexNet - ReLU 무엇을 해결했을까? ● Gradient Vanishing ○ 0.xxx ... ● 느린 처리속도 ※ 주의: Network 구성시 마지막 Layer(보통 FC)는 ReLU가 아닌 Sigmoid/Tanh를 사용
  • 6. AlexNet - Dropout ● AlexNet에서 사용한 방법 ○ Output에 0.5를 곱해준다. (Multiply) ● 현재 사용되는 방법 ○ 일정 확률로 Output을 0으로 만든다.
  • 7. AlexNet - Dropout ● AlexNet에서 사용한 방법 ○ Output에 0.5를 곱해준다. (Multiply) ● 현재 사용되는 방법 ○ 일정 확률로 Output을 0으로 만든다.
  • 8. AlexNet - Data Augmentation ● 한정된 수량의 Dataset을 보유했을때 Data의 수량을 늘려주는 Technique ● Neural Network의 구조로 인해 늘어난 Parameter에 비례하여 Data의 양을 늘릴 필요가 있음
  • 9. AlexNet - Data Augmentation (Methods) 일반적인 방법 ● Flip (반전) ● Rotation (회전) ● Scale (스케일링, 크기조정) ● Crop (잘라내기) ● Translation (좌표조정) ● (Gaussian) Noise (노이즈 삽입)
  • 10. AlexNet - Data Augmentation 특수한 방법 ● Constant - Unknown 부분을 특정 색으로 채워넣기 ● Edge - Boundary의 색을 Extend ● Reflect - Boundary를 기준으로 좌표대비 RGB값을 Reflect하여 채워넣기 ● Wrap - Boundary를 기준으로 이미지를 반복 이것 외에도 다양한 방법들이 있음!
  • 11. AlexNet - Data Augmentation ● 한정된 수량의 Dataset을 보유했을때 Data의 수량을 늘려주는 Technique ● Neural Network의 구조로 인해 늘어난 Parameter에 비례하여 Data의 양을 늘릴 필요가 있음
  • 12. AlexNet - Data Augmentation ● 한정된 수량의 Dataset을 보유했을때 Data의 수량을 늘려주는 Technique ● Neural Network의 구조로 인해 늘어난 Parameter에 비례하여 Data의 양을 늘릴 필요가 있음
  • 14. VGG - Improvements ● 3x3 Conv. Filter의 사용 ○ Parameter 수를 줄여주는 역할 ● 더 많은 Layer의 사용을 보여줌 ○ 그래도 ILSVRC 2013 대회에서 우승하지는 못함
  • 16. GoogLeNet - Improvements ● 실제 성능의 향상 ○ ILSVRC 2014 우승 ○ 실제 사람의 performance에 가까운 성능 (Top 5, 6.67%) ● Fully Connected Layer의 제거 ○ 이름에 ‘LeNet’이 들어가게 된 이유 (Google + LeNet) ○ Parameter 개수 감소 ● Inception Module의 첫 소개 ○ Filter Concatenate ○ (Actual Inception) 1x1 Conv. Filter ■ Parameter 개수 감소
  • 17. GoogLeNet - Inception Module 참고링크: Google Inception Model. (한국어) - 결국 기본적인 컨셉은 Divide and Conquer
  • 19. ResNet - Improvement ● Even deeper ○ ILSVRC 2015 우승 ○ 152개의 Layer 구조 ■ Residual Learning ● Batch Normalization의 더 다양한 사용법 ● Dropout을 사용하지 않은 구조
  • 20. ResNet - Residual Learning ● Input을 Activation에 넣는 것이 아닌, 학습 후 Input과의 차이만을 학습 ○ Identity Mapping ● 참조링크 ○ Identity Mappings in Deep Residual Networks (한국어)
  • 21. ResNet - Residual Learning (Shortcut Connections) 실험 결과는, original 빼고 다 별로였음
  • 22. ResNet - Residual Learning(Activation Functions)
  • 23. ResNet - Residual Learning(Activation Functions)
  • 24. ResNet - Current Problem ● Layer 갯수를 무한정 늘릴 수 있는것은 아님 ○ 아직도 Gradient Vanishing 문제는 존재 ○ 다만 Gradient Vanishing이 생기는 지점을 Delay 했을 뿐
  • 25. Other Neural Networks ● Xception - Inception + Depthwise Separable Conv. ● MobileNet ● DenseNet ● NAS/ENAS Network - AutoML/AutoKeras ● FlowNet ● Siamese Network ● 등등...