SlideShare a Scribd company logo

Google net

•Download as PPTX, PDF•
•
B
Brian Kim

Slide share for deep learning google network.

Data & Analytics
1 of 57
Korea University, Department of Computer Science & Radio Communication Engineering 2016010646 Bumsoo Kim Google NetEverything about GoogleNet and its architecture Bumsoo Kim Presentation 2017 1
Bumsoo Kim Presentation 2017 2 Whatis‘GoogleNet’? GoogleNet
Bumsoo Kim Presentation 2017 3 Whatis‘GoogleNet’? GoogleNet
Bumsoo Kim Presentation 2017 4 Whatis‘GoogleNet’? GoogleNet
Bumsoo Kim Presentation 2017 5 Whatis‘GoogleNet’? GoogleNet
Bumsoo Kim Presentation 2017 6 Whatis‘GoogleNet’? GoogleNet ILSVRC(ImageNet) 2014 Winning Model Google, “Going Deeper with Convolutions”(CVPR2015) Top5 Error Rate = 6.7% Revolution of Depth (22 layers)
Bumsoo Kim Presentation 2017 7 Howdeepis‘GoogleNet’? RevolutionofDepth
Bumsoo Kim Presentation 2017 8 Deeperisbetter? RevolutionofDepth The winning models of 2014 was distinguishable by its depth!
Bumsoo Kim Presentation 2017 9 Deeperisbetter? RevolutionofDepth The winning models of 2014 was distinguishable by its depth! So, does deeper network is the key to better performance?
Bumsoo Kim Presentation 2017 10 Deeperisbetter? RevolutionofDepth The winning models of 2014 was distinguishable by its depth! So, does deeper network is the key to better performance? The answer is…
Bumsoo Kim Presentation 2017 11 Deeperisbetter? RevolutionofDepth The winning models of 2014 was distinguishable by its depth! So, does deeper network is the key to better performance? The answer is… NO!
Bumsoo Kim Presentation 2017 12 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
Bumsoo Kim Presentation 2017 13 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
Bumsoo Kim Presentation 2017 14 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
Bumsoo Kim Presentation 2017 15 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
Bumsoo Kim Presentation 2017 16 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
Bumsoo Kim Presentation 2017 17 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
Bumsoo Kim Presentation 2017 18 ZF-Netdidnotprovideafundamentalsolution ObstaclesofDepth ZF-net fundamentally failed in increasing depth
Bumsoo Kim Presentation 2017 19 ZF-Netdidnotprovideafundamentalsolution ObstaclesofDepth ZF-net fundamentally failed in increasing depth 2014 top models succeeded in addressing this problem
Bumsoo Kim Presentation 2017 20 ZF-Netdidnotprovideafundamentalsolution ObstaclesofDepth ZF-net fundamentally failed in increasing depth 2014 top models succeeded in addressing this problem HOW?
Bumsoo Kim Presentation 2017 21 UniquemoduleofGooglenet Inception
Bumsoo Kim Presentation 2017 22 NetworkinNetwork Inceptionnodules
Bumsoo Kim Presentation 2017 23 NetworkinNetwork Inceptionnodules NIN (Network In Network) – Min Lin, 2013, Network in Network local receptive linear features local receptive linear+non-linear features non-linear features
Bumsoo Kim Presentation 2017 24 NetworkinNetwork Inceptionnodules NIN (Network In Network) – Min Lin, 2013, Network in Network non-linear features 90% of computation
Bumsoo Kim Presentation 2017 25 NetworkinNetwork Inceptionnodules NIN (Network In Network) – Min Lin, 2013, Network in Network Inception Average Pooling
Bumsoo Kim Presentation 2017 26 1x1Convolutions Inceptionnodules
Bumsoo Kim Presentation 2017 27 1x1Convolutions Inceptionnodules Original model
Bumsoo Kim Presentation 2017 28 1x1Convolutions Inceptionnodules Original model Various filters for various feature scales
Bumsoo Kim Presentation 2017 29 1x1Convolutions Inceptionnodules Original model Expensive unit : Larger filter sizes are very expensive!!
Bumsoo Kim Presentation 2017 30 1x1Convolutions Inceptionnodules Original model 1x1 Convolution
Bumsoo Kim Presentation 2017 31 1x1Convolutions Inceptionnodules 1x1 Convolution Dimension reduction Hebbian Principle Neurons that fire together, wire together 동일 차원에서 activate된 neuron들은 비슷한 성질을 가진 것들로 묶일 수 있다. c2 (large) c3 (small)
Bumsoo Kim Presentation 2017 32 Dowereallyneed5x5filters? Inception Factorizing Convolutions 5*5 = 25 → 2*(3*3) = 18 => 28% decrease of parameters!
Bumsoo Kim Presentation 2017 33 Dowereallyneed5x5filters? Inception Factorizing Convolutions 5*5 = 25 → 2*(3*3) = 18 => 28% decrease of parameters!
Bumsoo Kim Presentation 2017 34 HowEffectiveisit? Inception 130M 4.5B
Bumsoo Kim Presentation 2017 35 Whydoweneedit? AuxiliaryClassifier Gradient Vanishing Problem ReLU doesn’t provide a fundamental solution Overlapping multiple ReLUs cause the same problem!
Bumsoo Kim Presentation 2017 36 Whatiswrongaboutit? GradientVanishing Vanishing Gradients on Sigmoids * Deep Networks use activation function for non-linearity. *“Saturated”: Weight ‘W’ is too large. => ‘z’is 0 or 1 => z*(1-z) = 0 z = 1/(1 + np.exp(-np.dot(W, x))) # forward pass dx = np.dot(W.T, z*(1-z)) # backward pass: local gradient for x dW = np.outer(z*(1-z), x) # backward pass: local gradient for W * Local gradient is small : 0 <= z*(1-z) <= 0.25 (when z=0.5) : top layers are not trained
Bumsoo Kim Presentation 2017 37 Detailsarenotexplainedinthepaper AuxiliaryClassifier Training Deeper Convolutional Networks with Deep SuperVision(2014, Liwei Chang, Chen-Yu Lee)
Bumsoo Kim Presentation 2017 38 BeforeGradientVanishes! AuxiliaryClassifier Training Deeper Convolutional Networks with Deep SuperVision(2014, Liwei Chang, Chen-Yu Lee) 깊이가 깊어질수록 vanishing될 우려가 높아지므로, 얕은 깊이에서도 업데이트!
Bumsoo Kim Presentation 2017 39 EmpiricalDecision AuxiliaryClassifier Training Deeper Convolutional Networks with Deep SuperVision(2014, Liwei Chang, Chen-Yu Lee) 넣는 위치는 이론적 근거 없이 순수하게 “경험적”으로 판단한다.
Bumsoo Kim Presentation 2017 40 HowEffectiveisit? AuxiliaryClassifier Training Deeper Convolutional Networks with Deep SuperVision(2014, Liwei Chang, Chen-Yu Lee)
Bumsoo Kim Presentation 2017 41 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 42 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 43 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 44 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 45 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 46 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 47 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 48 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 49 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 50 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 51 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 52 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 53 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 55 Trainityourself! GoogleNet
Bumsoo Kim Presentation 2017 56 Trainityourself! GoogleNet
Thank You Bumsoo Kim Presentation 2017 57

Recommended

GoogLeNet.pptx
GoogLeNet.pptxGoogLeNet.pptx
GoogLeNet.pptxssuser2624f71
 
GoogLeNet Insights
GoogLeNet InsightsGoogLeNet Insights
GoogLeNet InsightsAuro Tripathy
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNNChanuk Lim
 
Resnet.pptx
Resnet.pptxResnet.pptx
Resnet.pptxYanhuaSi
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural networkFerdous ahmed
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningMohamed Loey
 
ResNet basics (Deep Residual Network for Image Recognition)
ResNet basics (Deep Residual Network for Image Recognition)ResNet basics (Deep Residual Network for Image Recognition)
ResNet basics (Deep Residual Network for Image Recognition)Sanjay Saha
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNNShuai Zhang
 

Recommended

GoogLeNet.pptx
GoogLeNet.pptxGoogLeNet.pptx
GoogLeNet.pptxssuser2624f71
 
GoogLeNet Insights
GoogLeNet InsightsGoogLeNet Insights
GoogLeNet InsightsAuro Tripathy
 
Mask R-CNN
Mask R-CNNMask R-CNN
Mask R-CNNChanuk Lim
 
Resnet.pptx
Resnet.pptxResnet.pptx
Resnet.pptxYanhuaSi
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural networkFerdous ahmed
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningMohamed Loey
 
ResNet basics (Deep Residual Network for Image Recognition)
ResNet basics (Deep Residual Network for Image Recognition)ResNet basics (Deep Residual Network for Image Recognition)
ResNet basics (Deep Residual Network for Image Recognition)Sanjay Saha
 
Introduction to CNN
Introduction to CNNIntroduction to CNN
Introduction to CNNShuai Zhang
 
CNN Machine learning DeepLearning
CNN Machine learning DeepLearningCNN Machine learning DeepLearning
CNN Machine learning DeepLearningAbhishek Sharma
 
Convolutional Neural Networks : Popular Architectures
Convolutional Neural Networks : Popular ArchitecturesConvolutional Neural Networks : Popular Architectures
Convolutional Neural Networks : Popular Architecturesananth
 
Recurrent Neural Network
Recurrent Neural NetworkRecurrent Neural Network
Recurrent Neural NetworkMohammad Sabouri
 
AlexNet
AlexNetAlexNet
AlexNetBertil Hatt
 
Cnn
CnnCnn
CnnNirthika Rajendran
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural networkMojammilHusain
 
Resnet
ResnetResnet
Resnetashwinjoseph95
 
Convolutional Neural Network
Convolutional Neural NetworkConvolutional Neural Network
Convolutional Neural NetworkVignesh Suresh
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketakiKetaki Patwari
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network Yan Xu
 
Machine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkMachine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkRichard Kuo
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Abdulrazak Zakieh
 
Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Terry Taewoong Um
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksChristian Perone
 
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic SegmentationSemantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation岳華 杜
 
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...Universitat Politècnica de Catalunya
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Simplilearn
 
LeNet-5
LeNet-5LeNet-5
LeNet-5佳蓉 倪
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationDat Nguyen
 
Wasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 IWasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 ISungbin Lim
 
FreeAnchor
FreeAnchorFreeAnchor
FreeAnchorBrian Kim
 
20190708 bumsookim yolact
20190708 bumsookim yolact20190708 bumsookim yolact
20190708 bumsookim yolactBrian Kim
 

More Related Content

What's hot

CNN Machine learning DeepLearning
CNN Machine learning DeepLearningCNN Machine learning DeepLearning
CNN Machine learning DeepLearningAbhishek Sharma
 
Convolutional Neural Networks : Popular Architectures
Convolutional Neural Networks : Popular ArchitecturesConvolutional Neural Networks : Popular Architectures
Convolutional Neural Networks : Popular Architecturesananth
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural networkMojammilHusain
 
Convolutional Neural Network
Convolutional Neural NetworkConvolutional Neural Network
Convolutional Neural NetworkVignesh Suresh
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketakiKetaki Patwari
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network Yan Xu
 
Machine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkMachine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkRichard Kuo
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Abdulrazak Zakieh
 
Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Terry Taewoong Um
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksChristian Perone
 
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic SegmentationSemantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation岳華 杜
 
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...Universitat Politècnica de Catalunya
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Simplilearn
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationDat Nguyen
 
Wasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 IWasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 ISungbin Lim
 

What's hot (20)

CNN Machine learning DeepLearning
CNN Machine learning DeepLearningCNN Machine learning DeepLearning
CNN Machine learning DeepLearning
 
Convolutional Neural Networks : Popular Architectures
Convolutional Neural Networks : Popular ArchitecturesConvolutional Neural Networks : Popular Architectures
Convolutional Neural Networks : Popular Architectures
 
Recurrent Neural Network
Recurrent Neural NetworkRecurrent Neural Network
Recurrent Neural Network
 
AlexNet
AlexNetAlexNet
AlexNet
 
Cnn
CnnCnn
Cnn
 
Convolutional neural network
Convolutional neural networkConvolutional neural network
Convolutional neural network
 
Resnet
ResnetResnet
Resnet
 
Convolutional Neural Network
Convolutional Neural NetworkConvolutional Neural Network
Convolutional Neural Network
 
CNN and its applications by ketaki
CNN and its applications by ketakiCNN and its applications by ketaki
CNN and its applications by ketaki
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network
 
Machine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural NetworkMachine Learning - Convolutional Neural Network
Machine Learning - Convolutional Neural Network
 
Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)Convolutional Neural Network (CNN)
Convolutional Neural Network (CNN)
 
Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
 
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic SegmentationSemantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
Semantic Segmentation - Fully Convolutional Networks for Semantic Segmentation
 
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
Image classification on Imagenet (D1L4 2017 UPC Deep Learning for Computer Vi...
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
 
LeNet-5
LeNet-5LeNet-5
LeNet-5
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance Segmentation
 
Wasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 IWasserstein GAN 수학 이해하기 I
Wasserstein GAN 수학 이해하기 I
 

More from Brian Kim

20190708 bumsookim yolact
20190708 bumsookim yolact20190708 bumsookim yolact
20190708 bumsookim yolactBrian Kim
 
20190718 bumsookim 2_attention
20190718 bumsookim 2_attention20190718 bumsookim 2_attention
20190718 bumsookim 2_attentionBrian Kim
 
Spectral cnn
Spectral cnnSpectral cnn
Spectral cnnBrian Kim
 
Representation learning
Representation learningRepresentation learning
Representation learningBrian Kim
 
Mastering the game of go with deep neural networks and tree searching
Mastering the game of go with deep neural networks and tree searchingMastering the game of go with deep neural networks and tree searching
Mastering the game of go with deep neural networks and tree searchingBrian Kim
 
Compressing neural language models by sparse word representation
Compressing neural language models by sparse word representationCompressing neural language models by sparse word representation
Compressing neural language models by sparse word representationBrian Kim
 

More from Brian Kim (8)

FreeAnchor
FreeAnchorFreeAnchor
FreeAnchor
 
20190708 bumsookim yolact
20190708 bumsookim yolact20190708 bumsookim yolact
20190708 bumsookim yolact
 
20190718 bumsookim 2_attention
20190718 bumsookim 2_attention20190718 bumsookim 2_attention
20190718 bumsookim 2_attention
 
Spectral cnn
Spectral cnnSpectral cnn
Spectral cnn
 
Representation learning
Representation learningRepresentation learning
Representation learning
 
Mastering the game of go with deep neural networks and tree searching
Mastering the game of go with deep neural networks and tree searchingMastering the game of go with deep neural networks and tree searching
Mastering the game of go with deep neural networks and tree searching
 
Compressing neural language models by sparse word representation
Compressing neural language models by sparse word representationCompressing neural language models by sparse word representation
Compressing neural language models by sparse word representation
 
Dcgan
DcganDcgan
Dcgan
 

Recently uploaded

9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home ServiceSapana Sha
 
GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]📊 Markus Baersch
 
ASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel CanterASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel Cantervoginip
 
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一F La
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024thyngster
 
B2 Creative Industry Response Evaluation.docx
B2 Creative Industry Response Evaluation.docxB2 Creative Industry Response Evaluation.docx
B2 Creative Industry Response Evaluation.docxStephen266013
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...limedy534
 
Customer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxCustomer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxEmmanuel Dauda
 
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一F sss
 
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptxAmazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptxAbdelrhman abooda
 
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdfHuman37
 
办理(UWIC毕业证书)英国卡迪夫城市大学毕业证成绩单原版一比一
办理(UWIC毕业证书)英国卡迪夫城市大学毕业证成绩单原版一比一办理(UWIC毕业证书)英国卡迪夫城市大学毕业证成绩单原版一比一
办理(UWIC毕业证书)英国卡迪夫城市大学毕业证成绩单原版一比一F La
 
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...Florian Roscheck
 
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degreeyuu sss
 
Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Colleen Farrelly
 
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...Jack DiGiovanna
 
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理e4aez8ss
 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxthyngster
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfBoston Institute of Analytics
 

Recently uploaded (20)

9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service
 
GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]
 
ASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel CanterASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel Canter
 
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
 
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
 
B2 Creative Industry Response Evaluation.docx
B2 Creative Industry Response Evaluation.docxB2 Creative Industry Response Evaluation.docx
B2 Creative Industry Response Evaluation.docx
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
 
Customer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxCustomer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptx
 
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
办理学位证中佛罗里达大学毕业证,UCF成绩单原版一比一
 
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptxAmazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
Amazon TQM (2) Amazon TQM (2)Amazon TQM (2).pptx
 
20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf20240419 - Measurecamp Amsterdam - SAM.pdf
20240419 - Measurecamp Amsterdam - SAM.pdf
 
办理(UWIC毕业证书)英国卡迪夫城市大学毕业证成绩单原版一比一
办理(UWIC毕业证书)英国卡迪夫城市大学毕业证成绩单原版一比一办理(UWIC毕业证书)英国卡迪夫城市大学毕业证成绩单原版一比一
办理(UWIC毕业证书)英国卡迪夫城市大学毕业证成绩单原版一比一
 
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...From idea to production in a day – Leveraging Azure ML and Streamlit to build...
From idea to production in a day – Leveraging Azure ML and Streamlit to build...
 
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
 
Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024
 
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
Building on a FAIRly Strong Foundation to Connect Academic Research to Transl...
 
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
 

Google net

  • 1. Korea University, Department of Computer Science & Radio Communication Engineering 2016010646 Bumsoo Kim Google NetEverything about GoogleNet and its architecture Bumsoo Kim Presentation 2017 1
  • 2. Bumsoo Kim Presentation 2017 2 Whatis‘GoogleNet’? GoogleNet
  • 3. Bumsoo Kim Presentation 2017 3 Whatis‘GoogleNet’? GoogleNet
  • 4. Bumsoo Kim Presentation 2017 4 Whatis‘GoogleNet’? GoogleNet
  • 5. Bumsoo Kim Presentation 2017 5 Whatis‘GoogleNet’? GoogleNet
  • 6. Bumsoo Kim Presentation 2017 6 Whatis‘GoogleNet’? GoogleNet ILSVRC(ImageNet) 2014 Winning Model Google, “Going Deeper with Convolutions”(CVPR2015) Top5 Error Rate = 6.7% Revolution of Depth (22 layers)
  • 7. Bumsoo Kim Presentation 2017 7 Howdeepis‘GoogleNet’? RevolutionofDepth
  • 8. Bumsoo Kim Presentation 2017 8 Deeperisbetter? RevolutionofDepth The winning models of 2014 was distinguishable by its depth!
  • 9. Bumsoo Kim Presentation 2017 9 Deeperisbetter? RevolutionofDepth The winning models of 2014 was distinguishable by its depth! So, does deeper network is the key to better performance?
  • 10. Bumsoo Kim Presentation 2017 10 Deeperisbetter? RevolutionofDepth The winning models of 2014 was distinguishable by its depth! So, does deeper network is the key to better performance? The answer is…
  • 11. Bumsoo Kim Presentation 2017 11 Deeperisbetter? RevolutionofDepth The winning models of 2014 was distinguishable by its depth! So, does deeper network is the key to better performance? The answer is… NO!
  • 12. Bumsoo Kim Presentation 2017 12 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
  • 13. Bumsoo Kim Presentation 2017 13 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
  • 14. Bumsoo Kim Presentation 2017 14 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
  • 15. Bumsoo Kim Presentation 2017 15 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
  • 16. Bumsoo Kim Presentation 2017 16 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
  • 17. Bumsoo Kim Presentation 2017 17 DeeperisNOTbetter ObstaclesofDepth Overfitting Unbalance & Aliasing
  • 18. Bumsoo Kim Presentation 2017 18 ZF-Netdidnotprovideafundamentalsolution ObstaclesofDepth ZF-net fundamentally failed in increasing depth
  • 19. Bumsoo Kim Presentation 2017 19 ZF-Netdidnotprovideafundamentalsolution ObstaclesofDepth ZF-net fundamentally failed in increasing depth 2014 top models succeeded in addressing this problem
  • 20. Bumsoo Kim Presentation 2017 20 ZF-Netdidnotprovideafundamentalsolution ObstaclesofDepth ZF-net fundamentally failed in increasing depth 2014 top models succeeded in addressing this problem HOW?
  • 21. Bumsoo Kim Presentation 2017 21 UniquemoduleofGooglenet Inception
  • 22. Bumsoo Kim Presentation 2017 22 NetworkinNetwork Inceptionnodules
  • 23. Bumsoo Kim Presentation 2017 23 NetworkinNetwork Inceptionnodules NIN (Network In Network) – Min Lin, 2013, Network in Network local receptive linear features local receptive linear+non-linear features non-linear features
  • 24. Bumsoo Kim Presentation 2017 24 NetworkinNetwork Inceptionnodules NIN (Network In Network) – Min Lin, 2013, Network in Network non-linear features 90% of computation
  • 25. Bumsoo Kim Presentation 2017 25 NetworkinNetwork Inceptionnodules NIN (Network In Network) – Min Lin, 2013, Network in Network Inception Average Pooling
  • 26. Bumsoo Kim Presentation 2017 26 1x1Convolutions Inceptionnodules
  • 27. Bumsoo Kim Presentation 2017 27 1x1Convolutions Inceptionnodules Original model
  • 28. Bumsoo Kim Presentation 2017 28 1x1Convolutions Inceptionnodules Original model Various filters for various feature scales
  • 29. Bumsoo Kim Presentation 2017 29 1x1Convolutions Inceptionnodules Original model Expensive unit : Larger filter sizes are very expensive!!
  • 30. Bumsoo Kim Presentation 2017 30 1x1Convolutions Inceptionnodules Original model 1x1 Convolution
  • 31. Bumsoo Kim Presentation 2017 31 1x1Convolutions Inceptionnodules 1x1 Convolution Dimension reduction Hebbian Principle Neurons that fire together, wire together 동일 차원에서 activate된 neuron들은 비슷한 성질을 가진 것들로 묶일 수 있다. c2 (large) c3 (small)
  • 32. Bumsoo Kim Presentation 2017 32 Dowereallyneed5x5filters? Inception Factorizing Convolutions 5*5 = 25 → 2*(3*3) = 18 => 28% decrease of parameters!
  • 33. Bumsoo Kim Presentation 2017 33 Dowereallyneed5x5filters? Inception Factorizing Convolutions 5*5 = 25 → 2*(3*3) = 18 => 28% decrease of parameters!
  • 34. Bumsoo Kim Presentation 2017 34 HowEffectiveisit? Inception 130M 4.5B
  • 35. Bumsoo Kim Presentation 2017 35 Whydoweneedit? AuxiliaryClassifier Gradient Vanishing Problem ReLU doesn’t provide a fundamental solution Overlapping multiple ReLUs cause the same problem!
  • 36. Bumsoo Kim Presentation 2017 36 Whatiswrongaboutit? GradientVanishing Vanishing Gradients on Sigmoids * Deep Networks use activation function for non-linearity. *“Saturated”: Weight ‘W’ is too large. => ‘z’is 0 or 1 => z*(1-z) = 0 z = 1/(1 + np.exp(-np.dot(W, x))) # forward pass dx = np.dot(W.T, z*(1-z)) # backward pass: local gradient for x dW = np.outer(z*(1-z), x) # backward pass: local gradient for W * Local gradient is small : 0 <= z*(1-z) <= 0.25 (when z=0.5) : top layers are not trained
  • 37. Bumsoo Kim Presentation 2017 37 Detailsarenotexplainedinthepaper AuxiliaryClassifier Training Deeper Convolutional Networks with Deep SuperVision(2014, Liwei Chang, Chen-Yu Lee)
  • 38. Bumsoo Kim Presentation 2017 38 BeforeGradientVanishes! AuxiliaryClassifier Training Deeper Convolutional Networks with Deep SuperVision(2014, Liwei Chang, Chen-Yu Lee) 깊이가 깊어질수록 vanishing될 우려가 높아지므로, 얕은 깊이에서도 업데이트!
  • 39. Bumsoo Kim Presentation 2017 39 EmpiricalDecision AuxiliaryClassifier Training Deeper Convolutional Networks with Deep SuperVision(2014, Liwei Chang, Chen-Yu Lee) 넣는 위치는 이론적 근거 없이 순수하게 “경험적”으로 판단한다.
  • 40. Bumsoo Kim Presentation 2017 40 HowEffectiveisit? AuxiliaryClassifier Training Deeper Convolutional Networks with Deep SuperVision(2014, Liwei Chang, Chen-Yu Lee)
  • 41. Bumsoo Kim Presentation 2017 41 Trainityourself! GoogleNet
  • 42. Bumsoo Kim Presentation 2017 42 Trainityourself! GoogleNet
  • 43. Bumsoo Kim Presentation 2017 43 Trainityourself! GoogleNet
  • 44. Bumsoo Kim Presentation 2017 44 Trainityourself! GoogleNet
  • 45. Bumsoo Kim Presentation 2017 45 Trainityourself! GoogleNet
  • 46. Bumsoo Kim Presentation 2017 46 Trainityourself! GoogleNet
  • 47. Bumsoo Kim Presentation 2017 47 Trainityourself! GoogleNet
  • 48. Bumsoo Kim Presentation 2017 48 Trainityourself! GoogleNet
  • 49. Bumsoo Kim Presentation 2017 49 Trainityourself! GoogleNet
  • 50. Bumsoo Kim Presentation 2017 50 Trainityourself! GoogleNet
  • 51. Bumsoo Kim Presentation 2017 51 Trainityourself! GoogleNet
  • 52. Bumsoo Kim Presentation 2017 52 Trainityourself! GoogleNet
  • 53. Bumsoo Kim Presentation 2017 53 Trainityourself! GoogleNet
  • 54. Bumsoo Kim Presentation 2017 Trainityourself! GoogleNet
  • 55. Bumsoo Kim Presentation 2017 55 Trainityourself! GoogleNet
  • 56. Bumsoo Kim Presentation 2017 56 Trainityourself! GoogleNet
  • 57. Thank You Bumsoo Kim Presentation 2017 57