SlideShare a Scribd company logo

Fine tuning a convolutional network for cultural event recognition

•
•
Universitat Politècnica de Catalunya
Universitat Politècnica de Catalunya

https://imatge.upc.edu/web/publications/fine-tuning-convolutional-network-cultural-event-recognition This thesis explores good practices for improving the performance of an existing convnet trained with a dataset of clean data when an additional dataset of noisy data is available. We develop techniques to clean the noisy data with the help of the clean one, a family of solutions that we will refer to as denoising, and then we explore the best sorting of the clean and noisy datasets during the fine-tuning of a convnet. Then we study strategies to select the subset of images of the clean data that will improve the classification performance, a practice we will efer to as fracking. Next, we determine how many layers are actually better to fine-tune in our convnet, given our amount of data. And finally, we compare the classic convnet architecture where a single network is fine-tuned to solve a multi-class problem with the case of fine-tuning a convnet for binary classification for each considered class.

Technology
1 of 54
Download to read offline
FINE-TUNING A CONVOLUTIONAL NETWORK FOR CULTURAL EVENT RECOGNITION ADVISORS: Andrea Calafell Xavier Giró-i-Nieto Amaia Salvador 20/07/2015 AUTHOR: Matthias Zeppelzauer
OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 2
MOTIVATION: Cultural Heritage 3Chinese New year
MOTIVATION: Cultural Heritage 4Carnival Rio
Classic onsite explorers 5
Onsite social media is big data... 6
...and online explorers need our help 7
CHALEARN: Looking at People 8 TRAINING SET 5,875 VALIDATION SET 2,332 TEST SET 3,569 50 EVENTS
MOTIVATION: Goals 9 ● Improve the results obtained in ChaLearn Challenge. ● Exploit the noisy data collected from Flickr
STATE OF THE ART: CaffeNet 10 Content Visual Time stamp Context Geolocation Text Zaharieva’15 X X X Mattivi’11 X X Bossard’13 X X Cao’08 X X X Sutanto’13 X Schinas’12 X X Brenner’13 X X Nguyen’13 X X MediaEval Social Event Detection
STATE OF THE ART: CaffeNet 11 CaffeNet ARCHITECTURE [Khrizevsky’12] SOFTWARE [Jia’14] DATA [Deng’09]
STATE OF THE ART: CNN ARCHITECTURE 12 Convolutional Neural Network architecture Babenko et al, Neural codes for image retrieval. In Computer Vision-ECCV, 2014
STATE OF THE ART: Object+Scene CNNs 13 Object-Scene Convolutional Neural Network for event recognition Wang et al, Object-scene convolutional neural networks for event recognition in images. In CVPRW, 2015
OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 14
BASELINE: Fine-tuning a ConvNet 15 50
BASELINE: ChaLearn @ CVPRW 2015 16 Awarded with the 2nd prize of the Cultural Event Recognition Challenge in the ChaLearn Workshop at CVPR 2015 Salvador. A, Giro-i-Nieto. X, Calafell, A, et al, Cultural Event Recognition with Visual ConvNets and Temporal Models. In CVPRW, 2015
BASELINE: ChaLearn @ CVPRW 2015 17 Awarded with the 2nd prize of the Cultural Event Recognition Challenge in the ChaLearn Workshop at CVPR 2015 Salvador. A, Giro-i-Nieto. X, Calafell, A, et al, Cultural Event Recognition with Visual ConvNets and Temporal Models. In CVPRW, 2015
OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 18
Convnets require to be trained with... 19 a large amount of labeled images
but clean data is expensive... 20 and downloading noisy data in an unsupervised fashion is easier and cheaper.
NOISY DATA: Flickr Dataset 21 FLICKR DATASET 4,068 50 EVENTS
DATASET BIAS 22 Dataset bias when fine-tuning with ChaLearn or Flickr dataset:
OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 23
DENOISING THE FLICKR DATASET 24 Mosaic of Queens Day from ChaLearn Mosaic of Queens Day from Flickr
DENOISING THE FLICKR DATASET 25Example event: Annual Buffalo Roundup Fine-tuned model with ChaLearn New subset from
BASELINE: Dataset ordering during fine-tuning 26 CaffeNet FINE-TUNING JOINT:
DENOISING THE FLICKR DATASET 27 Joint fine-tuning of the clean and noisy datasets: 0.6136
BASELINE: Dataset ordering during fine-tuning 28 CaffeNet FINE-TUNING: FINE-TUNING:
DENOISING THE FLICKR DATASET 29 Sequential fine-tuning of the clean and noisy datasets: 0.6136
BASELINE: Dataset ordering during fine-tuning 30 CaffeNet FINE-TUNING:FINE-TUNING:
DENOISING THE FLICKR DATASET 31 Sequential fine-tuning of the noisy and clean datasets: 0.6136 +1,3%
OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 32
FRACKING MINING +/- SAMPLES 33
FRACKING THE TRAINING DATASET 34Example event: Pingxi Lantern Festival Fine-tuned model with ChaLearn New subset from hard negatives hard positive
BASELINE: Dataset ordering during fine-tuning 35 CaffeNet FINE-TUNING: Fine-tuning with fracking subset from:
FRACKING THE TRAINING DATASET 36 Results of fine-tuning using fracking in images from ChaLearn: baseline: 0.61365 +0,9%
OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 37
FINE-TUNING DEEPER LAYERS ONLY 38 Layer 2 responds to corners and other edge/color conjunctions.
FINE-TUNING DEEPER LAYERS ONLY 39 Layer 3 has more complex invariances, capturing similar textures Zeiler et al, Visualizing and Understanding Convolutional Networks, In Computer Vision-ECCV 2014,
FINE-TUNING DEEPER LAYERS ONLY 40 50 Andrej Karpathy. Convolutional neural networks for visual recognition. In Stanford CS class CS231n. FC6 FC7 FC8
FINE-TUNING DEEPER LAYERS ONLY 41 Results of only fine-tuning the deeper layers: +3% 0.61365
FINE-TUNING DEEPER LAYERS ONLY 42 Results of only fine-tuning the deeper layers : +4% 0.6136
OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 43
BASELINE: ChaLearn @ CVPRW 2015 44 Awarded with the 2nd prize of the Cultural Event Recognition Challenge in the ChaLearn Workshop at CVPR 2015 Salvador. A, Giro-i-Nieto. X, Calafell, A, et al, Cultural Event Recognition with Visual ConvNets and Temporal Models. In CVPRW, 2015
ENSEMBLE OF EVENT DETECTORS 45 SINGLE CONVNET FOR THE 50 EVENTS:
ENSEMBLE OF EVENT DETECTORS 46 ONE CONVNET FOR EACH EVENTS:
ENSEMBLE OF EVENT DETECTORS 47 Results of ensemble of binary : +6,6% 0.6136
OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 48
CONLUSIONS 49 ● The Flickr dataset helped us to improve the score by swapping the order in which we were using the clean and noisy datasets CaffeNet FINE-TUNING:FINE-TUNING: +1,3%
CONLUSIONS 50 ● The network actually succeeds in improving his performance by learning from its own mistakes when applying fracking. +0,9% CaffeNet FINE-TUNING: Fine-tuning with fracking subset from:
CONLUSIONS 51 ● The results are better if we keep the weights learned in the earlier layers from a very large dataset. 50 +4%
CONLUSIONS 52 ● Fine-tuning one convnet for each class increases the score. +6,6%
FUTURE WORK 53 ● Mix our solutions with a fine-tuned network with PLACES, and with other local solutions. SCENE CNN (PLACES) LOCAL NOW ● Compete (and try to win) ChaLearn @ ICCV 2015 !!
FINE-TUNING A CONVOLUTIONAL NETWORK FOR CULTURAL EVENT RECOGNITION ADVISORS: Andrea Calafell Xavier Giró-i-Nieto Amaia Salvador 20/07/2015 AUTHOR: Matthias Zeppelzauer

Recommended

Region-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object RetrievalRegion-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object Retrieval
Universitat Politècnica de Catalunya
 
Convolutional Features for Instance Search
Convolutional Features for Instance SearchConvolutional Features for Instance Search
Convolutional Features for Instance Search
Universitat Politècnica de Catalunya
 
Deep and Young Vision Learning at UPC BarcelonaTech (NIPS 2016)
Deep and Young Vision Learning at UPC BarcelonaTech (NIPS 2016)Deep and Young Vision Learning at UPC BarcelonaTech (NIPS 2016)
Deep and Young Vision Learning at UPC BarcelonaTech (NIPS 2016)
Universitat Politècnica de Catalunya
 
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Universitat Politècnica de Catalunya
 
Deep image retrieval learning global representations for image search
Deep image retrieval learning global representations for image searchDeep image retrieval learning global representations for image search
Deep image retrieval learning global representations for image search
Universitat Politècnica de Catalunya
 
Intepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural NetworksIntepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural Networks
Universitat Politècnica de Catalunya
 

Recommended

Region-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object RetrievalRegion-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object Retrieval
Universitat Politècnica de Catalunya
 
Convolutional Features for Instance Search
Convolutional Features for Instance SearchConvolutional Features for Instance Search
Convolutional Features for Instance Search
Universitat Politècnica de Catalunya
 
Deep and Young Vision Learning at UPC BarcelonaTech (NIPS 2016)
Deep and Young Vision Learning at UPC BarcelonaTech (NIPS 2016)Deep and Young Vision Learning at UPC BarcelonaTech (NIPS 2016)
Deep and Young Vision Learning at UPC BarcelonaTech (NIPS 2016)
Universitat Politècnica de Catalunya
 
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Content-Based Image Retrieval (D2L6 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Content-based Image Retrieval - Eva Mohedano - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Deep Learning for Computer Vision: Image Retrieval (UPC 2016)
Universitat Politècnica de Catalunya
 
Deep image retrieval learning global representations for image search
Deep image retrieval learning global representations for image searchDeep image retrieval learning global representations for image search
Deep image retrieval learning global representations for image search
Universitat Politècnica de Catalunya
 
Intepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural NetworksIntepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural Networks
Universitat Politècnica de Catalunya
 
Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
Universitat Politècnica de Catalunya
 
Class Weighted Convolutional Features for Image Retrieval
Class Weighted Convolutional Features for Image Retrieval Class Weighted Convolutional Features for Image Retrieval
Class Weighted Convolutional Features for Image Retrieval
Universitat Politècnica de Catalunya
 
Object Detection Methods using Deep Learning
Object Detection Methods using Deep LearningObject Detection Methods using Deep Learning
Object Detection Methods using Deep Learning
Sungjoon Choi
 
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
Simone Ercoli
 
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
Universitat Politècnica de Catalunya
 
SeRanet introduction
SeRanet introductionSeRanet introduction
SeRanet introduction
Kosuke Nakago
 
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
Universitat Politècnica de Catalunya
 
Object Detection Beyond Mask R-CNN and RetinaNet I
Object Detection Beyond Mask R-CNN and RetinaNet IObject Detection Beyond Mask R-CNN and RetinaNet I
Object Detection Beyond Mask R-CNN and RetinaNet I
Wanjin Yu
 
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Universitat Politècnica de Catalunya
 
Deep image retrieval - learning global representations for image search - ub ...
Deep image retrieval - learning global representations for image search - ub ...Deep image retrieval - learning global representations for image search - ub ...
Deep image retrieval - learning global representations for image search - ub ...
Universitat de Barcelona
 
Scaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution AlgorithmsScaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution Algorithms
Xiaoyong Zhu
 
DeepFix: a fully convolutional neural network for predicting human fixations...
DeepFix: a fully convolutional neural network for predicting human fixations...DeepFix: a fully convolutional neural network for predicting human fixations...
DeepFix: a fully convolutional neural network for predicting human fixations...
Universitat Politècnica de Catalunya
 
Computer vision for transportation
Computer vision for transportationComputer vision for transportation
Computer vision for transportation
Wanjin Yu
 
Visual Search and Question Answering II
Visual Search and Question Answering IIVisual Search and Question Answering II
Visual Search and Question Answering II
Wanjin Yu
 
Adaptive object detection using adjacency and zoom prediction
Adaptive object detection using adjacency and zoom predictionAdaptive object detection using adjacency and zoom prediction
Adaptive object detection using adjacency and zoom prediction
Universitat Politècnica de Catalunya
 
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance Segmentation
Dat Nguyen
 
Data-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural netsData-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural nets
Balázs Kégl
 
Surveillance scene classification using machine learning
Surveillance scene classification using machine learningSurveillance scene classification using machine learning
Surveillance scene classification using machine learning
Utkarsh Contractor
 

More Related Content

What's hot

Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
Universitat Politècnica de Catalunya
 
Class Weighted Convolutional Features for Image Retrieval
Class Weighted Convolutional Features for Image Retrieval Class Weighted Convolutional Features for Image Retrieval
Class Weighted Convolutional Features for Image Retrieval
Universitat Politècnica de Catalunya
 
Object Detection Methods using Deep Learning
Object Detection Methods using Deep LearningObject Detection Methods using Deep Learning
Object Detection Methods using Deep Learning
Sungjoon Choi
 
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
Simone Ercoli
 
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
Universitat Politècnica de Catalunya
 
SeRanet introduction
SeRanet introductionSeRanet introduction
SeRanet introduction
Kosuke Nakago
 
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
Universitat Politècnica de Catalunya
 
Object Detection Beyond Mask R-CNN and RetinaNet I
Object Detection Beyond Mask R-CNN and RetinaNet IObject Detection Beyond Mask R-CNN and RetinaNet I
Object Detection Beyond Mask R-CNN and RetinaNet I
Wanjin Yu
 
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Universitat Politècnica de Catalunya
 
Deep image retrieval - learning global representations for image search - ub ...
Deep image retrieval - learning global representations for image search - ub ...Deep image retrieval - learning global representations for image search - ub ...
Deep image retrieval - learning global representations for image search - ub ...
Universitat de Barcelona
 
Scaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution AlgorithmsScaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution Algorithms
Xiaoyong Zhu
 
DeepFix: a fully convolutional neural network for predicting human fixations...
DeepFix: a fully convolutional neural network for predicting human fixations...DeepFix: a fully convolutional neural network for predicting human fixations...
DeepFix: a fully convolutional neural network for predicting human fixations...
Universitat Politècnica de Catalunya
 
Computer vision for transportation
Computer vision for transportationComputer vision for transportation
Computer vision for transportation
Wanjin Yu
 
Visual Search and Question Answering II
Visual Search and Question Answering IIVisual Search and Question Answering II
Visual Search and Question Answering II
Wanjin Yu
 
Adaptive object detection using adjacency and zoom prediction
Adaptive object detection using adjacency and zoom predictionAdaptive object detection using adjacency and zoom prediction
Adaptive object detection using adjacency and zoom prediction
Universitat Politècnica de Catalunya
 
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance Segmentation
Dat Nguyen
 

What's hot (20)

Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
Visual Saliency Prediction with Deep Learning - Kevin McGuinness - UPC Barcel...
 
Class Weighted Convolutional Features for Image Retrieval
Class Weighted Convolutional Features for Image Retrieval Class Weighted Convolutional Features for Image Retrieval
Class Weighted Convolutional Features for Image Retrieval
 
Object Detection Methods using Deep Learning
Object Detection Methods using Deep LearningObject Detection Methods using Deep Learning
Object Detection Methods using Deep Learning
 
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
Vision and Multimedia Reading Group: DeCAF: a Deep Convolutional Activation F...
 
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
Image Retrieval (D4L5 2017 UPC Deep Learning for Computer Vision)
 
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
 
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
CNN vs SIFT-based Visual Localization - Laura Leal-Taixé - UPC Barcelona 2018
 
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
Interpretability of Convolutional Neural Networks - Eva Mohedano - UPC Barcel...
 
SeRanet introduction
SeRanet introductionSeRanet introduction
SeRanet introduction
 
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
Deep Visual Saliency - Kevin McGuinness - UPC Barcelona 2017
 
Object Detection Beyond Mask R-CNN and RetinaNet I
Object Detection Beyond Mask R-CNN and RetinaNet IObject Detection Beyond Mask R-CNN and RetinaNet I
Object Detection Beyond Mask R-CNN and RetinaNet I
 
Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)Deep Learning for Computer Vision: Object Detection (UPC 2016)
Deep Learning for Computer Vision: Object Detection (UPC 2016)
 
Deep image retrieval - learning global representations for image search - ub ...
Deep image retrieval - learning global representations for image search - ub ...Deep image retrieval - learning global representations for image search - ub ...
Deep image retrieval - learning global representations for image search - ub ...
 
Scaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution AlgorithmsScaling up Deep Learning Based Super Resolution Algorithms
Scaling up Deep Learning Based Super Resolution Algorithms
 
DeepFix: a fully convolutional neural network for predicting human fixations...
DeepFix: a fully convolutional neural network for predicting human fixations...DeepFix: a fully convolutional neural network for predicting human fixations...
DeepFix: a fully convolutional neural network for predicting human fixations...
 
Computer vision for transportation
Computer vision for transportationComputer vision for transportation
Computer vision for transportation
 
Visual Search and Question Answering II
Visual Search and Question Answering IIVisual Search and Question Answering II
Visual Search and Question Answering II
 
Adaptive object detection using adjacency and zoom prediction
Adaptive object detection using adjacency and zoom predictionAdaptive object detection using adjacency and zoom prediction
Adaptive object detection using adjacency and zoom prediction
 
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
Deep 3D Analysis - Javier Ruiz-Hidalgo - UPC Barcelona 2018
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance Segmentation
 

Similar to Fine tuning a convolutional network for cultural event recognition

Data-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural netsData-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural nets
Balázs Kégl
 
Surveillance scene classification using machine learning
Surveillance scene classification using machine learningSurveillance scene classification using machine learning
Surveillance scene classification using machine learning
Utkarsh Contractor
 
thesis_final.pdf
thesis_final.pdfthesis_final.pdf
thesis_final.pdf
ssusere02009
 
Deep Learning for Computer Vision (2/4): Object Analytics @ laSalle 2016
Deep Learning for Computer Vision (2/4): Object Analytics @ laSalle 2016Deep Learning for Computer Vision (2/4): Object Analytics @ laSalle 2016
Deep Learning for Computer Vision (2/4): Object Analytics @ laSalle 2016
Universitat Politècnica de Catalunya
 
CNNs: from the Basics to Recent Advances
CNNs: from the Basics to Recent AdvancesCNNs: from the Basics to Recent Advances
CNNs: from the Basics to Recent Advances
Dmytro Mishkin
 
Object Discovery using CNN Features in Egocentric Videos
Object Discovery using CNN Features in Egocentric VideosObject Discovery using CNN Features in Egocentric Videos
Object Discovery using CNN Features in Egocentric Videos
Marc Bolaños Solà
 
Computing Challenges at the Large Hadron Collider
Computing Challenges at the Large Hadron ColliderComputing Challenges at the Large Hadron Collider
Computing Challenges at the Large Hadron Collider
inside-BigData.com
 
2019 Project Showcase - Alexander Adam Laurence
2019 Project Showcase - Alexander Adam Laurence2019 Project Showcase - Alexander Adam Laurence
2019 Project Showcase - Alexander Adam Laurence
AlexanderAdamLaurenc
 
Deep Video Object Tracking 2020 - Xavier Giro - UPC TelecomBCN Barcelona
Deep Video Object Tracking 2020 - Xavier Giro - UPC TelecomBCN BarcelonaDeep Video Object Tracking 2020 - Xavier Giro - UPC TelecomBCN Barcelona
Deep Video Object Tracking 2020 - Xavier Giro - UPC TelecomBCN Barcelona
Universitat Politècnica de Catalunya
 
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
Sandia National Laboratories: Energy & Climate: Renewables
 
Report face recognition : ArganRecogn
Report face recognition : ArganRecognReport face recognition : ArganRecogn
Report face recognition : ArganRecogn
Ilyas CHAOUA
 
Applications of Machine Learning to Location-based Social Networks
Applications of Machine Learning to Location-based Social NetworksApplications of Machine Learning to Location-based Social Networks
Applications of Machine Learning to Location-based Social Networks
Joan Capdevila Pujol
 
Cancer uk 2015_module1_ouellette_ver02
Cancer uk 2015_module1_ouellette_ver02Cancer uk 2015_module1_ouellette_ver02
Cancer uk 2015_module1_ouellette_ver02
Neuro, McGill University
 
Huawei STW 2018 public
Huawei STW 2018 publicHuawei STW 2018 public
Huawei STW 2018 public
Alan Smeaton
 
Paris Data Ladies #14
Paris Data Ladies #14Paris Data Ladies #14
Paris Data Ladies #14
Nina Bertrand
 
Loughborough research forum 2010 data overload presentation
Loughborough research forum 2010 data overload presentationLoughborough research forum 2010 data overload presentation
Loughborough research forum 2010 data overload presentation
Nicola Louise Beddall-Hill
 
Underwater sparse image classification using deep convolutional neural networks
Underwater sparse image classification using deep convolutional neural networksUnderwater sparse image classification using deep convolutional neural networks
Underwater sparse image classification using deep convolutional neural networks
Mohamed Elawady
 
What is wrong with data challenges
What is wrong with data challengesWhat is wrong with data challenges
What is wrong with data challenges
Balázs Kégl
 
Dissertation final report
Dissertation final reportDissertation final report
Dissertation final report
Smriti Tikoo
 
IntroML_3_DataVisualisation
IntroML_3_DataVisualisationIntroML_3_DataVisualisation
IntroML_3_DataVisualisation
Elio Laureano
 

Similar to Fine tuning a convolutional network for cultural event recognition (20)

Data-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural netsData-driven hypothesis generation using deep neural nets
Data-driven hypothesis generation using deep neural nets
 
Surveillance scene classification using machine learning
Surveillance scene classification using machine learningSurveillance scene classification using machine learning
Surveillance scene classification using machine learning
 
thesis_final.pdf
thesis_final.pdfthesis_final.pdf
thesis_final.pdf
 
Deep Learning for Computer Vision (2/4): Object Analytics @ laSalle 2016
Deep Learning for Computer Vision (2/4): Object Analytics @ laSalle 2016Deep Learning for Computer Vision (2/4): Object Analytics @ laSalle 2016
Deep Learning for Computer Vision (2/4): Object Analytics @ laSalle 2016
 
CNNs: from the Basics to Recent Advances
CNNs: from the Basics to Recent AdvancesCNNs: from the Basics to Recent Advances
CNNs: from the Basics to Recent Advances
 
Object Discovery using CNN Features in Egocentric Videos
Object Discovery using CNN Features in Egocentric VideosObject Discovery using CNN Features in Egocentric Videos
Object Discovery using CNN Features in Egocentric Videos
 
Computing Challenges at the Large Hadron Collider
Computing Challenges at the Large Hadron ColliderComputing Challenges at the Large Hadron Collider
Computing Challenges at the Large Hadron Collider
 
2019 Project Showcase - Alexander Adam Laurence
2019 Project Showcase - Alexander Adam Laurence2019 Project Showcase - Alexander Adam Laurence
2019 Project Showcase - Alexander Adam Laurence
 
Deep Video Object Tracking 2020 - Xavier Giro - UPC TelecomBCN Barcelona
Deep Video Object Tracking 2020 - Xavier Giro - UPC TelecomBCN BarcelonaDeep Video Object Tracking 2020 - Xavier Giro - UPC TelecomBCN Barcelona
Deep Video Object Tracking 2020 - Xavier Giro - UPC TelecomBCN Barcelona
 
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
02 stein intro_4th-pv_modeling_workshop_2015-10-22_sand2015-8571_c
 
Report face recognition : ArganRecogn
Report face recognition : ArganRecognReport face recognition : ArganRecogn
Report face recognition : ArganRecogn
 
Applications of Machine Learning to Location-based Social Networks
Applications of Machine Learning to Location-based Social NetworksApplications of Machine Learning to Location-based Social Networks
Applications of Machine Learning to Location-based Social Networks
 
Cancer uk 2015_module1_ouellette_ver02
Cancer uk 2015_module1_ouellette_ver02Cancer uk 2015_module1_ouellette_ver02
Cancer uk 2015_module1_ouellette_ver02
 
Huawei STW 2018 public
Huawei STW 2018 publicHuawei STW 2018 public
Huawei STW 2018 public
 
Paris Data Ladies #14
Paris Data Ladies #14Paris Data Ladies #14
Paris Data Ladies #14
 
Loughborough research forum 2010 data overload presentation
Loughborough research forum 2010 data overload presentationLoughborough research forum 2010 data overload presentation
Loughborough research forum 2010 data overload presentation
 
Underwater sparse image classification using deep convolutional neural networks
Underwater sparse image classification using deep convolutional neural networksUnderwater sparse image classification using deep convolutional neural networks
Underwater sparse image classification using deep convolutional neural networks
 
What is wrong with data challenges
What is wrong with data challengesWhat is wrong with data challenges
What is wrong with data challenges
 
Dissertation final report
Dissertation final reportDissertation final report
Dissertation final report
 
IntroML_3_DataVisualisation
IntroML_3_DataVisualisationIntroML_3_DataVisualisation
IntroML_3_DataVisualisation
 

More from Universitat Politècnica de Catalunya

Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Universitat Politècnica de Catalunya
 
Deep Generative Learning for All
Deep Generative Learning for AllDeep Generative Learning for All
Deep Generative Learning for All
Universitat Politècnica de Catalunya
 
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
Universitat Politècnica de Catalunya
 
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoTowards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
Universitat Politècnica de Catalunya
 
The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021
Universitat Politècnica de Catalunya
 
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Universitat Politècnica de Catalunya
 
Open challenges in sign language translation and production
Open challenges in sign language translation and productionOpen challenges in sign language translation and production
Open challenges in sign language translation and production
Universitat Politècnica de Catalunya
 
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in VideosGeneration of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
Universitat Politècnica de Catalunya
 
Discovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in MinecraftDiscovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in Minecraft
Universitat Politècnica de Catalunya
 
Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...
Universitat Politècnica de Catalunya
 
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Universitat Politècnica de Catalunya
 
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Universitat Politècnica de Catalunya
 
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Universitat Politècnica de Catalunya
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Universitat Politècnica de Catalunya
 
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Universitat Politècnica de Catalunya
 
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Universitat Politècnica de Catalunya
 
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Universitat Politècnica de Catalunya
 
Curriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object SegmentationCurriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object Segmentation
Universitat Politècnica de Catalunya
 
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Universitat Politècnica de Catalunya
 
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Universitat Politècnica de Catalunya
 

More from Universitat Politècnica de Catalunya (20)

Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
Deep Generative Learning for All - The Gen AI Hype (Spring 2024)
 
Deep Generative Learning for All
Deep Generative Learning for AllDeep Generative Learning for All
Deep Generative Learning for All
 
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
 
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoTowards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
 
The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021
 
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
 
Open challenges in sign language translation and production
Open challenges in sign language translation and productionOpen challenges in sign language translation and production
Open challenges in sign language translation and production
 
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in VideosGeneration of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
 
Discovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in MinecraftDiscovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in Minecraft
 
Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...
 
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
 
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
 
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
 
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
 
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
 
Curriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object SegmentationCurriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object Segmentation
 
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
 
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
 

Recently uploaded

Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
UiPathCommunity
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
christinelarrosa
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Neo4j
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
zjhamm304
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
Safe Software
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE RolesWhat is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
DianaGray10
 

Recently uploaded (20)

Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid ResearchHarnessing the Power of NLP and Knowledge Graphs for Opioid Research
Harnessing the Power of NLP and Knowledge Graphs for Opioid Research
 
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...QA or the Highway - Component Testing: Bridging the gap between frontend appl...
QA or the Highway - Component Testing: Bridging the gap between frontend appl...
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Essentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation ParametersEssentials of Automations: Exploring Attributes & Automation Parameters
Essentials of Automations: Exploring Attributes & Automation Parameters
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE RolesWhat is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
 

Fine tuning a convolutional network for cultural event recognition

  • 1. FINE-TUNING A CONVOLUTIONAL NETWORK FOR CULTURAL EVENT RECOGNITION ADVISORS: Andrea Calafell Xavier Giró-i-Nieto Amaia Salvador 20/07/2015 AUTHOR: Matthias Zeppelzauer
  • 2. OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 2
  • 6. Onsite social media is big data... 6
  • 7. ...and online explorers need our help 7
  • 8. CHALEARN: Looking at People 8 TRAINING SET 5,875 VALIDATION SET 2,332 TEST SET 3,569 50 EVENTS
  • 9. MOTIVATION: Goals 9 ● Improve the results obtained in ChaLearn Challenge. ● Exploit the noisy data collected from Flickr
  • 10. STATE OF THE ART: CaffeNet 10 Content Visual Time stamp Context Geolocation Text Zaharieva’15 X X X Mattivi’11 X X Bossard’13 X X Cao’08 X X X Sutanto’13 X Schinas’12 X X Brenner’13 X X Nguyen’13 X X MediaEval Social Event Detection
  • 11. STATE OF THE ART: CaffeNet 11 CaffeNet ARCHITECTURE [Khrizevsky’12] SOFTWARE [Jia’14] DATA [Deng’09]
  • 12. STATE OF THE ART: CNN ARCHITECTURE 12 Convolutional Neural Network architecture Babenko et al, Neural codes for image retrieval. In Computer Vision-ECCV, 2014
  • 13. STATE OF THE ART: Object+Scene CNNs 13 Object-Scene Convolutional Neural Network for event recognition Wang et al, Object-scene convolutional neural networks for event recognition in images. In CVPRW, 2015
  • 14. OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 14
  • 15. BASELINE: Fine-tuning a ConvNet 15 50
  • 16. BASELINE: ChaLearn @ CVPRW 2015 16 Awarded with the 2nd prize of the Cultural Event Recognition Challenge in the ChaLearn Workshop at CVPR 2015 Salvador. A, Giro-i-Nieto. X, Calafell, A, et al, Cultural Event Recognition with Visual ConvNets and Temporal Models. In CVPRW, 2015
  • 17. BASELINE: ChaLearn @ CVPRW 2015 17 Awarded with the 2nd prize of the Cultural Event Recognition Challenge in the ChaLearn Workshop at CVPR 2015 Salvador. A, Giro-i-Nieto. X, Calafell, A, et al, Cultural Event Recognition with Visual ConvNets and Temporal Models. In CVPRW, 2015
  • 18. OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 18
  • 19. Convnets require to be trained with... 19 a large amount of labeled images
  • 20. but clean data is expensive... 20 and downloading noisy data in an unsupervised fashion is easier and cheaper.
  • 21. NOISY DATA: Flickr Dataset 21 FLICKR DATASET 4,068 50 EVENTS
  • 22. DATASET BIAS 22 Dataset bias when fine-tuning with ChaLearn or Flickr dataset:
  • 23. OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 23
  • 24. DENOISING THE FLICKR DATASET 24 Mosaic of Queens Day from ChaLearn Mosaic of Queens Day from Flickr
  • 25. DENOISING THE FLICKR DATASET 25Example event: Annual Buffalo Roundup Fine-tuned model with ChaLearn New subset from
  • 26. BASELINE: Dataset ordering during fine-tuning 26 CaffeNet FINE-TUNING JOINT:
  • 27. DENOISING THE FLICKR DATASET 27 Joint fine-tuning of the clean and noisy datasets: 0.6136
  • 28. BASELINE: Dataset ordering during fine-tuning 28 CaffeNet FINE-TUNING: FINE-TUNING:
  • 29. DENOISING THE FLICKR DATASET 29 Sequential fine-tuning of the clean and noisy datasets: 0.6136
  • 30. BASELINE: Dataset ordering during fine-tuning 30 CaffeNet FINE-TUNING:FINE-TUNING:
  • 31. DENOISING THE FLICKR DATASET 31 Sequential fine-tuning of the noisy and clean datasets: 0.6136 +1,3%
  • 32. OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 32
  • 33. FRACKING MINING +/- SAMPLES 33
  • 34. FRACKING THE TRAINING DATASET 34Example event: Pingxi Lantern Festival Fine-tuned model with ChaLearn New subset from hard negatives hard positive
  • 35. BASELINE: Dataset ordering during fine-tuning 35 CaffeNet FINE-TUNING: Fine-tuning with fracking subset from:
  • 36. FRACKING THE TRAINING DATASET 36 Results of fine-tuning using fracking in images from ChaLearn: baseline: 0.61365 +0,9%
  • 37. OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 37
  • 38. FINE-TUNING DEEPER LAYERS ONLY 38 Layer 2 responds to corners and other edge/color conjunctions.
  • 39. FINE-TUNING DEEPER LAYERS ONLY 39 Layer 3 has more complex invariances, capturing similar textures Zeiler et al, Visualizing and Understanding Convolutional Networks, In Computer Vision-ECCV 2014,
  • 40. FINE-TUNING DEEPER LAYERS ONLY 40 50 Andrej Karpathy. Convolutional neural networks for visual recognition. In Stanford CS class CS231n. FC6 FC7 FC8
  • 41. FINE-TUNING DEEPER LAYERS ONLY 41 Results of only fine-tuning the deeper layers: +3% 0.61365
  • 42. FINE-TUNING DEEPER LAYERS ONLY 42 Results of only fine-tuning the deeper layers : +4% 0.6136
  • 43. OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 43
  • 44. BASELINE: ChaLearn @ CVPRW 2015 44 Awarded with the 2nd prize of the Cultural Event Recognition Challenge in the ChaLearn Workshop at CVPR 2015 Salvador. A, Giro-i-Nieto. X, Calafell, A, et al, Cultural Event Recognition with Visual ConvNets and Temporal Models. In CVPRW, 2015
  • 45. ENSEMBLE OF EVENT DETECTORS 45 SINGLE CONVNET FOR THE 50 EVENTS:
  • 46. ENSEMBLE OF EVENT DETECTORS 46 ONE CONVNET FOR EACH EVENTS:
  • 47. ENSEMBLE OF EVENT DETECTORS 47 Results of ensemble of binary : +6,6% 0.6136
  • 48. OUTLINE 1. Motivation and State of the art 2. Baseline 3. Study of the dataset bias 4. Denoising 5. Fracking 6. Fine-tuning deeper layers only 7. Ensemble of event detectors 8. Conclusions and future work 48
  • 49. CONLUSIONS 49 ● The Flickr dataset helped us to improve the score by swapping the order in which we were using the clean and noisy datasets CaffeNet FINE-TUNING:FINE-TUNING: +1,3%
  • 50. CONLUSIONS 50 ● The network actually succeeds in improving his performance by learning from its own mistakes when applying fracking. +0,9% CaffeNet FINE-TUNING: Fine-tuning with fracking subset from:
  • 51. CONLUSIONS 51 ● The results are better if we keep the weights learned in the earlier layers from a very large dataset. 50 +4%
  • 52. CONLUSIONS 52 ● Fine-tuning one convnet for each class increases the score. +6,6%
  • 53. FUTURE WORK 53 ● Mix our solutions with a fine-tuned network with PLACES, and with other local solutions. SCENE CNN (PLACES) LOCAL NOW ● Compete (and try to win) ChaLearn @ ICCV 2015 !!
  • 54. FINE-TUNING A CONVOLUTIONAL NETWORK FOR CULTURAL EVENT RECOGNITION ADVISORS: Andrea Calafell Xavier Giró-i-Nieto Amaia Salvador 20/07/2015 AUTHOR: Matthias Zeppelzauer