SlideShare a Scribd company logo

Performance evaluation of GANs in a semisupervised OCR use case

•
•
F
Florian Wilhelm

This document discusses using generative adversarial networks (GANs) for a semi-supervised optical character recognition (OCR) use case involving vehicle identification numbers (VINs). It describes the text spotting pipeline, challenges with limited training data, data augmentation techniques, and implementing a GAN for character detection. Evaluation shows the semi-supervised GAN approach outperforms other methods, achieving over 99% accuracy on VIN detection and recognition from images using only 85 labeled examples. Key learnings include that custom solutions can outperform off-the-shelf tools for specific tasks, and GANs are well-suited for problems with limited labeled data when combined with data augmentation.

Data & Analytics
1 of 37
Download to read offline
Performance Evaluation of GANs in a semi-supervised OCR Use Case Florian Wilhelm London, 2018-10-11
Special Interests • Mathematical Modelling • Recommendation Systems • Data Science in Production • Python Data Stack • Maintainer of PyScaffold Dr. Florian Wilhelm Principal Data Scientist @ inovex @FlorianWilhelm FlorianWilhelm florianwilhelm.info 2 Florian Tanten Master Thesis @ inovex October 2017 - May 2018
IT-project house for digital transformation: ‣ Agile Development & Management ‣ Web · UI/UX · Replatforming · Microservices ‣ Mobile · Apps · Smart Devices · Robotics ‣ Big Data & Business Intelligence Platforms ‣ Data Science · Data Products · Search · Deep Learning ‣ Data Center Automation · DevOps · Cloud · Hosting ‣ Trainings & Coachings Using technology to inspire our clients. And ourselves. inovex offices in Karlsruhe · Cologne · Munich · Pforzheim · Hamburg · Stuttgart. www.inovex.de
4 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
5https://www.autocheck.com/vehiclehistory/autocheck/en/vinbasics Vehicle Identification Number (VIN) Unique identifier like a fingerprint of a vehicle serial number country security code model year assembly plant details flexible fuel vehicles manufacturer
6 Use Case VIN: WF0DXXGAKDEJ37385 VIN-Decoder Manufacturer: BMW Model: X3 Year: 2013-03-21 Engine power: 143 PS Equipment: - Xenon Lights ... Information about the car: Spotting the vehicle identification number (VIN) in images of vehicle registration documents
7 OCR -Libraries PyOCR Commercial software Open source tools
8 „VSSZZZGJZHR03G533“ ??? + OCR with Tesseract
9 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
Character detection & extraction Character recognition 11Girshick et al. (2014), „Region-Based Convolutional Networks for Accurate Object Detection and Segmentation“ Methodology in Text Spotting Sliding Window Computer Vision Tools Others - Connected components - Stroke width transform - Edge detection - SVM - Learning with HOG - CNN - Region proposal - Hypotheses CNN pooling Character or word CNN CNN + RNN SVM Nearest Neighbor High-performer current studies CNN = Convolutional Neural Network SVM = Support Vector Machine HOG = Histogram of oriented Gradients RNN = Recurrent Neural Networks RL = Reinforcement Learning 379Character Recognition ... Spotting = Detection + Recognition
12https://en.wikipedia.org/wiki/Convolutional_neural_network; http://intellabs.github.io/ParallelJavaScript/ Convolutional Neural Network Max pooling with a 2x2 filter and stride = 2Convolution with 3x3 kernel and stride = 1
14 Agenda 1. Use Case 2. Data and Pipeline 3. Semi-supervised Learning 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
15 Objectives - ~170 images of vehicle registration documents b) Semi-supervised method a) Supervised method 2. Comparison of classifiers 1. Implementation of a prototype „XLG0H200NA0A10348“ Dataset: Text Spotting
16 End-to-End Text Spotting Pipeline Sliding window Character Detector (2 classes) Chararacter Recognizer (36 classes) Only one window per character All windows Non Maximum Suppression All windows with characters Region of Interest Extractor Image depicting only VIN X L G 0 H 2 0 N A 10 04 43 80
17 Small Dataset What to do about that? 1. Data Generation 2. Data Augmentation
18 Data Augmentation Data augmentation: Datasets: Original image labeled manually as „0“ 2 classes 36 classes Chararacter Recognizer (36 classes) Label: „0“ Character Detector (2 classes) Label: „character“ Label: „no character“
19 170 images of vehicle registration documents Training set 85 images 85 images Training sets of classifiers Testing sets of classifiers Testing sets of pipeline 85 images RecognizerDetector ~ 42000 images 2 classes ~ 8000 images 36 classes ~ 42000 images 2 classes ~ 8000 images 36 classes RecognizerDetector Data Augmentation Data Augmentation Testing set Datasets
20 Classifiers 1. Supervised Convolutional Neural Network 2. Semi-supervised Generative Adversarial Network Generator Discriminator Input Feature extraction Classification
21 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
22 Yann LeCun Director of Facebook AI Research, Prof at NYU “... (GANs) and the variations that are now being proposed is the most interesting idea in the last 10 years in ML, in my opinion.“ Ian J. Goodfellow @ Google Brain
23 Generative Adversarial Network Generator (G) Discriminator (D) Goal: Generate images, which seem to be realistic Goal: Differentiate between fake and real images
24 Generative Adversarial Network Generator (G) Discriminator (D) Is D correct? „D classified the generated image as 10% real“ „yes“ A B . . . 8 9 F Real imagesReal labeled images
25Goodfellow et al. (2014), Generative Adversarial Networks Mathematical formulation Discriminator output for real images Discriminator output for fake images Discriminator calculates likelihood [0,1] for an image being real Maximizing discriminator loss Minimizing generator loss Objective function Training (alternating)
26 Example of generated images Training images: Generated images during learning process:
27 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
28 Semi-supervised Learning Supervised Learning Unsupervised Learning Semi-supervised Learning • Makes use of unlabeled data • Combines supervised and unsupervised learning
29 Semi-supervised GAN for Character Detection Real labeled images Real unlabeled images Generator Discriminator
30 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
31 Character Detector (2 classes) 60,00% 70,00% 80,00% 90,00% 100,00% 20 50 100 200 400 700 1000 5000 15000 30000 42000 DCNN DCNN pretrained „Character“ „No character“ Manually generated images with CAPTCHA methods Pretraining of DCNN Size of labeled training set Accuracy Bildschirmfoto 2018-04-24 um 17.48.20Bildschirmfoto 2018-04-24 um 17.48.20
32 Character Detector (2 classes) 60,00% 70,00% 80,00% 90,00% 100,00% 20 50 100 200 400 700 1000 5000 15000 30000 42000 DCNN DCNN pretrained Supervised GAN Generator Discriminator Real labeled images C C F C C F Supervised GAN Size of labeled training set Accuracy Bildschirmfoto 2018-04-24 um 17.48.20 Bildschirmfoto 2018-04-24 um 17.48.20
33 Character Detector (2 classes) 60,00% 70,00% 80,00% 90,00% 100,00% 20 50 100 200 400 700 1000 5000 15000 30000 42000 DCNN DCNN pretrained Supervised GAN Semi-supervised GAN Discriminator C C F Generator F Real labeled images CC Real unlabeled images Semi-supervised GAN Size of labeled training set Accuracy Bildschirmfoto 2018-04-24 um 17.48.20
34 Character Recognizer (36 classes) 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% 80,00% 90,00% 100,00% 36 72 108 200 300 400 600 800 1000 5000 8000 60,00% 70,00% 80,00% 90,00% 100,00% 20 50 100 200 400 700 1000 5000 15000 30000 42000 DCNN DCNN pretrained SupervisedGAN Character DetectorCharacter Recognizer Size of labeled training set Accuracy Size of labeled training set Accuracy Bildschirmfoto 2018-04-24 um 17.48.20
.. 35 End-to-End Text Spotting Pipeline Sliding window Character Detector (2 classes) Chararacter Recognizer (36 classes) Non Maximum Suppression Region of Interest Extractor Accuracy = 99.94% 85 images 1. 2. 85. .
36 Google Cloud Vision API Sliding window Character Detector (2 classes) Chararacter Recognizer (36 classes) Non Maximum Suppression Region of Interest Extractor 85 images ∅ Levenshtein distance = 4.49 85 images of VINs .. . Our ApproachGoogle Cloud Vision API vs. ∅ Levenshtein distance = 0.011 Levenshtein distance: Classification Label AYZ33 XYZ321 = 3
37 Key Learnings • Custom solutions can tremendously outperform off-the-shelve software in a specific use-case • Semi-supervised GANs can be successfully applied in use-cases with little data • With simple data augmentation techniques having only little data might be enough
38 Bibliography - Krizhevsky et al. (2012) „ImageNet Classication with Deep Convolutional Neural Networks“ - Girshick et al. (2014), „Region-Based Convolutional Networks for Accurate Object Detection and Segmentation“ - Girshick et al. (2015), „Fast R-CNN“ - Girshick et al. (2015), „Faster R-CNN“ - He et al. (2017), „Mask-R-CNN“ - Goodfellow et al. (2014) „Generative Adversarial Networks"
Thank you! Florian Wilhelm Principal Data Scientist inovex GmbH Schanzenstraße 6-20 Kupferhütte 1.13 51063 Köln florian.wilhelm@inovex.de

Recommended

Uncertainty Quantification in AI
Uncertainty Quantification in AIUncertainty Quantification in AI
Uncertainty Quantification in AI
Florian Wilhelm
 
Uncertainty in Deep Learning
Uncertainty in Deep LearningUncertainty in Deep Learning
Uncertainty in Deep Learning
Roberto Pereira Silveira
 
Modeling uncertainty in deep learning
Modeling uncertainty in deep learning Modeling uncertainty in deep learning
Modeling uncertainty in deep learning
Sungjoon Choi
 
Uncertainty Modeling in Deep Learning
Uncertainty Modeling in Deep LearningUncertainty Modeling in Deep Learning
Uncertainty Modeling in Deep Learning
Sungjoon Choi
 
Multimodal Residual Learning for Visual Question-Answering
Multimodal Residual Learning for Visual Question-AnsweringMultimodal Residual Learning for Visual Question-Answering
Multimodal Residual Learning for Visual Question-Answering
NAVER D2
 
Usage of Generative Adversarial Networks (GANs) in Healthcare
Usage of Generative Adversarial Networks (GANs) in HealthcareUsage of Generative Adversarial Networks (GANs) in Healthcare
Usage of Generative Adversarial Networks (GANs) in Healthcare
GlobalLogic Ukraine
 
Bayesian Deep Learning
Bayesian Deep LearningBayesian Deep Learning
Bayesian Deep Learning
RayKim51
 
Gan 발표자료
Gan 발표자료Gan 발표자료
Gan 발표자료
종현 최
 

Recommended

Uncertainty Quantification in AI
Uncertainty Quantification in AIUncertainty Quantification in AI
Uncertainty Quantification in AI
Florian Wilhelm
 
Uncertainty in Deep Learning
Uncertainty in Deep LearningUncertainty in Deep Learning
Uncertainty in Deep Learning
Roberto Pereira Silveira
 
Modeling uncertainty in deep learning
Modeling uncertainty in deep learning Modeling uncertainty in deep learning
Modeling uncertainty in deep learning
Sungjoon Choi
 
Uncertainty Modeling in Deep Learning
Uncertainty Modeling in Deep LearningUncertainty Modeling in Deep Learning
Uncertainty Modeling in Deep Learning
Sungjoon Choi
 
Multimodal Residual Learning for Visual Question-Answering
Multimodal Residual Learning for Visual Question-AnsweringMultimodal Residual Learning for Visual Question-Answering
Multimodal Residual Learning for Visual Question-Answering
NAVER D2
 
Usage of Generative Adversarial Networks (GANs) in Healthcare
Usage of Generative Adversarial Networks (GANs) in HealthcareUsage of Generative Adversarial Networks (GANs) in Healthcare
Usage of Generative Adversarial Networks (GANs) in Healthcare
GlobalLogic Ukraine
 
Bayesian Deep Learning
Bayesian Deep LearningBayesian Deep Learning
Bayesian Deep Learning
RayKim51
 
Gan 발표자료
Gan 발표자료Gan 발표자료
Gan 발표자료
종현 최
 
Machine learning for_finance
Machine learning for_financeMachine learning for_finance
Machine learning for_finance
Stefan Duprey
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial Networks
MLReview
 
Generative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical ImagingGenerative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical Imaging
Sanghoon Hong
 
Machine learning the next revolution or just another hype
Machine learning the next revolution or just another hypeMachine learning the next revolution or just another hype
Machine learning the next revolution or just another hype
Jorge Ferrer
 
(141205) Masters_Thesis_Defense_Sundong_Kim
(141205) Masters_Thesis_Defense_Sundong_Kim(141205) Masters_Thesis_Defense_Sundong_Kim
(141205) Masters_Thesis_Defense_Sundong_Kim
Sundong Kim
 
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
MLconf
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
Yunjey Choi
 
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
MLconf
 
Semantic Analysis to Compute Personality Traits from Social Media Posts
Semantic Analysis to Compute Personality Traits from Social Media PostsSemantic Analysis to Compute Personality Traits from Social Media Posts
Semantic Analysis to Compute Personality Traits from Social Media Posts
Giulio Carducci
 
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
MLconf
 
Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...
宏毅 李
 
Performance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use casePerformance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use case
inovex GmbH
 
Basic Generative Adversarial Networks
Basic Generative Adversarial NetworksBasic Generative Adversarial Networks
Basic Generative Adversarial Networks
Dong Heon Cho
 
Capitalico / Chart Pattern Matching in Financial Trading Using RNN
Capitalico / Chart Pattern Matching in Financial Trading Using RNNCapitalico / Chart Pattern Matching in Financial Trading Using RNN
Capitalico / Chart Pattern Matching in Financial Trading Using RNN
Alpaca
 
Deep Generative Models
Deep Generative Models Deep Generative Models
Deep Generative Models
Chia-Wen Cheng
 
Bayesian Networks with R and Hadoop
Bayesian Networks with R and HadoopBayesian Networks with R and Hadoop
Bayesian Networks with R and Hadoop
Ofer Mendelevitch
 
Kernel, RKHS, and Gaussian Processes
Kernel, RKHS, and Gaussian ProcessesKernel, RKHS, and Gaussian Processes
Kernel, RKHS, and Gaussian Processes
Sungjoon Choi
 
[GAN by Hung-yi Lee]Part 1: General introduction of GAN
[GAN by Hung-yi Lee]Part 1: General introduction of GAN[GAN by Hung-yi Lee]Part 1: General introduction of GAN
[GAN by Hung-yi Lee]Part 1: General introduction of GAN
NAVER Engineering
 
[系列活動] 一日搞懂生成式對抗網路
[系列活動] 一日搞懂生成式對抗網路[系列活動] 一日搞懂生成式對抗網路
[系列活動] 一日搞懂生成式對抗網路
台灣資料科學年會
 
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial NetworksIntroduction to Generative Adversarial Networks
Introduction to Generative Adversarial Networks
BennoG1
 
AI and Deep Learning
AI and Deep Learning AI and Deep Learning
AI and Deep Learning
Subrat Panda, PhD
 
Machine learning quality for production
Machine learning quality for productionMachine learning quality for production
Machine learning quality for production
yusuke shibui
 

More Related Content

What's hot

Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial Networks
MLReview
 
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
MLconf
 
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
MLconf
 
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
MLconf
 
Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...
宏毅 李
 
Performance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use casePerformance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use case
inovex GmbH
 
[GAN by Hung-yi Lee]Part 1: General introduction of GAN
[GAN by Hung-yi Lee]Part 1: General introduction of GAN[GAN by Hung-yi Lee]Part 1: General introduction of GAN
[GAN by Hung-yi Lee]Part 1: General introduction of GAN
NAVER Engineering
 
[系列活動] 一日搞懂生成式對抗網路
[系列活動] 一日搞懂生成式對抗網路[系列活動] 一日搞懂生成式對抗網路
[系列活動] 一日搞懂生成式對抗網路
台灣資料科學年會
 

What's hot (20)

Machine learning for_finance
Machine learning for_financeMachine learning for_finance
Machine learning for_finance
 
Tutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial NetworksTutorial on Theory and Application of Generative Adversarial Networks
Tutorial on Theory and Application of Generative Adversarial Networks
 
Generative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical ImagingGenerative Adversarial Networks and Their Applications in Medical Imaging
Generative Adversarial Networks and Their Applications in Medical Imaging
 
Machine learning the next revolution or just another hype
Machine learning the next revolution or just another hypeMachine learning the next revolution or just another hype
Machine learning the next revolution or just another hype
 
(141205) Masters_Thesis_Defense_Sundong_Kim
(141205) Masters_Thesis_Defense_Sundong_Kim(141205) Masters_Thesis_Defense_Sundong_Kim
(141205) Masters_Thesis_Defense_Sundong_Kim
 
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
Aaron Roth, Associate Professor, University of Pennsylvania, at MLconf NYC 2017
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
Irina Rish, Researcher, IBM Watson, at MLconf NYC 2017
 
Semantic Analysis to Compute Personality Traits from Social Media Posts
Semantic Analysis to Compute Personality Traits from Social Media PostsSemantic Analysis to Compute Personality Traits from Social Media Posts
Semantic Analysis to Compute Personality Traits from Social Media Posts
 
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
Hanie Sedghi, Research Scientist at Allen Institute for Artificial Intelligen...
 
Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...Generative Adversarial Network and its Applications to Speech Processing an...
Generative Adversarial Network and its Applications to Speech Processing an...
 
Performance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use casePerformance evaluation of GANs in a semisupervised OCR use case
Performance evaluation of GANs in a semisupervised OCR use case
 
Basic Generative Adversarial Networks
Basic Generative Adversarial NetworksBasic Generative Adversarial Networks
Basic Generative Adversarial Networks
 
Capitalico / Chart Pattern Matching in Financial Trading Using RNN
Capitalico / Chart Pattern Matching in Financial Trading Using RNNCapitalico / Chart Pattern Matching in Financial Trading Using RNN
Capitalico / Chart Pattern Matching in Financial Trading Using RNN
 
Deep Generative Models
Deep Generative Models Deep Generative Models
Deep Generative Models
 
Bayesian Networks with R and Hadoop
Bayesian Networks with R and HadoopBayesian Networks with R and Hadoop
Bayesian Networks with R and Hadoop
 
Kernel, RKHS, and Gaussian Processes
Kernel, RKHS, and Gaussian ProcessesKernel, RKHS, and Gaussian Processes
Kernel, RKHS, and Gaussian Processes
 
[GAN by Hung-yi Lee]Part 1: General introduction of GAN
[GAN by Hung-yi Lee]Part 1: General introduction of GAN[GAN by Hung-yi Lee]Part 1: General introduction of GAN
[GAN by Hung-yi Lee]Part 1: General introduction of GAN
 
[系列活動] 一日搞懂生成式對抗網路
[系列活動] 一日搞懂生成式對抗網路[系列活動] 一日搞懂生成式對抗網路
[系列活動] 一日搞懂生成式對抗網路
 
Introduction to Generative Adversarial Networks
Introduction to Generative Adversarial NetworksIntroduction to Generative Adversarial Networks
Introduction to Generative Adversarial Networks
 

Similar to Performance evaluation of GANs in a semisupervised OCR use case

Monitoring AI with AI
Monitoring AI with AIMonitoring AI with AI
Monitoring AI with AI
Stepan Pushkarev
 
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Databricks
 
Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribu...
Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribu...Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribu...
Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribu...
polochau
 

Similar to Performance evaluation of GANs in a semisupervised OCR use case (20)

AI and Deep Learning
AI and Deep Learning AI and Deep Learning
AI and Deep Learning
 
Machine learning quality for production
Machine learning quality for productionMachine learning quality for production
Machine learning quality for production
 
Machine Vision On Embedded Platform
Machine Vision On Embedded Platform Machine Vision On Embedded Platform
Machine Vision On Embedded Platform
 
Machine vision Application
Machine vision ApplicationMachine vision Application
Machine vision Application
 
深度學習在AOI的應用
深度學習在AOI的應用深度學習在AOI的應用
深度學習在AOI的應用
 
RaymondResume2015v5
RaymondResume2015v5RaymondResume2015v5
RaymondResume2015v5
 
小數據如何實現電腦視覺，微軟AI研究首席剖析關鍵
小數據如何實現電腦視覺，微軟AI研究首席剖析關鍵小數據如何實現電腦視覺，微軟AI研究首席剖析關鍵
小數據如何實現電腦視覺，微軟AI研究首席剖析關鍵
 
Monitoring AI with AI
Monitoring AI with AIMonitoring AI with AI
Monitoring AI with AI
 
Data Summer Conf 2018, “Monitoring AI with AI (RUS)” — Stepan Pushkarev, CTO ...
Data Summer Conf 2018, “Monitoring AI with AI (RUS)” — Stepan Pushkarev, CTO ...Data Summer Conf 2018, “Monitoring AI with AI (RUS)” — Stepan Pushkarev, CTO ...
Data Summer Conf 2018, “Monitoring AI with AI (RUS)” — Stepan Pushkarev, CTO ...
 
230208 MLOps Getting from Good to Great.pptx
230208 MLOps Getting from Good to Great.pptx230208 MLOps Getting from Good to Great.pptx
230208 MLOps Getting from Good to Great.pptx
 
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
 
[Connect(); // Japan 2016] Microsoft の AI 開発最新アップデート ～ Cognitive Services からA...
[Connect(); // Japan 2016] Microsoft の AI 開発最新アップデート ～ Cognitive Services からA...[Connect(); // Japan 2016] Microsoft の AI 開発最新アップデート ～ Cognitive Services からA...
[Connect(); // Japan 2016] Microsoft の AI 開発最新アップデート ～ Cognitive Services からA...
 
Obscenity Detection in Images
Obscenity Detection in ImagesObscenity Detection in Images
Obscenity Detection in Images
 
Computer vision for transportation
Computer vision for transportationComputer vision for transportation
Computer vision for transportation
 
Machine Learning AND Deep Learning for OpenPOWER
Machine Learning AND Deep Learning for OpenPOWERMachine Learning AND Deep Learning for OpenPOWER
Machine Learning AND Deep Learning for OpenPOWER
 
Intelligent Thumbnail Selection
Intelligent Thumbnail SelectionIntelligent Thumbnail Selection
Intelligent Thumbnail Selection
 
B4UConference_machine learning_deeplearning
B4UConference_machine learning_deeplearningB4UConference_machine learning_deeplearning
B4UConference_machine learning_deeplearning
 
Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribu...
Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribu...Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribu...
Human-Centered AI: Scalable, Interactive Tools for Interpretation and Attribu...
 
Session 1 - The Current Landscape of Big Data Benchmarks
Session 1 - The Current Landscape of Big Data BenchmarksSession 1 - The Current Landscape of Big Data Benchmarks
Session 1 - The Current Landscape of Big Data Benchmarks
 
20170402 Crop Innovation and Business - Amsterdam
20170402 Crop Innovation and Business - Amsterdam20170402 Crop Innovation and Business - Amsterdam
20170402 Crop Innovation and Business - Amsterdam
 

More from Florian Wilhelm

Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
Florian Wilhelm
 
Unlocking the Power of Integer Programming
Unlocking the Power of Integer ProgrammingUnlocking the Power of Integer Programming
Unlocking the Power of Integer Programming
Florian Wilhelm
 
Forget about AI and do Mathematical Modelling instead!
Forget about AI and do Mathematical Modelling instead!Forget about AI and do Mathematical Modelling instead!
Forget about AI and do Mathematical Modelling instead!
Florian Wilhelm
 
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
Florian Wilhelm
 
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Florian Wilhelm
 
Bridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to ProductionBridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to Production
Florian Wilhelm
 
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Florian Wilhelm
 
PyData Meetup Berlin 2017-04-19
PyData Meetup Berlin 2017-04-19PyData Meetup Berlin 2017-04-19
PyData Meetup Berlin 2017-04-19
Florian Wilhelm
 

More from Florian Wilhelm (15)

Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Unlocking the Power of Integer Programming
Unlocking the Power of Integer ProgrammingUnlocking the Power of Integer Programming
Unlocking the Power of Integer Programming
 
WALD: A Modern & Sustainable Analytics Stack
WALD: A Modern & Sustainable Analytics StackWALD: A Modern & Sustainable Analytics Stack
WALD: A Modern & Sustainable Analytics Stack
 
Forget about AI and do Mathematical Modelling instead!
Forget about AI and do Mathematical Modelling instead!Forget about AI and do Mathematical Modelling instead!
Forget about AI and do Mathematical Modelling instead!
 
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
An Interpretable Model for Collaborative Filtering Using an Extended Latent D...
 
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
Honey I Shrunk the Target Variable! Common pitfalls when transforming the tar...
 
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
Matrix Factorization for Collaborative Filtering Is Just Solving an Adjoint L...
 
Bridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to ProductionBridging the Gap: from Data Science to Production
Bridging the Gap: from Data Science to Production
 
How mobile.de brings Data Science to Production for a Personalized Web Experi...
How mobile.de brings Data Science to Production for a Personalized Web Experi...How mobile.de brings Data Science to Production for a Personalized Web Experi...
How mobile.de brings Data Science to Production for a Personalized Web Experi...
 
Deep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
Deep Learning-based Recommendations for Germany's Biggest Vehicle MarketplaceDeep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
Deep Learning-based Recommendations for Germany's Biggest Vehicle Marketplace
 
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
Deep Learning-based Recommendations for Germany's Biggest Online Vehicle Mark...
 
Declarative Thinking and Programming
Declarative Thinking and ProgrammingDeclarative Thinking and Programming
Declarative Thinking and Programming
 
Which car fits my life? - PyData Berlin 2017
Which car fits my life? - PyData Berlin 2017Which car fits my life? - PyData Berlin 2017
Which car fits my life? - PyData Berlin 2017
 
PyData Meetup Berlin 2017-04-19
PyData Meetup Berlin 2017-04-19PyData Meetup Berlin 2017-04-19
PyData Meetup Berlin 2017-04-19
 
Explaining the idea behind automatic relevance determination and bayesian int...
Explaining the idea behind automatic relevance determination and bayesian int...Explaining the idea behind automatic relevance determination and bayesian int...
Explaining the idea behind automatic relevance determination and bayesian int...
 

Recently uploaded

Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
LuisMiguelPaz5
 
Simplify hybrid data integration at an enterprise scale. Integrate all your d...
Simplify hybrid data integration at an enterprise scale. Integrate all your d...Simplify hybrid data integration at an enterprise scale. Integrate all your d...
Simplify hybrid data integration at an enterprise scale. Integrate all your d...
varanasisatyanvesh
 
原件一样(UWO毕业证书）西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书）西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书）西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书）西安大略大学毕业证成绩单留信学历认证
pwgnohujw
 
如何办理(WashU毕业证书）圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(WashU毕业证书）圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证如何办理(WashU毕业证书）圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(WashU毕业证书）圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
acoha1
 
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Klinik kandungan
 
Abortion pills in Riyadh Saudi Arabia| +966572737505 | Get Cytotec, Unwanted Kit
Abortion pills in Riyadh Saudi Arabia| +966572737505 | Get Cytotec, Unwanted KitAbortion pills in Riyadh Saudi Arabia| +966572737505 | Get Cytotec, Unwanted Kit
Abortion pills in Riyadh Saudi Arabia| +966572737505 | Get Cytotec, Unwanted Kit
Abortion pills in Riyadh +966572737505 get cytotec
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
yulianti213969
 
Abortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
Abortion pills in Doha {{ QATAR }} +966572737505) Get CytotecAbortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
Abortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 

Recently uploaded (20)

Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024Northern New England Tableau User Group (TUG) May 2024
Northern New England Tableau User Group (TUG) May 2024
 
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
Huawei Ransomware Protection Storage Solution Technical Overview Presentation...
 
Simplify hybrid data integration at an enterprise scale. Integrate all your d...
Simplify hybrid data integration at an enterprise scale. Integrate all your d...Simplify hybrid data integration at an enterprise scale. Integrate all your d...
Simplify hybrid data integration at an enterprise scale. Integrate all your d...
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
 
原件一样(UWO毕业证书）西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书）西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书）西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书）西安大略大学毕业证成绩单留信学历认证
 
Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshare
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Case Study 4 Where the cry of rebellion happen?
Case Study 4 Where the cry of rebellion happen?Case Study 4 Where the cry of rebellion happen?
Case Study 4 Where the cry of rebellion happen?
 
如何办理(WashU毕业证书）圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(WashU毕业证书）圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证如何办理(WashU毕业证书）圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(WashU毕业证书）圣路易斯华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
 
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
 
jll-asia-pacific-capital-tracker-1q24.pdf
jll-asia-pacific-capital-tracker-1q24.pdfjll-asia-pacific-capital-tracker-1q24.pdf
jll-asia-pacific-capital-tracker-1q24.pdf
 
Abortion pills in Riyadh Saudi Arabia| +966572737505 | Get Cytotec, Unwanted Kit
Abortion pills in Riyadh Saudi Arabia| +966572737505 | Get Cytotec, Unwanted KitAbortion pills in Riyadh Saudi Arabia| +966572737505 | Get Cytotec, Unwanted Kit
Abortion pills in Riyadh Saudi Arabia| +966572737505 | Get Cytotec, Unwanted Kit
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting Techniques
 
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptxRESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
RESEARCH-FINAL-DEFENSE-PPT-TEMPLATE.pptx
 
Pentesting_AI and security challenges of AI
Pentesting_AI and security challenges of AIPentesting_AI and security challenges of AI
Pentesting_AI and security challenges of AI
 
社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
 
Abortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
Abortion pills in Doha {{ QATAR }} +966572737505) Get CytotecAbortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
Abortion pills in Doha {{ QATAR }} +966572737505) Get Cytotec
 
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital AgeCredit Card Fraud Detection: Safeguarding Transactions in the Digital Age
Credit Card Fraud Detection: Safeguarding Transactions in the Digital Age
 
Introduction to Statistics Presentation.pptx
Introduction to Statistics Presentation.pptxIntroduction to Statistics Presentation.pptx
Introduction to Statistics Presentation.pptx
 

Performance evaluation of GANs in a semisupervised OCR use case

  • 1. Performance Evaluation of GANs in a semi-supervised OCR Use Case Florian Wilhelm London, 2018-10-11
  • 2. Special Interests • Mathematical Modelling • Recommendation Systems • Data Science in Production • Python Data Stack • Maintainer of PyScaffold Dr. Florian Wilhelm Principal Data Scientist @ inovex @FlorianWilhelm FlorianWilhelm florianwilhelm.info 2 Florian Tanten Master Thesis @ inovex October 2017 - May 2018
  • 3. IT-project house for digital transformation: ‣ Agile Development & Management ‣ Web · UI/UX · Replatforming · Microservices ‣ Mobile · Apps · Smart Devices · Robotics ‣ Big Data & Business Intelligence Platforms ‣ Data Science · Data Products · Search · Deep Learning ‣ Data Center Automation · DevOps · Cloud · Hosting ‣ Trainings & Coachings Using technology to inspire our clients. And ourselves. inovex offices in Karlsruhe · Cologne · Munich · Pforzheim · Hamburg · Stuttgart. www.inovex.de
  • 4. 4 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
  • 5. 5https://www.autocheck.com/vehiclehistory/autocheck/en/vinbasics Vehicle Identification Number (VIN) Unique identifier like a fingerprint of a vehicle serial number country security code model year assembly plant details flexible fuel vehicles manufacturer
  • 6. 6 Use Case VIN: WF0DXXGAKDEJ37385 VIN-Decoder Manufacturer: BMW Model: X3 Year: 2013-03-21 Engine power: 143 PS Equipment: - Xenon Lights ... Information about the car: Spotting the vehicle identification number (VIN) in images of vehicle registration documents
  • 9. 9 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
  • 10. Character detection & extraction Character recognition 11Girshick et al. (2014), „Region-Based Convolutional Networks for Accurate Object Detection and Segmentation“ Methodology in Text Spotting Sliding Window Computer Vision Tools Others - Connected components - Stroke width transform - Edge detection - SVM - Learning with HOG - CNN - Region proposal - Hypotheses CNN pooling Character or word CNN CNN + RNN SVM Nearest Neighbor High-performer current studies CNN = Convolutional Neural Network SVM = Support Vector Machine HOG = Histogram of oriented Gradients RNN = Recurrent Neural Networks RL = Reinforcement Learning 379Character Recognition ... Spotting = Detection + Recognition
  • 11. 12https://en.wikipedia.org/wiki/Convolutional_neural_network; http://intellabs.github.io/ParallelJavaScript/ Convolutional Neural Network Max pooling with a 2x2 filter and stride = 2Convolution with 3x3 kernel and stride = 1
  • 12. 14 Agenda 1. Use Case 2. Data and Pipeline 3. Semi-supervised Learning 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
  • 13. 15 Objectives - ~170 images of vehicle registration documents b) Semi-supervised method a) Supervised method 2. Comparison of classifiers 1. Implementation of a prototype „XLG0H200NA0A10348“ Dataset: Text Spotting
  • 14. 16 End-to-End Text Spotting Pipeline Sliding window Character Detector (2 classes) Chararacter Recognizer (36 classes) Only one window per character All windows Non Maximum Suppression All windows with characters Region of Interest Extractor Image depicting only VIN X L G 0 H 2 0 N A 10 04 43 80
  • 15. 17 Small Dataset What to do about that? 1. Data Generation 2. Data Augmentation
  • 16. 18 Data Augmentation Data augmentation: Datasets: Original image labeled manually as „0“ 2 classes 36 classes Chararacter Recognizer (36 classes) Label: „0“ Character Detector (2 classes) Label: „character“ Label: „no character“
  • 17. 19 170 images of vehicle registration documents Training set 85 images 85 images Training sets of classifiers Testing sets of classifiers Testing sets of pipeline 85 images RecognizerDetector ~ 42000 images 2 classes ~ 8000 images 36 classes ~ 42000 images 2 classes ~ 8000 images 36 classes RecognizerDetector Data Augmentation Data Augmentation Testing set Datasets
  • 18. 20 Classifiers 1. Supervised Convolutional Neural Network 2. Semi-supervised Generative Adversarial Network Generator Discriminator Input Feature extraction Classification
  • 19. 21 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
  • 20. 22 Yann LeCun Director of Facebook AI Research, Prof at NYU “... (GANs) and the variations that are now being proposed is the most interesting idea in the last 10 years in ML, in my opinion.“ Ian J. Goodfellow @ Google Brain
  • 21. 23 Generative Adversarial Network Generator (G) Discriminator (D) Goal: Generate images, which seem to be realistic Goal: Differentiate between fake and real images
  • 22. 24 Generative Adversarial Network Generator (G) Discriminator (D) Is D correct? „D classified the generated image as 10% real“ „yes“ A B . . . 8 9 F Real imagesReal labeled images
  • 23. 25Goodfellow et al. (2014), Generative Adversarial Networks Mathematical formulation Discriminator output for real images Discriminator output for fake images Discriminator calculates likelihood [0,1] for an image being real Maximizing discriminator loss Minimizing generator loss Objective function Training (alternating)
  • 24. 26 Example of generated images Training images: Generated images during learning process:
  • 25. 27 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
  • 26. 28 Semi-supervised Learning Supervised Learning Unsupervised Learning Semi-supervised Learning • Makes use of unlabeled data • Combines supervised and unsupervised learning
  • 27. 29 Semi-supervised GAN for Character Detection Real labeled images Real unlabeled images Generator Discriminator
  • 28. 30 Agenda 1. Use Case 2. Text Spotting 3. Data and Pipeline 4. Generative Adversarial Networks 5. Semi-supervised Learning 6. Results
  • 29. 31 Character Detector (2 classes) 60,00% 70,00% 80,00% 90,00% 100,00% 20 50 100 200 400 700 1000 5000 15000 30000 42000 DCNN DCNN pretrained „Character“ „No character“ Manually generated images with CAPTCHA methods Pretraining of DCNN Size of labeled training set Accuracy Bildschirmfoto 2018-04-24 um 17.48.20Bildschirmfoto 2018-04-24 um 17.48.20
  • 30. 32 Character Detector (2 classes) 60,00% 70,00% 80,00% 90,00% 100,00% 20 50 100 200 400 700 1000 5000 15000 30000 42000 DCNN DCNN pretrained Supervised GAN Generator Discriminator Real labeled images C C F C C F Supervised GAN Size of labeled training set Accuracy Bildschirmfoto 2018-04-24 um 17.48.20 Bildschirmfoto 2018-04-24 um 17.48.20
  • 31. 33 Character Detector (2 classes) 60,00% 70,00% 80,00% 90,00% 100,00% 20 50 100 200 400 700 1000 5000 15000 30000 42000 DCNN DCNN pretrained Supervised GAN Semi-supervised GAN Discriminator C C F Generator F Real labeled images CC Real unlabeled images Semi-supervised GAN Size of labeled training set Accuracy Bildschirmfoto 2018-04-24 um 17.48.20
  • 32. 34 Character Recognizer (36 classes) 0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% 80,00% 90,00% 100,00% 36 72 108 200 300 400 600 800 1000 5000 8000 60,00% 70,00% 80,00% 90,00% 100,00% 20 50 100 200 400 700 1000 5000 15000 30000 42000 DCNN DCNN pretrained SupervisedGAN Character DetectorCharacter Recognizer Size of labeled training set Accuracy Size of labeled training set Accuracy Bildschirmfoto 2018-04-24 um 17.48.20
  • 33. .. 35 End-to-End Text Spotting Pipeline Sliding window Character Detector (2 classes) Chararacter Recognizer (36 classes) Non Maximum Suppression Region of Interest Extractor Accuracy = 99.94% 85 images 1. 2. 85. .
  • 34. 36 Google Cloud Vision API Sliding window Character Detector (2 classes) Chararacter Recognizer (36 classes) Non Maximum Suppression Region of Interest Extractor 85 images ∅ Levenshtein distance = 4.49 85 images of VINs .. . Our ApproachGoogle Cloud Vision API vs. ∅ Levenshtein distance = 0.011 Levenshtein distance: Classification Label AYZ33 XYZ321 = 3
  • 35. 37 Key Learnings • Custom solutions can tremendously outperform off-the-shelve software in a specific use-case • Semi-supervised GANs can be successfully applied in use-cases with little data • With simple data augmentation techniques having only little data might be enough
  • 36. 38 Bibliography - Krizhevsky et al. (2012) „ImageNet Classication with Deep Convolutional Neural Networks“ - Girshick et al. (2014), „Region-Based Convolutional Networks for Accurate Object Detection and Segmentation“ - Girshick et al. (2015), „Fast R-CNN“ - Girshick et al. (2015), „Faster R-CNN“ - He et al. (2017), „Mask-R-CNN“ - Goodfellow et al. (2014) „Generative Adversarial Networks"
  • 37. Thank you! Florian Wilhelm Principal Data Scientist inovex GmbH Schanzenstraße 6-20 Kupferhütte 1.13 51063 Köln florian.wilhelm@inovex.de