SlideShare a Scribd company logo
© 2021 EJ Technology Consultants
Tools and Strategies for Quickly
Building Effective Image Datasets
Evan Juras
Computer Vision Engineer
© 2021 EJ Technology Consultants
Purpose
• Demonstrate practical strategies and tools for quickly building image
datasets
• Train higher-quality models with less time investment
• Focus on object detection models
2
mAP
%
mAP
%
© 2021 EJ Technology Consultants
Object Detection Neural Networks
• Object detection models
locate and identify objects in
images
• Use CNNs to find features in
images and correlate them to
known objects they’ve been
trained on
• For this presentation,
“training” is transfer learning
and fine-tuning
3
Visualizing and Comparing Convolutional Neural Networks
arXiv: 1412.6631v2
© 2021 EJ Technology Consultants
Applying Object Detection Models
• Scope of application determines amount of
training images needed
• Constrained applications with limited variety of
visual conditions:
• Lower generalization of model is okay
• Fewer training images needed
• Applications with wide variety of visual
conditions:
• Need to be accurate at different light levels,
angles, coloration, distance from camera, etc.
• Larger number of images are needed
4
Playing card detector – low variance
Self-driving car – high variance
© 2021 EJ Technology Consultants
How can we Quickly Create an Object Detection Dataset?
• Problem: to train a new object detection model, a large image dataset is needed
• Manually gathering and labeling images is time consuming
5
• Solution: Use these strategies to accelerate dataset
creation!
• Using datasets already available online
• Capturing images from video
• Using sleek annotation and automated methods
tools to quickly label data
• Synthetic image generation and data
augmentation
My example – Bison distance detector
© 2021 EJ Technology Consultants
Using Online Image Datasets
• Datasets available online can be a good starting point
• Academic datasets such as Open Images Dataset,
ImageNet, COCO, etc.
• User-contributed datasets from TensorFlow, Kaggle,
or other sources
• Issues with online datasets
• Can be filled with poor quality images or label data
• Datasets not be available or appropriate for your
application
• Copyright and licensing: need to be careful using
copyrighted material for commercial purposes
6
© 2021 EJ Technology Consultants
Capturing Images from Video
• Gathering images from the camera used by the application significantly improves accuracy
7
• Various methods for capturing images specific
to your application:
• Set up cameras in situations similar to actual
in-situ application, record video, and grab
frames from video
• Use multiple cameras in multiple locations
and aggregate videos to a central server
• Use online video or live camera streams of
objects you are interested in
© 2021 EJ Technology Consultants
Capturing Images from Video, Continued
Python script to extract individual frames from a video:
github.com/EdjeElectronics/Image-Dataset-Tools#FrameGrabber
8
© 2021 EJ Technology Consultants
Image Annotation Tools
• Many annotation tools are available to help accelerate the image labeling process
• Paid annotation tools:
• V7 Darwin
• Supervisely
• Hive – pay humans to annotate
your data
• Lionbridge – pay humans to
annotate your data
• Free annotation tools:
• CVAT
• LabelImg
9
Video annotation tool from V7
© 2021 EJ Technology Consultants
Automated Labeling Methods
• Supervised automated labeling
• Train a model off a small portion of the
dataset
• Use that model to automatically label the
rest of the dataset, while manually rejecting
poor labels and re-labeling them
• github.com/EdjeElectronics/Image-Dataset-
Tools#AutoLabeler
• Other creative methods
• Tryolabs example uses OpenPose to
automatically locate positions of heads in a
frame
10
Face mask detection
in street camera
video streams using
AI: behind the
curtain
(Courtesy of
Tryolabs)
Automatically
labeling raccoon
images
© 2021 EJ Technology Consultants
Synthetic Image Generation and Data Augmentation
11
Synthetic Image Generation Data Augmentation
• Tools are available to synthetically generate
images
• Mindtech
• KineticVision
Synthetic scenario video from Mindtech
• Generate new images from existing ones
• Increase visual variety of images
• Resolve class imbalances
• Great for 2D objects and perspectives
Check out my data augmentation talk from EVS2020!
© 2021 EJ Technology Consultants
Limitations of These Methods
• Still need some manual groundwork involved in collecting images
• Searching for datasets, setting up video recording, auditing labels for accuracy
• Other limitations
• Difficult to use for uncommon objects or applications
• Won’t bring model up to highest accuracy possible – for best performance, need to
carefully curate and refine dataset, which takes time
12
© 2021 EJ Technology Consultants
Conclusions
• There are various methods for quickly building an object detection dataset
• Online datasets
• Fast image capture
• Sleek annotation tools and automated labeling
• Synthetic image generation and data augmentation
• Depending on application, may still need to manually curate dataset for best
accuracy
• Same concepts can be applied to image classification models
13
© 2021 EJ Technology Consultants
Example of Resource Slide
14
Resource Links
Browser-based free annotation tool (CVAT):
https://cvat.org
Using TensorFlow’s built-in datasets:
https://www.tensorflow.org/datasets/overview
Handy scripts for working with image datasets:
https://github.com/EdjeElectronics/Image-Dataset-Tools
Contact Information
Website: www.ejtech.io
Email: evan.juras@ejtech.io
References
[1]: M. Zeiler, R. Fergus (2013). Visualizing and
Understanding Convolutional Networks. arXiv:1311.2901
[2]: W. Yu, K. Yang, Y. Bai, H. Yao, Y. Rui (2014). Visualizing and
Comparing Convolutional Neural Networks.
arXiv:1412.6631v2

More Related Content

Similar to “Tools and Strategies for Quickly Building Effective Image Datasets,” a Presentation from BDTI

Biometric Recognition using Deep Learning
Biometric Recognition using Deep LearningBiometric Recognition using Deep Learning
Biometric Recognition using Deep Learning
SahithiKotha2
 
Real Time Moving Object Detection for Day-Night Surveillance using AI
Real Time Moving Object Detection for Day-Night Surveillance using AIReal Time Moving Object Detection for Day-Night Surveillance using AI
Real Time Moving Object Detection for Day-Night Surveillance using AI
IRJET Journal
 
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr..."Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
Edge AI and Vision Alliance
 
A Ensemble Learning-based No Reference QoE Model for User Generated Contents
A Ensemble Learning-based No Reference QoE Model for User Generated ContentsA Ensemble Learning-based No Reference QoE Model for User Generated Contents
A Ensemble Learning-based No Reference QoE Model for User Generated Contents
Duc Nguyen
 
BTP_ppt
BTP_pptBTP_ppt
BTP_ppt
Porika Kapil
 
Building a Custom Vision Model
Building a Custom Vision ModelBuilding a Custom Vision Model
Building a Custom Vision Model
Nicholas Toscano
 
Traffic Violation Detector using Object Detection
Traffic Violation Detector using Object DetectionTraffic Violation Detector using Object Detection
Traffic Violation Detector using Object Detection
shri ram murti smarak college of engineering,technology & research
 
A survey on Measurement of Objective Video Quality in Social Cloud using Mach...
A survey on Measurement of Objective Video Quality in Social Cloud using Mach...A survey on Measurement of Objective Video Quality in Social Cloud using Mach...
A survey on Measurement of Objective Video Quality in Social Cloud using Mach...
IRJET Journal
 
Computer Vision with Deep Learning
Computer Vision with Deep LearningComputer Vision with Deep Learning
Computer Vision with Deep Learning
Capgemini
 
Object detection presentation
Object detection presentationObject detection presentation
Object detection presentation
AshwinBicholiya
 
Deep Learning Projects - Anomaly Detection Using Deep Learning
Deep Learning Projects - Anomaly Detection Using Deep LearningDeep Learning Projects - Anomaly Detection Using Deep Learning
Deep Learning Projects - Anomaly Detection Using Deep Learning
DezyreAcademy
 
“Applying the Right Deep Learning Model with the Right Data for Your Applicat...
“Applying the Right Deep Learning Model with the Right Data for Your Applicat...“Applying the Right Deep Learning Model with the Right Data for Your Applicat...
“Applying the Right Deep Learning Model with the Right Data for Your Applicat...
Edge AI and Vision Alliance
 
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
Edge AI and Vision Alliance
 
USING IMAGE CLASSIFICATION TO INCENTIVIZE RECYCLING
USING IMAGE CLASSIFICATION TO INCENTIVIZE RECYCLINGUSING IMAGE CLASSIFICATION TO INCENTIVIZE RECYCLING
USING IMAGE CLASSIFICATION TO INCENTIVIZE RECYCLING
IRJET Journal
 
lokesh_UX_Designer_v5
lokesh_UX_Designer_v5lokesh_UX_Designer_v5
lokesh_UX_Designer_v5
Lokesh S
 
Computer architecture for vision system
Computer architecture for vision systemComputer architecture for vision system
Computer architecture for vision system
AkashPatil334
 
FYP-Final-External
FYP-Final-ExternalFYP-Final-External
FYP-Final-External
Ahmed Rik
 
Kyligence Cloud 4 - Feature Focus: AI-Augmented Engine
Kyligence Cloud 4 - Feature Focus: AI-Augmented EngineKyligence Cloud 4 - Feature Focus: AI-Augmented Engine
Kyligence Cloud 4 - Feature Focus: AI-Augmented Engine
SamanthaBerlant
 
Deep learning with_computer_vision
Deep learning with_computer_visionDeep learning with_computer_vision
Deep learning with_computer_vision
Anand Narayanan
 
Mini Project- Face Recognition
Mini Project- Face RecognitionMini Project- Face Recognition

Similar to “Tools and Strategies for Quickly Building Effective Image Datasets,” a Presentation from BDTI (20)

Biometric Recognition using Deep Learning
Biometric Recognition using Deep LearningBiometric Recognition using Deep Learning
Biometric Recognition using Deep Learning
 
Real Time Moving Object Detection for Day-Night Surveillance using AI
Real Time Moving Object Detection for Day-Night Surveillance using AIReal Time Moving Object Detection for Day-Night Surveillance using AI
Real Time Moving Object Detection for Day-Night Surveillance using AI
 
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr..."Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
"Enabling Ubiquitous Visual Intelligence Through Deep Learning," a Keynote Pr...
 
A Ensemble Learning-based No Reference QoE Model for User Generated Contents
A Ensemble Learning-based No Reference QoE Model for User Generated ContentsA Ensemble Learning-based No Reference QoE Model for User Generated Contents
A Ensemble Learning-based No Reference QoE Model for User Generated Contents
 
BTP_ppt
BTP_pptBTP_ppt
BTP_ppt
 
Building a Custom Vision Model
Building a Custom Vision ModelBuilding a Custom Vision Model
Building a Custom Vision Model
 
Traffic Violation Detector using Object Detection
Traffic Violation Detector using Object DetectionTraffic Violation Detector using Object Detection
Traffic Violation Detector using Object Detection
 
A survey on Measurement of Objective Video Quality in Social Cloud using Mach...
A survey on Measurement of Objective Video Quality in Social Cloud using Mach...A survey on Measurement of Objective Video Quality in Social Cloud using Mach...
A survey on Measurement of Objective Video Quality in Social Cloud using Mach...
 
Computer Vision with Deep Learning
Computer Vision with Deep LearningComputer Vision with Deep Learning
Computer Vision with Deep Learning
 
Object detection presentation
Object detection presentationObject detection presentation
Object detection presentation
 
Deep Learning Projects - Anomaly Detection Using Deep Learning
Deep Learning Projects - Anomaly Detection Using Deep LearningDeep Learning Projects - Anomaly Detection Using Deep Learning
Deep Learning Projects - Anomaly Detection Using Deep Learning
 
“Applying the Right Deep Learning Model with the Right Data for Your Applicat...
“Applying the Right Deep Learning Model with the Right Data for Your Applicat...“Applying the Right Deep Learning Model with the Right Data for Your Applicat...
“Applying the Right Deep Learning Model with the Right Data for Your Applicat...
 
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
“Practical Image Data Augmentation Methods for Training Deep Learning Object ...
 
USING IMAGE CLASSIFICATION TO INCENTIVIZE RECYCLING
USING IMAGE CLASSIFICATION TO INCENTIVIZE RECYCLINGUSING IMAGE CLASSIFICATION TO INCENTIVIZE RECYCLING
USING IMAGE CLASSIFICATION TO INCENTIVIZE RECYCLING
 
lokesh_UX_Designer_v5
lokesh_UX_Designer_v5lokesh_UX_Designer_v5
lokesh_UX_Designer_v5
 
Computer architecture for vision system
Computer architecture for vision systemComputer architecture for vision system
Computer architecture for vision system
 
FYP-Final-External
FYP-Final-ExternalFYP-Final-External
FYP-Final-External
 
Kyligence Cloud 4 - Feature Focus: AI-Augmented Engine
Kyligence Cloud 4 - Feature Focus: AI-Augmented EngineKyligence Cloud 4 - Feature Focus: AI-Augmented Engine
Kyligence Cloud 4 - Feature Focus: AI-Augmented Engine
 
Deep learning with_computer_vision
Deep learning with_computer_visionDeep learning with_computer_vision
Deep learning with_computer_vision
 
Mini Project- Face Recognition
Mini Project- Face RecognitionMini Project- Face Recognition
Mini Project- Face Recognition
 

More from Edge AI and Vision Alliance

“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
Edge AI and Vision Alliance
 
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr..."Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
Edge AI and Vision Alliance
 
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
Edge AI and Vision Alliance
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
Edge AI and Vision Alliance
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
Edge AI and Vision Alliance
 
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
Edge AI and Vision Alliance
 
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
Edge AI and Vision Alliance
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a..."OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
Edge AI and Vision Alliance
 
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
Edge AI and Vision Alliance
 
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
Edge AI and Vision Alliance
 
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
Edge AI and Vision Alliance
 
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
Edge AI and Vision Alliance
 
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
Edge AI and Vision Alliance
 
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
Edge AI and Vision Alliance
 
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
Edge AI and Vision Alliance
 
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
Edge AI and Vision Alliance
 
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
Edge AI and Vision Alliance
 
“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...
Edge AI and Vision Alliance
 

More from Edge AI and Vision Alliance (20)

“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
“Squeezing the Last Milliwatt and Cubic Millimeter from Smart Cameras Using t...
 
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr..."Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
"Maximize Your AI Compatibility with Flexible Pre- and Post-processing," a Pr...
 
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
“Addressing Tomorrow’s Sensor Fusion and Processing Needs with Cadence’s Newe...
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
“Silicon Slip-ups: The Ten Most Common Errors Processor Suppliers Make (Numbe...
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
 
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
“How Arm’s Machine Learning Solution Enables Vision Transformers at the Edge,...
 
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
“Nx EVOS: A New Enterprise Operating System for Video and Visual AI,” a Prese...
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a..."OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
"OpenCV for High-performance, Low-power Vision Applications on Snapdragon," a...
 
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
“Deploying Large Models on the Edge: Success Stories and Challenges,” a Prese...
 
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
“Scaling Vision-based Edge AI Solutions: From Prototype to Global Deployment,...
 
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
“What’s Next in On-device Generative AI,” a Presentation from Qualcomm
 
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
“Learning Compact DNN Models for Embedded Vision,” a Presentation from the Un...
 
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
“Introduction to Computer Vision with CNNs,” a Presentation from Mohammad Hag...
 
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
“Selecting Tools for Developing, Monitoring and Maintaining ML Models,” a Pre...
 
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
“Building Accelerated GStreamer Applications for Video and Audio AI,” a Prese...
 
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
“Understanding, Selecting and Optimizing Object Detectors for Edge Applicatio...
 
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
“Introduction to Modern LiDAR for Machine Perception,” a Presentation from th...
 
“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...“Vision-language Representations for Robotics,” a Presentation from the Unive...
“Vision-language Representations for Robotics,” a Presentation from the Unive...
 

Recently uploaded

Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 

Recently uploaded (20)

Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Artificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic WarfareArtificial Intelligence and Electronic Warfare
Artificial Intelligence and Electronic Warfare
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 

“Tools and Strategies for Quickly Building Effective Image Datasets,” a Presentation from BDTI

  • 1. © 2021 EJ Technology Consultants Tools and Strategies for Quickly Building Effective Image Datasets Evan Juras Computer Vision Engineer
  • 2. © 2021 EJ Technology Consultants Purpose • Demonstrate practical strategies and tools for quickly building image datasets • Train higher-quality models with less time investment • Focus on object detection models 2 mAP % mAP %
  • 3. © 2021 EJ Technology Consultants Object Detection Neural Networks • Object detection models locate and identify objects in images • Use CNNs to find features in images and correlate them to known objects they’ve been trained on • For this presentation, “training” is transfer learning and fine-tuning 3 Visualizing and Comparing Convolutional Neural Networks arXiv: 1412.6631v2
  • 4. © 2021 EJ Technology Consultants Applying Object Detection Models • Scope of application determines amount of training images needed • Constrained applications with limited variety of visual conditions: • Lower generalization of model is okay • Fewer training images needed • Applications with wide variety of visual conditions: • Need to be accurate at different light levels, angles, coloration, distance from camera, etc. • Larger number of images are needed 4 Playing card detector – low variance Self-driving car – high variance
  • 5. © 2021 EJ Technology Consultants How can we Quickly Create an Object Detection Dataset? • Problem: to train a new object detection model, a large image dataset is needed • Manually gathering and labeling images is time consuming 5 • Solution: Use these strategies to accelerate dataset creation! • Using datasets already available online • Capturing images from video • Using sleek annotation and automated methods tools to quickly label data • Synthetic image generation and data augmentation My example – Bison distance detector
  • 6. © 2021 EJ Technology Consultants Using Online Image Datasets • Datasets available online can be a good starting point • Academic datasets such as Open Images Dataset, ImageNet, COCO, etc. • User-contributed datasets from TensorFlow, Kaggle, or other sources • Issues with online datasets • Can be filled with poor quality images or label data • Datasets not be available or appropriate for your application • Copyright and licensing: need to be careful using copyrighted material for commercial purposes 6
  • 7. © 2021 EJ Technology Consultants Capturing Images from Video • Gathering images from the camera used by the application significantly improves accuracy 7 • Various methods for capturing images specific to your application: • Set up cameras in situations similar to actual in-situ application, record video, and grab frames from video • Use multiple cameras in multiple locations and aggregate videos to a central server • Use online video or live camera streams of objects you are interested in
  • 8. © 2021 EJ Technology Consultants Capturing Images from Video, Continued Python script to extract individual frames from a video: github.com/EdjeElectronics/Image-Dataset-Tools#FrameGrabber 8
  • 9. © 2021 EJ Technology Consultants Image Annotation Tools • Many annotation tools are available to help accelerate the image labeling process • Paid annotation tools: • V7 Darwin • Supervisely • Hive – pay humans to annotate your data • Lionbridge – pay humans to annotate your data • Free annotation tools: • CVAT • LabelImg 9 Video annotation tool from V7
  • 10. © 2021 EJ Technology Consultants Automated Labeling Methods • Supervised automated labeling • Train a model off a small portion of the dataset • Use that model to automatically label the rest of the dataset, while manually rejecting poor labels and re-labeling them • github.com/EdjeElectronics/Image-Dataset- Tools#AutoLabeler • Other creative methods • Tryolabs example uses OpenPose to automatically locate positions of heads in a frame 10 Face mask detection in street camera video streams using AI: behind the curtain (Courtesy of Tryolabs) Automatically labeling raccoon images
  • 11. © 2021 EJ Technology Consultants Synthetic Image Generation and Data Augmentation 11 Synthetic Image Generation Data Augmentation • Tools are available to synthetically generate images • Mindtech • KineticVision Synthetic scenario video from Mindtech • Generate new images from existing ones • Increase visual variety of images • Resolve class imbalances • Great for 2D objects and perspectives Check out my data augmentation talk from EVS2020!
  • 12. © 2021 EJ Technology Consultants Limitations of These Methods • Still need some manual groundwork involved in collecting images • Searching for datasets, setting up video recording, auditing labels for accuracy • Other limitations • Difficult to use for uncommon objects or applications • Won’t bring model up to highest accuracy possible – for best performance, need to carefully curate and refine dataset, which takes time 12
  • 13. © 2021 EJ Technology Consultants Conclusions • There are various methods for quickly building an object detection dataset • Online datasets • Fast image capture • Sleek annotation tools and automated labeling • Synthetic image generation and data augmentation • Depending on application, may still need to manually curate dataset for best accuracy • Same concepts can be applied to image classification models 13
  • 14. © 2021 EJ Technology Consultants Example of Resource Slide 14 Resource Links Browser-based free annotation tool (CVAT): https://cvat.org Using TensorFlow’s built-in datasets: https://www.tensorflow.org/datasets/overview Handy scripts for working with image datasets: https://github.com/EdjeElectronics/Image-Dataset-Tools Contact Information Website: www.ejtech.io Email: evan.juras@ejtech.io References [1]: M. Zeiler, R. Fergus (2013). Visualizing and Understanding Convolutional Networks. arXiv:1311.2901 [2]: W. Yu, K. Yang, Y. Bai, H. Yao, Y. Rui (2014). Visualizing and Comparing Convolutional Neural Networks. arXiv:1412.6631v2