Implementing Vision DL Solution using Open Source Tools
•
0 likes•129 views
This document outlines the steps to implement a vision-based deep learning solution using open source tools, including data collection, annotation, training and inference. It discusses collecting data through web crawling, live video recording, and image capture. Data is then preprocessed, labeled, and annotated using open source tools. A YOLO object detection model is trained on labeled data using the DarkFlow framework. The trained model is then deployed for inference on edge devices using Intel OpenVINO. Challenges discussed include the need for large amounts of varied data, iterative tuning, and automation of annotation.
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Implementing Vision DL Solution using Open Source Tools
Similar to Implementing Vision DL Solution using Open Source Tools (20)
More from Women in Analytics Conference
More from Women in Analytics Conference (10)
Recently uploaded
Recently uploaded (20)
Implementing Vision DL Solution using Open Source Tools
- 1. Unearth the Journey of Implementing Vision based Deep Learning Solution using Open Source Tools Neethu Elizabeth Simon Software Engineer Intel Corporation Phoenix, AZ
- 7. Video – Eye of IoT
- 9. Dried Chili Peppers Cinnamon Sticks Solution Data Set
- 10. Implementation • Build Model • Improve Models to meet performance & accuracy objectives • Deploy Model • Make Predictions by applying the Trained Model on new Data • Label Data• Data Acquisition & Organization • Preprocessing Collect Annotation TrainingInference
- 11. Implementation – Data Collection Collect Annotation TrainingInference Data Acquisition & Organization • Web Crawling/Scraping - Download pictures & videos Python Packages Request, Parcel, Beautiful Soup Frameworks/Tools Scrapy, Pyspider, Mechanical Soup
- 12. Implementation – Data Collection Collect Annotation TrainingInference Data Acquisition & Organization • Web Crawling/Scraping - Download pictures & videos • Live Video Recording/Image Capture from actual retail store Python Packages Request, Parcel, Beautiful Soup Frameworks/Tools Scrapy, Pyspider, Mechanical Soup
- 13. Implementation – Data Collection Collect Annotation TrainingInference Pre-processing • Data Cleanup • Remove Bad/Unrelated Data
- 14. Implementation – Data Annotation Collect Annotation TrainingInference Label Data • Establish ground truth
- 15. Implementation – Data Annotation Collect Annotation TrainingInference Open Source Tools • Bbox - https://github.com/puzzledqs/BBox-Label-Tool • LabelImg - https://github.com/tzutalin/labelImg • VGG Image Annotator (VIA) http://www.robots.ox.ac.uk/~vgg/software/via/ • Computer Vision Annotation Tool (CVAT) https://github.com/opencv/cvat Paid Tools – Labelbox, Dataloop, RectLabel
- 17. Implementation – Training Collect Annotation TrainingInference dried_chilli_pepper cinnamon_stick Lots of Labeled Data !!! Model Weights Forward Backward
- 18. Implementation – Training Collect Annotation TrainingInference Data • Type/Amount – Visual, Sensors • Labeled Datasets – PASCAL VOC, COCO, KITTI, ImageNet or ILSVRC, SUN • Custom Data
- 19. Implementation – Training Collect Annotation TrainingInference Compute Platform • Complexity Determined by Data • Cloud vs Edge • Intel® Xeon vs Core vs Atom Data • Type/Amount – Visual, Sensors • Labeled Datasets – PASCAL VOC, COCO, KITTI, ImageNet or ILSVRC, SUN • Custom Data Intel Core + GPU
- 20. Implementation – Training Collect Annotation TrainingInference Object Detection Algorithm • YOLO – You Only Look Once • SSD, R-CNN, Fast R-CNN, Faster RCNN, R-FCN Compute Platform • Complexity Determined by Data • Cloud vs Edge • Intel® Xeon vs Core vs Atom Data • Type/Amount – Visual, Sensors • Labeled Datasets – PASCAL VOC, COCO, KITTI, ImageNet or ILSVRC, SUN • Custom Data Intel Core + GPU YOLO
- 21. Implementation – Training Collect Annotation TrainingInference Deep Learning Framework • DarkNet - C & CUDA - https://pjreddie.com/darknet/ • DarkFlow - Tensorflow implementation of DarkNet https://github.com/thtrieu/darkflow • Others – PyTorch, Caffe Object Detection Algorithm • YOLO – You Only Look Once • SSD, R-CNN, Fast R-CNN, Faster RCNN, R-FCN Compute Platform • Complexity Determined by Data • Cloud vs Edge • Intel® Xeon vs Core vs Atom Data • Type/Amount – Visual, Sensors • Labeled Datasets – PASCAL VOC, COCO, KITTI, ImageNet or ILSVRC, SUN • Custom Data YOLO Intel Core + GPU DarkFlow
- 22. Implementation – Training Collect Annotation TrainingInference Deep Learning Framework • DarkNet - C & CUDA - https://pjreddie.com/darknet/ • DarkFlow - Tensorflow implementation of DarkNet https://github.com/thtrieu/darkflow • Others – PyTorch, Caffe Object Detection Algorithm • YOLO – You Only Look Once • SSD, R-CNN, Fast R-CNN, Faster RCNN, R-FCN Compute Platform • Complexity Determined by Data • Cloud vs Edge • Intel® Xeon vs Core vs Atom Data • Type/Amount – Visual, Sensors • Labeled Datasets – PASCAL VOC, COCO, KITTI, ImageNet or ILSVRC, SUN • Custom Data YOLO Intel Core + GPU DarkFlow Images - 1000 Images Training Time - 10 hrs Model Size – 800 MB Average loss – ~0.9
- 23. Implementation – Training Collect Annotation TrainingInferencedried_chilli_pepper cinnamon_stick Lots of Labeled Data !!! Model Weights Forward Backward python flow --train --model cfg/yolov2-2c.cfg --load bin/yolov2.weights --annotation all_Spice_Annotations/ --dataset all_Spice_Images/ DarkFlow
- 24. Implementation – Inference Collect Annotation TrainingInference Model Weights Forward dried_chilli_pepper Edge Device – Laptop/NUC/Mobile ???? Intel® OpenVINO Toolkit https://software.intel.com/en- us/openvino-toolkit Open Source Tool for Inference Support Different Models (Caffe/Tensorflow/MXNet/ONNX/Kaldi) Model Optimization Speed Deploymentpython flow --imgdir all_test_images/ --model cfg/yolov2-2c.cfg --load new_model_weights DarkFlow
- 25. Results [{"label": "cinnamon_stick", "confidence": 0.87, "topleft": {"x": 42, "y": 22}, "bottomright": {"x": 208, "y": 113}}] [{"label": "cinnamon_stick", "confidence": 0.06, "topleft": {"x": 93, "y": 216}, "bottomright": {"x": 384, "y": 402}}]
- 26. Results [{"label": "dried_chilli_pepper", "confidence": 0.95, "topleft": {"x": 0, "y": 25}, "bottomright": {"x": 761, "y": 352}}] [{"label": "dried_chilli_pepper", "confidence": 0.02, "topleft": {"x": 357, "y": 93}, "bottomright": {"x": 513, "y": 446}}, {"label": "cinnamon_stick", "confidence": 0.03, "topleft": {"x": 21, "y": 6}, "bottomright": {"x": 568, "y": 472}}]
- 28. Implementation Challenges • More is Better • Object Orientation, angle, lighting • Remove Bad/Unrelated Data • Require SME knowledge to define bad data & remove unrelated data Collect Annotation TrainingInference
- 29. [{"label": "cinnamon_stick", "confidence": 0.87, "topleft": {"x": 42, "y": 22}, "bottomright": {"x": 208, "y": 113}}]
- 30. Implementation Challenges • Manual vs Automation • Time consuming • Expensive • Difficult to Scale • More is Better • Object Orientation, angle, lighting • Remove Bad/Unrelated Data • Require SME knowledge to define bad data & remove unrelated data Collect Annotation TrainingInference
- 31. Implementation Challenges • Iterative Learning • Varied Data parameters & tuning • Hyperparamter optimization & data overfitting • High Compute • Manual vs Automation • Time consuming • Expensive • Difficult to Scale • More is Better • Object Orientation, angle, lighting • Remove Bad/Unrelated Data • Require SME knowledge to define bad data & remove unrelated data Collect Annotation TrainingInference
- 32. Implementation Challenges • Iterative Learning • Varied Data parameters & tuning • Hyperparamter optimization & data overfitting • High Compute • Constantly Update Models • Complexity in Hardware design – combination of CPU/GPU/Accelerators • Data Privacy & Security • Manual vs Automation • Time consuming • Expensive • Difficult to Scale • More is Better • Object Orientation, angle, lighting • Remove Bad/Unrelated Data • Require SME knowledge to define bad data & remove unrelated data Collect Annotation TrainingInference
- 33. Neethu Elizabeth Simon Software Engineer Intel Corporation Phoenix, AZ