AI PLATFORM and AI SOLUTION AUTOMOTIVE, ADAS (Advanced driver-assistance systems) & INFOTAINMENT, AUTONOMOUS & SEMI AUTONOMOUS VEHICLES, SPORTS & ENTERTAINMENT, AR / VR, BALL & PLAYER TRACKING, COMMERCIAL INSERTION, CONSUMER, GAMING, CAMERAS, WEARABLES, SMART PHONES, ROBOTICS & MACHINE VISION, QUALITY ASSURANCE & INSPECTION, POSITIONING & GUIDANCE, HEALTHCARE, RADIOLOGY, MEDICAL ADHERENCE, SECURITY & SURVEILLANCE, BIOMETRICS, IMAGE & VIDEO ANALYTICS, AI in GUIDED DRONE is BASED SURVEILLANCE, AGRICULTURE,...

1. Using Deep Learning for Computer Vision
Applications
Farshid PirahanSiah, PhD
August 2018

2. Computer Vision with Deep Learning market
AI PLATFORM and AI SOLUTION
AUTOMOTIVE, ADAS (Advanced driver-assistance systems) &
INFOTAINMENT, AUTONOMOUS & SEMI AUTONOMOUS VEHICLES,
SPORTS & ENTERTAINMENT, AR / VR, BALL & PLAYER TRACKING,
COMMERCIAL INSERTION, CONSUMER, GAMING, CAMERAS,
WEARABLES, SMART PHONES, ROBOTICS & MACHINE VISION,
QUALITY ASSURANCE & INSPECTION, POSITIONING & GUIDANCE,
HEALTHCARE, RADIOLOGY, MEDICAL ADHERENCE, SECURITY &
SURVEILLANCE, BIOMETRICS, IMAGE & VIDEO ANALYTICS, AI in
GUIDED DRONE is BASED SURVEILLANCE, AGRICULTURE,...

3. Computer Vision with Deep Learning market
“ The AI ​​market for computer vision is expected to expand from US $ 3.62 billion
in 2018 to US $ 25.32 billion by 2023. Market is expected to grow by 47.54% at
CAGR (Composite Annual Growth Rate) from 2018 to 2023.”
Industry Sector
● Automobile
● Sports, Entertainment
● Consumer
● Robot, machine vision
● Health care
● Security, monitoring
● Agriculture
Main Innovator
● LEAP MOTION
● ALGOLUX
● CLARIFAI
● DITTO LABS
● MAD STREET DEN

4. Computer Vision with Deep Learning market
Deep Learning Market Analysis By Application (Image Recognition, Voice
Recognition, Video Surveillance) 2018 - 2025
“ The global deep learning market size was valued at USD 272.0 million in 2016.
Increasing applicability in the autonomous vehicles and healthcare industries is
expected to contribute to the industry growth significantly. This technology is
gaining prominence on account of its complex data driven applications including
voice and image recognition. It offers a huge investment opportunity as it can be
leveraged over other technologies to overcome the challenges of high data
volumes, high computing power, and improvement in data storage.”

5. Computer Vision with Deep Learning market
Deep Learning Market Analysis By Application (Image Recognition, Voice
Recognition, Video Surveillance) 2018 - 2025
Industry Sector
● Automobile
● Sports, Entertainment
● Consumer
● Robot, machine vision
● Health care
● Security, monitoring
● Agriculture
Main Innovator
● LEAP MOTION
● ALGOLUX
● CLARIFAI
● DITTO LABS
● MAD STREET DEN

6. What's new in OpenCV 4 (August 2018)
C++11
Deep Learning in OpenCV 4:
Caffe
TensorFlow
Torch
Darknet
Some updates:
Hardware-accelerated Video I/O module
HighGUI module
Graph API module
Point Cloud module
Tracking, Calibration, and Stereo Modules

7. Compile OpenCV 4 - part 1
Download (Cmake & Visual studio 2017 & Windows 10 (64 bit))
https://github.com/opencv/opencv
https://github.com/opencv/opencv_contrib
Create build folder
Using Cmake to generate visual studio project
Use the path of … opencv_contrib-mastermodules
Open “OpenCV.sln” project

8. Compile OpenCV 4 - part 2
Build
Debug
Release
Install
Set the environment variable
Add “C:opencv4buildinstallx64vc15bin” to windows path

9. Setup and configure your project (debug mode (64 bit))
Setup visual studio project to use OpenCV 4 (you only need install folder inside the build folder of
compiled OpenCV 4)
For the debug mode (64 bit)
Add the include folder of OpenCV 4 to include path of your visual studio project
C:opencv4buildinstallinclude
Add the lib folder of OpenCV 4 to lib path of your visual studio project
C:opencv4buildinstallx64vc15lib
Put list of lib files for the debug mode (64 bit) (linker->input->additional dependencies)
opencv_aruco400d.lib;opencv_bgsegm400d.lib;opencv_bioinspired400d.lib;opencv_calib3d400d.lib;opencv_ccalib400d.lib;o
pencv_core400d.lib;opencv_datasets400d.lib;opencv_dnn400d.lib;opencv_dnn_objdetect400d.lib;opencv_dpm400d.lib;ope
ncv_face400d.lib;opencv_features2d400d.lib;opencv_flann400d.lib;opencv_fuzzy400d.lib;opencv_hfs400d.lib;opencv_highg
ui400d.lib;opencv_imgcodecs400d.lib;opencv_imgproc400d.lib;opencv_img_hash400d.lib;opencv_line_descriptor400d.lib;o
pencv_ml400d.lib;opencv_objdetect400d.lib;opencv_optflow400d.lib;opencv_phase_unwrapping400d.lib;opencv_photo400d
.lib;opencv_plot400d.lib;opencv_reg400d.lib;opencv_rgbd400d.lib;opencv_saliency400d.lib;opencv_shape400d.lib;opencv_
stereo400d.lib;opencv_stitching400d.lib;opencv_structured_light400d.lib;opencv_superres400d.lib;opencv_surface_matchin
g400d.lib;opencv_text400d.lib;opencv_tracking400d.lib;opencv_video400d.lib;opencv_videoio400d.lib;opencv_videostab400
d.lib;opencv_xfeatures2d400d.lib;opencv_ximgproc400d.lib;opencv_xobjdetect400d.lib;opencv_xphoto400d.lib

10. Setup and configure your project (release mode (64 bit))
Setup visual studio project to use OpenCV 4 (you only need install folder inside the build folder of
compiled OpenCV 4)
For the released mode (64 bit)
Add the include folder of OpenCV 4 to include path of your visual studio project
C:opencv4buildinstallinclude
Add the lib folder of OpenCV 4 to lib path of your visual studio project
C:opencv4buildinstallx64vc15lib
Put list of lib files for the released mode (64 bit) (linker->input->additional dependencies)
opencv_aruco400.lib;opencv_bgsegm400.lib;opencv_bioinspired400.lib;opencv_calib3d400.lib;opencv_ccalib400.lib;opencv
_core400.lib;opencv_datasets400.lib;opencv_dnn400.lib;opencv_dnn_objdetect400.lib;opencv_dpm400.lib;opencv_face400
.lib;opencv_features2d400.lib;opencv_flann400.lib;opencv_fuzzy400.lib;opencv_hfs400.lib;opencv_highgui400.lib;opencv_i
mgcodecs400.lib;opencv_imgproc400.lib;opencv_img_hash400.lib;opencv_line_descriptor400.lib;opencv_ml400.lib;opencv
_objdetect400.lib;opencv_optflow400.lib;opencv_phase_unwrapping400.lib;opencv_photo400.lib;opencv_plot400.lib;opencv
_reg400.lib;opencv_rgbd400.lib;opencv_saliency400.lib;opencv_shape400.lib;opencv_stereo400.lib;opencv_stitching400.lib
;opencv_structured_light400.lib;opencv_superres400.lib;opencv_surface_matching400.lib;opencv_text400.lib;opencv_tracki
ng400.lib;opencv_video400.lib;opencv_videoio400.lib;opencv_videostab400.lib;opencv_xfeatures2d400.lib;opencv_ximgpro
c400.lib;opencv_xobjdetect400.lib;opencv_xphoto400.lib

11. Simple program to test
#include "stdafx.h"
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
int main()
{
cv::String filePath = "C:/opencv4/OpenCV4VS2017/OpenCV4VS2017/opencv-logo.png";
cv::Mat src = cv::imread(filePath);
cv::imshow("test", src);
cv::waitKey(10000);
return 0;
}

12. Using Caffe model in OpenCV 4 - part 1
Download
1. Synset_words.txt
https://raw.githubusercontent.com/HoldenCaulfieldRye/caffe/master/
data/ilsvrc12/synset_words.txt
1. Bvlc_googlenet.caffemodel
http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel
1. Bvlc_googlenet.prototxt
https://raw.githubusercontent.com/opencv/opencv_extra/master/test
data/dnn/bvlc_googlenet.prototxt

13. Using caffe model in OpenCV 4 - part 2
static void getMaxClass(const cv::Mat &probBlob, int *classId, double *classProb)
{
cv::Mat probMat = probBlob.reshape(1, 1); //reshape the blob to 1x1000 matrix
cv::Point classNumber;
cv::minMaxLoc(probMat, NULL, classProb, NULL, &classNumber);
*classId = classNumber.x;
}
static std::vector<cv::String> readClassNames(const char *filename = "C:/opencv4/OpenCV4VS2017/DeepLearningFiles/synset_words.txt")
{
std::vector<cv::String> classNames;
std::ifstream fp(filename);
if (!fp.is_open())
{
std::cerr << "File with classes labels not found: " << filename << std::endl;
exit(-1);
}
std::string name;
while (!fp.eof())
{
std::getline(fp, name);
if (name.length())
classNames.push_back(name.substr(name.find(' ') + 1));
}
fp.close();
return classNames;
}

14. Using caffe model in OpenCV 4 - part 3
● float scale = 1.0;
● cv::Scalar meanG = cv::Scalar(104, 117, 123);
● bool swapRB = true;
● int inpWidth = 224;
● int inpHeight = 224;
● cv::String model = path + "bvlc_googlenet.caffemodel";
● cv::String config = path + "bvlc_googlenet.prototxt";
● cv::String nameClass = path + "synset_words.txt";
● cv::String nameImage = path + "space_shuttle.jpg";
● cv::String framework;
● int backendId = 0;
● int targetId = 0;
● cv::dnn::Net net;
● cv::Mat blob;
● cv::Mat img = imread(nameImage);
● net = cv::dnn::readNet(model, config);
● net.setPreferableBackend(backendId);
● net.setPreferableTarget(targetId);
● cv::dnn::blobFromImage(img, blob, scale, cv::Size(inpWidth,
inpHeight), meanG, swapRB, false);
● net.setInput(blob);
● cv::Mat prob = net.forward();
●
● cv::Point classIdPoint;
● double confidence;
● minMaxLoc(prob.reshape(1, 1), 0, &confidence, 0,
&classIdPoint);
● int classId = classIdPoint.x;
● std::vector<double> layersTimes;
● double freq = cv::getTickFrequency() / 1000;
● double t = net.getPerfProfile(layersTimes) / freq;
● std::string label = cv::format("Inference time: %.2f ms", t);
● putText(img, label, cv::Point(0, 15),
cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 255, 0));
● cv::imshow("https://www.youtube.com/tiziran", img);
● std::vector<cv::String> classNames = readClassNames();
● std::cout << "Best class: #" << classId << " '" <<
classNames.at(classId) << "'" << std::endl;
● std::cout << "Probability: " << confidence * 100 << "%" <<
std::endl;
● cv::waitKey(10000);

15. Using TensorFlow model in OpenCV 4 - part 1
TensorFlow API only for OpenCV 4
Download the inception model based on TensorFlow
● https://storage.googleapis.com/download.tensorflow.org/mo
dels/inception5h.zip

16. Using TensorFlow model in OpenCV 4 - part 2
● float scale = 1.0;
● cv::Scalar meanG = cv::Scalar(104, 117, 123);
● bool swapRB = true;
● int inpWidth = 224;
● int inpHeight = 224;
● cv::String model = path + "tensorflow_inception_graph.pb";
● cv::String nameImage = path + "space_shuttle.jpg";
● cv::String framework;
● int backendId = 0;
● int targetId = 0;
● cv::dnn::Net net;
● cv::Mat blob;
● cv::Mat img = imread(nameImage);
● net = cv::dnn::readNetFromTensorflow(model);
● net.setPreferableBackend(backendId);
● net.setPreferableTarget(targetId);
● cv::dnn::blobFromImage(img, blob, scale, cv::Size(inpWidth,
inpHeight), meanG, swapRB, false);
● net.setInput(blob);
● cv::Mat prob = net.forward();
● cv::Point classIdPoint;
● double confidence;
● minMaxLoc(prob.reshape(1, 1), 0, &confidence, 0,
&classIdPoint);
● int classId = classIdPoint.x;
● std::vector<double> layersTimes;
● double freq = cv::getTickFrequency() / 1000;
● double t = net.getPerfProfile(layersTimes) / freq;
● std::string label = cv::format("Inference time: %.2f ms", t);
● putText(img, label, cv::Point(0, 15),
cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 255, 0));
● cv::imshow("https://www.youtube.com/tiziran", img);
● std::vector<cv::String> classNames =
readClassNamesTensorFlow();
● std::cout << "Best class: #" << classId << " '" <<
classNames.at(classId) << "'" << std::endl;
● std::cout << "Probability: " << confidence * 100 << "%" <<
std::endl;
● cv::waitKey(10000);

17. Previous Videos, Courses, Classes, Training, Tutorials
❖ Compile full OpenCV 4 (main and
contribute) from source
➢ https://www.youtube.com/watc
h?v=VK70YdaMD44
❖ Setup and configure Visual Studio
2017 project (C++) for using
OpenCV 4 library
➢ https://www.youtube.com/watc
h?v=J0Exttz4_m4
❖ Using Caffe model in OpenCV 4
➢ https://www.youtube.com/watc
h?v=lj7AxKDoP9I
❖ Using TensorFlow model in
OpenCV 4
➢ https://www.youtube.com/watc
h?v=5JqX0CbxtNk

18. Linkedin groups
Computer Vision, Deep Learning, OpenCV 3, C++,
DIGITS, Caffe, TensorFlow, Theano, Torch
https://www.linkedin.com/groups/10320678

19. Telegram
Deep Learning in Computer Vision
(Channel) https://t.me/Dl4cv (Group) https://t.me/joinchat/CZ8vFgt1IdhUXkG615lsWA

20. WhatsApp Group
WhatsApp Group
Practical Deep Learning
https://chat.whatsapp.com/LE5FYddBYl68iAaTZnTuYq

21. Facebook group
Deep Learning, Computer Vision, Robotics
https://www.facebook.com/groups/185926728115336/

22. Slide share
Slide share
https://www.slideshare.net/pirahansiah

23. YouTube channel
https://www.youtube.com/tiziran

24. GitHub
Download source code from github
https://github.com/pirahansiah/opencv4

25. Outline
1 Compile full OpenCV 4 (main and contribute) from
source
https://www.youtube.com/watch?v=VK70YdaMD44
3 Using Caffe model in OpenCV 4
https://www.youtube.com/watch?v=lj7AxKDoP9I
2 Setup and configure Visual Studio 2017 project (C++)
for using OpenCV 4 library
https://www.youtube.com/watch?v=J0Exttz4_m4
4 Using TensorFlow model in OpenCV 4
https://www.youtube.com/watch?v=5JqX0CbxtNk

26. Reference - I
● Effect of Augmented Datasets to Train
Convolutional Neural Networks
● Pattern image significance for camera
calibration
● https://doi.org/10.1109/SCORED.2017.830544
0
● using an ant colony optimization algorithm for
image edge detection as a threshold
segmentation for ocr system
● http://www.jatit.org/volumes/Vol95No21/1Vol9
5No21.pdf
● GSFT-PSNR: Global Single Fuzzy Threshold
Based on PSNR for OCR Systems
● http://oldver.ijcsns.com/June.2016-Volume.4-
No.6/Article01.pdf
● Camera calibration for multi-modal robot
vision based on image quality assessment
● https://doi.org/10.1109/ASCC.2015.7360336
● Adaptive Image Thresholding Based on the
Peak Signal-to-noise Ratio
● http://www.academia.edu/download/44161592
/Adaptive_Image_Thresholding_Based_on_th
e20160328-31366-1wyb1jc.pdf
● peak signal-to-noise ratio based on threshold
method for image segmentation
● www.jatit.org/volumes/Vol57No2/4Vol57No2.p
df
● Simultaneous Localization And Mapping
Trends And Humanoid Robot Linkages
● http://ejournal.ukm.my/apjitm/article/download/
4164/2308

27. Reference - II
● character and object recognition based on
global feature extraction
● https://pdfs.semanticscholar.org/2d22/66bfb2f
5ba5cb40b5946b82c620dc713d793.pdf
● License plate recognition with multi-threshold
based on entropy
● https://doi.org/10.1109/ICEEI.2011.6021627
● Character recognition based on global feature
extraction
● https://doi.org/10.1109/ICEEI.2011.6021649
● Comparison single thresholding method for
handwritten images segmentation
● https://doi.org/10.1109/ICPAIR.2011.5976918
● TafreshGrid: Grid computing in Tafresh
university
● https://doi.org/10.1109/ICCSN.2011.6014009
● Multi-threshold approach for license plate
recognition system
● http://waset.org/publications/3636
● Adaptive image segmentation based on peak
signal-to-noise ratio for a license plate
recognition system
● https://doi.org/10.1109/ICCAIE.2010.5735125
● An evaluation of classification techniques
using enhanced Geometrical Topological
Feature Analysis
● https://scholar.google.com/scholar?oi=bibs&cl
uster=3038184255311332521&btnI=1&hl=en
● Augmented Optical Flow Methods for Video
● http://www.academia.edu/download/45596617
/farshid.pdf