When AOI meets AI

1. Department of Bio-Industrial Mechatronics Engineering
When AOI Meets AI
Ta-Te Lin
Department of Bio-Industrial Mechatronics Engineering,
National Taiwan University
National Taiwan University

2. Brief Introduction to AI
Source: https://www.youtube.com/watch?v=2E4t75uF6JI Source: https://vimeo.com/192179727
Source: https://www.youtube.com/watch?v=rKHFPsA8JjM Source:
https://www.youtube.com/watch?v=rVlhMGQgDkY&start_radi
o=1&list=RDQMaBMndpuioWw
2

3. Brief Introduction to AI
Development of AI
3
Artificial Intelligence
Machine Learning
Deep Learning
1980s1950s 1960s 1970s 1990s 2000s 2010s 2020s

4. Deep Learning
Neural Network - Emulating Human Brain
Source: https://www.edureka.co/blog/what-is-deep-learning
4
Brief Introduction to AI

5. Development of Deep Learning Models
5
Brief Introduction to AI
Source: http://condor.depaul.edu/ntomuro/courses/578/notes/1-IntroNNs.pdf
 ImageNet: about 15M labelled
high resolution images, 22K
categories
 Large Scale Visual Recognition
Challenge (ILSVRC)
 AlexNet: the first deep neural
networks trained on GPUs
The Breakthrough (2012)
https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks

6. Why AI Is Booming
6
AI
DATA/IMAGES
COMPUTING
POWER
OPEN SOURCEINTERNET
ALGORITHMS
Key Factors Driving the
Artificial Intelligence Boom

7. CPU & GPU Computation Power
Source: https://blog.inten.to/hardware-for-deep-learning-part-3-gpu-8906c1644664
7
Why AI Is Booming

8. GPU Parallel Processing
CPU
MULTIPLE CORES
GPU
THOUSANDS OF CORES
GPUs are the current workhorses of Deep Learning
8
Why AI Is Booming

9. Development of Deep Learning Models
Source: Canzian et al., 2017; https://arxiv.org/abs/1605.07678
9
Why AI Is Booming

10. Development of Deep Learning Models
Source: Bianco et al., 2018; https://arxiv.org/pdf/1810.00736.pdf 10
Why AI Is Booming

11. 11
When AOI Meets AI

12. 12
When AOI Meets AI
Industries
• Electric
components and
equipment
• Manufacturing
• Semiconductors
• Machinery parts
• Material production
• Packaging
• Printing
• Agriculture and
food
• Health care and life
science
• Logistics
• Monitoring and
surveillance
• etc.
Applications
• Gauging and
measurement
• 3D measurement
• Bar code and data
code reading
• OCR
• Object detection
• Object recognition
• Print inspection
• Surface inspection
• Defect detection
• Completeness
check
• Robotic guidance
• etc.
Machine Vision
Algorithms
• Basic processing
• 1D & 2D
measurement
• Color analysis
• Segmentation
• Matching
• Shape finding
• Pattern recognition
• Feature extraction
and analysis
• OCR
• Registration
• Calibration
• Blob analysis
• Morphology
• etc.
Strength of AI
• Complex
background
• Size and shape
variation
• Distortion
• Classification
• Object
detection
• Feature
extraction
• etc.

13. 13
When AOI Meets AI
Source: Yu et al. (2017). Fully Convolutional Networks for Surface Defect Inspection in
Industrial Environment. Lecture Notes in Computer Science, vol 10528. Springer, Cham
An Example of Surface Defect Inspection Using Deep Learning

14. 14
When AOI Meets AI
Source: Leta et al. (2008).
PCB Inspection Using Conventional and Deep Learning Methods
Source: Huang and Wei (2018).

15. 15
When AOI Meets AI
The problem of fruit or weed detection
Bulanon et al. (2002)
Payne et al. (2013)
Payne et al. (2012)
Bakhshipour et al. (2017)
Deep
Learning

16. Smart Farm Machinery Applying Deep Learning
16
When AOI Meets AI
Source: http://agrobot.com/

17. 17
When AOI Meets AI
Source: http://smartmachines.bluerivertechnology.com/
Smart Farm Machinery Applying Deep Learning

18. Greenhouse Pest Inset Monitoring System
18
When AOI Meets AI
Source: http://agrobot.com/
Light intensity sensor
Temperature/humidity
sensor
Sticky paper
trap
Lighting panel Camera

19. Greenhouse Pest Inset Monitoring System
19
When AOI Meets AI
Pest detection and recognition using deep learning

20. Convolutional layer
Filter size: 3*3
No. of filters: 32
Convolutional layer
Filter size: 3*3
No. of filters: 32
Convolutional layer
Filter size: 3*3
No. of filters: 64
Flattened
layer
Fully connected
layer
Neurons: 128
Softmax layer
class 1
class 2
class n
Feature extraction Learning layer
Prediction layer
Feature compression
Deep Learning Approach for Image Classification (Deep Classification)
Traditional Approach for Image Classification (Shallow Classification)
Image
Preprocessing
Feature
Extraction
Generic
Classifiers
class 1
class 2
class n
20
When AOI Meets AI

21. Convolutional layer
Filter size: 3*3
No. of filters: 32
Convolutional layer
Filter size: 3*3
No. of filters: 32
Convolutional layer
Filter size: 3*3
No. of filters: 64
Flattened
layer
Fully connected
layer
Neurons: 128
Softmax layer
class 1
class 2
class n
Feature extraction Learning layer
Prediction layer
Feature compression
Deep Learning Approach for Image Classification (Deep Classification)
Traditional Approach for Image Classification (Shallow Classification)
Image
Preprocessing
Feature
Extraction
Generic
Classifiers
class 1
class 2
class n
21
When AOI Meets AI

22. When AOI Meets AI
Selvaraju et al. (2017)
Visual Explanations from Deep Learning Networks

23. MachineVision + Deep Learning
23
When AOI Meets AI
Proprietary Open Source
License Fee
Technical Support
Stability
Rapid Development
No Royalty Fees
User Community and Forum
Complexity vs. Flexibility
Programming Language

24. Deep Learning Software
24
When AOI Meets AI
Software Initial
Release
Open
Source
Language CUDA
Support
Actively
Developed
Dlib 2002 Yes C++ Yes Yes
Theano 2007 Yes Python Yes No
Caffe 2013 No Python, C++ Yes No
TensorFlow 2015 Yes C++, Python Yes Yes
Chainer 2015 Yes Python Yes Yes
Keras 2015 Yes Python, R Yes Yes
Apache MXNet 2015 Yes C++, Python, Matlab, R, etc. Yes Yes
Microsoft Cognitive
Toolkit (CNTK)
2016 Yes Python, C++, C#/.NET Yes Yes
PyTorch 2016 Yes Python Yes Yes
Matlab Yes Matlab Yes Yes
Wolfram Mathematica Yes Wolfram Language Yes Yes

25. Deep Learning Software
25
When AOI Meets AI
100
75
50
25
0
Apr. 2014 Jun. 2016 Nov. 2017 May 2019Average
Source: Google Trend, Worldwide, 4/9/14-5/9/19, Machine Learning & Artificial Intelligence

26. Deep Learning Software
26
When AOI Meets AI
Source: https://towardsdatascience.com/deep-learning-framework-power-scores-2018-23607ddf297a

27. Synergy of Advanced Technologies
27
Future Trends in AIAOI

28. Integration of AI with AOI Applications
Artificial
Intelligence
Machine
Learning
Deep
Learning
Data/Images
Embedded
System
CPU
GPU
FPGA
Cloud
Computing
Edge
Computing
Computation
Hardware
AOI Applications
Algorithm
and Software
28
Future Trends in AIAOI

29. Synergy of Advanced Technologies
29
Future Trends in AIAOI
AOI
AI
Robotics
Sensors
IoT
Data
Analytics
Cloud
Computing
3D Imaging
Spectral Imaging
Embedded and Mobile System

30. 30
Future Trends in AIAOI
CLOUD SERVICE
USERS
APP
Service Oriented AIAOI
Data Analytics
Cloud Computing
Data Storage
Data Collaboration
Production Optimization

31. Rises of AOI Startup
https://kitov.ai/
http://www.sualab.com/
https://www.uveye.com/
https://www.birds.ai/
https://www.aquifi.com/
31
Future Trends in AIAOI

32. AI Startups in Israel
Source: https://www.startuphub.ai/ 32
Future Trends in AIAOI

33. AI Startups in Israel
Source: https://www.startuphub.ai/ 33
Future Trends in AIAOI
 Over 950 active startups utilizing
or developing AI technologies
 51% of AI startups are utilizing
machine learning technologies, of
which 21% are utilizing deep
learning technologies
 28% of AI startups are still
building their algorithms while
searching for data partners
 84% of AI startups offer a purely
software-based solution, while
16% offer a mixed offering of
hardware and software
Nov. 2018
Nov. 2018

34. Machine Vision Startups in Israel
Source: https://www.startuphub.ai/ 34
Future Trends in AIAOI
 Over 245 active startups utilizing
or developing computer vision
technologies
 71% of AI startups offer software-
based solutions, while 29% offer a
mixed offering of hardware and
software
 The typical startup takes 7.6 years
to exit following their
establishment
Nov. 2018
0 10 20 30 40 50 60 70
Number of Startups
Computer Vision Technology
Healthcare
Automotive
Agriculture
The Most Concentrated Sub-sector of the Computer Vision Startups

35. 35
Future Trends in AIAOI
1960s 2019
20191983
COMPUTATION + COMMUNICATION

36. 36
Future Trends in AIAOI
“The more people who use an AI, the smarter it gets.
The smarter it gets, the more people who use it. The
more people who use it, the smarter it gets. And so
on. Once a company enters this virtuous cycle, it
tends to grow so big so fast that it overwhelms any
upstart competitors.”
- Kevin Kelly

37. 37
When AOI Meets AI

38. 38
AIAOI
会い青い

39. Thanks for Listening!
39