Foundations of AI Lecture - Computer Vision

1. Dr V. Camilleri - vanessa.camilleri@um.edu.mt December 2022
Foundations of Ai
Principles of Computer Vision

2. Hi …
I’m Vanessa Camilleri
I’m a lecturer at the Department of AI, Faculty of ICT
I can be contacted via vanessa.camilleri@um.edu.mt
My main interests are in the
fi
elds of Creative
Computing & Education, which include VR, AR & MR,
Games & Game AI, and ML for Education.

4. Quite a vast topic!
Computer
Vision …
So to help our understanding we will be
focusing on 1 scenario…
a self-driving ambulance

5. • How can we capture the world around us?
• How can we make it easier for the machines to capture this
visual data?
• How can the machines make sense out of this data? How will
they learn?
What is Computer Vision?

7. Computer Vision helps
machines make sense out
of received visual data.

8. • What is the best camera for
computer vision use?
• Depends on the use & context
• Some general characteristics
may include:
• low-latency,
• adequate low-light and light
transition performance,
• IO connections, and
• weatherproo
fi
ng
Cameras
How do machines capture visual data?

9. Activity time
Work in groups (use chat/social media/…)
Focus on Practical Applications of Computer
Vision and discuss current developments in
one of these research
fi
elds: facial recognition,
self-driving cars, AR & MR, healthcare, etc.

10. • Acquisition
• Processing
• Segmentation
• Feature Extraction
• Classi
fi
cation
• Result Aggregation
Stages of Computer Vision

11. • The process of acquiring images
• 2D media
• 3D media
• An engineering discipline focusing on automating/digitising the
human vision system
Acquisition
Stages of Computer Vision

12. • Intrinsic Parameters:
• Focal length
• Principal Point
• Lens Distortion
• Extrinsic Parameters:
• Rotation
• Translation (relative to other cameras or original position)
Camera Parameters for Acquisition
Stages of Computer Vision

13. • A numeric representation of an image on a 2-D Grid
• Each element is referred to as a pixel and its value represents the
shade or colour of that segment
Image Digitisation
Stages of Computer Vision

14. • Colour in Images is
represented by:
• Bilevel images; pixels are
either 0 or 1
• Grayscale images; pixel
values range from 0 to 255
• RGB images; 3 channels/
values per pixel
representing red, green or
blue
Image Digitisation
Stages of Computer Vision

15. • How can the machine ‘understand’ the 2D space of the image?
• How can the machine be endowed with a 3D understanding of the
complexity of the environment?
2D images to 3D Scenes
Stages of Computer Vision

16. 2D images to 3D Scenes
Stages of Computer Vision

17. Which objects are where…
Object Detection

18. AI & Recent Developments
Object Detection

19. How Object Detection Works
Object Detection
• Image Processing Techniques
• OpenCV is a popular tool for image processing tasks
• Deep Learning Methods
• Supervised
• Unsupervised learning

20. Historical Evolution
Object Detection
• Before 2014 - Traditional object detection
• Viola Jones Detector (2001)
• HOG Detector (2006)
• DPM (2008)
• After 2014 - Deep learning detection
• Two stage object detection (2014-2021)
• One stage object detection (2016-2022)

21. One Stage vs. Two Stage Deep Learning Object Detectors
Object Detection
• Object detector solves two subsequent tasks:
• Task #1: Find an arbitrary number of objects (possibly even zero),
and
• Task #2: Classify every single object and estimate its size with a
bounding boxAfter 2014 - Deep learning detection
• Two stage
• (1) object region proposal with conventional Computer Vision
methods or deep networks, followed by
• (2) object classi
fi
cation based on features extracted from the
proposed region with bounding-box regression

22. Two Stage Object Detection
• Classi
fi
cation: Assigns a label to the whole image
• Usually denoted by a bounding box
• Detection: Applies classi
fi
cation and localisation to many objects
instead of just a single dominant object

23. One Stage vs. Two Stage Deep Learning Object Detectors
Object Detection
• One stage
• One-stage detectors predict bounding boxes over the images
without the region proposal step. This process consumes less time
and can therefore be used in real-time applications.
• The most popular one-stage detectors include the YOLO, SSD,
and RetinaNet. The latest real-time detectors are YOLOv7 (2022),
YOLOR (2021) and YOLOv4-Scaled (2020).
• Most popular benchmark is Microsoft COCO dataset. Di
ff
erent
models are typically evaluated according to a Mean Average
Precision (MAP) metric.

24. Yolo v2

25. Applications
Object Detection
• Computer Vision gives the machines the sense of sight—it allows
them to “see” and explore the world thanks to Machine Learning and
Deep Learning algorithms.This powerful technology has quickly
found applications across multiple industries, becoming an
indispensable part of technological development and digital
transformation.
• But how exactly do businesses bene
fi
t from the use of Computer
Vision?

26. Examples of Computer Vision Applications
• Manufacturing Sector
• Product Assembly Automation
• Defect Detection
• 3D Vision System
• Computer vision-guided Die Cutting
• Predictive Maintenance
• Safety and Security Standards
• Barcode Analysis
• Inventory Management

27. Examples of Computer Vision Applications
• Transportation
• Detecting Tra
ffi
c and Tra
ffi
c Signs
• Pedestrian Detection
• Tra
ffi
c Flow Analysis
• Parking Management
• License Plate Recognition
• Road Condition Management
• Automatic Tra
ffi
c Incident Detection
• Driver Monitoring

28. Examples of Computer Vision Applications
• Healthcare
• Improved Medical Imaging
• Better Diagnostic Applications
• Cancer Screening
• Surgery Assistance
• Research and Identifying Trends
• Retention Management in Clinical Trials
• Training
• Injury Prevention

29. Examples of Computer Vision Applications
• Agriculture
• Quality Inspection of Agricultural Food Products
• Image-based Plant Disease and Pests Detection
• Weed Detection
• Soil Sampling and Mapping with Drones
• Livestock Management
• E
ffi
cient Yield Analysis
• Grading and Sorting of Crops
• Phenotyping
• Indoor Farming

30. Examples of Computer Vision Applications
• Retail
• Visual Search for Enhanced Customer Experience
• Product Recommendations
• AR-based Try before you Buy
• Fitting Rooms with Magic Mirrors
• Automating Categorisation
• Improved Search Accuracy
• Better Inventory Management

31. Examples of Computer Vision Applications
• Sports and Fitness
• Real time Action Management
• E
ffi
cient Ball Tracking (Tennis and other sport)
• Training and Development Analytics
• Prevention of Life Threatening Situations

32. It’s all about making sense…
Challenges of Computer Vision
But what about a caption for…

33. What about this…
Challenges of Computer Vision

34. Activity time
Work in groups (use chat/social media/…)
Go back to the self-driving ambulance
scenario. Think about Computer Vision.
1. How can di
ff
erent applications of CV be
used during an accident?
2. What services can CV o
ff
er for the
ambulance?