Detection of potholes using cnn models

1. Realtime Pothole
Detection System using
improved CNN Models
K.MD. ZAHID PARVEZ(1602-20-737-076)
BATHINI AJAY GOUD(1602-20-737-310)
Review-2
Internal Guide –
Ms. B. Leelavathy
Assistant Professor
Vasavi College Of Engineering

2. Existing work presented in the base
 Model Selection:
 YOLOv5 CNN model chosen as the foundation for pothole detection.
 Image Processing:
 Utilization of convolutional methods for effective image processing.
 Model Training and Evaluation:
 Training of YOLOv5 models (YOLOv5m6, YOLOv5s6, YOLOv5n6) with subsequent
evaluation based on mAP@0.5.
 Notable mAP@0.5 results: 80.8%, 82.2%, and 82.5% for YOLOv5m6, YOLOv5s6, and
YOLOv5n6 respectively.
 Findings and Recommendations:
 Identification of minor errors, such as misclassifying manholes, trees, and shadows as
potholes.
 Recommendations for further research, including improving image processing during
night-time and enhancing detection performance for long-distance objects.

3. Existing work presented in the base

4. Our Proposed Work
Data Collection and Preprocessing
Choosing Framework, Architecture and Model
Training of Custom Dataset and Initial Results
Post-Tuning Strategies
Results and Improvements
A comprehensive website integrated with Google Maps for analysing
the damages reported

5. Data Collection and Preprocessing:
Maximally Stable Extremal Regions (MSER)
Source: Kaggle
 Explanation: The dataset of 700+ images was sourced from Kaggle, a platform for machine
learning datasets and competitions.
Labeling with LabelMe, storing labels in JSON
 Explanation: Image annotation was performed using LabelMe, a popular tool for
annotating images with bounding boxes.
 The annotated data, containing information about the location of objects in the images,
was stored in JSON format.
 This structured data is crucial for training a computer vision model.

6. Data Collection and Preprocessing:
Augmentation of images using Albumentations for dataset diversification
 Explanation: Albumentations is a powerful Python library for image augmentation. Image
augmentation involves applying various transformations to the original images to create new
training samples, thereby enhancing the model's ability to generalize. The augmentation techniques
applied include:
 Random cropping: Extracting random regions from the images to introduce variability.
 Grayscale conversion: Transforming images to grayscale to account for different color representations.
 Saturation adjustment: Changing the intensity of colors to handle variations.
 Brightness adjustment: Adjusting the overall brightness of images for robustness.
 Hue adjustment: Modifying the hue of images to simulate different lighting conditions.
 Shadow adjustment: Introducing shadows in images to account for diverse scenarios.
 Night-time simulation: Adjusting images to represent nighttime conditions, ensuring the model's adaptability.
 Purpose: Augmentation enhances the model's performance by exposing it to a wider range of
scenarios, ensuring better generalization and robustness in real-world conditions.

7. Choosing Framework, Architecture and
Model
Maximally Stable Extremal Regions (MSER)
Framework: PyTorch
 Chosen for flexibility, ease of use, and strong community support. Enables seamless
integration with SSD for training and deployment.
Architecture: ResNet
 ResNet variants as the backbone can provide better feature extraction capabilities as
our project has large set of features
Model: SSD
 Used SSD for predicting the bounding boxes at different scales in a single pass
through a network rather than two stage.
 Checked with different Momentum values

8. Choosing Framework, Architecture and
Model

9. Training of Custom Dataset and Initial
Results
Model Training
 Trained our custom dataset on top of SSD with different Momentum, Loss function, Learning
Rate
Initial Model Evaluation:
 Presentation of Metrics: Introduced initial performance metrics, showcasing the model's
achievements.
 Acknowledgment of Challenge: Recognized the challenge of limited accuracy and set the
stage for improvement.

10. Post-Tuning
Maximally Stable Extremal Regions (MSER)
 Plan for Post-Tuning:
 Developed a post-tuning plan with a focus on improving night-time detection.
 Real-time Camera Feed Handling:
 Incorporated real-time camera feeds with IP access, facilitating live streaming for on-the-fly
detection.
 Integration of Bounding Boxes:
 Implemented a process to merge overlapping bounding boxes, enhancing the precision of
object detection.

11. Results and Analysis
Momentum Learning Rate Loss Function Achieved mAP@50 Achieved Train Loss
0.1 0.0005 SGD Loss Function 88.676 0.6764
0.5 0.0000002 SGD Loss Function 13.15 39.944
0.9 0.01 SGD Loss Function 85.85 0.976
0.9 0.001 AGD Loss Function 49.9 1.892
0.1 0.001 AGD Loss Function 25.769 0.545
SGD: Stochastic Gradient Descent – measure of predicted output and actual output.
AGD: Accelerated Gradient Descent – predicts the curvature of loss faster than SGD.
After training with different Hyper Parameters we achieved 88.676 mAP@50 and 0.6764 Train
Loss with Momentum 0.1, Learning Rate 0.0005, SGD Loss Function.

12. 0.9
Momentu
0.01 AGD Loss
Function
85.85
mAP
0.976
Train Loss
0.9
Momentu
0.001 LR AGD Loss
Function
49.9 mAP 1.892 Loss
Function
0.1
Momentu
0.001 AGD Loss
Function
25.769
mAP
0.545 Train
Loss

13. 0.1
Momentu
0.0005
LR
SGD Loss
Function
88.676 mAP 0.6764
Train Loss
0.5
Momentum
0.0000002
LR
SGD Loss Function 13.15 mAP 39.944 Train Loss

14. Use cases Achieved
 Detection of Potholes in different lighting conditions including Night time.
 Detection of Potholes of different sizes, shapes and dimensions
 Detection of false objects like Trees, Shadows, Vehicles etc..

15. Use cases Achieved
 Detection of Potholes in different lighting conditions including Night time.
 Detection of Potholes of different sizes, shapes and dimensions
 Detection of false objects like Trees, Shadows, Vehicles etc..

16. Use cases Achieved
 Comprehensive website integrated with Google Maps API for analyzing results.

17. Use cases Achieved
 Comprehensive website integrated with Google Maps API for analyzing results.

18. Improvements:
Maximally Stable Extremal Regions (MSER)
 Enabling real-time camera feeds with IP access, facilitating live streaming for detection.

19. Software and Hardware Requirements
Maximally Stable Extremal Regions (MSER)
Software Requirements
Python
OpenCV
PyTorch
Albumentation
LabelMe
Ultrlaytics
SSD
Google Colab
Hard Requirements
Physical Camera with IP Streaming
System with GPU (If Google Colab is not used)

20. Team B-13