Exploration of U-Net and Support Vector Machine classification methods for UAV multispectral image segmentation Recently, many solutions have been introduced to accurately and automatically analyze data acquired with Unmanned Aerial Vehicles (UAVs), in particular by relying on algorithms based on Artificial Intelligence (AI) techniques. Among these, the most popular are those belonging to the category of neural networks. These techniques allow the development of ad-hoc and end-to-end solutions for the classification and segmentation of different object categories through the analysis of high-resolution multispectral images. In our research, two main methodologies have been explored for the automatic segmentation of crop rows from multispectral images acquired with UAVs. The first is based on Support Vector Machines, know to handle well overfitting issues, and the other through the implementation of “U-Net”, a state-of-the-art Convolution Neural Network

1. Artificial Intelligence Algorithms for Automatic
Segmentation in Multispectral UAV Imagery
Alex Bojeri
NOI Techpark Südtirol/Alto Adige, Bolzano, 12th November 2021 – SFScon2021

2. Project idea
 droneONtrap project founded by
"DIVA Project" Horizon 2020
 Support tool for farmers to reduce
waste of water and fertilizers
 Integrate UAV multispectral data
and phototraps images
 Main goals:
▪ Economical savings
▪ Better quality of goods
▪ Improvement of working conditions
▪ Preserve the environment
FOS Spa
 Leading company with 100% ownership of a group of
SMEs
 Fields of expertize:
▪ IT
▪ IoT
▪ Electronic design
 Collaborations with main italian research institutes
 Currently incubated in NOI Techpark Südtirol/Alto
Adige
2
Overview
MAVTech Srl
 Research oriented spin-off company of Politecnico
di Torino
 Main activities
▪ Production of Remotely Piloted Aircraft
Systems (SAPR)
▪ Innovative aeronautical solutions for aerial
surveillance and tactical operations (including
precision agricolture)
 Authorized SAPR operator accredited by ENAC
 Currently incubated in NOI Techpark Südtirol/Alto
Adige

3. Objective
 Need to recognize rows within
orchards to evaluate
vegetational health status on
single parcels
 Accurate segmentation between
rows pixels and the background
 Avoid false classifications caused
by:
▪ Grass
▪ Shadows
▪ Other objects
3

4. Methodology: General Approach
4
Unmanned Aerial Vehicle
▪MAVTech Q4E quadrotor
▪MTOW ~ 3,9 [kg]
▪Payload ≤ 600 [gr]
▪Flight Height 40÷45 [m]
▪Endurance ~ 20÷25 [min]
▪GNSS RTK
Multispectral sensor
▪Micasense RedEdge-M
▪5 spectral bands (475 [nm]
÷ 840 [nm])
▪Integrated GPS and light
conditions sensor
▪Calibrated Reflectance
Panel
Equipment Proposed solution
a) Support Vector Machine
(SVM)
 Few hyperparameters
 Small amount of training
samples
 Different kernel types
 Implementation thorough
open-source libraries (i.e.
scikit-learn python
module)
b) Convolutional Neural
Network (CNN)
 Self-learning with error
backpropagation
 Full classifier architecture
customization
 Spatial coherence
management
 Huge amount of training
samples
 Open-source PyTorch
module

5. SVM
 Machine learning techinique proposed by V.
Vapnik et al. in '90s
 Powerful both with high- and low-dimensional
feature spaces
 Definition of hyperplanes to best separate classes
 Needs few training samples
 Classifier:
▪ Input features: 5
▪ Output: binary classification value
5
CNN
 Suitable for image classification/segmentation
 U-Net, proposed by Ronnenberger et al. in 2015
for biomedical RGB image segmentation
 Classifier:
▪ Input channels: 5
▪ Output channels: binary map
▪ Contraction (left) and expansion (right) layers
▪ Training size: 70%
▪ Validation size: 30%
▪ Optimizer: Adaptive Moment Estimation (Adam)
▪ Training loss: BCEWithLogitsLoss
▪ Validation accuracy: Dice-score

6. Processing Flowchart
Support Vector Machine Convolutional Neural Network
6
Input Data:
Multispectral + CHM
Rasters Cropping
Cropped
Multispectral Data
Cropped
CHM Data
Data Scaling
Data Flattening
SVM Prediction
CHM Thresholding
Morfology Filtering
Predicted
Binary Map
Morphology Filtered
Binary Map
Data Reshaping
CHM Thresholded
Binary Map
Input Data:
Multispectral + CHM
Rasters Cropping
Cropped
Multispectral Data
Cropped
CHM Data
Data Reshaping
Reshaped
Multispectral Data
Reshaped
CHM Data
Data Splitting
……………
CNN
Prediction
……………
Data Blending
CHM Thresholding
Predicted
Binary Map
Morfology Filtering
Morphology Filtered
Binary Map
CHM Thresholded
Binary Map
……

7. Experimental Setup
7
Multispectral
Orthoimagery
Random Points
Data Sampling
Training/Validation
Samples
SVM
Traning Algorithm
SVM
Trained Model
Test Samples
SVM
Testing Algorithm
SVM
Performance Results
SVM Classifier
Dataset Manual
Annotation
Test
Ground-truth maps
Training/Validation
Ground-truth maps
CNN
Training Algorithm
CNN
Trained Model
CNN
Testing Algorithm
CNN
Performance Results
CNN Classifier
~2000 samples

8. Results: Support Vector Machine
 Direct classfication without filtering and thresholding:
F1-score = 0,60 (on average)
▪ Many False Negatives (FNs) → Reduction of Recall → F1-score
decreases
▪ Presence of False Positives (FPs) → Precision score decreases →
F1-score overall reduction
 Best average result: F1-score = 0,815 with CHM
thresholding and morphological filtering
▪ Recall increases from 0,508 to 0,817
▪ Precision decreases from 0,919 to 0,845
8
SVM
CHM thresholding
Morphology filter
False Negatives
False Positives

9. Results: U-Net
 Direct classification: F1-score = 0,845
 U-Net and CHM thresholding: F1-score = 0,772
▪ Significant decrease of Recall from 0,811 to 0,682
 U-Net and morphological filtering: F-score = 0,858
▪ Slightly better, removes little spots of misclassified areas
 In general, accuracy results affected by manual
mislabelling errors:
9
U-Net
Morphology filter
Prediction mismatches

10. Results: Accuracy and Speed Comparison
 U-Net overall achieves higher F1-
score even without post-processing
 SVM requires relevant post-
processing to be practically
applicable in real scenarios
 Overall 8,66 [ha] analized in ~123[min]
(~2 hours)
 On average, SVM requires less time to be
trained and to predict (~3x faster than
U-Net)
▪ SVM: 0,244 [ha/min]
▪ U-Net: 0,095 [ha/min]
 On GPU: U-Net ~10x faster than SVM
 Practical usecase:
▪ SVM: ~4,1 [min/ha]
▪ U-Net: ~10,6 [min/ha]
10
SVM classifier U-Net classifier
CHM thresholding
Morphology filters
Morphology filters

11. Conclusions and Future Perspective
11
 Accurate tool for automatically
segmenting rows in multispectral
orthoimagery
 Avoidance of inter-row and shadows
misclassifications
 SVM perform pixel-wise classification
→ High-resolution images not suitable
for SVM segmentation
 Post-processing avoids most
classification errors
 Multispectral data are sometimes
unnecessary for image classification,
but useful to retrieve multiple
information from the same data
source
 Some additional improvements are
needed to complete the methodology:
▪ Integration of further data to reinforce
the model accuracy
▪ Automatic adjustment of output
threshold for U-Net classifier

12. Thank You For Your Attention!
Any Questions?
@SFScon #SFScon #SFScon21
NOI Techpark Südtirol/Alto Adige, Bolzano, 12th November 2021 – SFScon2021

13. 13
Multispectral
Dataset
Campo "1" Campo "2" Campo "3" Campo "4" Campo "5"
 Renazzo (FE),
Italy
 Surveyed in July
and September
2020
 Pears ~ 1,324 [ha]
 7743 x 8394 [px]1
 Organic
cultivation
 Dosso (FE), Italy
 Surveyed in July
and September
2020
 Pears ~ 1 [ha]
 6050 x 65721
 Traditional
cultivation
 Bagnacavallo di
Villa Prati (RA),
Italy
 Surveyed in
September 2020
 Apples ~ 3,433
[ha]
 11548 x 95031
 No training data
 Baricella (BO),
Italy
 Surveyed in July
2020
 Pears ~ 0,335 [ha]
 4821 x 51551
 Discontinued
cultivation
 San Cesario sul
Panaro (MO), Italy
 Surveyed in July
2020
 Pears ~ 0,491 [ha]
 5471 x 57551
 Discontinued
cultivation
1Ground Sample Distance ≈ 3 [cm/px]