Machine vision system for the automatic segmentation of plants under different lighting conditions

1. Machine vision system for the automatic segmentation of plants
under different lighting conditions
Mehran Mesbahzade
June 2019
1
Sajad Sabzi , Yousef Abbaspour-Gilandeh , Hossein Javadikia
http://dx.doi.org/10.1016/j.biosystemseng.2017.06.021

2.  A machine vision system is suggested for segmenting different plants.
 126 features were extracted of 6 colour spaces of RGB, CMY, HIS, HSV, YIQ, YCbCr.
 Hybrid artificial neural network-differential had better performance than other algorithms.
 The efficiency of meta-heuristic classifier systems is better than statistical classifiers.
 Most of classes were segmented in the third channel of improved YCbCr colour space.
Highlights
2

3. With increasing population and the occurrence of successive droughts in many parts of the world, many governments are
urging the implementation of precision agriculture. The term of precision agriculture can have different definitions based on
different goals. But generally, the set of actions that causes maximum exploitation of agricultural land and/or the optimal use
of inputs is called precision agriculture. Using modern methods and technologies such as image processing, video
processing, artificial intelligence and machine vision is essential for implementing precision agriculture.
1.Introduction
3

4. The different stages used to design the machine vision system in this research are shown in the flowchart in Fig. 1.
2. Materials and methods
4
Fig. 1-Segmentation flowcharts of different plants under various
lighting conditions.

5.  RGB colour space
 CMY colour space
 HSV colour space
 HSI colour space
 YIQ colour space
 YCbCr colour space
 Improved YCbCr colour space
2.2. Colour spaces
5

6. 2.2. Colour spaces
Figure 2 shows 9 images at each of the light intensities
related to the photographic conditions.
Fig. 2-Sample images of different light conditions and
different growth stages,
(a): light intensity 1820 lux, (b): light intensity 1096 lux,
(c): light intensity 156 lux, (d): light intensity 327 lux,
(e): light intensity 1542 lux, (f): light intensity 1998 lux,
(j): light intensity 750 lux, (h): light intensity 450 lux,
(i): light intensity 1650 lux.
6

7.  Features related to mean and standard deviation
 Green indexes
Table 3 shows the extracting features related to this section. As it can be observed, the average and the standard deviation
of the 1st, 2nd and 3rd channels, and the average of the three channels, using RGB, CMY, HSI, HSV, YIQ and YCbCr colour
spaces, makes a total of 7 6 ¼ 42 features for each object.
2.3. Extracting colour features
7

8.  Genetic algorithm
Genetic algorithm is an important optimization method that is inspired by nature. Indeed, this algorithm has been
established based on natural evolutionary principles.
 Differential evolution algorithm
Differential evolution algorithm is also based on a population and has random behavioral tendencies like many
optimization algorithms and was proposed by Storn and Price (1996).
 Particle swarm algorithm
Particle swarm algorithm is a meta-heuristic algorithm that mimics the mass movement of birds for optimizing different
problems. This algorithm was first proposed by Kennedy and Eberhart (1995).
2.4. Selection of effective features
8

9. Following selection of effective features, classification is required. There are different methods for classification that are
generally divided into the two categories of metaheuristics and statistical methods.
Meta-heuristics method work based on spatial search and therefore they have generally better results than statistical
methods.
2.5. Classification
9

10. 3.1. Selection of effective features
3.2. Classification
3. Results and discussion
10

11. Figure 3(a)
shows the performance of classifier system in classifying the selective features
by the mentioned methods.
Figure 3(b)
shows an accuracy box plot graph related to the different methods.
Figure 3(c)
shows the coefficient of determining hybrid artificial neural network-harmony
search classifier system in classifying 9 different states of photography using
the inputs determined by the three mentioned methods.
Fig. 3. Performance comparison three methods used to extract features.
(a): mean square error related to each iteration,
(b): box plot graph related to classification accuracy for each method,
(c): box plot graph related to coefficient of determination for each method.
11

12. Figure 4
shows a box plot graph of the classifier system error related to 20
repetitions based on three criteria of Mean Square Error, Root Mean
Square Error and Mean Absolute Error.
Fig. 4 -Classifier errors for set of selective features by ANN-DE, ANNGA
and ANN-PSO
(a): mean square error,
(b): Root mean square error,
(c): Mean absolute error.
12

13. 3. Results and discussion
Table 6
shows the results related to the best classifier by the method of hybrid artificial neural network-harmony search algorithm.
Table 7
shows the results of classification training using the hybrid artificial neural network-harmony search classifier
13

14. 3. Results and discussion
Table 8
shows the results related to the classifier test. Among 7,170 samples, it wrongly
classified only 45 samples.
Table 9
shows the results of classification based on KNN classification for the number of
different neighbors.
The results of this classification can be observed in Table 10.
14

15. 3. Results and discussion
Table 11
shows the results related to these three performance criteria for the two classifiers. As it can be seen, sensitivity, specificity and
accuracy of the classifier system of hybrid artificial neural network harmony search have the values of 100 for the classes of 3 and
4. Value of 100% sensitivity means that classifier system has not classified any sample related to the intended class wrongly in
another class.
15

16. Figure 5
shows ROC graph related to training and test data using hybrid artificial neural network-
harmony search method.
(a): training,
(b): testing datausing hybrid ANN-HS classifier.
16

17. 17
Figure 6
shows ROC graph related to test data using K-nearest neighbor
method. In this figure, the graph related to seventh class has lowest
level area and this means that classification system puts highest
number of wrongly classified samples into this class.
Fig. 6 - ROC graph related to testing data using KNN classifier.

18. 3.4. Selection of colour space and appropriate thresholds
3.3.2. Investigating the performance of classifier systems
Table 12
shows the result of those two researches compared to this research. It can be seen that the segmentation precision of
Hernandez-Hernandez et al. (2016) was 97% and of Tang et al. (2016) was 92.5%.
Table 13
shows the channels for determining threshold and threshold value for different photography states.
18

19. Figures 7 and 8
show sample image of each state, images in the selected
channels in Table 13 and the histogram related to these
channels. It can be seen that the histograms have two peaks,
one peak related to the background and the other to the
objects.
Fig. 7-Original image, referred channels in Table 13,
and histogram related to each channel.
(a)-(f) are related to photography states 1e6 respectively.
19

20. 20
Fig. 8 -Original image, referred channels in Table 13,
and histogram related to each channel.
(a)-(c) are related to photography states 7 to 9 respectively.

21. 4. Conclusions
Figures 9 and 10
show sample image, segmented image and
binary image related to all 9 different states.
21

22. 4. Conclusions
22
Fig. 10-Original image, segmented image and binary image.
(a)-(d) are related to photography states 6-9 respectively.

23. The End
23