This document presents a method for classifying road environments using images from a vehicle-mounted camera. It extracts color and texture features from subregions of road images and classifies them using k-NN and artificial neural networks (ANN). For a four-class problem distinguishing off-road, urban, major road, and motorway classes, the accuracy is around 80%. For a two-class problem distinguishing off-road and on-road, the accuracy increases to around 90% using ANN classification. The method achieves a near real-time classification rate of 1Hz by classifying one video frame per second.

1. Advisor : Yin-Fu Huang
Automatic Road Environment
Classification
Intelligent Transportation Systems, IEEE Transactions on
Student : Chen-Ju Lai
1

2. Outline
 Introduction
 Road image subregions
 Feature extraction and representation
 Feature classification
 Experimental result
 Conclusion
2

3. Introduction
 Input video frame :640X480 resolution
Standard digital video camera(30-fps video)
 Feature extraction :color ,texture
Classification method :k-NN and ANN
 Four class problem , accuracy : ~80%
{off−road, urban, major/trunk road,multilane motorway/carriageway}
Two class problem , accuracy : ~90%
{off−road, on−road}
 A near real time classification rate of 1Hz. (i.e.,one frame
classification per second)
3

4. Introduction
 Colour based off-road environment and terrain type
classification[3]
 Texture and neural network for road segmentation[7]
 Detection and classification of highway lanes using
vehicle motion trajectories[31]
4

5. Road image subregions
5

6. Feature extraction and representation
 Color Features
 Choice color space
6

7. Feature extraction and representation
 Color Features
 Each such channel of each subregion as the normalized histogram
distribution.
 Mean , standard deviation , and entropy.
A given color value (indexed k = 1, . . . , L) occurs with
probability pk
 Each color channel is summarized as a color feature vector of
combining the histogram (quantized to 10 “bins”)
 13-value , 7 channel , 91-D color descriptor for each image frame.
7

8. Feature extraction and representation
 Texture Features
 grey-level co-occurrence matrix (GLCM) statistics
 localized orientation is defined with as {N,S, E,W,NW,NE,SW,SE}
Original image Co-occurrence Matrix
8

9. Feature extraction and representation
 Texture Features
 grey-level co-occurrence matrix (GLCM) statistics
Co-occurrence Matrix Stochastic Matrix
9

10. Feature extraction and representation
 Texture Features
 grey-level co-occurrence matrix (GLCM) statistics
10

11. Feature extraction and representation
 Texture Features
 grey-level co-occurrence matrix (GLCM) statistics
M(i, j) is the (i, j)th entry in GLCM M with dimension(colsx
rows)
11

12. Feature extraction and representation
 Texture Features
 grey-level co-occurrence matrix (GLCM) statistics
horizontal standard deviation and mean (σI, μI ),
vertical standard deviation and mean (σJ, μJ ).
12

13. Feature extraction and representation
 Texture Features
 Gabor filters allow the study of the localized spatial distribution of
the texture.
 The magnitude of the Gabor filter response identifies varying local
texture frequencies and orientations in the image.
 Use for the extraction of more gradual (low-frequency) textures
and more generally create discriminative texture descriptors.
13

14. Feature extraction and representation
 Texture Features
 The use of only the (N,E) GLCM directions is within this visual
discriminatory context.
 The Gabor filter in use is itself summarized as a quantized
histogram
(10 “bins”)
 Mean, standard deviation, and entropy
 23-value ,3 subregions , 69-D texture feature descriptor for each
image frame.
14

15. Feature extraction and representation
 Edge-Derived Features
 Three additional edge-derived features specific to the road-
edge subregion.
 Use Canny edge detector
 The set of edges, connected contours , and straight line detected
is then summarized by the entropy.
15

16. Feature Classification
 A 163-D combined feature vector per image frame.
 K-NN and ANN
 Training set : 800 image frames (200 per class ,for four
classes)
 Four-class
 classes = {off−road, urban, major/trunk road,
multilane motorway/carriageway}
 Two-class
 classes = {off−road, on−road}
 {on−road} = {{urban} ∪ {major/trunk road} ∪
{multilane motorway/carriageway}}
16

17. Experimental result
 Two test video sequences
Video sequence Duration Consist of
Video sequence 1 40-s 10-s segments of
{off-road, urban, major/trunk
road, multilane
motorway/carriageway}
Video sequence 2 50-s 10-s segments of
{urban, major/trunk road,
multilane
motorway/carriageway}
Test image frame 20-s segments of {off-road}
Road environment
600(20-s) urban
600(20-s) major/trunk road
30-fps video
600(20-s) multilane
motorway/carriageway
900(30-s) off-road
17

18. Experimental result
 K-NN classification
Fig. 2. Video sequence 1. Classification results using k-NN varying
parameter k.
18

19. Experimental result
 K-NN classification
Fig. 3. Video sequence 2. Classification results using k-NN varying
parameter k.
19

20. Experimental result
 ANN classification
 We employ a classical two-layer network topology with H hidden
nodes.
 163 input node , 2 or 4 output node (two class or four class).
 The ANN is trained using I iterations.
 The general range of parameter H ={10, . . . , 60} and I = {150, . . . ,
700}.
20

21. Experimental result
 ANN classification
21

22. Experimental result
 ANN classification
22 Fig. 4. Examples of successful ANN classification for four road
environments

23. Experimental result
 ANN classification
23 Fig. 5. Examples of successful ANN classification for two road
environments (ANN configuration: H = 15 Nodes; I = 200).

24. Experimental result
 Extended Sequence Results
 full video sequences (representing the complete set of viable data
gathered over several hours in varying environments)
24

25. Experimental result
 Misclassification
25

26. Conclusion
 An ANN classifier gives ∼90%–97% successful
classification for two class ; ∼80%–85% for four-class.
 A k-NN classifier implies the inherent feature overlap
within the current feature space ,so it’s difficult to classify.
 Future work
 Subregion optimization
 Alternative computationally efficient texture measures
 The efficient of varying weather and lighting condition on
performance
26