This document discusses methods for traffic sign detection and classification using neural networks. It describes gathering and labeling a large dataset of street images containing various traffic signs and objects. A YOLO algorithm is used to detect regions of interest within images for classification by an R-CNN neural network. Results are evaluated based on accuracy and types of failures, such as false positives. Future work involves improving the dataset size and quality to increase detection accuracy.

1. •
•
•
•
Guide:
Prof. Sahana M P
Group - 61

2. •
•
•
•

4. 1.GATHERING AND ANALYZING DATA
2.TRAFFIC SIGN DETECTION
3.CLASSIFICATION
4.RESULTS
5.FUTURE PROSPECTIVES

5. 1. GATHERING AND ANALYZING DATA
• THE FIRST STEP TO TAKE WAS TO DEFINE THE ROAD SIGNS AND OBJECTS FOR THE
DATABASE. THE DATABASE BUILDS UP ON THE GERMAN TRAFFIC SIGN DETECTION
BENCHMARK(GTSDB), THEREFORE THE OBJECTS IN THE DATABASE USED IN THE
REPOSITORY ARE SIMILAR PICTURES OF EVERYDAY TRAFFIC SITUATIONS IN GERMANY. IN
ORDER TO BUILD THE DATABASE THAT WOULD BE ABLE TO DETECT A LARGER AMOUNT OF
ROAD SIGNS IT WAS NECESSARY TO LABEL A MUCH LARGER NUMBER OF PICTURES. THE
GOAL IS TO DISTINGUISH BETWEEN MORE THAN 150 ROAD SIGNS, TRAFFIC LIGHTS AND
MORE THAN 15 PHYSICAL OBJECTS SUCH AS PEDESTRIANS, CARS AND MOTORCYCLES.
• IN THE SECOND STEP THE PICTURES NEEDED TO BE GENERATED AND LABELLED. IN ORDER
TO GENERATE THE PICTURES A SIMPLE COMMAND-LINE TOOL IS DEVELOPED THAT ALLOWS
VIDEOS TO BE READ IN AND EXTRACT THOSE FRAME BY FRAME. IN THE TOOL IT IS
POSSIBLE TO ALTERNATE THE NUMBER OF FRAMES TAKEN. FOR HIGHWAY FOOTAGE EVERY
5TH FRAME IS TAKEN, WHILE FOR URBAN FOOTAGE THE NUMBER OF EXTRACTED FRAMES
IS SET TO EVERY 15TH FRAME.
• THE IMAGES ARE LABELED WITH THE OPEN-SOURCE TOOL LABELIMG FOR WINDOWS. IN
ORDER TO ENSURE THAT THE IMAGES WILL BE LABELED CORRECTLY NUMBERS WERE
USED AS LABELS, THESE WILL BE LATER TRANSLATED BACK TO DESCRIPTIONS OF THE
OBJECT. THE USE OF NUMBERS REDUCES THE LIKELINESS OF TYPOS IN THE LABELING
PROCESS.

6. • AS REAL-LIFE TRAFFIC SITUATIONS ARE USED AS INPUT OF THE PROCESS THE DATASET
NEEDS TO BE REVIEWED AND ANALYSED REGULARLY. THEREFORE, ANOTHER TOOL WAS
DEVELOPED (DATASETCLI.PY) TO MANAGE THE LARGE AMOUNTS OF DATA. THE TOOL
OFFERS MULTIPLE OPTIONS FOR THE DATABASE. ALL FUNCTIONS REQUIRE A PATH TO THE
ROOT FOLDER OF YOUR DATA, WHICH SHOULD CONTAIN ONLY IMAGES AND XML-LABEL
FILES.
• AS REAL-LIFE TRAFFIC SITUATIONS ARE USED AS INPUT OF THE PROCESS THE DATASET
NEEDS TO BE REVIEWED AND ANALYSED REGULARLY. THEREFORE, ANOTHER TOOL WILL BE
DEVELOPED (DATASETCLI.PY) TO MANAGE THE LARGE AMOUNTS OF DATA. THE TOOL
OFFERS MULTIPLE OPTIONS FOR THE DATABASE. ALL FUNCTIONS REQUIRE A PATH TO THE
ROOT FOLDER OF YOUR DATA, WHICH SHOULD CONTAIN ONLY IMAGES AND XML-LABEL
FILES.
Root Data
Images
Labels – XML

7. • EXPORT OF CLASSES
THIS FUNCTION ALLOWS THE EXPORT OF SINGLE OR MULTIPLE CLASSES FROM THE
DATABASE AS A ZIP-FILE. IT ALLOWS USERS TO BUILD MODELS THAT EITHER EXCEED A
CERTAIN NUMBER OF IMAGES IN THE DATABASE OR TO LIMIT THE DATABASE TO CERTAIN
KINDS OF OBJECTS. THE FUNCTION ALLOWS FOR EXAMPLE TO EXTRACT ONLY ROAD SIGNS
WITH A SPEED LIMIT BUT NO OTHER ROAD SIGNS OR OBJECTS. DURING THE EXPORT, A
CSV-FILE IS GENERATED AND ADDED WITHIN THE ZIP-FILE TO ENSURE THE CONTENT OF
EXPORTED CLASS IS CORRECT. THE ZIP HOLDS IMAGES AS (PNG-FILES) AS WELL AS
LABELS (XML-FILES)
• GENERATE CSV-FILE FOR DATASET
ANOTHER FUNCTION IS THE GENERATION OF CSV FILES. CSV FILES ARE USED AS THE
BASIC INPUT FILE FOR MOST TRAINING PROCESSES, CONTAINING THE PATH OF THE
IMAGE'S FILES TOGETHER WITH THEIR LABEL AND ROI. THE FUNCTIONS SELECTS ALL
IMAGES WITH THE SELECTED CLASSES AND SEARCHES FOR THE CORRESPONDING XML
LABEL FILE TO WRITE ONE ENTRY INTO THE RESULTING CSV.
• GENERATE DIAGRAM FOR DATASET
A VERY BASIC MATPLOTLIB GRAPH TO VISUALIZE THE DIFFERENT CLASSES EXISTING IN
THE DATASET, TOGETHER WITH THEIR FREQUENCY RELATIVE TO THE WHOLE DATASET.
THESE PLOTS CAN BE GREAT TO GATHER AN INITIAL INTUITION OF HOW THE DATASET IS
STRUCTURED BUT BECOME FAST CONFUSING WHEN TOO MANY DIFFERENT CLASSES ARE
PRESENT IN THE DATASET.

8. 2. TRAFFIC SIGN DETECTION
• FOR THE DETECTION OF THE FRAME, OR THE IMAGE OF THE ROAD, WE MAKE USE OF THE YOLO
ALGORITHM – YOU ONLY LOOK ONCE
•
•
•
•

9. •
•
•
•
•

10. METHODS CONTINUED…
•
•
•
•

11. 3. CLASSIFICATION
• FOR THE CLASSIFICATION, WE MAKE USE OF THE DATA PROVIDED BY THE YOLO DETECTION. IN THIS
METHOD WE MAKE USE OF THE NEURAL NETWORK – R-CNN.
• R-CNN IS REGIONAL CONVOLUTION NEURAL NETWORK.
• COMBINES RECTANGULAR REGION PROPOSALS WITH CONVOLUTIONAL NEURAL NETWORK FEATURES
•
•
YOLO

12. 4. RESULTS
• THE RESULTS NEED TO BE DISTINGUISHED IN MULTIPLE WAYS. SUCH AS ACCURACY AND THE
KIND OF FAILURES THAT OCCUR.
• THE FAILURES THAT OCCUR IN OBJECT-DETECTION CAN BE SEPARATED INTO FOUR DIFFERENT
MODES.
• THOSE WILL BE DESCRIBED QUICKLY, AS A STARTING POINT FOR FURTHER READINGS INTO THE
SUBJECT.
• TRUE POSITIVE
THE OBJECT DETECTOR CORRECTLY IDENTIFIES AN OBJECT. HENCE, THIS IS THE RESULT WE
ARE SEEKING FOR AND NO FAILURE OCCURRED. IN THE CASE OF THIS OBJECT DETECTION THIS
RESULT BECAME MORE RELIABLE THE CLOSER THE ROAD SIGN WAS.
• FALSE POSITIVE
A FALSE POSITIVE DETECTION APPEARS WHEN A SIGN IS MARKED INCORRECTLY. INCORRECTLY
MARKED SIGNS USUALLY OCCUR IF THOSE ARE TOO SMALL, SO FOR EXAMPLE IN A LARGE
DISTANCE. THE OTHER MAIN REASON IS AN INSUFFICIENT NUMBER OF LABELS, SAVED IN THE
DATABASE.
• TRUE NEGATIVE
TRUE NEGATIVE DETECTIONS ARE NOT VERY SPECTACULAR, YET IMPORTANT. IN THIS CASE THE
DETECTOR CORRECTLY DOESN'T GIVE OUT ANY KIND OF RESULT.
• FALSE NEGATIVE
THE OPPOSITE TO TRUE-NEGATIVE ARE FALSE-NEGATIVE DETECTIONS. IN THIS CASE A SIGN OR
OBJECT IS JUST MISSED BY THE DETECTOR. ALSO, OFTEN RELATING TO LOW AMOUNTS OF

13. 5. FUTURE ENHANCEMENTS
•
•
•