This document presents a proposed automatic toll e-ticketing system using machine learning and convolutional neural networks for vehicle number plate recognition. The system aims to recognize license plates in real-time to process toll payments electronically without needing RFID tags. It uses a raspberry pi camera to capture plate images which are then processed by a CNN model and OCR to extract the characters and match them to registered vehicles. The system is intended to provide a more convenient alternative to FASTags by identifying plates automatically while allowing prepayment of tolls through a mobile app. It achieved an average accuracy of 92% in testing and claims to overcome limitations of existing systems.

1. Automatic Toll E-Ticketing System by Advance
Vehicle Number Plate Recognition Using
Machine learning by Convolutional Neural
Network
Presented by
R.Ramya[513415104039]
N.Janarthanan[513415104701]
S.R.Archith Krishnan[513415104005]
Under the Guidance of,
Dr.K.Selva Bhuvaneswari,
Assistant Professor,
Department of Computer Science Engineering,
University College of
Engineering,Kancheepuram,
Kanchipuram.

2. Abstract
➢ The growing affluence of urban India has made the ownership of vehicles a
necessity. This has resulted in an unexpected civic problem - that of traffic control
and vehicle identification in Highway Toll gates.
➢ The Vehicle Number Plate Recognition System (VNPR) plays an important role in
addressing these issues.
➢ There are numerous VNPR systems available today which are based on different
methodologies.
➢ In this paper, we attempt to review the various techniques and their usage. The
VNPR system has been implemented using Machine Learning.

3. Introduction
❏ Automatic License Plate Recognition (ALPR) is a technology which automatically
recognizes number from vehicle’s license number plate. Vehicle license number is a
unique identity as no two vehicles can have same license number.
❏ Text Classification is a technique in Machine Learning , which is used sort out the the
characters in the Number Plate. In ALPR, the characters in the image of the number
plate are written in a text file which can be edited.
❏ Thus the unique number of the vehicle is recognized by a machine. In this paper,
identification of Indian number plate is proposed which is based on Machine
Learning .
❏ Indian number plate consists of 9 or 10 characters of English language and numbers.
Isolated images of all the alphabets of English language i.e. from A–Z and numerals
from 0–9

4. Scope and Goal
➢ The legislature has now made it compulsory for all vehicles made or sold after first
December 2017 to have a FASTag.
➢ FASTag is right now dynamic in more than 300 toll plazas the nation over.
➢ At this developing world , our answer has an agreeable force on the clients , If all the
Fastag are replaced by scanning license plates system, clients will be fulfilled and
they don't have to glue any labels over the windscreens.
➢ People need not wait for paying in each tolls , they can prepay and proceed their
journey without any pressure.

5. Objective
❏ This algorithm can recognize plates using the pictures taken at various angles,
various distances and different times of the day ,thus under various
illumination conditions.
❏ It plays a major role in real time applications such as automatic monitoring of
Traffic Rules and maintaining law enforcement on public roads
❏ It should reduce the noise rate and should improve the performance and
Processing speed .
❏ Should Produce 75% accuracy.

6. Literature Review
Gajendra Sharma (2018),“Performance Analysis of Vehicle Number Plate Recognition
System Using Template Matching Algorithm”,Journal of Information Technology &
Software Engineering(2018),Page No: 2-9.
❏ Observation: This system uses VNPR system which is a Digital Image Processing
technique which is broadly used to identify the Vehicle number plate Using Optical
Character Recognition(OCR) in Digital Image Processing.
❏ Limitations:
1. The Processing speed is very slow and hence it takes more time.
2. The Accuracy of recognition is quite low which is only 67.98%.
3. The Phase correction is levied to the maximum extent of 63.46%

7. Literature Review
Kajal A.Mandhare, Pornima P. Bagate, Pankaj Patil(2015), “Automatic License Plate
Recognition”, International Journal Of Informative & Futuristic Research(2015),
Page no: 2217-2223
❏ Observation: This system also uses Automatic License Plate Recognition(ALPR) which
has a minimum Database Requirement. This System requires less Training sets and
also that the memory requirement is only minimum. At last this system can be able to
recognize image from any angle of distance.
❏ Limitations:
1. Needs high resolution digital camera.
2. Sensitive to environment conditions.

8. Literature Review
Lihong Zheng, Xianggjian He, Qiang Wu,Tom Hintz(2007),”Number Plate Recognition
Without Segmentation”, Research paper in Computing Technologies(2007),
Pages : 164-168
1. Observation: The method proposed in this paper is suitable for poor quality images in
real time applications. It gives a total different views in number plate recognition. It
can achieve faster recognition in complex conditions especially in dark and poor
conditions.
2. Limitations:
a. Some number plates are motion blurred
b. Some are overlapped by other vehicle’s body

9. Literature Review
M.A.Massoud, M.Sabee, M.Gergais, R.Bakhit (2013), “Automated new License Plate
Recognition in Egypt”,Alexandria Engineering Journal,Pages: 319-326
❏ Observation: The proposed technique involves Extraction of plate Region,
Recognition of plate Characters, and database Communication. A Video stream was
one of the most important features.The real time was capability , and that it did not
require any additional Sensor.
❏ Limitations:
❏ It has fixed illumination
❏ Fixed type of License Plate, Stationary backgrounds
❏ Limited Vehicle Speeds

10. Existing System
➢ FASTag is a device that uses Radio Frequency Identification (RFID) technology
➢ This system was first introduced by SANA SAID AL-GHAWI,S ASIF HUSSAIN,
MUNA AL RAHBI & ZAHID HUSSAIN OF Saudi University.
➢ It is used for making toll payments directly from the prepaid or savings account
linked to it.
➢ RFID tag is affixed on the windscreen of the vehicle and RFID reader is placed in
every toll ,which enables to drive through toll plazas, without much stoppings
for cash transactions.

11. Difficulties in Existing System
❏ Most of the vehicle Users are not interested in pasting labels on the
windshield of their vehicles.
❏ And there are numerous issues in Recharging these labels.
❏ The RFID Installation process is a hectic process and also an long one,Hence
users prefer not to undergo this system as it is much time consuming for the
installation alone.
❏ Not all the Toll Gates have the RFID Reader.

12. Proposed System
➢ Rather than gluing the Tag on the highest point of the windshield , here License
plates of the vehicles are recognised which are actually present in every vehicle by
default.
➢ Each client who have connected their Mobile number (related with bank) with their
automobile number in this application , the toll amount will be deducted from their
Bank accounts and E-Bills are generated.
➢ And also the customers can prepay their total toll amount before they start their
Journey through this application.
➢ In the event if the customers don’t have balance in their wallet they would be given
the option to pay through manual tolls.

13. Methodology
➢ When the Vehicle arrives the Pi- Camera captures the car’s number plate and send it to
the Raspberry pi, which processes the image [with the help of model trained by Machine
Learning(Tensor Flow)] and extracts the characters from the License Plate.
➢ There are two phases:
(i) Phase 1 – Training & Processing the input using Convolution Neural
Network Using Tensorflow.
(ii) Phase 2 – Recognition using Photo OCR Algorithm.
• The above two phases are correlated using Two modules in the android application. They
are:
(i)User Module.
(ii)Admin Module.

14. Phase 1-Training in Neural Network
➢ Neural networks take a different
approach to problem solving than that
of conventional computers.
➢ Neural networks process information in
a similar way the human brain does.
The network is composed of a large
number of highly interconnected
processing elements(neurones)
working in parallel to solve a specific
problem.

15. Phase 2- Photo OCR
The algorithm used in Machine Learning was Photo OCR
Photo OCR Pipeline
1. Text detection
2. Character segmentation
○ Splitting “ADD” for example
3. Character classification
○ First character “A”, second “D”, and so on

16. Processing Diagram(Phase 2)

17. Extracting Character(Phase 2)

18. System Architecture
Extraction
Model
RX6I GDU
Compare with DBFee ProcessingBill Generation
Input Model
CNN

19. Working
1. Firstly, image is feeded to CNN to extract image features.
2. The next step is to apply Recurrent Neural Network to these features followed by
the special decoding algorithm
3. This decoding algorithm takes lstm outputs from each time step and produces the
final labeling.

21. User Module
➢ The User has to first register with the application and should provide the following
documents :
○ Vehicle RC Copy
○ Any 1 KYC document
○ 1 Passport size Photo
➢ The KYC Completion will be done
➢ The User can also prepay their Upcoming Tolls through the app.
➢ Once the payment has been done for a specific toll, a bill is generated .

22. Pay for Upcoming Tolls

23. Admin Module
❏ After the License plate is scanned , the Admin has to verify in the database
whether the vehicle with this number has registered with the application or not.
❏ The Admin has to check whether this vehicle has already paid the toll amount or
not.
❏ Also the Admin has to check the Wallet Balance of the user , whether the available
balance is sufficient to make the Toll Payment in case of Prepaid Journey.

24. Acquiring Dataset

25. Login
➔ This Module provides Sign Up [for
new users] and Sign In [for already
registered users]
➔ In Sign In , OTP is sent to the
registered mobile number to verify
the user

26. Admin Registration
➔ The admin is the Toll Incharge.
➔ In this module, the admin has to
register with his personal details and
on which toll he is working on.

27. Admin Module
➔ From this module, the number plate
of the vehicle is obtained .
➔ The Toll Admin checks whether the
user already paid the toll or not.

28. Customer Registration
➔ In this Module, the customer registers
to the system .
➔ The customer provide all the personal
details and upload their photo, RC
book, KYC document for verification.

29. Customer Module
➔ Customer pays the Toll through this
module.
➔ When the customer gives the data of
from source ,to destination , mode of
travel and journey type , system
gives the total distance, total time
and total charges .
➔ After Payment Bill is generated.

30. Profile
➔ This Module shows the profile data of
the customer.
➔ Customer can add vehicle to their
account .
➔ Customer could add money to their
Wallet, for paying toll.
➔ Generated bills are displayed here.

31. Experimental Results
Actual Plate Predicted Plate Error Detection
(in char)
Processing
speed(in secs)
Accuracy(in
percentage)
HR26DK8337 HR26DK8337 0 2 100
TN 45 BK 9199 IN 45 BK 9199 1 2 90
AP65 AWD AP6 AND 2 2 80
JH 01R 1869 H 01R 1869 1 1 90
KL 01 CC 50 KL 01 CC 50 0 1 100
Average: 0.8 Average:1.6 sec Average: 92%

32. Performance Analysis Result
PC - Phase Correlation
NCC - Normalized Cross Correlation

33. Advantages
Thus by the proposed system we overcome the limitations availed in the existing systems.
1. The recharging process is easy and efficient.
2. There is no need to paste the labels above the wind shields.
3. We don’t need to carry any Cash and Transaction cards.
4. The Processing speed is very high and hence it takes less time to process.
5. The Accuracy of recognition is high which is only 75%.
6. Due to the usage of Machine learning ,The images cannot be blurred and the
segmentation is done with cent percentage.

34. References
[1]. Massoud MA, Sabee M, Gergais M, Bakhit R (2013) “Automated NEW LICENSE Plate Recognition
in Egypt”. Alexandria Engineering Journal 52: 319-326.
[2]. Fink O (2008) “Vulnerability Identification”.
[3]. Zheng, He, Wu, Hintz (2005) “Character Recognition of Car Number Plates”.International
Conference on Computer Vision 1: 33-39.
[4]. Kim KK, Kim KI, Kim JB, Kim HJ (2000) “Learning Based Approach for License Plate Recognition,
Neural Networks for Signal Processing”. IEEE Signal Processing Society Workshop 2: 614-623.
[5]. Zheng Y, Li H, Doermann DS (2004) “Machine Printed Text and Handwriting Identification in Noisy
Document Images”. IEEE Transactions on Pattern Analysis and Machine Intelligence 26: 337-353
[6]. Zhao B, Liu Y, Xia SW (2000)” Support Vector Machine and Its Application in Handwritten Numeral
Recognition”. Proceedings 15th International Conference on Pattern Recognition 2: 720-723.