Final Semester Major Project ppt on the topic "Billing System Based on Gesture Recognition"

1. BILLING SYSTEM
Submitted To:
Ms. Adwitiya Sinha
Submitted By:
Akshit Bansal(10104669)
Saumya Tripathi(9910103583)
USING HAND GESTURES

2. OUR PROJECT: BILLING SYSTEM USING
HAND GESTURES
 Billing system is developed taking into consideration problems faced
by a disabled person
 The person can go to the shop
 Select items from the menu
 Enter it’s quantity using hand gesture
 Confirm/Cancel order
 Add more items as per the requirement
 Pay the bill
 Give his feedback

3. OBJECTIVES
 To make a unique billing system that is developed for people with
hearing and/or speech disability
 It is based on gesture recognition because the disabled person
lacks the ability to communicate
 The system generates the bill
 The person can also give his feedback

4. KEY FEATURES
 The Billing system is being implemented using openCV in Microsoft
Visual Studio 2010
 Person enters a shop, if he wants to place an order he has to select
an item from the menu
 Selection will be based on color recognition. He has to select the
items using blue color
 Person can enter the quantity of the selected items. The quantity will
be entered using gesture recognition

5.  Algorithms and techniques used are:
1) Contouring
2) Convex Hull Detection
3) Defects calculation
4) Thresholding
5) Fingers Earth’s Movers Distance(FEMD)
 Person can perform the following functions:
1) Cancel order
2) Confirm order
3) Add more items
4) Give feedback

6. ADVANTAGES TO THE DISABLED PERSONS
 People with some disabilities can communicate easily with the help
of sign languages
 Disabled people can easily place their orders using our billing system
and it can be understood by any person
 The disabled person can:
1) Go to the shop
2) Select the item he wants to order
3) Enter it’s quantity
4) Confirm/Cancel order
5) Pay the bill with the help of the system
 He can avoid the tedious and time consuming task of explaining
himself to the sales person

7. SIGNIFICANCE OF PROBLEM
 Now the disabled person is no more dependent on anyone else like
an interpreter for explaining himself to the other people.
 The system does not depend on anyone particular sign language.
So anyone can use this system with some basic English language
knowledge.
 No use of database for this system reduces the image processing
time and makes the system more efficient.

8. ALGORITHMS USED
 Hand segmentation
1) Is the detection of hand from the whole image
2) Background subtraction is used to find the hand area
3) The system acquires image of the background
4) Then it starts capturing the image when hand is shown in front of
the camera
5) Now the previous image is subtracted from the new one
6) Hence the background is removed and we obtain image of the
hand

9.  Thresholding
1) Used to separate out the regions of image as per the users
requirement
2) The RGB image is converted into gray scale image
3) The gray scale is then converted into black and white image
 Contouring
1) Contour is a curve that connects points of an image with same
color or intensity
2) To identify hand gestures we detect contours of all objects that
exist in the threshold image
3) Identify the contour with the maximum area

10.  Fingertip detection
1) No of fingers can be detected by constructing a polygon of the
hand contour
2) Then a convex hull is made around the polygon
3) Now the defects are calculated
4) No of fingers = defects – 1
 FEMD algorithm
1) No of fingers can be detected by measuring the distance of all
contour points from the centre
2) The angle between the fingers are calculated
3) No of fingers = angles + 1

11. TOOLS AND TECHNOLOGIES USED
 openCV
1) Has C++, C, Python and Java interfaces and supports Windows,
Linux, Mac OS, iOS and Android.
2) openCV was designed for computational efficiency and with a
strong focus on real-time applications.
3) The library can take advantage of multi-core processing.
 MICROSOFT VISUAL STUDIO 2010
It is used to develop console and graphical user interface
applications along with Windows Forms or WPF applications, web
sites, web applications, and web services in both native code
together with managed code for all platforms supported by
Microsoft Windows.

12. PROJECT REQUIREMENTS
 Functional Requirements
1) The system shall be able to select the item as pointed by the user
2) Acquire and process the images of the numerals
3) Show the list of items available in the shop
4) confirm or cancel the order
5) Calculate the bill of the user and display the results

13.  Non-Functional Requirements
1) System requirements - 32 bits windows, webcam and blue colored
objects
2) Performance - The system shall not consume much time to process
the image and response
3) Security - The system has no security constraint
4) Reliability - The system shall be able to process the images almost
accurately
5) Maintainability - System is easy to maintain

14. FINDINGS
1) Background of an image plays a very important role while
developing any system based on image processing.
2) The efficiency of most of the image processing algorithms and
techniques depends significantly on the background.
3) The color of an image, its brightness and the distance of the object
from the webcam in an image also need to be paid full heed to
maintain the accuracy.

15. Billing system using “Hand Gestures” has
been divided into a number of modules

16.  List of modules is as follows:
 Load image of background.
 background subtraction.
 convert RGB-black and white.
 contouring
 polygon formation
 Convex hull detection
 defects calculation
 fingers = defects - 1
 quantity determined.

17.  FEMD algorithm
 angle calculation
 fingers = angles + 1
 select item to buy(tea, coffee, samosa)
 selection via(centroid formation)
 enter the quantity
 confirm/cancel order
 add more items
 price = quantity * price per unit
 give feedback

18. CONCLUSION
 A billing system based on the gesture recognition has been
developed that serves the basic purpose of placing an order for a
disabled person.
 The person can select the item of his choice, show the quantity
needed, confirm the order, add more items if he wishes to and get
the bill he has to pay all just with the help of gestures.
 The person also has the option of cancelling the order if in case he
changes his mood and wants to cancel it.
 He can also give his valuable feedback thus helping the place
improve its quality and standard.