This document describes a project that developed an Android application called "Real Time Object Detection Through Voice Assistant" to help visually impaired users. The app uses machine learning and computer vision techniques like CNNs, OpenCV, and optical character recognition to detect objects and text in real-time using a device camera. It then provides audio feedback by translating the detections to speech using text-to-speech conversion. The app was implemented in Java using Android Studio and works without an internet connection by running entirely on the mobile device. Screenshots demonstrate its user interface and capabilities for real-time object and text detection with voice outputs.

1. Real Time Object Detection Through Voice
Assistant
Guided By:
Renuka Bantupalli
Team Members
MEGHANA TIYYALI (18Q71A0560)
SAILAJA KHUNTIA (18Q71A0585)
BHAGYASHREE TAMANG (18Q71A0512)
YASH SHARMA (18Q71A0554)
VARA PRASAD (18Q71A0539)

2. Content
● Abstract
● Introduction
● Objectives
● Technology Platform Overview
● Implementation
● Application snapshots
● Live demonstration
● Conclusion

3. ABSTRACT :
• Mobile devices or portable smart gadgets are largely used in our day-to-day life now. These devices
can be very useful in helping people who are visually impaired and assist them to make their life
easier.
• This project presents an android application that recognizes objects using real time object and text
detection by scanning them.
• The application can run on the device without any remote server. Our solution uses a voice feedback
to tell the user about the detected object.The app doesn’t need any photograph to detect the object.
• To enable robust recognition, we first segment the object from the background using
tensorflow machine learning API based on iterative diagram cuts.
• We then formulate the recognition problem as an instance retrieval task and the user gets to know
the object through a text to speech method.
• The system is designed to help a visually impaired person know about the environment around
them. This app can be run on any low-end smartphone devices.

4. Logo and Interface:

5. Introduction
● With the recent rapid development of information technology (IT), a lot of
research has been carried out to solve inconveniences in everyday life, and as a
result, various conveniences for people have been provided .
● Nevertheless, there are still many inconveniences for the visually impaired.
The greatest inconveniences that a blind person feels in everyday life include
finding information about objects and indoor mobility problems .
● We analyze accurate object information and obtain a location using a deep
learning object recognition technique.
● In addition, voice recognition and voice guidance technologies are synthesized so
the visually impaired can know the location of the objects they want to find by
speaking to the system

6. Objectives:
The prime objective of “Real time object detection through voice
assistant” is to develop a full fledged Android application which could be
able identify and classify Real Time Based Common day-to-day Objects
and generate voice feedbacks.

7. Technology Platform:
● Mobile Platform : Android
● Application development framework :
Android
● Developing language : Java
● User Interface Language : XML
● Android API level : 10 and above
● Database : Sqlite

8. Object Text
What is the
object in
front of me
There is a
Laptop, Bag,
Flower Vase,
Read the
text in front
of me

9. Algorithms used in Object Detection :
CNN
The convolutional neural network, or CNN for brief, could also be a specialized kind of neural network model designed for working
with two-dimensional image data, although they're going to be used with one-dimensional and three-dimensional data.
Open CV stands for open source computer vision. it's a group of libraries in Python. it's a tool by which we will be able to manipulate
the pictures , like image scaling, etc. This supports and helps us in developing real time computing applications. It mainly
concentrates and targets on image processing, video capture and analysis. It includes several features like face detection and also
object detection.
.
Open CV
Training the data set
The data set is typically the gathering of knowledge. the info set could also be collection of images or alphabets or numbers or documents
and files too. the info set we used for the thing detection is that the collection of images of all the objects that are to be identified.
Several different images of every and each object is typically present within the data set. If there are more number of images like each object
within the datasets then the accuracy are often improved.

10. Voice Generation module for Detected Objects
• After the detection of an object, it is utmost important to acknowledge the person about the presence of that object on his/her way. For
the voice generation module PYTTSX3 which plays an important role. Pyttsx3 is a conversion library in Python which converts text into
speech.
• This library works well with both Python 2 and 3. To get reference to a pyttsx.
Engine instance, a factory function called as pyttsx.init() is invoked by an application. Pyttsx3 is a tool which converts text to speech
easily.
• This algorithm works as whenever an object is being detected, approximate distance is being calculated, with the help of cv2 library and
cv2.putText() function, the texts are getting displayed on to the screen.
• To identify the hidden text in an image, we use Python_Tesseract.
• Object is initialized by engine which is responsible for loading a speech engine driver from the pyttsx.drivers module.
• After construction, an object created by an engine is used by the application to register

11. :
Algorithms used in Character Recognition :
Optical Character Recognition (OCR) Algorithm
• It gives a machine power to read any text. OCR is the best way to recognize any text which make
visually impaired user able to hear text out loud. Camera takes individual frame using detect
methods and Text Recognizer.
• To read any text straight from camera, when any texts become available we have implemented
processor in Ocr Detector Processor which handle all the detections.
• From Text Recognizer Receive Detections will receive text blocks when text becomes available.

12. Implementation
Required tools and software to develop “Real time object detection through voice
assistant” application are mentioned below:
● Java Standard Edition(SE) 7 or above
● Android powered device for app running and debugging
● Device driver

13. Architecture
Activity Diagram

14. Use Case Diagram

15. Application Snapshots
The red mark icon represent the real
time object detection app, on clicking it
would open the application which result
the creation of main activity.

16. Snapshot (Continued)
This is the first screen or activity of the
project output.Now app will ask
permission for pictures and record
video.Click on allow to enable it.If it is
already enabled , permission request is
not appeared.

17. After app main activity launched , click on
the “Real time object detection through
voice assistant” button.By clicking on it
user can go to individual activity.
Snapshot (Continued)

18. After clicking on real time object detection
button, the following options appear on the
screen -
i) DetectText for Text detection
ii)DetectObject for Object detection.
Snapshot (Continued)

19. Live demonstration

20. :
Conclusion
The project exposed us to the latest technology in the area of mobile platform
development.Thus, this project successfully demonstrated a mobile based “real
time object detection through voice assistant” application.

21. Thank you
for listening!