9. A project to build a system that
converts visual input into audio signals
which may lead to a practical product
to help the blind or visually impaired
navigate
9
10. Technological Objectives
The main objective is to obtain a device for the whole Visually Impaired
(V.I.) community- totally blind, partially sighted, and people in a situation
of progressive loss of vision.
To integrate the developments of image segmentation, image
enhancement, 3D Real World perception, audio representation of space
into a unique basic prototype with the purpose of assisting visually
impaired people in mobility in outdoor spaces.
To develop new ways of interfacing with V. I. people- to develop an
acoustic interface (Ear-plugs for presentation) for the blinds.
To develop a system able to calculate a person’s head orientation in a
static and dynamic situation.
10
11. Technological Objectives
To integrate the 3D modelling system and the intelligent system for the
detection of risks and obstacles
To integrate in a common platform all the data acquired from the
environment from different sources- artificial vision system, 3D Real World
Recognition System.
To finally develop a prototype which can be commercialised
During the project a simple device able to make a direct transformation
from the acquisition system into an acoustic representation will be created
as a final product
11
12. Gadgets/Technology for V.I. Person 12
Brainport Vision Device
Electric lollipop that allows blind people to 'see'
using their tongue
Converts images captured by a tiny camera into
a series of electrical tingles, which can be felt on
the tongue
Nerves then send these messages to the brain,
which turn the tingles back into pictures
Costs around 6000 Pounds ( 5 Lakh INR)
13. Gadgets/Technology for V.I. Person
Devices that alert you when a pot of liquid begins to boil
Gadgets that make a sound when a cup or container you're pouring liquid
into is nearly full
Knives with an adjustable guide to help you cut slices of even thickness
Tactile watches and alarm clocks
13
15. Cyborg / Bionics
An organism “to which exogenous components have been added for the
purpose of adapting to new environments”.
Recent meaning of the term bionics refers to merging organism and
machine.
This approach results in a hybrid system combining biological and
engineering parts, which can also be referred as a cybernetic organism
(cyborg).
15
18. What is Image Processing?
In electrical engineering and computer science, image processing is any form of signal
processing for which the input is an image, such as a photograph or video frame; the
output of image processing may be either an image or, a set of characteristics or
parameters related to the image.
Most image-processing techniques involve treating the image as a two-dimensional
signal and applying standard signal-processing techniques to it.
In other words it is basically the transformation of data from a still or video camera
into either a decision or a new representation.
All such transformations are done for achieving some particular goal. The input data
may be a live video feed, the decision may be that a face has been detected, and a new
representation may be conversion of a color image into a gray scale image.
19. Our Innovation
Locating everyday items using Image-Video Processing.
Detection of potential hazards (location and motion of objects, persons,
obstacles).
Face Detection and Tagging.
Text to Speech Conversion.
Color Detection
Detecting and analyzing in-front objects (living and non-living) and
perform operations on them.
19
20. Approach
We are using OpenCV with Panda Board for this application.
Interfacing OpenCV and Panda Board along with webcam to generate real
time images.
Coding to find distance of objects or any obstacle and provide output as
an audio signal.
Provide with inbuilt images and search for them. Coding to match the
images and provide the respective name of object as sound signal.
Advance further to develop for general usage amongst people.
20
21. Advantage & Disadvantage
Advantage
Detection of known objects if they are lost.
Can read newspaper, books, or rather any reading material.
Can easily detect color and hence can be useful in various scenarios.
Face Detection and Face tagging mode.(He can easily know person and can
also save new face)
Most of the software are freeware, so cheap in cost.
Disadvantage
A fully-assembled device (Panda Board + Webcam) is to be carried for all the
operations.
Bulkier in comparison to Mobile.
21
22. Hardware/Software
The Panda Board is a low-power, low-cost single-board
computer development platform based on the Texas
Instruments OMAP4430 system on a chip (SoC). It is a community
supported development platform.
The Panda Board ES is a newer version based on the OMAP4460 SoC, with
the CPU and GPU running at higher clock rates. Like its predecessor, it is a
community supported development platform. We are working with Panda
Board ES.
22
24. Pandaboard Overview
OMAP4 (Cortex-A9) CPU based open development platform.
OMAP4430 Application processor
1GB low-power DDR2
Display HDMI v1.3 Connector (Type A) to drive HD displays, DVI-D Connector,
3.5" audio in/out and HDMI Audio out
Full size SD/MMC card
Built in 802.11 & Bluetooth v2.1+EDR
On board 10/100 Ethernet
Expansion: 1xUSB OTG, 2xUSB HS host ports, General purpose expansion
header
25. What OS does Pandaboard support?
Debian
Ubuntu
Android
Fedora
Sabayon
Xubuntu
Angstrom
26. OpenCV
OpenCV grew out of an Intel Research initiative to advance CPU-intensive
applications.
The intent behind OpenCV was to provide a platform that a student can
readily utilize for developing applications, instead of reinventing the basic
functions from scratch.
27. Why use OpenCV?
It is free.
It is fast.
Good documentation in the form of documentation, tutorials, user groups,
forums etc.
Lot of pre-built functions and algorithms to get a head start.
There is an active development on interfaces for other languages like
Ruby, Python, Matlab etc.
Pretty fast.
28. License
OpenCV is released under a BSD license, it is free for both academic and
commercial use.
There are only a few algorithms inside it that may have been patented and
those are not free.
36. Using MATLAB
inbuilt function
Using our own
algorithm, we applied
linear differentiator. It
works fast and is more
efficient in
comparison to
MATLAB INBUILT
ALGO
36EDGE
DETECTION
37. Application
This project is applied mainly in the commercial field of helping visually
impaired people with lower eyesight or completely BLIND
Industrial applications can be devised and enhanced like robots and
machineries
Concept can be used for Autonomous Cars
For Security, Identifying and Tracking
37
38. Projects Implemented
1. Capturing Image using hand gesture and uploading it directly to Google Drive (Internet needed)
2. Face Detection and Person Identity Detection for running lifetime of prog
3. Optical Character Recognition
4. Cartoonifying Real Time Images
5. Colour Detection and Diff Sound for Diff Colour
6. Ball Detection
7. Head Pose Estimation
8. Augmented Reality based Position Change Tracking
9. Combined Project of OpenGL and OpenCV for Image Operation
10. Number Recognition in real-time
11. Text to Speech Synthesis using C++
12. Kinect with WINDOWS
13. Kinect with Ubuntu
14. Sound Marking
39. References
A collaborative European project CASBliP, or Cognitive Aid System for
Blind People
Andhajan Mandal, Ahmedabad, Gujarat, India
Electric lollipop that allows blind people to 'see' using their tongue-
Brainport Vision Device
OMAP4460 Pandaboard ES System Reference Manual
Neil Harbisson-
http://www.ted.com/talks/neil_harbisson_i_listen_to_color.html
AMTS-www.amts.co.in
39
