Wireless and uninstrumented communication by gestures for deaf and mute based...
paper2
1. Abstract—Searching for lost objects is a common
irritating activity. In this paper we explore possibilities for
a system to aid the searching process to find lost items. The
goal is to design a low cost solution for home and private
use. The system uses the directional sensitivity of the
human hearing to assist the search. A mobile phone is used
as a terminal to activate the sound emitting tags by
Bluetooth. To meet the low power requirements upcoming
“Bluetooth low energy” technology needs to be used,
however a prototype is developed to demonstrate the
functionality of the designed system with regular
Bluetooth.
Keywords—Bluetooth low energy, localizing, lost items,
mobile phone.
I. INTRODUCTION
When you are in a hurry, you quickly want to get out
of the door but your keys are lost. People frequently lose
valuable objects and spent a lot of time searching for
them. It would be great if you could quickly locate your
lost keys without a time consuming search. In this paper
the possibilities for such a system are explored and a
prototype is developed. The goal is to design a low cost
system for home and private use.
There are numerous former papers about localizing
objects with many different applications and design
goals. The first category is indoor locating systems using
wireless protocols like rfid [1], Wi-Fi[2] and Bluetooth
[3]. They all operate identically by calculating the
distance based on the received signal strength from
several fixed points. The accuracy of such a system is
approximately 2m, which is not accurate enough to find
lost objects in a room. Furthermore using an
infrastructure to determine the position of a tag increases
the cost of the system drastically, while it limits the area
where it can be used to that confined space. Therefor
these systems are mainly used to track items in a
warehouse or people in a building.
There are other methods to find an object than
determining its absolute position. The directional
sensitivity of the human hearing can be used to find an
object emitting sound. There are cheap products already
available that use this finding technique. The simplest
version uses sound not only to localize the tag, but also
for the activation of it [4]. The huge downside to the
whistle activation is that it generates a lot of false
positives. The beeping sound is not only activated when
you whistle but reacts on all high frequency sounds,
which occur very frequently.
An alternative system uses a button press on one tag to
activate the sound on all other tags[5]. As long as you
have one item with a tag on you this system works very
well. However,since it uses a button press to activate the
sound on all tags, there is no way of implementing any
kind of security. When widely used it would offer an
easy opportunity for thieves to locate valuable objects.
Therefore the choice in this paper was to use sound
emitting tags with the mobile phone as the activation
device. This creates a lot of technical possibilities for
extra features and security without introducing extra
costs. A system using the mobile phone as a terminal was
already developed before to aid the visually impaired [6].
The target group had difficulties finding objects in
everyday life, so this system has lower usability
requirements for the battery lifetime. The tags used in the
system need to be recharged every 2-3 weeks, this is
impractical for a system not used every day.
This paper addresses new technology in an attempt to
drastically improve the battery lifetime on the tags.
Making the same system a helpful aid for everyone
searching for objects every once in a while. Due to the
wider availability of smart phones there are extra features
possible in the terminal due to bigger screens, gps, more
computing power and internet availability.
Additionally not just your own phone could be used to
find a tag, but a whole network of mobile phones could
be instructed to localize a tag[7]. This could be integrated
in the proposed system in this paper, but storing the last
seen location with gps will give the same result for
objects not moved by others.
Finding lost objects with a mobile phone
Tom Derks
Technische Universiteit Eindhoven (TU/e)
Phone +31655134098
E-mail: t.h.m.derks@student.tue.nl,tderks@gmail.com
2. II. FORMULATION OF THE PROBLEM
A. General
The goal of the project is to design a system aimed at
consumers to quickly find lost items. This implicates a
low-cost solution with cheap or no infrastructure if
feasible. In this chapter different localization methods
will be discussed together with other requirements for the
system, like power consumption, security and protocols.
B. Localization methods
There are three basic methods described in literature to
locate an object. The most common method is through
trilateration. Distances are estimated by calculating the
time of flight (TOF) or from the strength of the received
signal (RSSI) from at least three fixed nodes. With those
three distances a location can be calculated. To
determine the TOF of a signal accurate clocks are needed
in all nodes. Especially for indoor applications with
small timing differences due to the small distances it’s
very cost inefficient. The RSSI measurement is available
in a lot of cheap standard transceivers. However mapping
the RSSI value to a distance is very problematic. A lot of
factors influence the signal strength such as reflections
and variation in output power. Especially indoors the
reflections greatly influence the accuracy of this system.
Triangulation uses the angles to the tag from three
nodes opposed to the distance used in trilateration.
Standard transceivers are not capable of determining the
angle of a signal. Furthermore both triangulation and
trilateration require an infrastructure with fixed nodes to
be able to measure the data needed to calculate the
position. This infrastructure is expensive and limits the
system to be used only in the area where this
infrastructure exists.
The third method to localize uses a directional antenna
to scan for the origin of a beacon signal similar to radar.
In combination with a distance measurement as done in
trilateration the relative position can be calculated. This
method does not need an infrastructure and can thus be
used anywhere. However it needs a transceiver at the
terminal which can search for the angle of origin of a
signal. This eliminates the use of a standard transceiver
which makes it an expensive solution.
Both the triangulation, trilateration and the directional
method need a terminal with a screen to display the
location of the tag relative to the terminal or display its
location on a map.
The three methods above give the position of the lost
item. The actual location of the item is not of interest, it
only needs to be found by a person. Therefore an
alternative to the previous localization methods is using
the sound locating ability of the human hearing. This
method is similar to the directional method with a beacon
signal. It uses sound as a beacon signal and the human
hearing as the directional receiver. As we all have
experienced, it is very easy to locate where a continuous
sound is originating from. This eliminates the need of an
expensive transceiver in the terminal. It only needs to
transmit a trigger signal to enable the beacon sound.
C. Terminal
All localization methods require a terminal to either
show the location of the item or to activate the sound
signal. There are also tags available using whistling for
activation [4]. These tags have a lot of false positives and
there is no way to implement any kind of security.
A better alternative is to use a terminal most people
already carry on them, the mobile phone. Usage of the
mobile phone as the terminal also solves the problem of
the terminal itself being lost, since it can be called and
found by using the same sound technique. Using the
mobile phone as the terminal does limit the available
wireless protocols. Manufacturers will not add a separate
transceiver for one new application task of the phone.
D. Protocol
The most used wireless protocols today are zigbee,
Bluetooth and wifi, ordered by increasing energy
consumption. The only protocols currently available in
phones are bluetooth and wifi. Since bluetooth
communication still requires significant power, zigbee
seemed a valuable protocol to add into phones. Nokia
developed a low power protocol based on bluetooth
called wibree. This protocol is a low power, stripped
version of the current Bluetooth protocol.
Although not backwards compatible it was designed to
easily make dual-mode (Bluetooth+wibree) chips for
little added cost. Wibree was later added to Bluetooth 4.0
protocol and renamed to Bluetooth Low Energy(LE).
This means that the next generation of bluetooth devices
will have this dual mode chip, capable of low power
communication. Bluetooth low energy chips also should
become cheap, since they are expected to be the RF
successor of infra-red remotes.
E. Power
The tags that will be attached to the items need to
consume little power. They need to function on small
batteries like a coin cell for at least a year. When using
3. coin-cell batteries maximum current limitations apply.
The maximum peak current for the widely used CR2032
is 15mA. Current class II bluetooth modules have peak
currents of around 80mA. Bluetooth LE is designed to
operate on coin cell batteries, thus having a peak current
of only 15mA. Together with the shortened connection
setup time of max 3ms compared to 1 second average for
normal bluetooth it saves a lot of power by decreasing
the duty cycle of the power consuming transmitting state.
F. Security
Since the tracking is generally put on valuable items,
security is needed to prevent unauthorized access to the
locating ability. Without this measure the device would
be an easy target for thieves searching for valuable items.
Therefore there needs to be a security step before
activating the buzzer sound. This can be easily
implemented with a secret key stored on the tag and the
terminal. The bluetooth protocol already has connection
pairing with a secret pin code build in.
III. RESULTS
A. General
To implement the object finder system the tag and the
software for the mobile phone terminal have to be
developed. Bluetooth low energy chips are not available
yet. Therefore a prototype is developed with a normal
Bluetooth module, dropping the low power requirement
for the tag. The system architecture will be designed to
meet the power requirements when the normal Bluetooth
module is replaced with a low energy type.
The system consists of one or multiple tags and a
single terminal. The tag will be attached to the object that
needs to be found later. The terminal is a bluetooth
enabled smart-phone with a software interface to enable
or disable the buzzer on the tag. When the buzzer is
enabled the tag can easily be found if the object is within
hearing range. A block diagram of the system can be
seen in Figure 1.
B. Tag
The tag needs to turn on the buzzer sound when it is
activated with the phone terminal. The tag consists of a
buzzer to generate sound, a microcontroller for the logic
and a bluetooth module for the wireless communication.
As mentioned before Bluetooth low energy chips are
not available yet. Because a normal class 1 bluetooth
module (LM400) is used in the tag, dropping the low
power requirement for the demo model. For the logic the
simplest ic from Microchip is used with a hardware uart
port and debugging capabilities available (16F88).
The ic can be programmed and debugged in circuit by
using the PICKIT ICD2 usb programmer. The software
for the micro controller is written in C. With the ‘Hitech
C compiler in MPLAB’ the C code can be converted to
assembly code running on the chip.
The ic uses its hardware uart interface to communicate
with the Bluetooth module. The Bluetooth module can
switch between command and data mode. In data mode
the uart interface is a transparent Serial Port Profile
(SPP) Bluetooth connection. When in command mode,
AT commands can be send to configure the Bluetooth
device and manage the Bluetooth connections on the
device.
Bluetooth
Tag
Micro-
Controller
PIC16LF88
Buzzer
Bluetooth
Module
LM-400
UART
Mobile
Phone
(Bluetooth
enabled)
Figure 1: Block diagram of the system
4. Vdd
Buzzer
6
7
8
Rx
9
13
12
11
10
2
3
4
5
GND
A0
Vdd
17
16
15
14
1 18
16F88-L
MCLR
PGD
PGC
Tx
1 7
Rx
3
4
5
6
Vdd Gnd
9
10
11
12
Tx
2 8
LM-400
PGD
PGC
MCLR
ICD2
Figure 2: Electric scheme of the tag
Battery consumption on the phone is less of an issue
than it is for the tag, since phones have a much larger
battery capacity and can be charged frequently. To
preserve energy in the tag it does not listen for incoming
connections, but polls the phone to check if the buzzer
needs to be enabled. This way the tag’s Bluetooth
module can be turned off to sleep mode most of the time,
which consumes much less power than listening in
standby mode.
When the tag comes out of sleep mode to poll for the
activation status it sends the AT command to connect to
the paired mobile phone. After the device is connected
the Bluetooth module is put in data mode to request the
status message for the sound activation. Based on the
received status the buzzer is turned on or off. The
Bluetooth module is switched back to command mode to
send the disconnect command and goes back to sleep
mode.
C. Phone
To use the mobile phone as the terminal, custom
software needs to be installed on the device. Java
programs have been supported for a long time on almost
all common phones. Recent developments in
smartphones extended the features and possibilities of
these java applications (apps) with more extensive API’s,
callback hooks and background processes. Android has
been chosen as the platform since it’s the most widely
available smartphone operating system.
After the application is paired with the tag a
background Bluetooth listening service is started to listen
for incoming connections. When it receives the polling
messages from the tag it sends back the status message to
the tag. Based on the received status message the buzzer
on the tag is activated or deactivated.
D. Power
Although the power requirement for the prototype was
dropped, the goal was still to meet the power
requirements by only replacing the bluetooth module.
The preliminary product specifications of upcoming
Bluetooth chips like the nRF8001 are already available.
With the information of the datasheet the future power
consumption can be calculated. This example chip does
not have an included microcontroller, but current
consumption for a simple build in microcontroller will be
negligible. Therefore the formula for the average current
consumption is [8].
𝐼 𝑎𝑣(𝑇) =
( 𝐶𝑐𝑜𝑛 + 𝑃𝑖𝑝𝑒 𝑎𝑣𝑔)
𝑇
+ 𝑃 ∗ 𝐼 𝑅𝑋 𝑝𝑝𝑚 + 𝐼𝑖𝑑𝑙𝑒
𝐶 𝑐𝑜𝑛 = 14𝜇𝐶 Average charge of a connection event
𝑃𝑖𝑝𝑒 𝑎𝑣𝑔 = 14𝜇𝐶 Average charge of an active service
pipe
𝑃 = 300𝑝𝑝𝑚 𝑇𝑜𝑡𝑎𝑙 crystal tolerance
𝐼 𝑅𝑋 𝑝𝑝𝑚 = 0.015 𝜇𝐴 Additional current drain
With the average current consumption, the battery
duration using a standard coin-cell battery (CR2032) can
be calculated. This coin-cell battery has a capacity of
220mAh. In Figure 3 the battery duration is shown as a
function of the polling interval. With a polling interval of
12 seconds the battery lasts 3 years. Increasing the
interval gives little extra battery time, while it increases
the beep delay of the tag. When the tag is activated on
the phone, the worst case delay for the tag to beep is the
polling interval.
Figure 3: Battery duration of the tag
0
0.5
1
1.5
2
2.5
3
3.5
4
0 2 4 6 8 10 12 14 16 18 20 22 24
BatteryDuration(years)
Interval (s)
5. E. Buzzer tone frequency
For the buzzer the frequency best localizable by the
human hearing should be chosen to optimize the search.
The brain does the localization by comparing the sound
waves entering both ears. The human hearing does sound
localization with three different methods. These methods
can be divided in two categories. The first category
contains the interaural time differences (ITD). Sound
from the right side reaches the right ear earlier than the
left ear. The auditory system evaluates interaural time
differences from phase delays at low frequencies and
group delays at high frequencies. The second category is
the interaural level differences (ILD). Sound has a higher
level at the ear closest to the sound source. These level
differences are highly frequency dependent and increase
with increasing frequency. The best localizable sound
appears to be clicking sounds in the 2-8kHz
frequency range [9]. With such a signal the human
hearing can depend on time onset as well as level
differences.
F. Map location
When the tag is not in range of the terminal it cannot
be found by sound activation. Since most smart-phones
have a gps sensor, the location where the terminal lost
contact with the tag can be stored. When the phone stops
receiving the polling messages, the gps receiver is turned
on to store the current location. When the tag is out of
range to enable the buzzer locating method, the
approximate location can be displayed on google maps to
give the user an idea where their tag is (figure 4). It uses
the current gps location and auto-scales to show both the
location of the phone and the tag on the map
Figure 4: Last location visible on google maps
G. Other functionality
There are numerous other functionalities possible with
the system. Features can be added to the phone terminal
and tags to keep them together, so you always have all
your valuable items on you. When the tag goes out of
range or below a certain signal strength, the phone could
beep, reminding you not to forget your keys. This could
also be implemented the other way around; the tag will
beep to remind you to take your phone.
IV. CONCLUSION
A working prototype for the tag has been developed
with a corresponding mobile app so any android phone
can be used as a terminal. The power requirement on the
tag can be met with the current architecture as soon as
bluetooth low energy chips become available. Since
bluetooth low energy chips are likely to become cheap,
the goal of a low cost localizing system with a long
battery life time is achieved.
The beeper activation/deactivation and the map
location feature are implemented in the android app.
Additional features of the section “Other functionaly”
could still be added to this program.
V. RFERENTIES
[1] Guang-yao Jin, Xiao-yi Lu, Myong-Soon Park, “An Indoor
Localization Mechanism Using Active RFID Tag”, Department
of Computer Science and Engineering, College of Information
and Communications, Korea University, Seoul 136-701, Korea,
2006
[2] Chin-Heng Lim, Yahong Wan, Boon-Poh Ng, Chong-Meng
Samson Se, “A Real-Time Indoor WiFi Localization System
Utilizing Smart Antennas“, NTU, Singapore, 2007
[3] Miguel Rodriguez, Juan P. Pece, Carlos J. Escudero,“In-building
location using Bluetooth”, Departamento de Electr´onica e
Sistemas, Universidade da Coru˜na, 2005
[4] Keyfinder, http://www.seenontvproducts.net/keyfinder/
consulted on 05.09 2010.
[5] KeyRinger, http://www.keyringer.com/ consulted on 05.09 2010.
[6] Julie A. Kientz, Shwetak N. Patel, Arwa Z. Tyebkhan, Brian
Gane, Jennifer Wiley, Gregory D. Abowd, “Where’s My Stuff?
Design and Evaluation of a Mobile System for Locating Lost
Items for the Visually Impaired”, Georgia Institute of
Technology, Atlanta, Georgia, USA, 2006
[7] Christian Franka, Philipp Bolligera, Friedemann Matterna,
Wolfgang Kellererb, ”The sensor internet at work: Locating
everyday items using mobile phones” Institute for Pervasive
Computing, ETH Zurich, Switzerland, 2007
[8] nRF8001 Preliminary Product Specification,
http://www.nordicsemi.com/eng/nordic/download_resource/9215
/1/65440951, Page 39, consulted on 05.09 2010.
[9] Micheal D. Mann, “The nervous system in action”, page 8-15,
2007