1. Usability Study of the Finger Counter, a
Human-Computer Interface
Christen Ng and Jessie Burger
CRA DMP Project at Boston University
Abstract
The Finger Counter is a computer-vision system that identifies the number of fingers held up in
front of an inexpensive webcam in real time. Usability studies were conducted to determine the
limitations of the system. Tests performed included: a voice-interactive test, a drawing test, and a
questionnaire that gauged the ease of use of the system, as well as a limitations test that
established conservative functional ranges for use of the Finger Counter.
Introduction
Finger Counter is a computer-vision system that counts the numbers of fingers held up in front of
a video camera in real time. The system is designed as a simple and universal human-computer
interface: potential applications include educational tools for young children and supplemental
input devices, particularly for persons with disabilities. The interface is language independent
and requires minimal education and computer literacy. Finger Counter uses background
differencing and edge detection to locate the outline of the hand. The system then processes the
polar-coordinate representation of the pixels on the outline to identify and count fingers: fingers
are recognized as protrusions that meet particular threshold requirements. The system also logs
the frequency of different inputs over a given time interval. The Finger Counter interface was
implemented under Linux using Video4Linux. The system was tested extensively under various
lighting and background conditions to find the most favorable environment for the Finger
Counter to function. Such varying conditions were also conducive to establishing limitation
criteria for the system. The following is a compilation of the various tests conducted, their
description, and the results obtained:
I. Tests Performed
EXPERIMENT 1 - Voice Interactive Test
Demonstration and Explanation
Test subjects were given a brief introduction to the Finger Counter: how it works, its intended
purpose, the types of tests they would complete, and a demonstration on how to use it. They were
then given the opportunity to familiarize themselves with the program by playing around with it
before the formal testing began. During this trial period, advice was given to the participants as
to how to most effectively use the Finger Counter (e.g. hands parallel to camera, maximizing the

2. width bet
Once the
on the fo
minutes.
given onc
What fol
user. The
right win
shows ho
The test s
beyond in
hand pos
response
complied
ignored b
random t
asked to
failed to
Test Adm
tween their f
e subjects fel
llowing test
When test s
ce.
lows is a scr
e window dir
ndow shows
ow many fin
system was
nitiating the
se or type the
time in seco
d with by the
by the system
time interval
remove thei
comply with
ministration
fingers, orien
lt sufficiently
s were recor
ubjects were
reenshot of t
rectly below
raw video. T
gers the syst
completely a
program. Fo
e key on the
onds; the log
e deadline (th
m. After a re
l between 4 a
r hands from
h this reques
n
ntation and r
y comfortabl
rded. The dem
e part of a gr
the test in ac
w it shows the
The bottom r
tem recogniz
automated an
ollowing a r
keyboard; th
g files also d
he failures).
ecognition w
and 10 secon
m the field of
st.
rotation of h
le interacting
monstration
roup, the dem
tion. The top
e frame rate
right window
zes and the l
nd did not re
equest, a use
he system lo
etermined th
Misidentifie
was made or t
nds before m
f view betwe
hand placeme
g with the pr
n and explana
monstration
p left window
and other st
w is the "use
location of th
equire action
er was given
ogged the req
he number o
ed poses or i
the timer ran
making anoth
een tests; oft
ent in regard
rogram, thei
ation took be
and explana
w is for mes
tatus informa
er feedback w
heir fingertip
n from the ad
n up to 5 seco
quest made a
f requests th
incorrect key
n out, the sys
her request. U
tentimes, ho
ds to the cam
ir performan
etween 2 to
ation was on
ssages to the
ation. The to
window," wh
ps.
dministrator
onds to mak
and the user'
hat were not
y presses we
stem waited
Users were
wever, they
mera).
nces
5
nly
e
op
hich
r
ke the
's
ere
a

3. Each user was permitted some time to play with the drawing program before the test
administration. There was no time limit; users took from 30 seconds through approximately 3
minutes to try out the interface before going on to the test. In the "voice-interactive test," the
computer plays a recorded message asking the user to hold up a certain number of fingers or else
type a key from "1" through "5" on the keyboard. The audio message is supplemented by a
screen message in large type. A typical message might say "please hold up one finger" or "please
type 2 on the keyboard." To choose which message to play, the system generated a random
number from 1 through 10. Numbers from 1 through 5 would cause the system to ask for as
many fingers. Numbers from 6 through 10 would cause the system to request keys "1" through
"5" on the keyboard. Random numbers were chosen "with replacement," so a single test subject
could be asked to hold up some hand poses, or type some keys, a number of times and never be
requested to hold up other hand poses, or type other keys. There was no set number of requests;
some users got as few as 10 requests, while one got 33 requests. The test system was automated
and users were given up to 5 seconds to respond.
The Test Subjects
There were a total of 19 test subjects: 6 high school students, 4 college students, 7 graduate
students, and 2 professionals. The majority of college students, graduate students, and
professionals were acquaintances of the administrators. The high school students were
participants in a math and science summer camp. Many participants had strong technical
backgrounds, as their computer experience ranged from 6 to 30 years and daily computer use
ranged from 1 to 15 hours. Each test subject completed exactly one test.
Test Conditions
Multiple locations were used to conduct testing. The graduate students and professionals were
tested under florescent lighting, with the camera on a tripod looking up at a ceiling about two
meters above. The high school kids were tested in the graduate lounge, under florescent and
incandescent lighting, with the camera mounted on a tripod facing down half a meter from the
table. Subjects made hand gestures in the air above the camera when it was facing upwards and
on the table surface below the camera when it was facing downwards. The program ran on a
Toshiba Satellite Pro 6000 with an Intel Pentium III Mobile 1.2 GHz processor and 512 MB of
RAM. The Pentium III has a bus speed of 133 MHz.
EXPERIMENT 2 - Drawing Test
Demonstration and Explanation
The same test subjects participating in Experiment 1 performed this test, with the addition of five
more college students. They received the instructions described above. In addition, they were
told about the "drawing test," in which they would be asked to trace a circular template on the
screen, first using the Finger Counter interface and then using a computer pointing device.
Test Administration

4. Each Exp
interactiv
voice-int
The draw
fingers. T
how man
finger, th
correspon
finger, th
draws a b
varies wi
system d
boxes are
camera is
black. Th
For the "
of a whit
match the
colored b
pressed a
template,
way arou
reported
the circul
The follo
the circul
periment-2-t
ve test. Thus
teractive test
wing program
The program
ny fingers are
he system dra
nding to the
he boxes may
box centered
ith the distan
raws three o
e held up, th
s processed.
his keeps the
drawing test
te circle. Eac
e circular tem
boxes lined u
a key on the
, and the test
und, the test
the average
lar template.
owing is a sc
lar template.
test subject t
s, each user a
t before takin
m permits a u
m works as fo
e held up, it
aws a small,
position of t
y appear as a
d on the midp
nce between
or four small
e system era
Also at each
e screen from
t," instead of
ch test subjec
mplate. A te
up with some
keyboard to
t subject atte
administrato
difference b
.
creen shot of
. The right w
took the draw
already had p
ng the drawi
user to "draw
ollows: The
tracks the fi
, randomly c
the fingertip
a line or othe
point betwee
the fingertip
boxes corre
ases the scree
h frame, the
m becoming
f being comp
ct was asked
est subject fir
e point on th
initiate the
empted to tra
or pressed an
between each
f the drawing
window show
wing test imm
played with
ing test.
w" on the com
screen is ini
ingertip posi
colored box (
p in the field
er object. If
en the two fi
ps. If the use
esponding to
en. New box
system fade
overly clutte
pletely black
d to draw a c
rst moved hi
he circle outl
test. The scr
ace the circle
nother key, h
h pixel in the
g test. The le
ws the "user
mediately af
the drawing
mputer scree
itially black.
itions. If the
(essentially a
of view. As
two fingers
ingers; in thi
er holds up th
o the fingertip
xes are draw
es each color
ered with co
k, the initial
circle with hi
is or her fing
line. Then, o
reen cleared,
e. When the
halting the te
e user's draw
eft window s
feedback" w
fter taking th
g program an
en by movin
Once the sy
user is hold
a large pixel
s the user mo
are held up,
is case, the s
hree or four
p positions.
wn as each fra
red pixel a li
olors.
screen conta
is or her inde
gertip until th
one of the tes
, except for t
test subject
est. At that p
wing and the
shows the us
window.
he voice-
nd taken the
ng his or her
ystem determ
ding up one
l) on the scre
oves his or h
the system
size of the bo
fingers, the
Finally, if fi
ame from th
ittle toward
ained the out
ex finger to
he resulting
st administra
the circular
made it all t
point, the sys
nearest poin
ser's drawing
mines
een
her
ox
ive
he
tline
ators
the
stem
nt on
g and

5. After the "drawing test" with the Finger Counter, another drawing test was conducted using a
computer pointing device. Available devices included a laptop touchpad, a laptop stick pointer,
and a trackball pointing device. Each user was asked to choose the pointing device most
unfamiliar. As with the Finger Counter drawing test, the screen displayed a circular template to
which a test subject maneuvered the screen pointer (a commonly used arrow). Once the arrow
was pointing at part of the template, one of the test administrators began the test, at which point a
series of small colored boxes trailed the screen pointer as the user traced the circular template.
When the test subject had circumnavigated the template, the administrator stopped the test. The
system then computed the average distance using the same method as described above.
The Test Subjects
The test subjects for Experiment 3 included those described above in Experiment 2, as well as an
additional five college students. As with Experiment 2, each test subject completed exactly one
test. Three college students opted to use the AccuPoint input device as their unfamiliar pointing
device, one subject did not take this test, and the remaining elected to use the trackball.
Test Conditions
The test conditions were the same for the previously listed subjects. The three additional college
students were tested in their apartment under incandescent lighting, with the camera mounted on
a mini-tripod facing down, approximately half a meter above the table. One of the test subjects
completed the test under natural lighting conditions, aided by a portable lamp, with the camera
mounted on a tripod facing upwards.
EXPERIMENT 3 - Questionnaire
The test subjects who participated in Experiments 1 and 2 were asked to complete the following
questionnaire. Test subjects took from 1 to 5 minutes to do so.
Name ___________________________________________________
Telephone Number or Email
(in case we have follow-up questions) __________________________
Occupation _______________________________________________
How many years of computer experience do you have? years
How many hours per day do you use a computer (on average)? hours
How easy did you find a computer mouse to use (1 = very hard 10 = super easy)? 1 2 3 4 5 6 7 8 9 10
How natural to use is a mouse (1 = completely unnatural 10 = completely
intuitive)?
1 2 3 4 5 6 7 8 9 10
We asked you in the test to use a pointing device you were 1 2 3 4 5 6 7 8 9

6. unfamiliar with. How easy did you find the other pointing device to
use (1 = very hard 10 = super easy)?
10
How natural to use is the other pointing device (1 = completely
unnatural 10 = completely intuitive)?
1 2 3 4 5 6 7 8 9
10
How easy did you find the Finger Counter to use (1 = very hard 10
= super easy)?
1 2 3 4 5 6 7 8 9
10
How natural to use is the Finger Counter (1 = completely unnatural
10 = completely intuitive)?
1 2 3 4 5 6 7 8 9
10
We would appreciate any comments you might have on the Finger Counter.
In particular, what types of applications do you think would be useful with the Finger Counter?
Do you have any comments, criticisms, or suggestions regarding the Finger Counter or this
testing procedure?
Thanks for your time.
EXPERIMENT 4 - Limits of Pose Recognition
For this test, we placed a webcam in settings similar to the previous experiments, i.e., in the
graduate computer lab, mounted on a tripod which was placed on the floor with the camera
facing upwards. We ran the interface and had it capture frames as well as report poses
recognized. Hands were moved in the following ways:
1. Toward the camera
2. Away from the camera
3. Rotation in the direction of each Euler angle on axes going through the center of the palm
a. "Pitch": The hand is rotated so the fingertips are closer to the camera than the
palm and vice versa.
b. "Roll": The hand is rotated so that the side of the hand including the base of the
little finger is closer than the side including the base of the thumb and vice versa.
c. "Yaw": The hand is rotated in the plane parallel to the image plane, so that, from
the camera's perspective, the fingertips move from side to side while the palm
remains more or less fixed.
A digital video camera was set up on a tripod next to the webcam and was used to capture still
images of hand positions for "out of plane" rotations, i. e. pitch and roll. A hand was placed over
the camera so that the hand was oriented parallel to the camera lens, perpendicular to the bottom
of the image frame, and the system properly recognized the hand pose. All tests (distance, pitch,
roll, and yaw) were performed on the 3 test administrators using all 5 hand positions.
To determine the limits on the range of the distance of the hand position, the hand was moved

7. toward th
hand pos
hand pos
positions
camera u
the close
To determ
remainin
these ext
the last "
measured
the hand,
hand pos
or 5 finge
negative.
To determ
either clo
was kept
was set u
could be
the Finge
positions
the horiz
the point
average f
as positiv
he camera un
sition just be
sition just aft
s was also m
until recognit
st "bad" han
mine the ran
ng parallel to
remes at the
good" positi
d from the ve
, to a line seg
sitions using
ers. The yaw
.
mine the ran
oser to or far
perpendicul
up so that the
clearly dete
er Counter an
s were then a
ontal axis, w
where the th
fingertip pos
ve.
ntil recogniti
fore failure o
ter failure of
measured usin
tion failed, a
nd position, w
nge of the yaw
the camera
e point just b
ions were the
ertical axis o
gment drawn
1 finger, an
w angle in th
nge of the pit
rther away fr
lar to the bot
e camera len
rmined. Ima
nd the digita
analyzed usin
which was al
humb joins t
sition was us
ion failed. A
of the system
f the system,
ng a tape me
and two imag
were recorde
w of hand po
lens, until re
before recogn
en analyzed
of the image
n from the c
d to the tip o
e right direc
Yaw
tch of positio
rom the cam
ttom of the i
ns was parall
ages of the "g
al camera. Th
ng a softwar
long the defa
the hand to t
sed. Downw
An image wa
m, i.e. the clo
i.e. the farth
asure. The h
ges, one of t
ed, as well as
ositions, the
ecognition fa
nition failure
using a soft
, which was
enter of the
of the middle
ction was reg
w Example
ons, the hand
mera lens than
image, until
el to the side
good" and "b
he images fr
re angle mea
ault hand pos
the fingertips
ards pitch w
as taken, usin
osest possib
hest "bad" p
hand was the
the farthest "
s the distanc
e hand was ro
failed. An im
e and the poi
tware angle m
the same as
palm to the
e finger for h
garded as po
d was tilted
n the palm, w
recognition
e of the hand
bad" position
rom the digit
asuring tool.
sition, to a li
s. If the fing
was regarded
ng the Finge
le "good" po
osition. The
en moved aw
"good" hand
ces.
otated left an
mage was tak
int just after
measuring to
s the default
tip of the ind
hand positio
sitive, and th
so that the fi
while the han
failed. The
d and the ang
ns were capt
tal camera o
Angles wer
ine segment
gertips were
as negative,
r Counter, o
osition, and t
distance at
way from the
d position, on
nd right, whi
ken of both o
. The image
ool. Angles w
orientation o
dex finger fo
ons using 2, 3
he left as
fingertips we
nd orientatio
digital came
gle of pitch
tured using b
of the last "go
re measured
which ran f
not even, the
, upwards pi
of the
the
these
e
ne of
ile
of
s of
were
of
or
3, 4,
ere
on
era
both
ood"
from
from
e
itch

8. To determ
circular f
the came
up so tha
Images o
digital ca
analyzed
which wa
of the han
index fin
knuckles
a referen
positive.
II. Test
EXPERI
The follo
mine the ran
fashion (so th
era lens and p
at the lens wa
of the "good"
amera. The im
d using a soft
as along the
nd from the
nger if less th
or middle o
ce point. Le
Results
IMENT 1
owing table s
nge of the rol
hat the palm
perpendicula
as viewing th
" and "bad" p
mages from
tware angle m
default "lev
little finger
han 5 fingers
of the fingers
ft roll was re
shows the re
Pitc
ll of hand po
m of the hand
ar to the bott
he fingertips
positions we
the digital c
measuring to
vel" hand pos
to the thumb
s were held u
s, whichever
egarded as n
Ro
esults of the v
ch Example
ositions, the
d was turned
tom of the ca
s and the ang
ere captured
camera of the
ool. Angles w
sition, to a li
b if 5 fingers
up. If the han
r best represe
negative, whi
oll Example
voice-interac
hand was ro
up) while k
amera image
gle of roll co
using both t
e last "good
were measur
ine segment
s were in use
nd position u
ented the pla
ile roll to the
ctive test de
otated left an
keeping the h
e. The digita
ould be clear
the Finger C
" positions w
red from the
which ran th
e, or the littl
used less tha
ane of the ha
e right was r
scribed in E
nd right in a
hand parallel
al camera wa
rly determine
Counter and t
were then
e horizontal
hrough the p
e finger to th
an 4 fingers,
and, was use
regarded as
xperiment 1
l to
as set
ed.
the
axis,
plane
he
the
ed as
:

9. Prior to c
testing po
the make
Experime
RAM. Th
be attribu
large chu
bus, used
Experime
second fo
Experime
expect th
experime
The follo
in the pri
response
Finger Co
Keyboard
conducting E
ool. The only
e of the mach
ent 1 was ru
he Pentium I
utable to the
unks of data,
d for the earl
ent 1. The fr
or the earlier
ent 1. This d
hat the confid
ent and 0.44
owing chart s
ior experime
time while f
ounter
d
Experiment 1
y difference
hine running
n on a Dell I
IV has a bus
bus speed o
for instance
lier experime
rame rate of
r conducted
difference wo
dence interv
for Experim
shows the av
ent and Expe
five fingers
Figure 1
Mean
3.20s 2.9
2.21s 1.9
1, Stephen C
in the testin
g the program
Inspiron 820
s speed of 40
on the testing
e, images, co
ent, is three
the Finger C
experiment
ould very lik
al would be
ment 1.
verage respo
eriment 1. In
had the long
. Response T
95% Con
98s
99s
Crampton adm
ng protocols
m. The test p
00 with an In
00 MHz. The
g machines.
ould be sent
times faster
Counter inter
and approxim
kely affect th
similar, and
onse time by
n both experi
gest.
Times for the
nfidence Int
3.42
2.49
ministered th
were the tes
program on t
ntel Pentium
e difference
The bus spe
to and from
than the Pen
rface was ap
mately 6 fra
he mean resp
d, in fact, it w
finger for th
iments, one f
e Finger Cou
terval Range
2s
9s
he same test
sting conditi
the testing do
m IV 1.4 GHz
in results (S
ed would af
the CPU. Th
ntium III bus
pproximately
ames per sec
ponse time. O
was: 0.42s fo
he Finger Co
finger had th
unter
e
t to a differen
ons, specific
one prior to
z and 256 M
See Figure 1)
ffect how fas
he Pentium
s, used for
y 10 frames p
ond for
One would
or the earlier
ounter interf
he shortest
nt
cally
MB of
) may
st
IV
per
r
face

10. EXPERI
For the d
12.6 pixe
standard
polygons
dotted lin
and plus
EXPERI
From the
per day r
shows th
"naturaln
Counter w
IMENT 2
drawing test,
els for the Fi
deviations w
s show the m
nes connect t
show outlier
IMENT 3
e questionnai
ranged from
e mean and
ness." The m
was the next
Mouse
Finger C
Other Po
the average
inger Counte
were 9.8 pixe
middle quarti
the nearest s
rs.
ire response
1 to 15, with
standard dev
mouse ranked
t "easiest to
Counter
ointing Devi
e distance be
er and 6.3 pi
els and 3.8 p
iles (25% an
samples with
s, the years o
h a spike (6 r
viation of the
d very high f
use" and sec
ice
tween the dr
xels for the
pixels. Figur
d 75% confi
hin 1.5 times
of computer
respondents
e 1-to-10 ran
for both ease
cond most "n
Ease of Us
Mean
9.0
7.0
6.5
rawing and t
other pointin
re 3 shows a
idence) surro
s the inter-qu
r experience
) at 8 hours
nkings for "e
e of use and "
natural."
se "
SD Mea
1.3 9.2
2.2 7.0
2.1 6.1
the circular t
ng device. T
box plot of
ounding the
uartile range
ranged from
per day. The
ease of use"
"naturalness
"Naturalness
an SD
0.9
2.5
2.3
template was
The respectiv
the results. T
median, the
es, and the ci
m 6 to 30. Ho
e following t
and
s," the Finger
s"
D
9
5
3
s
ve
The
ircles
ours
table
r

11. EXPERI
The follo
Counter f
finger, w
value. Th
hand pos
IMENT 4
owing graphs
functions in
which each re
hese then det
stures, which
s display the
regards to o
ed bar displa
termined the
h is shown by
e most conse
our limitation
ays as the sm
e overall rang
y the shaded
ervative rang
ns test. A ran
mallest maxim
ge in which
d gray area.
ge of hand po
nge was foun
mum value a
the Finger C
ositions in w
nd individua
and the large
Counter func
which the Fin
ally for each
est minimum
ctioned for a
nger
h
m
all