Presentation of the paper "Blind Children Navigation through Gaming and Associated Brain Plasticity" Virtual Rehabilitation International Conference 2009, Haifa, Israel

1. Blind
Children
Naviga/on
through
Gaming
and
Associated
Brain
Plas/city
Jaime
Sanchez,
Angelo
Tadres,
Alvaro
Pascual-­‐Leone,
Lo:i
Merabet

2. Outline
• Introduc?on
• Research
purpose
• SoBware
descrip?on
• Usability
evalua?on
• Brain
plas?city
• Conclusion
• Future
Work

3. Introduc/on
• For
children
who
are
blind,
orien?ng
oneself
and
moving
about
in
an
unfamiliar
environment
is
an
experience
that
can
be
complex
and
extremely
variable
(Kulyukin
et
al.,
2004)
.
• People
who
are
blind
prefer
to
navigate
using
the
room’s
perimeter
(“shorelining”)
rather
than
the
center
of
a
room.
This
way
of
exploring
the
environment
can
lead
users
to
find
inefficient
solu?ons
to
problems
(Kulyukin
et
al.,
2004;
Sánchez
&
Elías,
2007;
Lahav
&
Mioduser
2004)
.

4. Introduc/on
• Having
a
mental
map
of
the
space
is
fundamental
for
the
efficient
development
of
orienta/on
and
mobility
techniques.
(Piaget,
1954,
1956,
1962)
• One
of
the
possibili?es
for
assis?ng
in
blind
naviga?on
is
through
the
use
of
audio-­‐based
games.
There
are
a
variety
of
studies
that
highlight
the
importance
of
the
use
of
video
games
for
learning.
(Squire,
2003;
Steinkuehler,
2004)
• The
possibility
of
using
educa?onal
video
games
opens
enormous
possibili?es
for
working
with
learners
who
are
blind.
It
provides
the
opportunity
to
develop
skills
that
might
be
more
complicated
(such
as
naviga/on)
and
to
do
so
in
a
mo/va/ng
and
challenging
mode.
(Go
&
Lee,
2007)

5. Research
purpose
• Design,
implement
and
evaluate
an
Audio-­‐based
virtual
Environment
Simulator
(AbES)
for
the
development
and
use
of
blind
users’
orienta?on
and
mobility
skills
in
closed
and
unfamiliar
spaces.
• Analyze
concrete
possibili?es
for
using
this
applica?on
to
study
changes
in
brain
ac?vity
during
naviga?on
through
gaming
combined
with
advanced
techniques
of
neuroimaging
and
neuroscience.
• Understanding
of
the
neurobiological
underpinnings
of
naviga?on
learning
and
the
impact
of
gaming
will
allow
the
design
of
biologically
op?mized,
neuroplas?city-­‐guided
gaming
interven?ons
for
neurorehabilita?on
and
learning.

6. SoPware
Descrip/on
• Simula?on
modes:
– Free
Naviga?on
– Path
Naviga?on
– Game
Mode

7. SoPware
Descrip/on
• Simula?on
modes:
– Free
Naviga/on
– Path
Naviga?on
– Game
Mode

8. SoPware
Descrip/on
• Simula?on
modes:
– Free
Naviga?on
– Path
Naviga/on
– Game
Mode

9. SoPware
Descrip/on
• Simula?on
modes:
– Free
Naviga?on
– Path
Naviga?on
– Game
Mode

10. Open
Door
Closed Door
Wall
Jewel
Static Object
Empty
Stairs
Player Monster
SoPware
Descrip/on

11. SoPware
Descrip/on
To Press Descrip/on
Turn
LeB H To
hear
the
verbalized
audio
of
the
cardinal
direc?on
you
are
facing
aBer
having
turned
leB.
Turn
Right K To
hear
the
verbalized
audio
of
the
cardinal
direc?on
you
are
facing
aBer
having
turned
right.
Walk Space
If
it
is
possible
to
advance,
the
sound
of
a
footstep
will
be
heard.
Another
sound
will
be
heard
that
signifies
having
bumped
into
something
if
advancing
is
not
possible.
Ac?on J This
command
is
used
to
ask
what
is
in
front
of
you,
as
well
as
to
open
doors.
Ask
Room F
This
command
allows
the
blind
user
to
know
in
what
room/floor
he/she
is
located,
and
provides
informa?on
about
his/her
current
orienta?on
and
informa?on
about
the
task
that
must
be
completed,
(if
any).

12. SoPware
descrip/on
• Development
– MicrosoB
Visual
Studio
.Net,
using
C#
language
– MicrosoB
.Net
Framwork
2.0
– MicrosoB
Windows
XP
• Minimal
Hardware
Requirements
– 512
Gb
Ram
– Intel
Pen?um
IV
processor
– 10
Mb
in
HD
– Stereo
Speakers
– Keyboard
– Mouse

13. Usability
Evalua/on
-­‐
Sample
• Six
children
ages
9
to
11
who
anend
the
Santa
Lucía
School
for
blind
children
in
San?ago,
Chile
• Was
made
up
of
2
girls
and
4
boys
• None
of
them
had
any
other
neurological
deficits
• 4
had
low
vision
and
2
were
blind
since
birth

14. Usability
Evalua/on
-­‐
Instruments
• SoBware
Usability
Elements
(SUE)
Ques?onnaire
was
used,
which
allowed
us
to
quan?fy
the
degree
to
which
the
sounds
used
in
the
simulator
were
recognizable
• Open
Ques?ons
Usability
(OQU)
Ques?onnaire
was
applied
to
the
users,
which
included
ques?ons
such
as:
What
informa?on
provided
in
the
soBware
allows
the
recogni?on
of
the
space?
Were
you
able
to
move
about
easily?
Why?
Could
you
form
an
image
of
the
space
traveled
through
the
sounds?
How?
• SoBware
Usability
for
Blind
Children
Ques?onnaire
(SUBC)
(Sánchez,
2003)
This
ques?onnaire
consists
of
18
items
for
which
the
users
must
define
to
what
degree
each
of
them
was
fulfilled,
using
a
scale
ranging
from
1
("a
linle")
to
10
("a
lot").

15. Usability
Evalua/on
-­‐
Procedure
• The
SUE
ques?onnaire
was
completed
during
20-­‐minute
sessions.
With
this
results,
the
soBware
was
redesigned
to
improve
the
soBware's
audio
interface
• With
this
new
version
of
AbES,
the
SUBC
was
completed
in
individual
45-­‐minute
sessions
– AbES
was
formally
presented
to
the
user
– Explaning
to
user
the
modes
of
interac?on
– The
keyboard
commands
and
the
task
that
user
had
to
perform
is
explain
• The
user
proceeded
to
answer
the
OQU
ques?onnaire

16. Usability
Evalua/on
-­‐
Results
• Results
of
the
SUE
ques?onnaire
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Step
on
the
floor
Step
on
the
stair
Sta?c
object
Wall
Closed
door
Open
the
door
End
of
task
Found
jewel
Monster
Ac/on
or
Object
Iden?fy
Don't
Iden?fy

17. Usability
Evalua/on
-­‐
Results
• Results
of
the
SUBC
ques?onnaire
1
2
3
4
5
6
7
8
9
10
Sa?sfac?on
Control
&
Use
Sounds
Image
Usability

18. Brain
plas/city
• In
parallel
to
the
development
and
valida?on
of
the
AbES,
we
are
also
inves?ga?ng
the
brain
mechanisms
associated
with
naviga/on
skill
by
carrying
out
virtual
naviga?on
tasks
within
a
neuroimaging
scanner
environment.
• Magne?c
resonance
imaging
(fMRI),
allow
us
to
follow
brain
ac?vity
related
to
behavioral
performance.
Func?onal
MRI
takes
advantage
of
the
fact
that
when
a
region
of
the
brain
is
highly
ac?ve,
there
is
an
oversupply
of
oxygenated
blood
to
that
region.

19. Brain
plas/city
A) Subject playing AbES in the scanner
AbES projected
on a screen
control keys
reflecting
mirror
subject
wearing
headphones
B) Brain activity related to navigation
visual
cortex
auditory
cortex
sensory-motor
cortex
frontal
cortex

20. Conclusion
• The
study
iden?fied
key
aspects
of
an
interface
necessary
to
grasp
and
construct
a
mental
spa?al
map
for
blind
users,
their
needs,
and
ways
of
interac?ng
with
virtual
technology.
They
are
considered
crucial
for
designing
interac?ve
interfaces
for
blind
children
that
can
determine
the
success
or
failure
of
audio-­‐based
virtual
environments.
• AbES
was
developed
by
using
user-­‐centered
design
methodologies,
resul?ng
in
a
soBware
with
a
high
degree
of
usability
and
user
acceptance.
This
represents
an
advancement
on
broader
work
inves?ga?ng
the
impact
of
the
use
of
this
tool
on
the
development
and
prac?ce
of
orienta?on
and
mobility
skills
for
legally
blind
users.

21. Conclusion
• One
of
the
modes
of
interac?on
in
AbES
is
audio-­‐based
gaming.
We
intended
for
users
to
be
able
to
play
and
enjoy
the
game
and
in
doing
so,
learn
to
navigate
their
surrounding
environment,
get
to
know
and
understand
the
space,
its
dimensions
and
the
corresponding
objects.
• We
hope
that
the
results
to
be
obtained
in
our
ongoing
research
will
shed
some
light
on
the
brain
mechanisms
associated
with
naviga?on
performance
and
learning
over
?me
with
blind
users
interac?ng
with
audiobased
interfaces
such
as
AbES.
• Learning
how
the
brain
and
related
sensory
mechanisms
provide
structure
for
naviga?on
in
blind
users,
and
what
areas
are
associated
with
the
skill
of
naviga?ng
through
space
and
learning
will
provide
some
clues
for
poten?al
use
in
training,
learning
and
rehabilita?on.

22. Future
work
• Obtain
more
qualita?ve
and
quan?ta?ve
data
on
the
behavior
of
the
users
while
working
with
a
simulator
such
as
AbES,
and
its
poten?al
as
a
tes?ng
and
monitoring
pla:orm
for
naviga?on
and
learning
skills.
• Establish
how
the
brain
processes
informa?on
when
the
user
plays
and
navigates
within
a
virtual
environment,
how
the
users
perceive
this
informa?on,
and
which
are
the
key
factors
that
lead
to
the
genera?on
of
a
spa?al
mental
map
of
the
navigated
space,
as
well
as
how
that
mental
map
of
the
virtual
naviga?on
is
transferred
to
real
naviga?on
skills.

23. Blind
Children
Naviga/on
through
Gaming
and
Associated
Brain
Plas/city
Jaime
Sanchez,
Angelo
Tadres,
Alvaro
Pascual-­‐Leone,
Lo:i
Merabet