ICVR2013 Workshop: Designing an effective rehabilitation simulation

Sergi Bermúdez i Badia
Inv. Assist. Prof., Universidade da Madeira
Marie Curie Research Fellow, Madeira – ITI
sergi.bermudez@uma.pt

NeuroRehabLab
http://neurorehabilitation.m-iti.org
Universidade da Madeira / Madeira – ITI

Sergi Bermúdez i Badia
Mónica S. Camerião
Athanasios Vourvopoulos
Ana Lúcia dos Santos Faria

Quality of Life Technologies Center
http://www.cmu.edu/qolt
Carnegie Mellon University

Daniel P. Siewiorek

University of Pittsburgh
Department of Occupational Therapy

Scott Bleakley

Myomo Inc
http://www.myomo.com
SPECS - http://specs.upf.edu
Universitat Pompeu Fabra

Steve Kelly
Ela Lewis
Paul F.M.J. Verschure

PERCRO - http://www.percro.org
Scuola Superiore Sant'Anna
Hospital del Mar
Medicina Física i Rehabilitació

ICVR 2013: Designing an effective rehabilitation simulation - 26/08/2013 - Philadelphia

Antonio Frisoli
Esther Duarte Oller

• Our approach (not necessarily the best one)
• The process for developing, designing and testing an
effective rehabilitation simulation
– Only Stroke
– Particular case of upper limb rehabilitation
– Not going to explain or justify the use of VR

• Metrics for simulated movement performance and
monitoring
• Personalization using the metrics
• Lessons learned from different studies
• Open source / game engines for rehabilitation
activities


1. The process for designing, developing and
testing an effective rehabilitation simulation
hypothesis
where
- Designing  defining
- Developing  formulating and implementing
- Testing  clinical validation


Optimal rehabilitation
guidelines

VR and Serious Games

HCI


Optimal rehabilitation
guidelines

Neuroscience

VR and Serious Games

HCI


1. Treatment frequency and intensity correlate with recovery
(Kwakkel et al., 2004; Sonoda, Saitoh, Nagai, Kawakita, & Kanada,
2004).
 deployment
2. Movement practice and repetition play a fundamental role in
recovery (Karni et al., 1995).
 simulated task / game mechanics
3. Specificity of rehabilitation training with respect to the deficits and
required functional outcomes has an impact on recovery (Krakauer
2006).
 Interface technology & personalization


- The brain has the capability to reorganize
itself in such a way that alternate brain areas
take over other functions. The best way to
stimulate this reorganization is still under
discussion, and several approaches have been
proposed.
Dobkin, Nat ClinPractNeurol 2008

- Stroke rehabilitation should focus on
maximizing cortical reorganization.


- VR can provide:
-

Fully controlled environments

-

Minimally supervised intensive
training

-

Task-specific movement reiteration

-

Individualized training

-

Feedback for reward and motivation


- Mirror Neuron System: neurons that are
active both, during the execution of goaloriented movements and during the
observation of the same action performed
by others.1-3
- There is strong evidence of the existence
of a Mirror Neuron System (MNS) in the
human brain.3-5

1

Rizzolatti & Fabbri-Destro, Exp Brain Res 2010

di Pellegrino et al., Exp Brain Res 1992; 2Gallese et al., Brain 1996; 3Rizzolatti & Fabbri-Destro, Exp Brain Res 2010;
4
Iacoboni et al., Science 1999; 5Buccino et al., Eur J Neurosci 2001

- VR can provide:
-

Fully controlled environments

-

Minimally supervised intensive
training

-

Task-specific movement reiteration

-

Individualized training

-

Feedback for reward and motivation

- Moreover:
-

MN can be activated through the
1
observation of artificial agents.

-

Evidence that a first-person
perspective is the optimal frame of
2
reference during action observation.
1

The Rehabilitation Gaming System: Spheroids

Gazzola et al., Neuroimage 2007; 2Maeda et al., J Neurophysiol 2000


BRAIN
- Generation of Sensory Motor Rhythms
- Use of remaining Cortico-Spinal Tracts

VR
- Close the sense-act loop
- Activation of Mirror Neuron System
- Optimal Rehabilitation Guidelines
- Feedback & motivation

Trade-off:
In general, the more specific an exercise is…
the more complex the interface is…
and the more difficult deployment is …


BRAIN

VR


Active movement against gravity/friction
Body movement

Finger flexion


Tangible interaction

Natural User Interfaces (NUI)


BRAIN

VR




Body movement
Finger flexion
No movement against gravity/friction
Passive assistance

Active assistance

* Armeo, Hocoma

* mpower 1000, myomo inc.

Adapted / Assistive Interfaces


BRAIN

VR


Body movement
Finger flexion
No movement against gravity/friction
Passive assistance

No active movement
Active assistance

Mental Imagery / Neurofeedback approaches

* Armeo, Hocoma

* mpower 1000, myomo inc.


Interaction based on:
- Movement (body kinematics):
- Range of Movement (ROM)
- Movement speed
- Movement latency
- Movement smoothness
- Muscle synergies
- Movement Coordination
- EMG

* international 10/20 system
* Armeo, Hocoma

Adapted / Assistive Interfaces
Brain Computer Interfaces (BCI)

- Autonomic Nervous System (EDA,
respiration, HR, etc…)
- Brain activity (EEG, NIR, fMRI, etc…)


- Limiting perceived factors in elderly
population:
- Perceived barriers due to
physical limitations1,2,3
- Lack of knowledge4,5
Study:
- 8 participants with upper limb deficits
( 4 male + 4 female, M= 44.8 yo, not all
strokes)
- 3 out of 8 subjects reported previous
computer experience.
- Mouse + keyboard vs.
Key-glove comparison.
1Opalinski,

J Technology in Human Services 2001; 2Peacock
et al., Eu J of Ageing 2007; 3Ng, Educational Gerontology
2008; 4Carpenter et al., Computers in Human Behavior
2007; 5Saunders, Educational Gerontology 2004.

Rubio et al., Presence 2012


- Limiting perceived factors in elderly
population:
- Perceived barriers due to
physical limitations1,2,3
- Lack of knowledge4,5

t-test, p = .02

Study:
- 8 participants with upper limb deficits
( 4 male + 4 female, M= 44.8 yo, not all
strokes)
- 3 out of 8 subjects reported previous
computer experience.
- Mouse + keyboard vs.
Key-glove comparison.
1Opalinski,

J Technology in Human Services 2001; 2Peacock
et al., Eu J of Ageing 2007; 3Ng, Educational Gerontology
2008; 4Carpenter et al., Computers in Human Behavior
2007; 5Saunders, Educational Gerontology 2004.

Rubio et al., Presence 2012


Originally developed by psychologists Robert M. Yerkes and
John Dillingham Dodson in 1908

Yerkes & Dodson , 1908


Cameirão et al., J Neuroeng Rehabil 2010

Hitting

Catching


Grasping

Automated procedure for
difficulty adjustment:
• Task individualization
• Optimized level of
performance
• Adapts to individual arms



- VR and real counterparts:
- Are they equivalent?
- Comparison with
control group
- The calibration task is used
to measure arm kinematics
and to set the baseline
parameters of the training for
every session.
- Range of Movement (ROM),
movement latency,
movement speed.


- Extracted from RGS
calibration
- Groups showed a
dissimilar pattern of
improvement in the
speed of the paretic
arm over time
(ANOVA, p<.05).
*p<.05, Mann-Whitney Test

Cameirão et al., Rest NeurNeursc, 2011.


12 stroke patients and 10 controls were exposed to random
combinations of the 4 game parameters

4-factor ANOVA to disclose main and interaction effects

Multiple regression to estimate constants (m0 ... m9)

- Not all VR
parameters may be
relevant
- Parameters interact
in a non-trivial an nonlinear manner.


- 9 stroke patients and 10
controls
- The model captures the
behavior of the individual
arms by different game
parameters, and to adapt
the difficulty level
accordingly.

Performance

*p<.05, T-Test

Training paradigm:
- Goal oriented and repetitive
actions
- Bimanual training (non-paretic
arm support)
- Parameterized (flying speed,
turning speed, acceptance radius,
distance between objects)

Bermúdez i Badia, Stroke Research & Treat, 2012

Motivation:
- Embedded in a game
- Extensive visual and sound
feedback
- Automatic computation of
training parameters (Avoid failure
and frustration)


A first study with 10 healthy participants
has shown that the NTT captures precise
quantitative kinematic information
during a NTT training session, including:
•

Range of Movement (ROM)

•

Movement smoothness

•

Arm coordination

•

Arm contribution to task
Bermúdez i Badia, Stroke Research & Treat, 2012


Kinematic measure = c0 + c1*speed + c2*turning + c3*acceptance + c4*distance
+ c5*speed*turning + c6*speed*acceptance + c7*speed*distance
+ c8*turning*acceptance + c9*turning*distance
+ c10*distance*acceptance + c11*speed2 + c12*turning2 + c13*acceptance2
+ c14*distance2
s

t

a

d

s*t

s*a

s*d

t*a

t*d

a*d

s2

t2

a2

d2

Movement
Smoothness
Range of Motion
Arm
Displacement
Arm coordination

- Not all parameters contribute to all movement kinematic measures
- We have a quantitative way of adapting parameters depending on a

desired kinematic training

higher level

RGS

Control
IOT

NSG
- N = 16 acute stroke patients
- 3 weekly 20 min sessions during 12 weeks

-

RGS group recovers faster than Controls, but not at follow-up  Accelerates recovery
RGS ≥ extended OT

-

Reduces therapy costs


N = 44 chronic stroke patients
5 weekly 30 min sessions during 4 weeks

1.

NUI interface

2.

Haptic interface (GRAB, PERCRO):

- Force-feedback
- Increase in the ecological validity of the task
3.

Exoskeleton (ARMEO, Hocoma):

- Support against gravity
- Control of trunk movements
Cameirão et al., Stroke, 2012.


- All groups improved
significantly
- The haptics group retained
during a longer period of
time
 include if possible
- The exoskeleton group
showed a “poorer”
improvement at proximal
movements
 not all assistance is
always helpful
 Functional vs. correct
movement
Cameirão et al., Stroke, 2012.

Between-group

Within-group


*p<.05, **p<.01,
Wilcoxon Test

- 9 naïve healthy subjects (26.4±4.2
years)
- 3 experimental conditions
Combined motor execution and motor
imagery in VR seems to be more
effective at:
-

1

Driving cortical sensorimotor areas1,
Increasing attention and alertness to
task2,
Engaging additional related networks
(cross-modal sensory processing3 or
short term memory).

Bermúdez et al., IEEE Trans Neur Sys Reh Eng, 2013.

Solodkin et al., 2004; 2 Egner et al., 2004; 3 Kanayama et al., 2007


-18 right-handed healthy subjects
(24±3 years)
- Active and imagination conditions
using RGS
Imagery of target catching was related
to activation of frontal, parietal,
temporal, cingulate and cerebellar
regions, consistent with:
-

object processing,
motor intention,
attention,
mirror mechanisms.
Prochnow et al., Eu Journal of Neuroscience, 2013.


- Virtual Environments are simulations that engage the senses of users through
real-time 3D graphics, audio and interaction to create an experience of presence
within an artificial world.
- Game Engines bring together large-scale software architectures and programming
paradigms to design and implement rendering, collision, physics and animation
processes.


Cost

Free

Platforms

Windows

Community

Large & commercial

Programming

Development Tools

Learning Curve

Fast

Customization

Source code
available

Torque was used to implement the RGS
http://www.rgs-project.eu


Cost

Free

Platforms

Win, Mac, Linux

Community

Developers

Programming

Python

Learning Curve

Slower

Customization

Source code
available

Panda3D was used to implement the NTT
http://neurorehabilitation.m-iti.org/NTT


Cost

~ 1500$ / developer

Platforms

All

Community

Very large &
commercial

Programming

Integrated
Development
Environment

Learning Curve

Very Fast

Customization

Using external
libraries

Unity is used in the RehabNet project
http://neurorehabilitation.m-iti.org/rehabnet


Cost

Free

~ 1500$ / developer

Free

Windows

All

Win, Mac, Linux

Large & commercial

Very large &
commercial

Developers

Programming

Development Tools

Integrated
Development
Environment

Python

Learning Curve

Fast

Very Fast

Slower

Customization

Source code
available

Using external
libraries

Source code
available

Platforms
Community


• A close dialog between health practicioners – neuroscientists –
technologists is necessary
• Our systems are not the end product, are the hypotheses
• Hypotheses need validation  impact assessment
• We believe that positive effects of gaming in rehabilitation training do not
emerge out of superficial “gamification” of therapies.
• Game / training mechanics need to include:
• Neuroscientific principles of recovery
• Difficulty vs skill needs to be quantified and well understood
• Game parameters need to be automatically personalized
• Stress levels need to be controlled to ensure maximal performance
and consequent maximal learning


For more information contact:
sergi.bermudez@uma.pt
or visit
http://neurorehabilitation.m-iti.org

ICVR2013 Workshop: Designing an effective rehabilitation simulation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to ICVR2013 Workshop: Designing an effective rehabilitation simulation

Similar to ICVR2013 Workshop: Designing an effective rehabilitation simulation (20)

Recently uploaded

Recently uploaded (20)

ICVR2013 Workshop: Designing an effective rehabilitation simulation