SlideShare a Scribd company logo
Close Range Depth Sensing
Cameras for Virtual Reality based
Hand Rehabilitation
Darryl Charles1, Katy Pedlow2, Suzanne McDonough2, Ka Shek3, and Therese Charles3
1Computer

Science Research Institute, School of Computing and Information Engineering,
University of Ulster, Coleraine, Northern Ireland BT52 1SA
2Centre for Health and Rehabilitation Technologies, School of Health Sciences,
University of Ulster, Jordanstown, Northern Ireland BT37 0QB
3SilverFish Studios, Coleraine, Northern Ireland BT52 2NR
Introduction
• Our interest is in low cost, technology based,
games enhanced physical rehabilitation
• Previous work has used webcams and Kinect
– Effective but has issues, e.g. lag and resolution

• Leap Camera released

– Low lag, high resolution tracking of fingers
– We investigated beta camera and sdk
– Developed several VR therapies for fingers.

• In this paper we present results from trials with
professional physiotherapists
Previous Research
• Video of previous webcam and augmented
reality games from Ulster ->
Leap Motion Camera/Controller
Leap Motion camera/controller
• Low cost (approx. £70 in the UK)
• Small (0.5” x 1.2” x 3” with a weight of 0.1
pounds)
• Built-in infrared LEDs to detect objects within
a dome of approximately 8 cubic feet above it
• Minimal latency and high spatial precision
(0.01mm)
Leap Demo
• On Screen (Visualizer + other examples)
Method
• Construct virtual simulations of 3 common rehab tasks
for hand and fingers
– Leap Motion and a natural interface controller
– Use a 3D game engine for construction
– Collaborate with commercial developers (SilverFish
Studios) and academics from the Centre for Health and
Rehabilitation Technologies (CHaRT)

• Trial simulated tasks with clinicians

– Give us understanding of the potential and limitations of
the Leap
– Obtain feedback from professionals at Regional Acquired
Brain Injury Unit (RABIU) at Musgrave Hospital in Belfast
before considering patient trials
Task 1 – Cotton Balls
Activity Description

Progression Parameters

Feedback Parameters

Cotton balls and a container are placed on a
table. The user is asked to pick the cotton balls
up off of the table and place them in the
container. The use is encouraged to use a pincer
grasp.
• Size of Container
• Distance of container from user
• Height of container
• Number of cotton balls
• Time required to place a set number of
cotton balls in the container
• Number of repetitions
Task 2 – Stacking Blocks
Activity Description

The user is given rectangular blocks
(wooden or plastic), and is asked to
build a tower by stacking blocks
vertically on the table.
Progression
• Distance of blocks from the user.
Parameters
• Number of blocks
• Size of blocks
Feedback Parameters • Number of repetitions in a set time
period
• Time: Time required to complete the
task
Task 3 - Nine Hole Peg Test (NHPT)
• The standardized equipment for the test typically
consists of:

– A board, in wood or plastic, with 9 holes (10 mm
diameter, 15 mm depth), placed apart by 32 mm
(Mathiowetz et al. 1985) or 50 mm (Heller et al. 1987)
– A container for the pegs. Initially the container was a
square box (100 x 100 x 10 mm) apart from the board.
The most current container is a shallow round dish at
the end of the board (Grice et al., 2003)
– 9 pegs (7 mm diameter, 32 mm length) (Mathiowetz
et al. 1985)
– Stopwatch
Virtual Tasks
• Video or live demo of software
Screenshots of three simulated rehab tasks. From left to right: Cotton Balls, Nine Hole Peg Test, and
Stacking Blocks.
Trial Participants
Participant

Q1. Occupation

1

Physiotherapist

2
3
4
5
6
7
8

Q2. Years of
Experience
10+

Q3. Use Games
for Rehab
No

Q4. Play Games

Occupational
Therapist
Student

10+

Yes

Occasionally

0

No

Never

Occupational
Therapist
Occupational
Therapist
Occupational
Therapist
Occupational
Therapist
Occupational
Therapist

1-2

Yes

Occasionally

3-5

Yes

Occasionally

3-5

Yes

Never

5-10

Yes

Occasionally

5-10

Yes

Never

Occasionally
Post Trial Questions
Background information

1. What is your occupation?
Physiotherapist / Occupational therapist / Other
2. How many years clinical experience do you have?
3. Do you use computer games for rehabilitation? YES/NO

System specific questions
(Answers on a Likert Scale 1 – 7)

4. Do you play games?
5. I feel that with practise I would become proficient in using the control interface
6. I feel that with practise I would become proficient in using the control interface
7. The tasks presented on the screen are easy to understand
8. The content on the screen is appropriate for the patient population
9. The prototypes provide a good illustration of all the functionalities I would require it to
have e.g. type of tasks, movements emphasised

Patient population related questions
(Answers on a Likert Scale 1 – 7)

10. I feel it would be easy to use this system in my clinical environment
11. I can see the benefit of this system for my general patient population
12. I can see the benefit of this system for the older patient population
13. I can see the benefit of this system for the younger patient population
14. I feel patients would be motivated to use this system
15. I feel my patients would benefit from this type of system in their home environment
16. I feel the system needs to be adapted to suit my patient population
Please state how it would need adapted (follow up comment)
Full Results
Mean Responses per Question
Variation per Participant
Participant

Q1. Occupation

Q2. Years of
Experience

Q3. Use
Games for
Rehab

Q4. Play Games

Q5 – Q16
Mean
Response

Interquartile
Range

1

Physiotherapist

10+

No

Occasionally

1.33

0.75

2

Occupational
Therapist

10+

Yes

Occasionally

2.25

0.75

3

Student

0

No

Never

2.67

2.5

4

Occupational
Therapist

1-2

Yes

Occasionally

4.00

2

5

Occupational
Therapist

3-5

Yes

Occasionally

1.75

1.75

6

Occupational
Therapist

3-5

Yes

Never

4.17

1

7

Occupational
Therapist

5-10

Yes

Occasionally

1.00

0

8

Occupational
Therapist

5-10

Yes

Never

3.58

3

6 Yes
2 No

5 Occasionally
3 Never

2.59 (Mean)

1.47 (Mean)

Summary
Summary of Results
• Mean response to all questions = 2.59 (scale 1-7)
– Six of respondents had a mean response to the
questions of less than 3

• 4 clinicians over 5 years experience provided a
(mean = 2.04)
– 4 other responded with a (mean = 3.14)

•
•
•
•

Tasks easy to understand (mean = 1.88)
Good illustration of rehab tasks (mean = 2.38)
Suitable for young patients (mean = 1.6)
Suitable for home environment (mean = 2.5)
Discussion
• Ease of use of system

– We spent most time on this – it is tricky!
– Some clinicians did the interface tricky – especially at the start

• Issues: its an unfamiliar UI, hand position must remain parallel to table
surface (+/- 10 degrees or so)
• Lessons: user needs time to attune, lots of positions cues required, VR
headset like Oculus Rift may help

– Several clinicians felt that older people could have problems
learning to use the system

• Clinicians felt that the system has potential but the tasks
should be converted into games and could be more tailored
to their treatements
• Clinicians were excited about the use of the system in the
home – especially since it is so cheap and easy to set up.
Participant Comments
• “Develop programmes for different
orientation, i.e. vertical and horizontal”
(Participant 1)
• “More functional activities; i.e. lift a cup,
bringing to a mouth. Puzzles - to incorporate
cognitive skills” (Participant 4)
• “May be set games for different age groups”
(Participant 7)
Conclusion
• The results were very encouraging
• We have learned a lot about designed virtual
rehab. software that use the Leap controller
• Next phase
– Move software to Unity3D
– Design games for the Leap in collaboration with
clinicians
– Trial games with patients

More Related Content

Similar to An Evaluation of the Leap Motion Depth Sensing Camera for Tracking Hand and Fingers Motion in Physical Therapy

QMRAS Project Presentation
QMRAS Project PresentationQMRAS Project Presentation
QMRAS Project Presentation
Gary Spencer
 
Introduction to Usability Testing for Survey Research
Introduction to Usability Testing for Survey ResearchIntroduction to Usability Testing for Survey Research
Introduction to Usability Testing for Survey Research
Caroline Jarrett
 
Tecnologías que mejoran el resultado en el proceso de rehabilitación de perso...
Tecnologías que mejoran el resultado en el proceso de rehabilitación de perso...Tecnologías que mejoran el resultado en el proceso de rehabilitación de perso...
Tecnologías que mejoran el resultado en el proceso de rehabilitación de perso...
Teletón Paraguay
 
OSCE paeds.pptx
OSCE paeds.pptxOSCE paeds.pptx
OSCE paeds.pptx
Smriti Arora
 
Final_Presentation.pptx
Final_Presentation.pptxFinal_Presentation.pptx
Final_Presentation.pptx
SudeekshaKoricherla
 
Lab 06-sol
Lab 06-solLab 06-sol
Lab 06-sol
Ahmad sohail Kakar
 
Modeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural NetworksModeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural Networks
Josh Patterson
 
Controlled Experiments - Shengdong Zhao
Controlled Experiments - Shengdong ZhaoControlled Experiments - Shengdong Zhao
Controlled Experiments - Shengdong Zhao
Michael Shilman
 
ISA15 - Influence of a HMD on UX and performance in a VR-based sports applica...
ISA15 - Influence of a HMD on UX and performance in a VR-based sports applica...ISA15 - Influence of a HMD on UX and performance in a VR-based sports applica...
ISA15 - Influence of a HMD on UX and performance in a VR-based sports applica...
Pedro Kayatt
 
1. Intro DS.pptx
1. Intro DS.pptx1. Intro DS.pptx
1. Intro DS.pptx
Anusuya123
 
Analytic emperical Mehods
Analytic emperical MehodsAnalytic emperical Mehods
Analytic emperical Mehods
M Surendar
 
A systematic approach towards designing low-cost motor and cognitive rehabili...
A systematic approach towards designing low-cost motor and cognitive rehabili...A systematic approach towards designing low-cost motor and cognitive rehabili...
A systematic approach towards designing low-cost motor and cognitive rehabili...
Sergi Bermudez i Badia
 
Scientific Research Steps Part 2
Scientific Research Steps Part 2Scientific Research Steps Part 2
Scientific Research Steps Part 2
Ainul Yaqin
 
Computer games for user engagement in Attention Deficit Hyperactivity Disorde...
Computer games for user engagement in Attention Deficit Hyperactivity Disorde...Computer games for user engagement in Attention Deficit Hyperactivity Disorde...
Computer games for user engagement in Attention Deficit Hyperactivity Disorde...
Karel Van Isacker
 
Topic 5 Conducting Surveys, Experiment of observation.pdf
Topic 5 Conducting Surveys, Experiment of observation.pdfTopic 5 Conducting Surveys, Experiment of observation.pdf
Topic 5 Conducting Surveys, Experiment of observation.pdf
RubenAgacio
 
Top Brainnovation harnessing Big Data
Top Brainnovation harnessing Big DataTop Brainnovation harnessing Big Data
Top Brainnovation harnessing Big Data
SharpBrains
 
Differences in-task-descriptions
Differences in-task-descriptionsDifferences in-task-descriptions
Differences in-task-descriptionsSameer Chavan
 
HCI 3e - Ch 9: Evaluation techniques
HCI 3e - Ch 9:  Evaluation techniquesHCI 3e - Ch 9:  Evaluation techniques
HCI 3e - Ch 9: Evaluation techniques
Alan Dix
 
(Fall 2012) Palm Vein Recognition (Cradle VS Non-cradle)
(Fall 2012) Palm Vein Recognition (Cradle VS Non-cradle)(Fall 2012) Palm Vein Recognition (Cradle VS Non-cradle)
(Fall 2012) Palm Vein Recognition (Cradle VS Non-cradle)
International Center for Biometric Research
 

Similar to An Evaluation of the Leap Motion Depth Sensing Camera for Tracking Hand and Fingers Motion in Physical Therapy (20)

QMRAS Project Presentation
QMRAS Project PresentationQMRAS Project Presentation
QMRAS Project Presentation
 
Introduction to Usability Testing for Survey Research
Introduction to Usability Testing for Survey ResearchIntroduction to Usability Testing for Survey Research
Introduction to Usability Testing for Survey Research
 
Tecnologías que mejoran el resultado en el proceso de rehabilitación de perso...
Tecnologías que mejoran el resultado en el proceso de rehabilitación de perso...Tecnologías que mejoran el resultado en el proceso de rehabilitación de perso...
Tecnologías que mejoran el resultado en el proceso de rehabilitación de perso...
 
OSCE paeds.pptx
OSCE paeds.pptxOSCE paeds.pptx
OSCE paeds.pptx
 
Final_Presentation.pptx
Final_Presentation.pptxFinal_Presentation.pptx
Final_Presentation.pptx
 
Lab 06-sol
Lab 06-solLab 06-sol
Lab 06-sol
 
Modeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural NetworksModeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural Networks
 
Controlled Experiments - Shengdong Zhao
Controlled Experiments - Shengdong ZhaoControlled Experiments - Shengdong Zhao
Controlled Experiments - Shengdong Zhao
 
ISA15 - Influence of a HMD on UX and performance in a VR-based sports applica...
ISA15 - Influence of a HMD on UX and performance in a VR-based sports applica...ISA15 - Influence of a HMD on UX and performance in a VR-based sports applica...
ISA15 - Influence of a HMD on UX and performance in a VR-based sports applica...
 
1. Intro DS.pptx
1. Intro DS.pptx1. Intro DS.pptx
1. Intro DS.pptx
 
Analytic emperical Mehods
Analytic emperical MehodsAnalytic emperical Mehods
Analytic emperical Mehods
 
A systematic approach towards designing low-cost motor and cognitive rehabili...
A systematic approach towards designing low-cost motor and cognitive rehabili...A systematic approach towards designing low-cost motor and cognitive rehabili...
A systematic approach towards designing low-cost motor and cognitive rehabili...
 
Scientific Research Steps Part 2
Scientific Research Steps Part 2Scientific Research Steps Part 2
Scientific Research Steps Part 2
 
Computer games for user engagement in Attention Deficit Hyperactivity Disorde...
Computer games for user engagement in Attention Deficit Hyperactivity Disorde...Computer games for user engagement in Attention Deficit Hyperactivity Disorde...
Computer games for user engagement in Attention Deficit Hyperactivity Disorde...
 
Topic 5 Conducting Surveys, Experiment of observation.pdf
Topic 5 Conducting Surveys, Experiment of observation.pdfTopic 5 Conducting Surveys, Experiment of observation.pdf
Topic 5 Conducting Surveys, Experiment of observation.pdf
 
Top Brainnovation harnessing Big Data
Top Brainnovation harnessing Big DataTop Brainnovation harnessing Big Data
Top Brainnovation harnessing Big Data
 
Differences in-task-descriptions
Differences in-task-descriptionsDifferences in-task-descriptions
Differences in-task-descriptions
 
Chapter 8 eval. tech. lesson 2
Chapter 8 eval. tech. lesson 2Chapter 8 eval. tech. lesson 2
Chapter 8 eval. tech. lesson 2
 
HCI 3e - Ch 9: Evaluation techniques
HCI 3e - Ch 9:  Evaluation techniquesHCI 3e - Ch 9:  Evaluation techniques
HCI 3e - Ch 9: Evaluation techniques
 
(Fall 2012) Palm Vein Recognition (Cradle VS Non-cradle)
(Fall 2012) Palm Vein Recognition (Cradle VS Non-cradle)(Fall 2012) Palm Vein Recognition (Cradle VS Non-cradle)
(Fall 2012) Palm Vein Recognition (Cradle VS Non-cradle)
 

More from Interactive Technologies and Games: Education, Health and Disability

Robotics and Education – EduRob Project Results Launch
Robotics and Education – EduRob Project Results LaunchRobotics and Education – EduRob Project Results Launch
Robotics and Education – EduRob Project Results Launch
Interactive Technologies and Games: Education, Health and Disability
 
Educational Robotics for Students with disabilities (EDUROB) - brochure
Educational Robotics for Students with disabilities (EDUROB) - brochureEducational Robotics for Students with disabilities (EDUROB) - brochure
Educational Robotics for Students with disabilities (EDUROB) - brochure
Interactive Technologies and Games: Education, Health and Disability
 
Can Computer-Assisted Training of Prerequisite Motor Skills Help Enable Commu...
Can Computer-Assisted Training of Prerequisite Motor Skills Help Enable Commu...Can Computer-Assisted Training of Prerequisite Motor Skills Help Enable Commu...
Can Computer-Assisted Training of Prerequisite Motor Skills Help Enable Commu...
Interactive Technologies and Games: Education, Health and Disability
 
Increasing Awareness of Alzheimer’s Disease through a Mobile Game (Beverley C...
Increasing Awareness of Alzheimer’s Disease through a Mobile Game (Beverley C...Increasing Awareness of Alzheimer’s Disease through a Mobile Game (Beverley C...
Increasing Awareness of Alzheimer’s Disease through a Mobile Game (Beverley C...
Interactive Technologies and Games: Education, Health and Disability
 
Game features of cognitive training (Michael P. Craven and Carlo Fabricatore)
Game features of cognitive training (Michael P. Craven and Carlo Fabricatore)Game features of cognitive training (Michael P. Craven and Carlo Fabricatore)
Game features of cognitive training (Michael P. Craven and Carlo Fabricatore)
Interactive Technologies and Games: Education, Health and Disability
 
Enhancing the measurement of clinical outcomes using Microsoft Kinect choices...
Enhancing the measurement of clinical outcomes using Microsoft Kinect choices...Enhancing the measurement of clinical outcomes using Microsoft Kinect choices...
Enhancing the measurement of clinical outcomes using Microsoft Kinect choices...
Interactive Technologies and Games: Education, Health and Disability
 
User involvement in design and application of virtual reality gamification to...
User involvement in design and application of virtual reality gamification to...User involvement in design and application of virtual reality gamification to...
User involvement in design and application of virtual reality gamification to...
Interactive Technologies and Games: Education, Health and Disability
 
Our virtual selves, our virtual morals – Mass Effect players’ personality and...
Our virtual selves, our virtual morals – Mass Effect players’ personality and...Our virtual selves, our virtual morals – Mass Effect players’ personality and...
Our virtual selves, our virtual morals – Mass Effect players’ personality and...
Interactive Technologies and Games: Education, Health and Disability
 
Support Dementia: using wearable assistive technology and analysing real-time...
Support Dementia: using wearable assistive technology and analysing real-time...Support Dementia: using wearable assistive technology and analysing real-time...
Support Dementia: using wearable assistive technology and analysing real-time...
Interactive Technologies and Games: Education, Health and Disability
 
Breast Cancer Diagnosis using a Hybrid Genetic Algorithm for Feature Selectio...
Breast Cancer Diagnosis using a Hybrid Genetic Algorithm for Feature Selectio...Breast Cancer Diagnosis using a Hybrid Genetic Algorithm for Feature Selectio...
Breast Cancer Diagnosis using a Hybrid Genetic Algorithm for Feature Selectio...
Interactive Technologies and Games: Education, Health and Disability
 
Keynote speakers – Dom Martinovs and Rachel Barrett, ‘ No One Left Behind’ pr...
Keynote speakers – Dom Martinovs and Rachel Barrett, ‘ No One Left Behind’ pr...Keynote speakers – Dom Martinovs and Rachel Barrett, ‘ No One Left Behind’ pr...
Keynote speakers – Dom Martinovs and Rachel Barrett, ‘ No One Left Behind’ pr...
Interactive Technologies and Games: Education, Health and Disability
 
Playing games with observation, dependency and agency in a new environment fo...
Playing games with observation, dependency and agency in a new environment fo...Playing games with observation, dependency and agency in a new environment fo...
Playing games with observation, dependency and agency in a new environment fo...
Interactive Technologies and Games: Education, Health and Disability
 
Me, My Game-Self, and Others: A Qualitative Exploration of the Game-Self (Nik...
Me, My Game-Self, and Others: A Qualitative Exploration of the Game-Self (Nik...Me, My Game-Self, and Others: A Qualitative Exploration of the Game-Self (Nik...
Me, My Game-Self, and Others: A Qualitative Exploration of the Game-Self (Nik...
Interactive Technologies and Games: Education, Health and Disability
 
A comparison of humanoid and non-humanoid robots in supporting the learning o...
A comparison of humanoid and non-humanoid robots in supporting the learning o...A comparison of humanoid and non-humanoid robots in supporting the learning o...
A comparison of humanoid and non-humanoid robots in supporting the learning o...
Interactive Technologies and Games: Education, Health and Disability
 
ITAG 2016 Keynote speaker - Fiorella Operto, ‘Robotics, A New Science’
ITAG 2016 Keynote speaker - Fiorella Operto, ‘Robotics, A New Science’ITAG 2016 Keynote speaker - Fiorella Operto, ‘Robotics, A New Science’
ITAG 2016 Keynote speaker - Fiorella Operto, ‘Robotics, A New Science’
Interactive Technologies and Games: Education, Health and Disability
 
Tell me what you want and I’ll show you what you can have: who drives design ...
Tell me what you want and I’ll show you what you can have: who drives design ...Tell me what you want and I’ll show you what you can have: who drives design ...
Tell me what you want and I’ll show you what you can have: who drives design ...
Interactive Technologies and Games: Education, Health and Disability
 
Using a blended pedagogical framework to guide the applications of games in n...
Using a blended pedagogical framework to guide the applications of games in n...Using a blended pedagogical framework to guide the applications of games in n...
Using a blended pedagogical framework to guide the applications of games in n...
Interactive Technologies and Games: Education, Health and Disability
 
Urban Games: playful storytelling experiences for city dwellers
Urban Games: playful storytelling experiences for city dwellersUrban Games: playful storytelling experiences for city dwellers
Urban Games: playful storytelling experiences for city dwellers
Interactive Technologies and Games: Education, Health and Disability
 
Game transfer Phenomena: the pervasiveness of sounds from video games and the...
Game transfer Phenomena: the pervasiveness of sounds from video games and the...Game transfer Phenomena: the pervasiveness of sounds from video games and the...
Game transfer Phenomena: the pervasiveness of sounds from video games and the...
Interactive Technologies and Games: Education, Health and Disability
 
Immersive Virtual Reality Simulation Deployment in a Lean Manufacturing Envir...
Immersive Virtual Reality Simulation Deployment in a Lean Manufacturing Envir...Immersive Virtual Reality Simulation Deployment in a Lean Manufacturing Envir...
Immersive Virtual Reality Simulation Deployment in a Lean Manufacturing Envir...
Interactive Technologies and Games: Education, Health and Disability
 

More from Interactive Technologies and Games: Education, Health and Disability (20)

Robotics and Education – EduRob Project Results Launch
Robotics and Education – EduRob Project Results LaunchRobotics and Education – EduRob Project Results Launch
Robotics and Education – EduRob Project Results Launch
 
Educational Robotics for Students with disabilities (EDUROB) - brochure
Educational Robotics for Students with disabilities (EDUROB) - brochureEducational Robotics for Students with disabilities (EDUROB) - brochure
Educational Robotics for Students with disabilities (EDUROB) - brochure
 
Can Computer-Assisted Training of Prerequisite Motor Skills Help Enable Commu...
Can Computer-Assisted Training of Prerequisite Motor Skills Help Enable Commu...Can Computer-Assisted Training of Prerequisite Motor Skills Help Enable Commu...
Can Computer-Assisted Training of Prerequisite Motor Skills Help Enable Commu...
 
Increasing Awareness of Alzheimer’s Disease through a Mobile Game (Beverley C...
Increasing Awareness of Alzheimer’s Disease through a Mobile Game (Beverley C...Increasing Awareness of Alzheimer’s Disease through a Mobile Game (Beverley C...
Increasing Awareness of Alzheimer’s Disease through a Mobile Game (Beverley C...
 
Game features of cognitive training (Michael P. Craven and Carlo Fabricatore)
Game features of cognitive training (Michael P. Craven and Carlo Fabricatore)Game features of cognitive training (Michael P. Craven and Carlo Fabricatore)
Game features of cognitive training (Michael P. Craven and Carlo Fabricatore)
 
Enhancing the measurement of clinical outcomes using Microsoft Kinect choices...
Enhancing the measurement of clinical outcomes using Microsoft Kinect choices...Enhancing the measurement of clinical outcomes using Microsoft Kinect choices...
Enhancing the measurement of clinical outcomes using Microsoft Kinect choices...
 
User involvement in design and application of virtual reality gamification to...
User involvement in design and application of virtual reality gamification to...User involvement in design and application of virtual reality gamification to...
User involvement in design and application of virtual reality gamification to...
 
Our virtual selves, our virtual morals – Mass Effect players’ personality and...
Our virtual selves, our virtual morals – Mass Effect players’ personality and...Our virtual selves, our virtual morals – Mass Effect players’ personality and...
Our virtual selves, our virtual morals – Mass Effect players’ personality and...
 
Support Dementia: using wearable assistive technology and analysing real-time...
Support Dementia: using wearable assistive technology and analysing real-time...Support Dementia: using wearable assistive technology and analysing real-time...
Support Dementia: using wearable assistive technology and analysing real-time...
 
Breast Cancer Diagnosis using a Hybrid Genetic Algorithm for Feature Selectio...
Breast Cancer Diagnosis using a Hybrid Genetic Algorithm for Feature Selectio...Breast Cancer Diagnosis using a Hybrid Genetic Algorithm for Feature Selectio...
Breast Cancer Diagnosis using a Hybrid Genetic Algorithm for Feature Selectio...
 
Keynote speakers – Dom Martinovs and Rachel Barrett, ‘ No One Left Behind’ pr...
Keynote speakers – Dom Martinovs and Rachel Barrett, ‘ No One Left Behind’ pr...Keynote speakers – Dom Martinovs and Rachel Barrett, ‘ No One Left Behind’ pr...
Keynote speakers – Dom Martinovs and Rachel Barrett, ‘ No One Left Behind’ pr...
 
Playing games with observation, dependency and agency in a new environment fo...
Playing games with observation, dependency and agency in a new environment fo...Playing games with observation, dependency and agency in a new environment fo...
Playing games with observation, dependency and agency in a new environment fo...
 
Me, My Game-Self, and Others: A Qualitative Exploration of the Game-Self (Nik...
Me, My Game-Self, and Others: A Qualitative Exploration of the Game-Self (Nik...Me, My Game-Self, and Others: A Qualitative Exploration of the Game-Self (Nik...
Me, My Game-Self, and Others: A Qualitative Exploration of the Game-Self (Nik...
 
A comparison of humanoid and non-humanoid robots in supporting the learning o...
A comparison of humanoid and non-humanoid robots in supporting the learning o...A comparison of humanoid and non-humanoid robots in supporting the learning o...
A comparison of humanoid and non-humanoid robots in supporting the learning o...
 
ITAG 2016 Keynote speaker - Fiorella Operto, ‘Robotics, A New Science’
ITAG 2016 Keynote speaker - Fiorella Operto, ‘Robotics, A New Science’ITAG 2016 Keynote speaker - Fiorella Operto, ‘Robotics, A New Science’
ITAG 2016 Keynote speaker - Fiorella Operto, ‘Robotics, A New Science’
 
Tell me what you want and I’ll show you what you can have: who drives design ...
Tell me what you want and I’ll show you what you can have: who drives design ...Tell me what you want and I’ll show you what you can have: who drives design ...
Tell me what you want and I’ll show you what you can have: who drives design ...
 
Using a blended pedagogical framework to guide the applications of games in n...
Using a blended pedagogical framework to guide the applications of games in n...Using a blended pedagogical framework to guide the applications of games in n...
Using a blended pedagogical framework to guide the applications of games in n...
 
Urban Games: playful storytelling experiences for city dwellers
Urban Games: playful storytelling experiences for city dwellersUrban Games: playful storytelling experiences for city dwellers
Urban Games: playful storytelling experiences for city dwellers
 
Game transfer Phenomena: the pervasiveness of sounds from video games and the...
Game transfer Phenomena: the pervasiveness of sounds from video games and the...Game transfer Phenomena: the pervasiveness of sounds from video games and the...
Game transfer Phenomena: the pervasiveness of sounds from video games and the...
 
Immersive Virtual Reality Simulation Deployment in a Lean Manufacturing Envir...
Immersive Virtual Reality Simulation Deployment in a Lean Manufacturing Envir...Immersive Virtual Reality Simulation Deployment in a Lean Manufacturing Envir...
Immersive Virtual Reality Simulation Deployment in a Lean Manufacturing Envir...
 

Recently uploaded

FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
Pierluigi Pugliese
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
RinaMondal9
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neo4j
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 

Recently uploaded (20)

FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024By Design, not by Accident - Agile Venture Bolzano 2024
By Design, not by Accident - Agile Venture Bolzano 2024
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 

An Evaluation of the Leap Motion Depth Sensing Camera for Tracking Hand and Fingers Motion in Physical Therapy

  • 1. Close Range Depth Sensing Cameras for Virtual Reality based Hand Rehabilitation Darryl Charles1, Katy Pedlow2, Suzanne McDonough2, Ka Shek3, and Therese Charles3 1Computer Science Research Institute, School of Computing and Information Engineering, University of Ulster, Coleraine, Northern Ireland BT52 1SA 2Centre for Health and Rehabilitation Technologies, School of Health Sciences, University of Ulster, Jordanstown, Northern Ireland BT37 0QB 3SilverFish Studios, Coleraine, Northern Ireland BT52 2NR
  • 2. Introduction • Our interest is in low cost, technology based, games enhanced physical rehabilitation • Previous work has used webcams and Kinect – Effective but has issues, e.g. lag and resolution • Leap Camera released – Low lag, high resolution tracking of fingers – We investigated beta camera and sdk – Developed several VR therapies for fingers. • In this paper we present results from trials with professional physiotherapists
  • 3. Previous Research • Video of previous webcam and augmented reality games from Ulster ->
  • 5. Leap Motion camera/controller • Low cost (approx. £70 in the UK) • Small (0.5” x 1.2” x 3” with a weight of 0.1 pounds) • Built-in infrared LEDs to detect objects within a dome of approximately 8 cubic feet above it • Minimal latency and high spatial precision (0.01mm)
  • 6. Leap Demo • On Screen (Visualizer + other examples)
  • 7. Method • Construct virtual simulations of 3 common rehab tasks for hand and fingers – Leap Motion and a natural interface controller – Use a 3D game engine for construction – Collaborate with commercial developers (SilverFish Studios) and academics from the Centre for Health and Rehabilitation Technologies (CHaRT) • Trial simulated tasks with clinicians – Give us understanding of the potential and limitations of the Leap – Obtain feedback from professionals at Regional Acquired Brain Injury Unit (RABIU) at Musgrave Hospital in Belfast before considering patient trials
  • 8. Task 1 – Cotton Balls Activity Description Progression Parameters Feedback Parameters Cotton balls and a container are placed on a table. The user is asked to pick the cotton balls up off of the table and place them in the container. The use is encouraged to use a pincer grasp. • Size of Container • Distance of container from user • Height of container • Number of cotton balls • Time required to place a set number of cotton balls in the container • Number of repetitions
  • 9. Task 2 – Stacking Blocks Activity Description The user is given rectangular blocks (wooden or plastic), and is asked to build a tower by stacking blocks vertically on the table. Progression • Distance of blocks from the user. Parameters • Number of blocks • Size of blocks Feedback Parameters • Number of repetitions in a set time period • Time: Time required to complete the task
  • 10. Task 3 - Nine Hole Peg Test (NHPT) • The standardized equipment for the test typically consists of: – A board, in wood or plastic, with 9 holes (10 mm diameter, 15 mm depth), placed apart by 32 mm (Mathiowetz et al. 1985) or 50 mm (Heller et al. 1987) – A container for the pegs. Initially the container was a square box (100 x 100 x 10 mm) apart from the board. The most current container is a shallow round dish at the end of the board (Grice et al., 2003) – 9 pegs (7 mm diameter, 32 mm length) (Mathiowetz et al. 1985) – Stopwatch
  • 11. Virtual Tasks • Video or live demo of software Screenshots of three simulated rehab tasks. From left to right: Cotton Balls, Nine Hole Peg Test, and Stacking Blocks.
  • 12. Trial Participants Participant Q1. Occupation 1 Physiotherapist 2 3 4 5 6 7 8 Q2. Years of Experience 10+ Q3. Use Games for Rehab No Q4. Play Games Occupational Therapist Student 10+ Yes Occasionally 0 No Never Occupational Therapist Occupational Therapist Occupational Therapist Occupational Therapist Occupational Therapist 1-2 Yes Occasionally 3-5 Yes Occasionally 3-5 Yes Never 5-10 Yes Occasionally 5-10 Yes Never Occasionally
  • 13. Post Trial Questions Background information 1. What is your occupation? Physiotherapist / Occupational therapist / Other 2. How many years clinical experience do you have? 3. Do you use computer games for rehabilitation? YES/NO System specific questions (Answers on a Likert Scale 1 – 7) 4. Do you play games? 5. I feel that with practise I would become proficient in using the control interface 6. I feel that with practise I would become proficient in using the control interface 7. The tasks presented on the screen are easy to understand 8. The content on the screen is appropriate for the patient population 9. The prototypes provide a good illustration of all the functionalities I would require it to have e.g. type of tasks, movements emphasised Patient population related questions (Answers on a Likert Scale 1 – 7) 10. I feel it would be easy to use this system in my clinical environment 11. I can see the benefit of this system for my general patient population 12. I can see the benefit of this system for the older patient population 13. I can see the benefit of this system for the younger patient population 14. I feel patients would be motivated to use this system 15. I feel my patients would benefit from this type of system in their home environment 16. I feel the system needs to be adapted to suit my patient population Please state how it would need adapted (follow up comment)
  • 15. Mean Responses per Question
  • 16. Variation per Participant Participant Q1. Occupation Q2. Years of Experience Q3. Use Games for Rehab Q4. Play Games Q5 – Q16 Mean Response Interquartile Range 1 Physiotherapist 10+ No Occasionally 1.33 0.75 2 Occupational Therapist 10+ Yes Occasionally 2.25 0.75 3 Student 0 No Never 2.67 2.5 4 Occupational Therapist 1-2 Yes Occasionally 4.00 2 5 Occupational Therapist 3-5 Yes Occasionally 1.75 1.75 6 Occupational Therapist 3-5 Yes Never 4.17 1 7 Occupational Therapist 5-10 Yes Occasionally 1.00 0 8 Occupational Therapist 5-10 Yes Never 3.58 3 6 Yes 2 No 5 Occasionally 3 Never 2.59 (Mean) 1.47 (Mean) Summary
  • 17. Summary of Results • Mean response to all questions = 2.59 (scale 1-7) – Six of respondents had a mean response to the questions of less than 3 • 4 clinicians over 5 years experience provided a (mean = 2.04) – 4 other responded with a (mean = 3.14) • • • • Tasks easy to understand (mean = 1.88) Good illustration of rehab tasks (mean = 2.38) Suitable for young patients (mean = 1.6) Suitable for home environment (mean = 2.5)
  • 18. Discussion • Ease of use of system – We spent most time on this – it is tricky! – Some clinicians did the interface tricky – especially at the start • Issues: its an unfamiliar UI, hand position must remain parallel to table surface (+/- 10 degrees or so) • Lessons: user needs time to attune, lots of positions cues required, VR headset like Oculus Rift may help – Several clinicians felt that older people could have problems learning to use the system • Clinicians felt that the system has potential but the tasks should be converted into games and could be more tailored to their treatements • Clinicians were excited about the use of the system in the home – especially since it is so cheap and easy to set up.
  • 19. Participant Comments • “Develop programmes for different orientation, i.e. vertical and horizontal” (Participant 1) • “More functional activities; i.e. lift a cup, bringing to a mouth. Puzzles - to incorporate cognitive skills” (Participant 4) • “May be set games for different age groups” (Participant 7)
  • 20. Conclusion • The results were very encouraging • We have learned a lot about designed virtual rehab. software that use the Leap controller • Next phase – Move software to Unity3D – Design games for the Leap in collaboration with clinicians – Trial games with patients