Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Towards Virtual Rehabilitation:
Hand Gesture Controlled Games to Improve
Cognitive and Motor Skills of Children with
Speci...
Nearly 34,000 of children
in Sri Lanka has
difficulties with cognitive
and motor skills
Support tools in
existence are eit...
Aim
 The aim of the research on a specialized approach to help the
children with special needs on their rehabilitation ex...
Objectives
 Track the hand gestures so precisely, that it can be used to identify the
improvements of the human body. Uni...
Research
 Literature review
 Identifying serious
games
 Creating taxonomy of serious games
 Evaluation of the related work
 Developing serious games for children with cerebral palsy:
case study and pilot trial [1]
Discusses a series of approach...
 Help Me! A Serious Game for Rehabilitation of Children
Affected by Cerebral Visual Impairment [2]
The system integrates ...
 Azahar: Serious games, improving autonomy [3]
Azahar is a platform for free, customizable applications and Serious
Games...
 Gesture based serious games for physical therapy [4]
This is a game developed by advancedmedialab.com in Saudi Arabia as...
 Visited Sri Lankan therapy units
 National Hospital of Sri Lanka
 Lady Ridgeway Hospital for
Children
 Evaluation of hand gesture recognition and hand gesture
controllers.
 Contribution
 Taxonomy built by categorizing the serious games.
 Identify the type of serious games that are suitable ...
 System Architecture
 Process
 Process Flow
 Algorithms
 Scoring
[5]
T = Theoretical minimum time spent to finish the level (in seconds)
Tx = Time spent to finish t...
 Coordinate conversion
 Technology
 Java 8
 Leap Motion Java SDK (Version 2.2.3)
 JavaFX 2
 MySQL
 Evaluation
 Six users
 Multiple Scenarios
 Usability feedback
 Critical evaluation of the results
 Results
 Gesture recognition
 Evaluation Form
 Sample (User B) :
 Conclusion
 Using a Natural User Interface, LeapPlay application was able to
improve the skills of disabled children us...
 Detailed comparison between hand gestures controllers lead to
the identification of the Leap Motion device as a good ges...
 Lessons learned
 Identified multiple ways to precision hand gesture tracking.
 Fatigue is a major concern for long dur...
 Proof of Concept
References
[1] Henschke, M., Hobbs, D., & Wilkinson, B. (2012, November).
Developing serious games for children with cere...
Thank You
Towards Virtual Rehabilitation - MCS (UCSC) Research Project 2016 - Presentation
Towards Virtual Rehabilitation - MCS (UCSC) Research Project 2016 - Presentation
Towards Virtual Rehabilitation - MCS (UCSC) Research Project 2016 - Presentation
Upcoming SlideShare
Loading in …5
×

Towards Virtual Rehabilitation - MCS (UCSC) Research Project 2016 - Presentation

290 views

Published on

Towards Virtual Rehabilitation - MCS (UCSC) Research Project 2016 - Presentation

Published in: Technology
  • Be the first to comment

Towards Virtual Rehabilitation - MCS (UCSC) Research Project 2016 - Presentation

  1. 1. Towards Virtual Rehabilitation: Hand Gesture Controlled Games to Improve Cognitive and Motor Skills of Children with Special Needs Thiwanka A. Makumburage 2013/MCS/048 2016 Supervisor Dr. G.D.S.P. Wimalaratne
  2. 2. Nearly 34,000 of children in Sri Lanka has difficulties with cognitive and motor skills Support tools in existence are either traditional or out of date. Healthcare professionals are very hard to find and their services are very expensive to be afforded What are Serious Games? Can we play games using hand gestures? Will games Improve human skills? Introduction
  3. 3. Aim  The aim of the research on a specialized approach to help the children with special needs on their rehabilitation exercise, using serious games controlled by hand gestures, to improve their cognitive functions and motor skills.
  4. 4. Objectives  Track the hand gestures so precisely, that it can be used to identify the improvements of the human body. Uniquely identifying fingers, joints of a hand leads to better information input to the serious game.  Identify the type of serious games that are suitable for the cognitive and motor skills development of a child.  Develop a process by utilizing real-time precision hand gesture tracking to bridge the hand gestures with the game actions.  Collection and analyze of the game play data, measure the improvements of children’s cognitive and motor skills.
  5. 5. Research  Literature review  Identifying serious games
  6. 6.  Creating taxonomy of serious games
  7. 7.  Evaluation of the related work
  8. 8.  Developing serious games for children with cerebral palsy: case study and pilot trial [1] Discusses a series of approaches used to develop appropriate serious games for children with cerebral palsy. They have used two sample games shown above, including data and feedback from one child with cerebral palsy who evaluated the games for entertainment, engagement and replay value. This research only focuses on improvements of the cognitive functions of the cerebral palsy effected children.
  9. 9.  Help Me! A Serious Game for Rehabilitation of Children Affected by Cerebral Visual Impairment [2] The system integrates an eye tracker system to correctly measure the performances of the child. The developed game was called HelpMe! And has two goals: to train the ability to see and touch an object at the same time and to train the problem-solving capabilities of the child.
  10. 10.  Azahar: Serious games, improving autonomy [3] Azahar is a platform for free, customizable applications and Serious Games designed to improve the quality of life and autonomy of people with autism and/or intellectual disability
  11. 11.  Gesture based serious games for physical therapy [4] This is a game developed by advancedmedialab.com in Saudi Arabia as a replacement for physiotherapy exercises. It’s an online flying game that can be played by a patient using hand and body gestures. The gestures used to play the game are the ones required to perform physical therapy. Difficulty level of the game can be changed automatically by the system or remotely by the therapist according to the recovery behavior of each patient.
  12. 12.  Visited Sri Lankan therapy units  National Hospital of Sri Lanka
  13. 13.  Lady Ridgeway Hospital for Children
  14. 14.  Evaluation of hand gesture recognition and hand gesture controllers.
  15. 15.  Contribution  Taxonomy built by categorizing the serious games.  Identify the type of serious games that are suitable for the cognitive and motor skills development of a child.  Develop a process by utilizing real-time precision hand gesture tracking to bridge the hand gestures with the game actions.  ‘Leap Play’ application, to improve the cognitive and motor skills of the children with special needs.  Children who participated for the evaluation of the research, got their cognitive and motor skills improved.
  16. 16.  System Architecture
  17. 17.  Process
  18. 18.  Process Flow
  19. 19.  Algorithms  Scoring [5] T = Theoretical minimum time spent to finish the level (in seconds) Tx = Time spent to finish the level (in seconds) N = Number of times repeated the level M = Maximum marks can be achieved from the level
  20. 20.  Coordinate conversion
  21. 21.  Technology  Java 8  Leap Motion Java SDK (Version 2.2.3)  JavaFX 2  MySQL
  22. 22.  Evaluation  Six users  Multiple Scenarios  Usability feedback  Critical evaluation of the results
  23. 23.  Results
  24. 24.  Gesture recognition
  25. 25.  Evaluation Form
  26. 26.  Sample (User B) :
  27. 27.  Conclusion  Using a Natural User Interface, LeapPlay application was able to improve the skills of disabled children using the serious games specially designed for therapy.  This project introduces a new way of utilizing hand gestures controls for the wellbeing of the society.  The literature review on serious games and hand gesture recognition lead to better understanding of the knowledge area.  Visits to the National Hospital of Sri Lanka and Lady Ridgeway Hospital for Children gave insight of the problem in Sri Lankan context.
  28. 28.  Detailed comparison between hand gestures controllers lead to the identification of the Leap Motion device as a good gesture tracker .  LeapPlay system was implemented according to the design with the limitations mentioned in the design chapter.  The project was evaluated successfully and the results were positive towards achieving the goal.  Finally this project will provide a strong case to support the fact that serious games can contribute towards the virtual rehabilitation.
  29. 29.  Lessons learned  Identified multiple ways to precision hand gesture tracking.  Fatigue is a major concern for long duration game play.  Future work  Integrate the system with haptic feedback gloves.
  30. 30.  Proof of Concept
  31. 31. References [1] Henschke, M., Hobbs, D., & Wilkinson, B. (2012, November). Developing serious games for children with cerebral palsy: case study and pilot trial. In Proceedings of the 24th Australian Computer-Human Interaction Conference (pp. 212-221). ACM [2] Ciman, M., Gaggi, O., Nota, L., Pinello, L., Riparelli, N., & Sgaramella, T. M. (2013). Help Me! A Serious Game for Rehabilitation of Children Affected by CVI. In WEBIST (pp. 257-262). [3] Azahar: Serious Games Improving Autonomy , http://seriousgamesmarket.blogspot.com/2015/09/azahar-serious- games-improving-autonomy.html [4] Serious Games for Physical Threapy , http://advancedmedialab.com/e- health/products/gesture-based-serious-games-for-physical-therapy.html [5] R.A.G.K. Ranathunga, L.A.V.N. Rajakaruna, S.A.T.N. Karunarathne, L.N.Y. Abeywardena,D.P. Nawinna, T. Halloluwa (2014) 'A Gamified Learning Tool for Sri Lankan Primary Schools ', PNCTM, Vol 3.
  32. 32. Thank You

×