Games For Upper-limb Stroke Rehabilitation (Seminar)


Published on

A one hour seminar I gave at my university (University of Ulster) in February 2010. It looks at how video games can be applied to stroke rehabilitation and showcases some work we have conducted in the field, including some webcam games.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Games For Upper-limb Stroke Rehabilitation (Seminar)

  1. 1. Games For Upper-limbStroke RehabilitationJames Burke Michael McNeill Philip Morrow Darryl Charles School of Computing and Information EngineeringSuzanne McDonoughJacqui CrosbieSchool of Life and Health Sciences<br />School of Computing & Information Engineering, University of Ulster, Northern Ireland<br />
  2. 2. Contents<br />Research Overview<br />Background on Stroke & Traditional Therapy<br />Technology for Stroke Rehabilitation<br />Previous Studies<br />Game Design for Rehabilitation<br />Outcome Measures<br />Previous University of Ulster Work<br />Webcam Games (with evaluation)<br />Augmented Reality Games<br />Final Remarks<br />
  3. 3. Research Overview<br />The Problem Area:<br />Stroke is a leading cause of severe physical disability.<br />1,100,000 people living in the UK with impairments as the result of a stroke.<br />111,000 first time strokes occurring each year.1<br />Often difficult to provide appropriate level of therapy to meet a patient’s rehabilitation needs.<br />Difficult to maintain patient motivation.2<br />British Heart Foundation & Stroke Association (2009) Stroke Statistics.<br />Burdea (2002) "Key note address: Virtual rehabilitation – benefits and challenges."<br />
  4. 4. Research Overview<br />The Proposed Solution:<br />Video games may be effective in optimising engagement & motivation.<br />Create a framework for the design of games for upper-limb stroke rehabilitation.<br />Evaluation of appropriate technology for home-based therapy.<br />Design, development and evaluation of rehabilitation games using novel technology.<br />
  5. 5. Research Overview<br />Rehabilitation Therapy<br />Gamesfor Stroke Rehabilitation<br />Mapping to rehabilitation<br />Tasks for rehabilitative systems<br />Technology<br />Game Design<br />
  6. 6. Stroke<br />Effects of stroke:<br />Attention and concentration deficiencies<br />Balance loss<br />Pain<br />Weakness and paralysis<br />Depression<br />Fatigue<br />Can make day-to-day activities difficult.<br />Upper limb remains weak in up to 66% cases.1<br />Van der Lee et al. (1999) “Forced use of the upper extremity in chronic stroke patients”.<br />
  7. 7. Stroke Rehabilitation<br />Rehabilitation programmes usually devised with realistic goals for that particular patient.<br />Phases of rehabilitation:<br />In hospital<br />Travel to specialised units<br />Home-based rehabilitation<br />May involve a visiting professional<br />Patients often do not receive optimal level of therapy.1<br />Burdea (2002) "Key note address: Virtual rehabilitation – benefits and challenges."<br />
  8. 8. Stroke Rehabilitation<br />Early and intensive practice of functional tasks show more positive outcomes for upper-limb rehabilitation.1<br />Typical movements practised:2<br />Reach towards object.<br />Grasp object.<br />Manipulate object.<br />Release object.<br />Single handed and bimanual tasks.<br />Tasks often perceived to be mundane and boring.<br />Therapy can be insufficient.<br />Kwakkel et al. (1999) "Intensity of leg and arm training after primary middle-cerebral-artery stroke”.<br />Carr & Shepherd (2002) “Stroke Rehabilitation: Guidelines for Exercise and Training to Optimize Motor Skill”.<br />
  9. 9. Technology for Stroke Rehab.<br />Benefits:<br />Rehabilitation systems can be novel, interesting, safe and customisable to a particular user’s abilities and interests.<br />Motivate, involve and immerse the user.<br />Potential for home use.1<br />No need for therapist to be present?<br />Session data can be recorded at home & uploaded to remote site.2<br />Limitations:<br />Clinically valid?1,2<br />Cost of equipment? Logistics?<br />Expertise required to setup/operate?<br />Rizzo & Kim (2005) “A SWOT analysis of the field of virtual reality rehabilitation and therapy”.<br />Burdea (2002) "Key note address: Virtual rehabilitation – benefits and challenges."<br />
  10. 10. Technology Interfaces<br />Input mechanisms<br />Electromagnetic sensors<br />Data-gloves<br />Mixed reality systems<br />Vision-based tracking (cameras)<br />Force feedback devices<br />Nintendo Wiiremote<br />Output devices<br />Head mounted displays (HMDs)<br />Projectors<br />Monitors<br />
  11. 11. Existing Studies<br />Bespoke:<br />Sensor-based systems (Crosbie 2004, Yeh 2005)<br />Xbox console with P5 glove (Morrow 2006)<br />Haptics & Robotics (Boian 2002, Guo 2007, Podobnik 2008) <br />Adapted:<br />Sony EyeToy(Rand 2004, Yavuzer 2008)<br />Some patients experienced difficulty playing games.<br />Lack of level grading and suitable challenge for games.<br />Clearly, rehab games need to be flexible to impairments.<br />Nintendo Wii(ongoing studies)<br />
  12. 12. Game Design for Stroke Rehab<br />Existing systems use few or no game design principles.<br />“Designers of rehabilitation tasks can benefit from examining the formulas that commercial game developers use…” 1<br />Video games are often highly engaging.<br />Since rehabilitation should be intensive, games may therefore offer high quality rehabilitation environment.<br />Commercial off-the-shelf games may already offer some benefits to people with stroke, but this could be improved!<br />Rizzo & Kim (2005) “A SWOT analysis of the field of virtual reality rehabilitation and therapy”.<br />
  13. 13. Game Design for Stroke Rehab<br />Play<br />Voluntary activity, rule-bound and meaningful.1<br />Manipulation that satisfies curiosity.3<br />Game<br />A system defined by rules, with a quantifiable outcome.2<br />A problem-solving activity, approached with a playful attitude.3<br />A series of meaningful choices.4<br />Game design<br />Determinant of game-play.<br />Aim to achieve meaningful play.2<br />Huizinga (1955) “Homo Ludens: A Study of the Play-Element in Culture”.<br />Salen & Zimmerman (2003) “Rules of Play: Game Design Fundamentals”.<br />Schell (2008) “The art of game design: a book of lenses”.<br />Sid Meier, game designer. Cited in 2.<br />
  14. 14. Game Design for Stroke Rehab<br />Why play?<br />Challenge and competition<br />Dynamic interactive experience<br />Social interaction<br />Locally or remotely (online)<br />Escapism<br />Emotional experience<br />Fun<br />T-shirt from<br />
  15. 15. Game Design Principles<br />Three important principles selected for rehabilitation:<br />Meaningful Play<br />Handling Failure<br />Appropriate level of challenge<br />Image from<br />
  16. 16. Game Design Principles<br />Meaningful Play<br />Feedback allows the player to measure their progress.<br />Progress towards goals.<br />Upgrading equipment or gaining a level.<br />Goals can be short-term and long-term.<br />Motivate and increase longevity.<br />Inputschoices<br />Player<br />Game<br />Meaningful play<br />Feedback<br />Discernable and integrated outcome<br />
  17. 17. Game Design Principles<br />Handling Failure<br />Failure is a prominent and expectedelement of video games.<br />Should only occur due to player’s incorrect choice.<br />Rehabilitation games should handle failure conservatively.<br />Reward all engagement.<br />Positive and encouraging feedback, even if performance is “poor”.<br />Scoring principles redesigned.<br />
  18. 18. Game Design Principles<br />1,2<br />Maintaining ChallengeAppropriately<br />Match patient abilities.<br />How to maintain challenge?<br />Speed, position and size of game elements can set level of challenge.<br />In-game calibration can determine suitable level of challenge for player’s abilities.<br />Csikszentmihalyi (1988) “Optimal Experience: Psychological Studies of Flow in Consciousness”.<br />Rabin (2005) “Introduction to Game Development”.<br />
  19. 19. Outcome Measures<br />Usability<br />Playability<br />Motivation<br />Engagement / Immersion<br />Enjoyment<br />Functional outcome<br />Assessed by health professional.<br />
  20. 20. Game Usability<br />In software engineering, usability refersto the ease of use of an application’suser interface.1<br />Measured in terms of efficiency, effectivenessand satisfaction.<br />Differs in relation to video games:<br />A good tool should be both easy to learn and easy to master.<br />A good game should be easy to learn but difficult to master.2<br />Game satisfaction emerges from both UI and playability.<br />Both games & software require effective feedback mechanisms.<br />ISO 9241 Part 11 : Guidance on usability -<br />Malone (1982) “Heuristics for designing enjoyable user interfaces: Lessons from computer games”<br />Image from<br />
  21. 21. Motivation<br />Goal-orientated behaviour<br />Intrinsic: Rewards inherent to the task itself.<br />The enjoyment of play, the resulting score, beating a previous high score, improving motor function.<br />Extrinsic: Rewards outside of the task.<br />Peer/therapist encouragement, competition, improving motor function.<br />
  22. 22. Engagement / Immersion<br />Engagement<br />Entry level to immersion: Time, effort and attention need to be invested. <br />Player adequately in control of the game and gaining appropriate feedback.<br />Engrossment<br />The emotional investment the player puts into the game. <br />Player is less self-conscious and less aware of their surroundings.<br />Total immersion<br />Player no longer thinks about the fact that they are playing the game.<br />Brown & Cairns (2004) “A Grounded Investigation of Game Immersion”.<br />
  23. 23. A Framework for Stroke Games<br />Design a framework as a guideline for designing stroke rehabilitation games.<br />Map game design principles to stroke rehabilitation:<br />Suitable level of difficulty.<br />Smooth learning curve.<br />Exercise appropriate range of motions.<br />Locus of movement.<br />Competition.<br />Short-term & long-term goals.<br /><ul><li>Simple game premise.
  24. 24. Appropriate time limits.
  25. 25. Focus on affected side.
  26. 26. Meaningful tasks (for ADLs).
  27. 27. Good usability & playability.
  28. 28. Effective feedback.</li></li></ul><li>A Framework for Stroke Games<br />Mapping Game Design to Rehabilitation<br />Tailored Experience<br />Meaningful Feedback<br />Rehabilitation System<br />Meaningful Play<br />Engagement<br />Enjoyment<br />Motivation<br />Challenge<br />Longevity<br />Improved Outcome?<br />
  29. 29. Previous UU VR Stroke Projects<br />Bilateral catch task:<br />Uses an electromagnetic sensorattached to a real physical basket.<br />User moves basket with both handsto catch falling oranges.<br />Adaptive ‘Whack-a-Mouse’ game:<br />Sensor attached to player’s hand.<br />Encourages arm movement and visual discrimination.<br />Not suitable for home use.<br />
  30. 30. Webcam Games<br />Low cost game system.<br />Contains multiple games which share astored player profile.<br />Single and bimanual arm rehabilitation.<br />Tracks a coloured glove or mitt with anystandard webcam - intuitive controls.<br />Play standing or seated.<br />Optional adaptive dynamic difficulty.<br />Clear and consistent user interface.<br />No attaching of wires required.<br />Potential for home rehabilitation.<br />
  31. 31. Webcam Games<br />L<br />R<br />Image Processed – Pixels identified for each glove<br />Player Movement Wearing Gloves<br />Image Captured by USB Webcam<br />Output to Monitor / Projector<br />
  32. 32. Webcam Games<br />Profile setup<br />Stores individual playerprofile, per player.<br />Stores player information:<br />Player identifier.<br />Affected side.<br />Game speed settings.<br />Game session duration.<br />
  33. 33. Webcam Games<br />Initial ability determination test:<br />Player must roll each ball as far as they possibly can.<br />Test done for each arm.<br />Allows system to determine range of movement and position game elements accordingly.<br />Stored in player profile.<br />Can be used as baseline test.<br />
  34. 34. Webcam Games<br />Single arm exercise:<br />“Rabbit Chase”<br />Player must catch a rabbit as it peers out of holes on the screen.<br />“Bubble Trouble”<br />Player must burst bubbles as they float around the screen.<br />Two arm exercise:<br />“Arrow Attack”<br />Player must touch corresponding arrows with both hands simultaneously as they enter boxes.<br />“Bubble Trouble” (two handed version)<br />Bubbles colour coded and show arrows.<br />
  35. 35. Webcam Games<br />Effective feedback<br />Large, bold, easily identifiable graphics.<br />“Hit” and “Miss” colour coded messages.<br />Particle effects to show a hit.<br />Identifiable sound effects for hits andmisses.<br />Encouraging messages, regardless ofperformance.<br />Score performance chart showsprogress over previous sessions.<br />
  36. 36. Webcam Games - Tools<br />Camera Calibration tool<br />Allows refining of colourdetection settings.<br />Improve automation ofcolour detection in future.<br />Log Analyser<br />Shows graphical representationof movements.<br />Allows replay of game sessions.<br />View scores.<br />Potential for remote viewing.<br />
  37. 37. Evaluation of Games<br />Initial phase – Evaluation with able-bodied users:<br />10 able bodied users playing webcam games.<br />Majority of participants enjoyed all games.<br />Players felt games were enjoyable, replayable and easy to play due to intuitive controls.<br />Adaptivity approved by those who noticed it (>80%), indicating the game would be less enjoyable without it.<br />One user expressed that the adaptivity was too aggressive.<br />> 80% agreed that the feedback was effective.<br />
  38. 38. Evaluation of Games<br />Phase 2 – Single play evaluation with stroke users:<br />Single play sessions to gain feedback on games and determine playability by people with stroke.<br />Eligibility determined by a process of screening for mental confusion and visual impairment. Participants also required some movement in the upper limb.<br />Three participants recruited with varying degrees of movement.<br />Played each game a number of times and filled out questionnaire on usability (including playability).<br />Also rated level of exertion on the Borg-10 scale.<br />
  39. 39. Evaluation of Games<br />
  40. 40. Evaluation of Games<br />All participants displayed enthusiasm and excitement during gameplay.<br />Some expressed interest in obtaining the games for home use.<br />All were able to play the games well despite varying levels of impairment.<br />Lowest score obtained in any game was a respectable 61%.<br />All rated games to be enjoyable and replayable.<br />Lowest score given for enjoyment of game was 7/10.<br />Feedback effective: all agree that they were able to identify when they had made a mistake and know how they made it.<br />All approved of adaptivity feature; however, felt that the games speeded up too quickly when performing well.<br />
  41. 41. Evaluation of Games<br />Phase 3 – Three week single user case studies:<br />Results not yet published.<br />Feedback from study very positive.<br />Included motivation questionnaire.<br />Performance results and motor function outcome are encouraging.<br />
  42. 42. Augmented Reality Games<br />Marker-Based AR.<br />Markers tracked via image from standard webcam.<br />Real-world image augmented with virtual elements.<br />Allows use of physical objects of varying size, shape and weight.<br />Rehabilitate high quality motor skills which are more transferable to activities of daily living?<br />Various libraries: ARToolKit, ARToolKitPlus, ARTag.<br />
  43. 43. Augmented Reality Games<br />“Brick ‘a’ Break”<br />Clone of Atari’s Breakout game.<br />Aim to encourage reach and grasp movement.<br />Player holds a small cube, representing a paddle.<br />No time limit for the player to clear a level – player’s score determined by time taken.<br />Player does not lose a life for missing a ball –short pause while the ball is re-spawned.<br />Future versions will contain multiple levels.<br />Different level layouts.<br />Different ball speeds.<br />Different playfield size.<br />
  44. 44. Augmented Reality Games<br />“Shelf Stack”<br />Designed to encourage reach, grasp, lift and release motor function.<br />Player has several different real world objects.<br />Prompted to pick up a particular object, move it to a specific location on the shelf, then return it to its position.<br />Points awarded for completion of selection-placement tasks and speed.<br />Score as highly as possible within time limit.<br />Requires cognitive skills to discriminate the correct object and placement of the object.<br />
  45. 45. Final Remarks<br />Ongoing & Future Work:<br />Publication of three week trial of games with people with stroke.<br />Development of framework for designing stroke rehabilitation games.<br />Development and evaluation of Augmented Reality games.<br />Conclusion:<br />Games have potential to engage and motivate during rehabilitation.<br />No need for computer or video game experience.<br />Low-cost off-the-shelf hardware may offer an opportunity for home rehabilitation in addition to traditional therapy.<br />