Presentation at the Games For Health 2012 Conference in Boston - The presentation outlines the development process of a cognitive training and assessment, highlighting the need to facilitate communication between developer and clnician by building a flexible application framework. In such framework it is of highest importance to be able to change tasks, content, timings, interface and instructions rapidly to adapt to different patient populations. For questions about this presentaton please contact Sebastian Koenig at koenig@assessim.com

1. AssesSim Office
A realistic virtual environment for
assessment of cognitive function
Sebastian Koenig
University of Southern California
Institute for Creative Technologies
skoenig@ict.usc.edu
www.AssesSim.com
www.virtualgamelab.com
Pictures are downloaded from Google Images
and rights belong to their respective owners.
June 2012 Games for Health Conference www.gamesforhealth.org

2. What I’m Playing
June 2012 Games for Health Conference www.gamesforhealth.org

3. Overview
• Cognitive Assessment
• Ecological Validity
• A Framework for Cognitive Assessment
• AssesSim Office
• The Future
• Summary
June 2012 Games for Health Conference www.gamesforhealth.org

4. Cognitive Assessment
• Evaluation of Brain Functions, Cognitive
Abilities
• Clinical Decision-Making
– Ability to perform complex functional tasks in
everyday life
– Examples: Driving a car, making a
breakfast, buying groceries, working in an office
June 2012 Games for Health Conference www.gamesforhealth.org

5. Cognitive Assessment
Current computerized tests to predict
cognitive function in a mostly non-
functional, abstract context.
June 2012 Games for Health Conference www.gamesforhealth.org

6. Cognitive Assessment
Complex situations which we are trying to assess
June 2012 Games for Health Conference www.gamesforhealth.org

7. Ecological Validity
• Extend to which a test is similar to the real world
• (Shadish, Cook & Campbell, 2001)
– Interaction
– Graphics
– Audio
– Content
• Main advantage of virtual reality applications
June 2012 Games for Health Conference www.gamesforhealth.org

8. Ecological Validity
Virtual Classroom, USC ICT Chaim Sheba Medical Center, Tel Aviv Virtual Office, USC ICT
Caren System, Walter Reed AMC
Game-Based Rehab Lab, USC ICT
Recent attempts to build Virtual Reality applications for assessment in
a functional, complex environment
June 2012 Games for Health Conference www.gamesforhealth.org

9. Framework
What is needed to actually build a wide range of functional virtual
environments for cognitive assessment?
• Game Engines- Unity, CryEngine, Unreal Engine, etc.
• 3D Models –
Autodesk, Modo, SketchUp, TuroboSquid, the3DStudio, Dexsoft
Games, etc.
• Powerful Hardware – PC, Smartphones, Tablets
• Interaction Devices – Kinect, PS Move, Wii, Leap, etc.
• Clinical Expertise
• Need for enhanced assessment and training
 All of these elements are readily available to build a flexible
interactive virtual environment
June 2012 Games for Health Conference www.gamesforhealth.org

10. Framework
June 2012 Games for Health Conference www.gamesforhealth.org

11. Framework
• Fully interactive environment
• Evidence-based, clinically relevant content
• Features, Tasks, timings changed easily
– Iterate quickly
– Experiment
– Collect data / feedback
June 2012 Games for Health Conference www.gamesforhealth.org

12. Framework
• Change context
• Individualize content
June 2012 Games for Health Conference www.gamesforhealth.org

13. Framework
• Cognitive and motor rehabilitation
JewelMine, Game-Based Rehab Lab, USC ICT
June 2012 Games for Health Conference www.gamesforhealth.org

14. AssesSim Office
June 2012 Games for Health Conference www.gamesforhealth.org

15. AssesSim Office
• Collaboration with NNL – Kessler Foundation and
USC ICT
• Assessment of Executive
Functions, Attention, Memory in TBI patients
• 16 Iterations
– Input devices (joystick, mouse/keyboard, kinect?)
– Instructions
– Data collection
– Tasks
– Difficulty, Timings
June 2012 Games for Health Conference www.gamesforhealth.org

16. AssesSim Office - Extensions
Once the framework is in place extensions towards a wide range of
functional context becomes possible. As such, the following extensions
have been built in the past six months:
• Navigation / Spatial Orientation
• Wheelchair Training
• Prospective Memory (To-Do-List)
• Attention (Divided / Sustained / Alertness)
• Executive Functions (Organize a meeting)
• Disaster Management
• Motor Rehabilitation (Balance / Reaching)
• Change each scenario over time (Generalization)
• …
June 2012 Games for Health Conference www.gamesforhealth.org

17. The Future
• From “One-offs” to toolkits
• Give more control to the clinician by using intuitive controls
and interfaces to set up tasks and content difficulty
Timeline Editors
June 2012 Games for Health Conference www.gamesforhealth.org

18. The Future
• Individualized assessment and training
• Validation vs. context-sensitivity
June 2012 Games for Health Conference www.gamesforhealth.org

19. The Future
• Collection of metrics
(Usability, Content, Difficulty)
• Quantitative analysis of patient and clinician
interactions with the developed system
• Conceptualizing data
• Assessment vs. Training
– User motivation
– Diversity of content, storyline
– Input from clinician or patient (or both?)
June 2012 Games for Health Conference www.gamesforhealth.org

20. Summary
• Ecological validity
• Flexible framework for cognitive assessment
• Toolkits versus/with individualized content
June 2012 Games for Health Conference www.gamesforhealth.org

21. Thanks to…
• USC ICT
– Skip Rizzo and Belinda Lange
• Kessler Foundation - NNL
– Denise Krch, John DeLuca, Nancy Chiaravalloti
• HIT Lab New Zealand
– Andreas Dünser, Greg Crucian, Christoph Bartneck
• Asklepios Rehabilitation Klinik, Schaufling, Germany
• Rocketbox Studios GmbH
– Markus Wojcik
• Craig Hospital, Denver
June 2012 Games for Health Conference www.gamesforhealth.org

22. Thank You
Sebastian Koenig
University of Southern California
Institute for Creative Technologies
skoenig@ict.usc.edu
www.AssesSim.com
www.virtualgamelab.com
June 12-1, 2013 November 5-6, 2012
Boston, MA www.gamesforhealtheurope.org
June 2012 Games for Health Conference www.gamesforhealth.org