Nicolas Van Labeke
Learning Sciences Research Institute
National Biomedical Research Unit in Hearing
www.lsri.nottingham.a...
Perceptual Learning & Auditory Training
• Perceptual learning is the relatively
permanent change of perception
following s...
Auditory Training
Time
Skill
Social
Cognitive
Pure tone
Discrimination
Phoneme
Discrimination
Speech
Intelligibility
The Problem
Sweetow & Sabes (2006, p555)
“Despite the high motivation and willingness of the subjects
in this project, ove...
Auditory Testing/Training (1)
STAR package – MRC Institute of Hearing Research, Nottingham
http://www.ihr.mrc.ac.uk/
Auditory Testing/Training (2)
Sound Express – TigerSpeech Technology
http://www.tigerspeech.com/tst_soundex.html
Auditory Testing/Training (3)
Hoop Nut – Scientific Learning® BrainApps™
http://www.scilearn.com/products/brainapps/hoop-n...
Psychophysical Procedures
Marvit et al. (2003)
The Solution (?)
• Combine auditory perceptual learning,
technology-enhanced learning and game
theories to create auditory...
?
Game-based Auditory Learning
Auditory Testing/Training Auditory Learning Game
• Flexible core mechanics
– Interaction wi...
Game-based Auditory Learning
Auditory Bubble
• Training for learning
 testing of learning
• Used pre-existing game
• Firs...
?
Game-based Auditory Learning
Performance & Psychophysical procedures
Auditory Testing/Training Auditory Learning Game
• ...
Intrinsic Integration
• How do we design effective learning games?
• Deploy learning material
• Motivational factors: chal...
Intrinsic Integration (Habgood 2007)
• Motivational effects of challenge, control, etc. are
realized through the core mech...
Game-based
Auditory Discrimination
Therapy
for Tinnitus
Tinnitus
• Perception of sound in the
absence of any
corresponding external
sound
• may be heard in one or both ears
or in...
Neuroscience of Tinnitus in 25s
tonotopy & neuroplasticity
Adjamian et al. (2009)
Herraiz et al (2009)
König et al. (2006)
Moderate quality = Further studies are likely to impact confidence in the estimate of effect and may
change the estimate.
...
Game-based ADT for Tinnitus
Management
• Continuation of NBRUH Tinnitus studies
• Evaluating different game-based paradigm...
Learning Material : Frequency
Discrimination
• Pure tone signal
• Stimuli parameters:
– s : 200ms
– is: 500ms
– Ft : “near...
Auditory Submarine
Interactive listening
2-Interval discrimination
Coerced gameplay
(Theoretical) Difference Limen
=
the smallest change in frequency
that can be detected subjectively
=
Gate size
0
2
4
6
8
10
12
14
16
18
0
200
400
600
800
1000
1200
1400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
L...
Gate 1
Gate 2
Gate 5
. . . . . .
Game Levels
• Level associated with
particular Frequency
Difference Limens
• Level consis...
Gate 1
Gate 2
VisualAuditory
VisualAuditory
Auditory
Gate 5
. . . . . .
Visual Elements
• Visual elements act
as support t...
Auditory Treasure Hunter
Interactive listening
3-Interval discrimination
Un-coerced strategic
gameplay
20
60
40
160 160
120
200
160
0
2
4
6
8
10
12
14
16
18
0
200
400
600
800
1000
1200
1400
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
L...
Game Levels
• Level associated with
particular Frequency
Difference Limens
• Un-coerced goals
– Target as amount of
gold t...
20
60
40
160 160
120
200
Visual Elements
• Visual elements act as
complement to
auditory task
– Missing information
(depth and value of
nuggets)
• ...
By Proximity
By Value
By Value
Study overview
• Cross-over design
– Comparison of treatments
(training phase 1)
– Comparison of game experience
(training...
Questions
• Game-based training paradigm
– Impact on auditory tasks, stimuli, processes and
performance?
Frequency Discrim...
Questions
• Intrinsic integration of core mechanics and
learning material
– Bespoke design vs. generalisation?
– Design gu...
Questions
• What aspects of the design of
educational games & other audio-
related computer systems are applicable
for aud...
References
Amitay, S., Hawkey, D. J., & Moore, D. R.
(2005). Auditory frequency discrimination
learning is affected by sti...
Auditory perceptual learning and game design   an example for tinnitus management
Auditory perceptual learning and game design   an example for tinnitus management
Auditory perceptual learning and game design   an example for tinnitus management
Auditory perceptual learning and game design   an example for tinnitus management
Auditory perceptual learning and game design   an example for tinnitus management
Auditory perceptual learning and game design   an example for tinnitus management
Auditory perceptual learning and game design   an example for tinnitus management
Auditory perceptual learning and game design   an example for tinnitus management
Upcoming SlideShare
Loading in …5
×

Auditory perceptual learning and game design an example for tinnitus management

887 views

Published on

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
887
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Auditory perceptual learning and game design an example for tinnitus management

  1. 1. Nicolas Van Labeke Learning Sciences Research Institute National Biomedical Research Unit in Hearing www.lsri.nottingham.ac.uk/nvl/ http://hearing.nihr.ac.uk/ Auditory perceptual learning and game design An example for Tinnitus management
  2. 2. Perceptual Learning & Auditory Training • Perceptual learning is the relatively permanent change of perception following sensory experience • Auditory training improves listening and language/literacy skills and reduces the handicap associated with hearing impairment • A wide range auditory training software exists
  3. 3. Auditory Training Time Skill Social Cognitive Pure tone Discrimination Phoneme Discrimination Speech Intelligibility
  4. 4. The Problem Sweetow & Sabes (2006, p555) “Despite the high motivation and willingness of the subjects in this project, over 20% initially enrolled in the project dropped out of the study. It is clear that not all patients will be willing to put forth the time or effort required to complete this, or any, rehabilitation program [...]. This cannot be blamed entirely on patients, however. Some audiologists may be reluctant to require patients to add further effort beyond the purchase of hearing devices.” Potentially the problem arises because the training is derived from auditory testing paradigms based on psychophysics & signal detection theory
  5. 5. Auditory Testing/Training (1) STAR package – MRC Institute of Hearing Research, Nottingham http://www.ihr.mrc.ac.uk/
  6. 6. Auditory Testing/Training (2) Sound Express – TigerSpeech Technology http://www.tigerspeech.com/tst_soundex.html
  7. 7. Auditory Testing/Training (3) Hoop Nut – Scientific Learning® BrainApps™ http://www.scilearn.com/products/brainapps/hoop-nut/
  8. 8. Psychophysical Procedures Marvit et al. (2003)
  9. 9. The Solution (?) • Combine auditory perceptual learning, technology-enhanced learning and game theories to create auditory learning games suitable for use outside the lab – training for learning  testing of learning – User-Centred approach of design – Design games that are intrinsically motivating – Wide range of age- and disability-related factors of people with hearing loss favours a casual game approach – Principles of auditory learning suggest the use of intrinsic integration (Habgood 2007)
  10. 10. ? Game-based Auditory Learning Auditory Testing/Training Auditory Learning Game • Flexible core mechanics – Interaction with stimuli (doing something with them) – Creating purpose and aim for individual actions  Better conditions for realisation of motivational factors  Engaging flow experience  Design Space? • Psychoacoustic-based: – Interaction limited to stimuli delivery/detection mechanism – Repetitive and monotonous – aimless and purposeless at short-term  Unfavourable conditions for realisation of motivational factors  Monotonous flow experience  Limited user engagement (hence extrinsic motivation) ? ??
  11. 11. Game-based Auditory Learning Auditory Bubble • Training for learning  testing of learning • Used pre-existing game • First iteration: – Define elements of auditory games – Graft auditory learning material on core mechanics Frozen Bubble http://www.frozen-bubble.org/
  12. 12. ? Game-based Auditory Learning Performance & Psychophysical procedures Auditory Testing/Training Auditory Learning Game • Performance on the game is well defined – Performance on auditory task is unknown – What/how to measure? How to tailor difficulty • Effects of divided attention – visual & auditory tasks • Performance on the auditory task is well defined – Measurement of auditory learning – Tailoring of auditory task difficulty • Limited cognitive load ? ? => Intrinsic Integration as design methodology
  13. 13. Intrinsic Integration • How do we design effective learning games? • Deploy learning material • Motivational factors: challenge, fantasy, curiosity, control, cooperation, competition, recognition (Malone & Lepper 1987) • Habgood (2007): Core mechanics – Rules of the game – Essential interactions required to create gaming experience – Explain individual preferences to games – Define different game genre & flow experience
  14. 14. Intrinsic Integration (Habgood 2007) • Motivational effects of challenge, control, etc. are realized through the core mechanics • Integrate the learning material with the core mechanics and NOT the fantasy “Deliver learning material through the parts of the game that are the most fun to play, riding on the back of the flow experience produced by the game, and not interrupting or diminishing its impact” “Embody the learning material within the structure of the gaming world and the player’s interactions with it, providing an external representation of the learning content that is explored through the core mechanics of the gameplay”
  15. 15. Game-based Auditory Discrimination Therapy for Tinnitus
  16. 16. Tinnitus • Perception of sound in the absence of any corresponding external sound • may be heard in one or both ears or in the middle of the head; may be difficult to pinpoint its exact location; may be low, medium or high-pitched; may be a single noise or two or more components; may be continuous or come and go. • Management use mix of counselling (self-help) and sound-based therapies – Tinnitus Masking (TM): sense of relief, trying to completely mask the tinnitus – Tinnitus Retraining Therapy (TRT): reaching mixing point with ambient noise; habituation & emotional response • Auditory Discrimination Therapy ? 230,000 adults in the UK have tinnitus to a degree that has a severe effect on their ability to lead a normal life. There are 470,000 adults with tinnitus that has a severe effect on their quality of life. 2.3 million adults have tinnitus that makes it difficult for them to sleep. 2.3 million adults have tinnitus that they find severely to moderately annoying. 4.7 million adults have experienced tinnitus for longer than five minutes - this does not include tinnitus experienced for a short time after exposure to loud noise. (RNID 2009)
  17. 17. Neuroscience of Tinnitus in 25s tonotopy & neuroplasticity Adjamian et al. (2009) Herraiz et al (2009) König et al. (2006)
  18. 18. Moderate quality = Further studies are likely to impact confidence in the estimate of effect and may change the estimate. Low quality = Further studies are very likely to change confidence and likely to change the estimate Hoare et al (2010)
  19. 19. Game-based ADT for Tinnitus Management • Continuation of NBRUH Tinnitus studies • Evaluating different game-based paradigms – STARv2 (baseline) • Active listening, trial-based tasks – Auditory Submarine • Interactive listening (sonar analogy) • system-induced gameplay (coerced decision-making) – Auditory Treasure Hunter • Interactive listening (metal detector analogy) • listener-induced gameplay (un-coerced, autonomous decision-making) • Impact of visual representation in gameplay and task performance – Distractor / support / scaffolding – Effectiveness of auditory learning games
  20. 20. Learning Material : Frequency Discrimination • Pure tone signal • Stimuli parameters: – s : 200ms – is: 500ms – Ft : “near” legion-edge (eg 2.5KHz) – Fc : 50% Ft , down to threshold (just noticeable difference) • Constant loudness • Identical signal duration • ADSR envelope for “clean” output (usually AR ramp 20ms) 2I-2AFC 3I-3AFC oddball
  21. 21. Auditory Submarine Interactive listening 2-Interval discrimination Coerced gameplay
  22. 22. (Theoretical) Difference Limen = the smallest change in frequency that can be detected subjectively = Gate size
  23. 23. 0 2 4 6 8 10 12 14 16 18 0 200 400 600 800 1000 1200 1400 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Level-by-level adaptive mechanism Level Delta (4%) Delta (10%) Delta (%20) Training Frequency: 2.5 KHz Staircase: 1-down, 1-up Base: 50% Step: 4% - 10% - 20% Theoretical Difference Limen (Hz) Game Level Runs
  24. 24. Gate 1 Gate 2 Gate 5 . . . . . . Game Levels • Level associated with particular Frequency Difference Limens • Level consist of running through 5 gates – Initial gates acting as “training” to current DL – Final gate acting as “validation” of training & access to next level • “Lives” introduced to limit repetitive failures • Time as a performance marker – Speed increase – booster mode
  25. 25. Gate 1 Gate 2 VisualAuditory VisualAuditory Auditory Gate 5 . . . . . . Visual Elements • Visual elements act as support to auditory task – Gate appearing only after some time in the run • Scaffolding = decay over time – by gate (50%) – by level (staircase) • Other visual elements? – E.g. Submarine fading out
  26. 26. Auditory Treasure Hunter Interactive listening 3-Interval discrimination Un-coerced strategic gameplay
  27. 27. 20 60 40 160 160 120 200
  28. 28. 160
  29. 29. 0 2 4 6 8 10 12 14 16 18 0 200 400 600 800 1000 1200 1400 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Level-by-level adaptive mechanism Level Delta (4%) Delta (10%) Delta (%20) Game Level Runs Theoretical Difference Limen (DELTA) Level n
  30. 30. Game Levels • Level associated with particular Frequency Difference Limens • Un-coerced goals – Target as amount of gold to collect • Define success and failure of level – Limited by number of “lives” (digging) – Time not a constraint (auditory and cognitive processing) • Layout of level (zones, nuggets) – Increase difficulty? – Increase playability? 6 4
  31. 31. 20 60 40 160 160 120 200
  32. 32. Visual Elements • Visual elements act as complement to auditory task – Missing information (depth and value of nuggets) • How to make sure they do not replace the auditory task? – “Detection” scheme for left and right – fading out of content, depth and location • Within level (actions) • Across level (staircase) – Memory?
  33. 33. By Proximity
  34. 34. By Value
  35. 35. By Value
  36. 36. Study overview • Cross-over design – Comparison of treatments (training phase 1) – Comparison of game experience (training phases 1-3) • Evaluation based on tinnitus- related outcomes measures AND desirability & flow experience – Desirability & semi-structured interview at Phase 1 – Usability at Phase 2 & 3 – Preference at Phase 3 • Iterative Design process – Prototype development – Alpha testing: usability with NBRUH/LSRI members (mostly normal hearing) – Beta testing: piloting and usability with participants of previous Tinnitus Study STARv2 Auditory Submarine Auditory Treasure Hunter Phase 1 Phase 2 Phase 3
  37. 37. Questions • Game-based training paradigm – Impact on auditory tasks, stimuli, processes and performance? Frequency Discrimination? Using other dimensions of auditory stimuli?
  38. 38. Questions • Intrinsic integration of core mechanics and learning material – Bespoke design vs. generalisation? – Design guideline and toolkit? – General principles? • Interaction with stimuli (i.e. doing something with them)  Creating purpose and aim for individual actions  Engaging flow experience  Better conditions for realisation of motivational factors • Compliance vs. engagement – Time-on-task as an outcome, not a requirement • “lifelong” commitment? – Impromptu return to task
  39. 39. Questions • What aspects of the design of educational games & other audio- related computer systems are applicable for auditory learning games? • How deep should the intrinsic integration between the auditory material and the core mechanics be? • How do we address the wide range of age- and disability-related factors of the hearing impaired?
  40. 40. References Amitay, S., Hawkey, D. J., & Moore, D. R. (2005). Auditory frequency discrimination learning is affected by stimulus variability. Perception & Psychophysics, 67(4), 691- 698. Moore, D. R., Rosenberg, J. F., & Coleman, J. S. (2005). Discrimination training of phonemic contrasts enhances phonological processing in mainstream school children. Brain and Language, 94(1), 72-85. Smith, G. E., Housen, P., Yaffe, K., Ruff, R., Kenniso n, R. F., Mahncke, H. W., Zelinski, E. M. (2009). A cognitive training program based on principles of brain plasticity: results from the Improvement in Memory with Plasticity-based Adaptive Cognitive Training (IMPACT) study. Journal of the American Geriatrics Society, 57(4), 594- 603. Sweetow, R. W., & Sabes, J. H. (2006). The need for and development of an adaptive Listening and Communication Enhancement (LACE) Program. Journal of the American Academy of Audiology, 17(8), 538-558. de Miranda, E. C., Gil, D., & Iório, M. C. M. (2008). Formal auditory training in elderly hearing aid users. Revista Brasileira de Otorrinolaringologia, 74(6). Fu, Q., & Galvin, J. J. (2007). Computer- Assisted Speech Training for Cochlear Implant Patients: Feasibility, Outcomes, and Future Directions. Seminars in Hearing, 28(2), 142-150. Moore, D. R., Ferguson, M. A., Halliday, L. F., & Riley, A. (2008). Frequency discrimination in children: Perception, learning and attention. Hearing Research, 238(1-2), 147-154. Malone, T. W., & Lepper, M. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. In Aptitude, Learning and Instruction: Conative and affective process analyses, Snow & Farr (Eds.), pp. 253- 223. Habgood, M. P. J. (2007). The Effective Integration of Digital Games and Learning Content. PhD thesis, University of Nottingham. Ross, M. (2005). Home-based auditory and speech reading training; A Review of Four Programs. Hearing loss Magazine, November/December, pp. 30- 34.

×