Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Multimodal Cognitive Load Assessment  for Athletes Natalie Ruiz *, Bo Yin*, Damian Farrow^, Lyndell Bruce^, Ronnie Taib*, ...
Skill Acquisition and Cognitive Load <ul><li>Cognitive Load Theory  [Sweller et. al. 1988] </li></ul><ul><ul><li>Degree of...
Impact of high cognitive load on human responses <ul><li>Working memory is shared by cognitive processes e.g. perception a...
Behavioural Indicators of High Load <ul><li>Possible behavioural data sources for cognitive load assessment: </li></ul><ul...
<ul><li>Netball AIS Decision Making Study </li></ul><ul><ul><li>Opportunistic data collection </li></ul></ul><ul><ul><li>P...
Analysis  -  Speech <ul><li>Hypothesis:  </li></ul><ul><ul><li>Speech signal features will vary as cognitive load changes ...
Results - Speech <ul><li>Analysis 1: Intrinsic Complexity (post hoc) </li></ul><ul><ul><li>Expert labelling of tasks based...
Discussion and Future Work <ul><li>So far, best result for speech: 65% accuracy on High vs Low </li></ul><ul><li>Explorato...
Application Contexts – Sports <ul><li>Multimodal assessment of cognitive load in sports contexts be applied to, for exampl...
Upcoming SlideShare
Loading in …5
×

Multimodal Cognitive Load Assessment

1,303 views

Published on

A paper presented at IACSS09

Published in: Sports, Technology, Education
  • Be the first to comment

Multimodal Cognitive Load Assessment

  1. 1. Multimodal Cognitive Load Assessment for Athletes Natalie Ruiz *, Bo Yin*, Damian Farrow^, Lyndell Bruce^, Ronnie Taib*, and Fang Chen* * NICTA – National ICT Australia ^ AIS – Australian Institute of Sport IACSS2009, Canberra
  2. 2. Skill Acquisition and Cognitive Load <ul><li>Cognitive Load Theory [Sweller et. al. 1988] </li></ul><ul><ul><li>Degree of mental demand while completing a task </li></ul></ul><ul><ul><li>Intrinsic, extraneous and germane load </li></ul></ul><ul><ul><li>Must not exceed capacity of working memory </li></ul></ul><ul><li>High cognitive load not conducive for learning </li></ul><ul><li>Aim: Assessment of cognitive load </li></ul><ul><ul><li>Monitor when load is too high or too low </li></ul></ul><ul><ul><li>Either extreme is problematic for skill acquisition and optimal performance </li></ul></ul><ul><ul><li>Performance != Cognitive load </li></ul></ul>
  3. 3. Impact of high cognitive load on human responses <ul><li>Working memory is shared by cognitive processes e.g. perception and production of responses </li></ul><ul><li>Resulting disturbance may not be perceptible to humans, but may be machine measurable </li></ul>Long-term memory Short-term memory Shared space (7±2) Visual processing (Visuospatial sketchpad) Linguistic processing (Phonological loop) Central executive Perception Response Muscular action Excitation + vocal tract configuration Gesture… Multi-sensory perception . . . . . . Disturbance e.g. latency, pitch, jittering
  4. 4. Behavioural Indicators of High Load <ul><li>Possible behavioural data sources for cognitive load assessment: </li></ul><ul><ul><li>Speech, </li></ul></ul><ul><ul><li>Mouse speed and pressure, </li></ul></ul><ul><ul><li>Linguistic or dialogue patterns, </li></ul></ul><ul><ul><li>Pen input, </li></ul></ul><ul><ul><li>Eye-gaze, </li></ul></ul><ul><ul><li>Manual gesture, etc… </li></ul></ul><ul><li>Advantages </li></ul><ul><ul><li>Implicit data collection </li></ul></ul><ul><ul><li>(Relatively) non-intrusive </li></ul></ul><ul><ul><li>Does not interfere with task flow </li></ul></ul><ul><ul><li>Can be collected/analysed in real time </li></ul></ul>
  5. 5. <ul><li>Netball AIS Decision Making Study </li></ul><ul><ul><li>Opportunistic data collection </li></ul></ul><ul><ul><li>Participants: 20 underage-expert subjects </li></ul></ul><ul><ul><li>Task: </li></ul></ul><ul><ul><ul><li>32 netball clips of 5-10s projected onto a wall </li></ul></ul></ul><ul><ul><ul><li>Competitive scenarios of varying complexity </li></ul></ul></ul><ul><ul><ul><li>Decision-making: who to pass, what kind of pass, confidence </li></ul></ul></ul><ul><ul><li>Available Data: </li></ul></ul><ul><ul><ul><li>Speech responses </li></ul></ul></ul><ul><ul><ul><li>Video of subject (front) </li></ul></ul></ul><ul><ul><ul><li>Eye-Gaze (crosshairs) </li></ul></ul></ul><ul><ul><ul><li>Performance/ Accuracy </li></ul></ul></ul><ul><ul><ul><li>Subjective ratings </li></ul></ul></ul>Exploratory Data Collection: Netball Decision Making
  6. 6. Analysis - Speech <ul><li>Hypothesis: </li></ul><ul><ul><li>Speech signal features will vary as cognitive load changes </li></ul></ul><ul><li>600 mins speech data </li></ul><ul><ul><li>120 mins effective speech </li></ul></ul><ul><li>Existing speech classification </li></ul><ul><li>system developed at NICTA </li></ul><ul><ul><li>At least 3 level classification (L/M/H) </li></ul></ul><ul><ul><li>Use half of the data for training </li></ul></ul><ul><ul><li>Use the other half for testing </li></ul></ul><ul><ul><li>Applications </li></ul></ul><ul><ul><ul><li>Traffic control, crisis mgt, etc... </li></ul></ul></ul><ul><ul><li>Accuracy in the range of 75%-80% </li></ul></ul><ul><ul><li>Details in the paper + references </li></ul></ul>
  7. 7. Results - Speech <ul><li>Analysis 1: Intrinsic Complexity (post hoc) </li></ul><ul><ul><li>Expert labelling of tasks based on possible choices, difficulty of pass, and pressure </li></ul></ul><ul><ul><li>Trained and tested six levels of load/difficulty; two-thirds training, one-third testing </li></ul></ul><ul><ul><li>~19% accuracy </li></ul></ul><ul><li>Analysis 2: Grouping Levels </li></ul><ul><ul><li>Grouped Levels 1&2 (Low) and 5&6 (High) to double the amount of data per model </li></ul></ul><ul><ul><li>Trained and tested 2 levels (Highest vs Lowest); </li></ul></ul><ul><ul><li>~65% accuracy </li></ul></ul><ul><li>Subjective ratings and performance scores </li></ul><ul><ul><li>Subjective rating scores only within bandwidth of 2-3 points on 7pt scale </li></ul></ul><ul><ul><li>Performance scores not as expected for the six load levels. </li></ul></ul><ul><li>Analysis 3: Best and Worst Performances </li></ul><ul><ul><li>3 best performing tasks (100% score) and 3 worst performing tasks (0% score) </li></ul></ul><ul><ul><li>Trained and tested 2 levels (High and Low) </li></ul></ul><ul><ul><li>56% accuracy  </li></ul></ul><ul><ul><li>Due to low granularity of scoring? </li></ul></ul>
  8. 8. Discussion and Future Work <ul><li>So far, best result for speech: 65% accuracy on High vs Low </li></ul><ul><li>Exploratory study served to highlight issues </li></ul><ul><ul><li>Surprising that results not due to small amounts of data </li></ul></ul><ul><ul><li>Necessary to administer tasks of extreme load levels (high and low) </li></ul></ul><ul><ul><li>Higher granularity in performance scores </li></ul></ul><ul><ul><li>Look into intrinsic load structures to quantify “designed load” for Netball DM tasks </li></ul></ul><ul><ul><li>Theory building: does poor task performance mean higher or lower load? </li></ul></ul><ul><li>Analysis of other measures yet to be done: </li></ul><ul><ul><li>Eye gaze, fixation and trajectory patterns, search strategies </li></ul></ul><ul><ul><li>Head movement, proxy of video frame movement </li></ul></ul><ul><ul><li>Manual gesture (manual annotation) </li></ul></ul>
  9. 9. Application Contexts – Sports <ul><li>Multimodal assessment of cognitive load in sports contexts be applied to, for example: </li></ul><ul><ul><li>Objective supplementary coaching tool </li></ul></ul><ul><ul><ul><li>Individualised training progressions </li></ul></ul></ul><ul><ul><ul><li>Individualised athlete assessment (own baselines) </li></ul></ul></ul><ul><ul><ul><li>Highly customisable to the type of skill being learned </li></ul></ul></ul><ul><ul><li>Ongoing performance monitoring </li></ul></ul><ul><ul><ul><li>Possible to assess individual athletes in team sports </li></ul></ul></ul><ul><ul><ul><li>Performance != experienced load level </li></ul></ul></ul><ul><ul><ul><li>In training and in the field. </li></ul></ul></ul><ul><ul><li>Talent identification programs </li></ul></ul>

×