Your SlideShare is downloading. ×
Multimodal Cognitive Load Assessment
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Multimodal Cognitive Load Assessment


Published on

A paper presented at IACSS09

A paper presented at IACSS09

Published in: Sports, Technology, Education

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Multimodal Cognitive Load Assessment for Athletes Natalie Ruiz *, Bo Yin*, Damian Farrow^, Lyndell Bruce^, Ronnie Taib*, and Fang Chen* * NICTA – National ICT Australia ^ AIS – Australian Institute of Sport IACSS2009, Canberra
  • 2. Skill Acquisition and Cognitive Load
    • Cognitive Load Theory [Sweller et. al. 1988]
      • Degree of mental demand while completing a task
      • Intrinsic, extraneous and germane load
      • Must not exceed capacity of working memory
    • High cognitive load not conducive for learning
    • Aim: Assessment of cognitive load
      • Monitor when load is too high or too low
      • Either extreme is problematic for skill acquisition and optimal performance
      • Performance != Cognitive load
  • 3. Impact of high cognitive load on human responses
    • Working memory is shared by cognitive processes e.g. perception and production of responses
    • Resulting disturbance may not be perceptible to humans, but may be machine measurable
    Long-term memory Short-term memory Shared space (7±2) Visual processing (Visuospatial sketchpad) Linguistic processing (Phonological loop) Central executive Perception Response Muscular action Excitation + vocal tract configuration Gesture… Multi-sensory perception . . . . . . Disturbance e.g. latency, pitch, jittering
  • 4. Behavioural Indicators of High Load
    • Possible behavioural data sources for cognitive load assessment:
      • Speech,
      • Mouse speed and pressure,
      • Linguistic or dialogue patterns,
      • Pen input,
      • Eye-gaze,
      • Manual gesture, etc…
    • Advantages
      • Implicit data collection
      • (Relatively) non-intrusive
      • Does not interfere with task flow
      • Can be collected/analysed in real time
  • 5.
    • Netball AIS Decision Making Study
      • Opportunistic data collection
      • Participants: 20 underage-expert subjects
      • Task:
        • 32 netball clips of 5-10s projected onto a wall
        • Competitive scenarios of varying complexity
        • Decision-making: who to pass, what kind of pass, confidence
      • Available Data:
        • Speech responses
        • Video of subject (front)
        • Eye-Gaze (crosshairs)
        • Performance/ Accuracy
        • Subjective ratings
    Exploratory Data Collection: Netball Decision Making
  • 6. Analysis - Speech
    • Hypothesis:
      • Speech signal features will vary as cognitive load changes
    • 600 mins speech data
      • 120 mins effective speech
    • Existing speech classification
    • system developed at NICTA
      • At least 3 level classification (L/M/H)
      • Use half of the data for training
      • Use the other half for testing
      • Applications
        • Traffic control, crisis mgt, etc...
      • Accuracy in the range of 75%-80%
      • Details in the paper + references
  • 7. Results - Speech
    • Analysis 1: Intrinsic Complexity (post hoc)
      • Expert labelling of tasks based on possible choices, difficulty of pass, and pressure
      • Trained and tested six levels of load/difficulty; two-thirds training, one-third testing
      • ~19% accuracy
    • Analysis 2: Grouping Levels
      • Grouped Levels 1&2 (Low) and 5&6 (High) to double the amount of data per model
      • Trained and tested 2 levels (Highest vs Lowest);
      • ~65% accuracy
    • Subjective ratings and performance scores
      • Subjective rating scores only within bandwidth of 2-3 points on 7pt scale
      • Performance scores not as expected for the six load levels.
    • Analysis 3: Best and Worst Performances
      • 3 best performing tasks (100% score) and 3 worst performing tasks (0% score)
      • Trained and tested 2 levels (High and Low)
      • 56% accuracy 
      • Due to low granularity of scoring?
  • 8. Discussion and Future Work
    • So far, best result for speech: 65% accuracy on High vs Low
    • Exploratory study served to highlight issues
      • Surprising that results not due to small amounts of data
      • Necessary to administer tasks of extreme load levels (high and low)
      • Higher granularity in performance scores
      • Look into intrinsic load structures to quantify “designed load” for Netball DM tasks
      • Theory building: does poor task performance mean higher or lower load?
    • Analysis of other measures yet to be done:
      • Eye gaze, fixation and trajectory patterns, search strategies
      • Head movement, proxy of video frame movement
      • Manual gesture (manual annotation)
  • 9. Application Contexts – Sports
    • Multimodal assessment of cognitive load in sports contexts be applied to, for example:
      • Objective supplementary coaching tool
        • Individualised training progressions
        • Individualised athlete assessment (own baselines)
        • Highly customisable to the type of skill being learned
      • Ongoing performance monitoring
        • Possible to assess individual athletes in team sports
        • Performance != experienced load level
        • In training and in the field.
      • Talent identification programs