Predicting and Explaining Individual Performance in Complex Tasks

Predicting and Explaining Individual Performance in Complex Tasks Marsha Lovett, Lynne Reder, Christian Lebiere, John Rehling, Baris Demiral This project is sponsored by the Department of the Navy, Office of Naval Research

Multi-Tasking ,[object Object],[object Object],[object Object],[object Object]

[object Object],Gobet & Ritter, 2000

[object Object],[object Object],[object Object]

Basic Goals of Project ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

How to predict task performance ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Example: Memory Task Performance ,[object Object]

Zero-Parameter Predictions ,[object Object],(Lovett, Daily, & Reder, 2000)

Challenges of Complex Tasks ,[object Object],[object Object],[object Object]

What about knowledge differences? ,[object Object],[object Object],[object Object],[object Object]

Individual Differences in ACT-R ,[object Object],[object Object],[object Object]

ACT-R & Individual Differences M1, M2, M3, … W1, W2, W3, … P1, P2, P3, …

Overview of Talk ,[object Object],[object Object],[object Object],[object Object]

P/M Tasks ,[object Object],[object Object],[object Object]

W affects Performance ,[object Object],[object Object]

Estimating W ,[object Object],[object Object]

Estimating PM ,[object Object],[object Object]

Joint Distribution of W and P/M W and P/M are tapping distinct characteristics

ACT-R & Individual Differences M1, M2 , M3, … W1, W2 , W3, … P1, P2 , P3, …

Specifics of our Approach ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

AMBR: Air Traffic Control Task ,[object Object],[object Object],[object Object],[object Object]

AMBR Task AC=aircraft, ATC=air traffice controller ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Issuing an AMBR Command ,[object Object],[object Object],[object Object],[object Object],[object Object]

General Methods ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Experiments 1 & 2 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Off-the-shelf ACT-R Model of AMBR ,[object Object],[object Object],[object Object],[object Object]

Model Captures Range of Performance

Model Predictions ,[object Object],70 21 Model scenarios with no errors 4 205 Model scenarios with errors Subject scenarios with no errors Subject scenarios with errors

Ind’l Diffs’ Impact on Hold Errors ,[object Object],# Hold Errors Parameter Value

Mean Errors by Scenario Scenario

Be Careful What (DM) you Model ,[object Object],[object Object],[object Object],[object Object]

Observable Behaviors ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Stochastic variation on the single-action level is part of subject and model behavior

The Details Are Inside ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Conclusion thus far… ,[object Object],[object Object],[object Object]

Modifications for Experiment 3 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Raw Characteristics of Data ,[object Object],[object Object],[object Object],[object Object]

Model Modifications ,[object Object],[object Object]

Model Predicts Hold Errors ,[object Object],[object Object],[object Object]

Model Predicts Number of Clicks

W, P/M affect RT click by click ,[object Object],[object Object],Hi-Hi Model & Subject Lo-Lo Model & Subject

W, P/M affect RT click by click ,[object Object],[object Object]

Conclusion thus far ,[object Object],[object Object],[object Object],[object Object],[object Object]

Theoretical Interlude: Spatial vs. Verbal WM ,[object Object],[object Object],[object Object],[object Object]

Opportunity to Test in Current Work ,[object Object],[object Object],[object Object],[object Object],R R R “ normal” “ normal” “ reversed”

Opportunity to Test in Current Work ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Spatial Span taps speed as well… ,[object Object],[object Object],[object Object],[object Object],Mem+Mouse Mouse Mouse -.70 .-56 -.16 Welcome-Tot -.53 -.61 -.20 Welcome-AC -.39 -.55 -.62 SpeedReq-AC PM SS MODS

Theoretical Interlude Conclusion ,[object Object],[object Object]

Strategic Variation Emerges ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Experiment 5 Details ,[object Object],[object Object],[object Object],[object Object],[object Object]

Modeling Specific AMBR Components Easy Scenarios Hard Scenarios Accuracy of first AC click Accuracy of first AC click

Modeling Specific AMBR Components Easy Scenarios Hard Scenarios RT to Correct AC click RT to Correct AC click

Model Predictions Match Data ,[object Object],[object Object],[object Object],[object Object]

Summary of Conclusions ,[object Object],[object Object],[object Object]

Features of Our Approach ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Predicting and Explaining Individual Performance in Complex Tasks

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Predicting and Explaining Individual Performance in Complex Tasks

Similar to Predicting and Explaining Individual Performance in Complex Tasks (20)

More from Ülger Ahmet

More from Ülger Ahmet (20)

Recently uploaded

Recently uploaded (20)

Predicting and Explaining Individual Performance in Complex Tasks