1. -2
-1
0
1
2
3
4
5
6
7
8
9
10
0 20 40 60 80
PERFORMANCE(N-BACKLEVEL))
AGE (IN YEARS)
PERFORMANCE GAINS BY AGE
Baseline Performance
Average Performance in last three sessions
Gain in Performance
Baseline (2nd Order Poly)
Last three sessions (2nd Order Poly)
Gain in performance (2nd Order Poly)
Individual Differences in Working Memory Training
Shafee Mohammed1; Benjamin Katz2; Chelsea M. Parlett1; Martin Buschkuehl3; and Susanne M. Jaeggi1
University of California, Irvine1, University of Michigan2, MIND Research Institute3
INTRODUCTION
RESEARCH QUESTIONS
• How do individual difference factors, such as age, baseline
ability, and n-back training type (spatial vs. verbal), influence
the outcome of training?
• How does one’s pattern of performance on the training task
influence transfer gains on untrained tasks?
• Data:
• 418 participants from varied age range, education level,
and geographic locations.
• At least 15 sessions of working memory training.
Anguera et al. (2012). Behavioral Brain Research.
Buschkuehl et al. (2014). Cognitive, Affective, and Behavioral Neuroscience.
Jaeggi et al. (2011). Proceedings of the National Academy of Sciences of the
United States of America.
Jaeggi et al. (2011). Paper presented at the Eighteenth Annual Cognitive
Neuroscience Society Meeting.
Jaeggi et al. (2014). Memory and Cognition.
Jaeggi et al. (2008). Proceedings of the National Academy of Sciences of the
United States of America.
Jaeggi et al. (2010). Intelligence.
Jonides et al. (2010). Presented at the Office of Naval Research - Contractor's
Meeting, Arlington, VA.
Katz et al. (under review).
Seidler et al. (2010). Technical Report No. M-CASTL 2010-01, University of
Michigan, Ann Arbor.
Stepankova et al. (2014). Developmental Psychology.
Tsai et al. (in preparation)
Zhang et al. (2014). 26th Annual Convention of the Association for Psychological
Science, San Francisco, CA.
METHOD
RESULTS
• Age, country in which the study was conducted, and baseline abilities
predict one’s ability to improve at WM training.
• Gender, N-back type, and supervision have no significant effect.
• Prediction accuracy is significantly different from chance with nearly
76% (0.02) (200 iterations) on an average.
DISCUSSION
CONCLUSIONS
• Baseline characteristics, and age are important factors.
• Age, and baseline performance, are good predictors of training
performance (R2 = 0.30).
• Further analysis will include non-linear mixed effect models and
cluster analyses.
• Working memory is critically important for success in school
and at work (Gathercole et al., 2003; Higgins et al., 2007).
• One’s performance on an adaptive and challenging
longitudinal working memory intervention may serve as a
useful assay to investigate cognitive plasticity.
Error Matrix
Target
Label
Predicted
Label
Count
0 0 34
0 1 10
1 1 33
1 0 12
Accuracy: 76% (0.02)
Authors’ note: MB is employed at MIND Research Institute, whose interest is related to
this work. SMJ has an indirect financial interest in the MIND Research Institute.
REFERENCES
Gathercole, S. E., Brown, L., & Pickering, S. J. (2003). Working memory assessments at school entry
as longitudinal predictors of National Curriculum attainment levels. Educational and Child
Psychology, 20(3), 109-122.
Higgins, D. M., Peterson, J. B., Pihl, R. O., & Lee, A. G. (2007). Prefrontal cognitive ability,
intelligence, Big Five personality, and the prediction of advanced academic and workplace
performance. Journal of Personality and Social Psychology, 93(2), 298.
Jaeggi, S. M., Buschkuehl, M., Shah, P., & Jonides, J. (2014). The role of individual differences in
cognitive training and transfer. Memory & Cognition, 42(3), 464-480.
PREDICTORS OF TRAINING
• Not every person improves equally on a WM training task.
• Identifying the factors that potentially contribute to explaining
such individual differences is the key to improving training.
• Data mining and machine learning can be viable approaches to
investigating plasticity.
Figure 2. Average performance on the first three sessions(blue), performance on last three sessions (gray), and
performance gain (purple) as a function of Age.
Figure 1. An example showing visual and auditory 2-back tasks. The stimuli are
presented chronologically from left to right as a visual stimulus or an auditory
stimulus respectively. Participants should match the current stimulus with the nth
stimulus (2nd in this case.)
Table 1. The accuracy of the algorithm to detect whether or not a
subject is in the top fifty percentile.
Table 4. Values in the descriptive table are raw scores and standardized units for regression. Numbers in bold are statistically
significant (in this case, p<0.01). Subjects aged 10-20, from US, Male, Supervised (as opposed to Unsupervised), Trained on
spatial n-back, are left out as controls.
Descriptive statistics Regression Results
Sample Characteristics Mean SD Beta SE
Age (Range 7-78 years) 22.51 16.11
Age Squared 765.68 1313.59
Age Centered 0 16.11 0.25 0.01
Age Centered Squared 258.89 627.57 -0.48 0.00
Gender (Female Proportion) 0.51 0.02 0.08
Baseline N-Back Level 2.64 0.95 0.11 0.05
Final N-Back level 3.74 1.64
Gain 1.11 1.044
Training Characteristics
Supervised participants 0.43 0.08 0.01
Location (US Participants) 0.81 -0.23 0.01
N-back Type -0.03 0.01
Spatial 0.35
Verbal 0.11
Object
Total Single N-back 0.60
Total Dual N-back 0.40
Total Observations 418 R-Squared 0.300
Gathering the data and cleaning
Descriptive statistical analysis
Data mining methods – Logistic classifier and Decision tree analysis
Non-linear mixed models (planned)
Nearest neighbor analysis (planned)
Class Probability
1 0.715
0 0.678
0 0.870
0 0.567
1 0.502
0 0.503
Table 2. The prediction probability of each
individual’s accuracy to detect whether or not
each subject is in the top fifty percentile. The
individual’s in the bold have a low predictive
probabilities.
Figure 3. The prediction accuracies over 200 iterations. Attained prediction range is 0.69 - 0.81.
-2
0
2
4
6
0 10 20 30 40 50 60 70 80
N-backlevel
Age in years
Training gains based on training locations
Non-US Participants
US Participants
Figure 4. The difference between the US (in red) and non-US (blue) participant training gains. The individuals
with negative post training gains are exclusively from the US.
N-BackLevel
Age
Figure 7. (right) shows a
decision tree model used to
predict and classify individuals.
Each individual starts at the
purple dot and moves along the
branches of the tree based on a
decision choice for each
condition. Only a single branch
is shown in full detail for
illustration purposes.
Figure 5. (left) The baseline
performance (top graph) and the
gains (bottom graph) of the top
50% of the participants (in green)
and bottom 50% of the participants
(in red).
TruePositiveRate
False Positive Rate
Schema
Number of examples 315
Number of features 7
Number of unpacked features 7
Number of classes 2
Model Properties
Number of Trees 10
Max tree depth 6
Training accuracy 0.92
Validation Accuracy 0.85
Training log-loss 0.27
Validation log-loss 0.39
Figure 6.(left) ROC curve obtained by aggregating all
predictions from 200 repetitions. Table 3.(above) Shows
the model characteristics and the validation estimates.
DATA SOURCES