Select technical and
psychometric information
Dr. Kevin McGrew
Institute for Applied Psychometrics (IAP)
© Institute for A...
Detailed
information can
be found in the
extensive 369
page WJ IV
Technical
Manual
Technical Manual Overview
• Chapter 1 – Overview
• Chapter 2 – Test Design and Development
Procedures
• Chapter 3 – Standa...
One of the WJ IV guiding design objectives was for the WJ IV
to retain a focus on psychometric quality that has been
assoc...
Contemporary CHC taxonomic model of human cognitive
abilities was used in the WJ IV test design blueprint
General
Intellig...
Propositions and
Supporting Evidence for
Use of WJ IV Test Scores
Matrix Included in First
Chapter of Technical
Manual
Technical Manual Overview
• Chapter 1 – Overview
• Chapter 2 – Test Design and Development
Procedures
• Chapter 3 – Standa...
The Rasch model was used during test development,
calibration, item pool equating, and scaling
Expert
content,
bias, and
sensitivity
review of
items
completed
(etc)
Statistical item bias (DIF) analyses (sex, race, ethnicity)
informed item selection and test construction
Technical Manual Overview
• Chapter 1 – Overview
• Chapter 2 – Test Design and Development
Procedures
• Chapter 3 – Standa...
National Norm Sample
• 7,416 participants
• Preschool (664)
• K-12 (3,891)
• College/University (775)
• Adult (2,086)
• Ag...
Technical Manual Overview
• Chapter 1 – Overview
• Chapter 2 – Test Design and Development
Procedures
• Chapter 3 – Standa...
Median
reliabilities
(across
all technical
age groups)
for WJ IV
COG clusters
General Intellectual Ability 0.97
Brief Inte...
Median reliabilities (across
all technical age groups) for WJ IV
OL clusters
Oral Language 0.90
Broad Oral Language 0.92
O...
Reading 0.95
Broad Reading 0.97
Basic Reading Skills 0.95
Reading Comprehension 0.93
Reading Comprehension-Extended 0.96
R...
Technical Manual Overview
• Chapter 1 – Overview
• Chapter 2 – Test Design and Development
Procedures
• Chapter 3 – Standa...
Sources of validity
evidence presented
•Representativeness of the WJ IV Test Content, Process,
and Construct Coverage
•Dev...
Sources of validity
evidence presented
•Representativeness of the WJ IV Test Content,
Process, and Construct Coverage
•Dev...
General
Intelligence (g)
Mathematical
achievement
(A3)
Reading
decoding (RD)
Reading
comprehension
(RC)
Reading speed
(RS)...
    WJ IV author classifications   Independent classifications
Battery/Test name   Broad Narrow   Broad Narrow
Tests of Co...
Test construct, process and content analysis 
presented for all COG, OL and ACH tests in 
the technical manual
(Reported f...
Figural-visual
Speed-fluency
Quantitative-numeric
Reading-writing
Auditory-linguistic
-2 -1 0 1 2
-2
-1
0
1
2
ORLVOC
NUMSE...
Sources of validity
evidence presented
•Representativeness of the WJ IV Test Content, Process, and Construct Coverage
•Dev...
Sources of validity
evidence presented
•Representativeness of the WJ IV Test Content, Process, and Construct Coverage
•Dev...
Stage 3:
Confirmatory
structural
model cross-
validation
phase
Stage 2:
Exploratory structural model
generation (MG) and e...
Stage 3:
Confirmatory
structural
model cross-
validation
phase
Stage 2:
Exploratory structural model
generation (MG) and e...
Wards cluster analysis of ages 9-13 year sample (model development sample A; n=785)
NUMREV
OBJNUM
VRBATN
MEMWRD
SENREP
NWD...
At most ages
7-10 principal
component
exploratory
solutions
were
extracted and
reviewed
At most ages
7-10 principal
component
exploratory
solutions
were
extracted and
reviewed
At most ages
7-10 principal
component
exploratory
solutions
were
extracted and
reviewed
-2 -1 0 1 2
-2
-1
0
1
2
ORLVOC
NUMSER
VRBATN LETPAT
PHNPRO
VAL VISUAL
GENINF
NUMREV
NUMPAT
NWDREP
STYREC
PICREC
ANLSYN
OBJ...
A
Two final WJ IV cross-
validated models reported
across age groups
A = Broad CHC top-down
model.
Results presented for a...
Two final WJ IV cross-
validated models finalized
across age groups
B = Broad+narrow CHC
bottom-up model.
Results presente...
Stage 3:
Confirmatory
structural
model cross-
validation
phase
Stage 2:
Exploratory structural model
generation (MG) and e...
Test EFA
Other exp.
methods CFA Cross-validation
WJ III No No Yes No
SB5 No No Yes No
DAS-II No No Yes No
KABC-II No No Ye...
Two final WJ IV cross-validated models across age groups: A = Broad CHC top-down model; B = Broad+narrow CHC bottom-up mod...
Sources of validity
evidence presented
•Representativeness of the WJ IV Test Content, Process, and Construct Coverage
•Dev...
15 independent WJ IV concurrent validity studies
differentiated by age/grade
WJIV COG compared to:
• WISC-IV, WAIS-IV, WPP...
Sources of validity
evidence presented
•Representativeness of the WJ IV Test Content, Process, and Construct Coverage
•Dev...
WJ IV Battery:  Select Technical and Psychometric Information Overview
WJ IV Battery:  Select Technical and Psychometric Information Overview
WJ IV Battery:  Select Technical and Psychometric Information Overview
WJ IV Battery:  Select Technical and Psychometric Information Overview
WJ IV Battery:  Select Technical and Psychometric Information Overview
WJ IV Battery:  Select Technical and Psychometric Information Overview
Upcoming SlideShare
Loading in...5
×

WJ IV Battery: Select Technical and Psychometric Information Overview

4,489

Published on

A brief overview of the content and psychometric information presented in the WJ IV battery technical manual.

Published in: Education, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,489
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "WJ IV Battery: Select Technical and Psychometric Information Overview"

  1. 1. Select technical and psychometric information Dr. Kevin McGrew Institute for Applied Psychometrics (IAP) © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  2. 2. Detailed information can be found in the extensive 369 page WJ IV Technical Manual
  3. 3. Technical Manual Overview • Chapter 1 – Overview • Chapter 2 – Test Design and Development Procedures • Chapter 3 – Standardization and Norm Development Procedures • Chapter 4 – Reliability • Chapter 5 – Validity © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  4. 4. One of the WJ IV guiding design objectives was for the WJ IV to retain a focus on psychometric quality that has been associated with the previous editions of Woodcock-Johnson batteries: •By providing a new, large, and nationally-representative standardization sample of the US population; •By updating items and simplifying test administration and interpretation procedures; •By augmenting the underlying scaling of speeded tests; and •By utilizing state-of-the art data collection, test development, and data analytic methods as models to facilitate progress in the field of applied test development. © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  5. 5. Contemporary CHC taxonomic model of human cognitive abilities was used in the WJ IV test design blueprint General Intelligence (g) Mathematical achievement (A3) Reading decoding (RD) Reading comprehension (RC) Reading speed (RS) Spelling ability (SG) English usage (EU) Writing ability (WA) Writing speed (WS) General verbal information (K0) Lexical knowledge (VL) Listening ability (LS) Induction (I) General sequential reasoning (RG) Quantitative reasoning (RQ) Memory span (MS) Associative memory (MA) Meaningful memory (MM) Naming facility (NA) Visualization (Vz) Visual Memory (MV) Spatial Scanning (SS) Phonetic coding (PC) Quantitative Knowledge (Gq) Quantitative Knowledge (Gq) Reading & Writing (Grw) Reading & Writing (Grw) Comp -Knowledge (Gc) Comp -Knowledge (Gc) Fluid Reasoning (Gf) Fluid Reasoning (Gf) Long-Term Retrieval (Glr) Long-Term Retrieval (Glr) Visual Processing (Gv) Visual Processing (Gv) Auditory Processing (Ga) Auditory Processing (Ga) Processing Speed (Gs) Processing Speed (Gs) Perceptual speed (P) Number Facility (N) General science info. (K1) Knowledge of culture (K2) Domain- Specific Knw. (Gkn) Domain- Specific Knw. (Gkn) Geography ach. (A5) Shading designates proposed changes in CHC model based on analysis of WJ IV COG, OL. ACH norm data (see Chapter 1 and Appendix A in WJ IV Technical Manual) Language development (LD) Working memory capacity (WM) Attentional Control (AC) Speed of Lexical access (LA) Word Fluency (FW) Verbal (print) lang. comp. (V) Short-Term Wk Mem. (Gwm) Short-Term Wk Mem. (Gwm) Memory for Sound Patterns (UM) Memory for Sound Patterns (UM)
  6. 6. Propositions and Supporting Evidence for Use of WJ IV Test Scores Matrix Included in First Chapter of Technical Manual
  7. 7. Technical Manual Overview • Chapter 1 – Overview • Chapter 2 – Test Design and Development Procedures • Chapter 3 – Standardization and Norm Development Procedures • Chapter 4 – Reliability • Chapter 5 – Validity © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  8. 8. The Rasch model was used during test development, calibration, item pool equating, and scaling
  9. 9. Expert content, bias, and sensitivity review of items completed
  10. 10. (etc) Statistical item bias (DIF) analyses (sex, race, ethnicity) informed item selection and test construction
  11. 11. Technical Manual Overview • Chapter 1 – Overview • Chapter 2 – Test Design and Development Procedures • Chapter 3 – Standardization and Norm Development Procedures • Chapter 4 – Reliability • Chapter 5 – Validity © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  12. 12. National Norm Sample • 7,416 participants • Preschool (664) • K-12 (3,891) • College/University (775) • Adult (2,086) • Ages 2-90+ years, Grades K.0-18.0 • 100 geographically diverse communities from 46 states and the District of Columbia • The Standards for Educational and Psychological Testing (aka., the Joint Standards) (American Psychological Association, American Educational Research Association, & National Council on Measurement in Education, 1999; in press) guided the norming and technical analyses © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  13. 13. Technical Manual Overview • Chapter 1 – Overview • Chapter 2 – Test Design and Development Procedures • Chapter 3 – Standardization and Norm Development Procedures • Chapter 4 – Reliability • Chapter 5 – Validity © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  14. 14. Median reliabilities (across all technical age groups) for WJ IV COG clusters General Intellectual Ability 0.97 Brief Intellectual Ability 0.94 Gf-Gc Composite 0.95 Comprehension-Knowledge 0.93 Comprehension-Knowledge- Extended 0.94 Fluid Reasoning 0.94 Fluid Reasoning- Extended 0.96 Short-Term Working Memory 0.91 Short-Term Working Memory- Extended 0.93 Cognitive Processing Speed 0.94 Auditory Processing 0.92 Long-Term Retrieval 0.97 Visual Processing 0.86 Quantitative Reasoning 0.94 Auditory Memory Span 0.90 Number Facility 0.90 Perceptual Speed 0.93 Cognitive Efficiency 0.95 Cognitive Efficiency-Extended 0.95 Reading Aptitude A 0.93 Reading Aptitude B 0.93 Math Aptitude A 0.94 Math Aptitude B 0.95 Writing Aptitude A 0.92 Writing Aptitude B 0.93© Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  15. 15. Median reliabilities (across all technical age groups) for WJ IV OL clusters Oral Language 0.90 Broad Oral Language 0.92 Oral Expression 0.89 Listening Comprehension 0.90 Phonetic Coding 0.95 Speed of Lexical Access 0.89 Vocabulary 0.93 © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  16. 16. Reading 0.95 Broad Reading 0.97 Basic Reading Skills 0.95 Reading Comprehension 0.93 Reading Comprehension-Extended 0.96 Reading Fluency 0.96 Reading Rate 0.96 Mathematics 0.96 Broad Mathematics 0.97 Math Calculation Skills 0.97 Math Problem Solving 0.95 Written Language 0.94 Broad Written Language 0.95 Basic Writing Skills 0.95 Written Expression 0.92 Academic Skills 0.97 Academic Applications 0.96 Academic Fluency 0.97 Academic Knowledge 0.95 Phoneme-Grapheme Knowledge 0.94 Brief Achievement 0.97 Broad Achievement 0.99 Median reliabilities (across all technical age groups) for WJ IV ACH clusters © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  17. 17. Technical Manual Overview • Chapter 1 – Overview • Chapter 2 – Test Design and Development Procedures • Chapter 3 – Standardization and Norm Development Procedures • Chapter 4 – Reliability • Chapter 5 – Validity © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  18. 18. Sources of validity evidence presented •Representativeness of the WJ IV Test Content, Process, and Construct Coverage •Developmental Patterns of WJ IV Ability Clusters •Internal Structure and Relations within the WJ IV •Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement •Performance of Clinical Samples on WJ IV Measures © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  19. 19. Sources of validity evidence presented •Representativeness of the WJ IV Test Content, Process, and Construct Coverage •Developmental Patterns of WJ IV Ability Clusters •Internal Structure and Relations within the WJ IV •Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement •Performance of Clinical Samples on WJ IV Measures © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  20. 20. General Intelligence (g) Mathematical achievement (A3) Reading decoding (RD) Reading comprehension (RC) Reading speed (RS) Spelling ability (SG) English usage (EU) Writing ability (WA) Writing speed (WS) General verbal information (K0) Lexical knowledge (VL) Listening ability (LS) Induction (I) General sequential reasoning (RG) Quantitative reasoning (RQ) Memory span (MS) Working memory capacity (WM) Associative memory (MA) Meaningful memory (MM) Speed of Lexical access (LA) Naming facility (NA) Visualization (Vz) Visual Memory (MV) Spatial Scanning (SS) Phonetic coding (PC) Quantitative Knowledge (Gq) Quantitative Knowledge (Gq) Reading & Writing (Grw) Reading & Writing (Grw) Comp -Knowledge (Gc) Comp -Knowledge (Gc) Fluid Reasoning (Gf) Fluid Reasoning (Gf) Short-Term Wk Mem. (Gwm) Short-Term Wk Mem. (Gwm) Long-Term Retrieval (Glr) Long-Term Retrieval (Glr) Visual Processing (Gv) Visual Processing (Gv) Auditory Processing (Ga) Auditory Processing (Ga) Processing Speed (Gs) Processing Speed (Gs) Perceptual speed (P) Number Facility (N) General science info. (K1) Knowledge of culture (K2) Domain- Specific Knw. (Gkn) Domain- Specific Knw. (Gkn) Geography ach. (A5) WJ IV author-based contemporary CHC broad and narrow ability WJ IV content coverage classification Language development (LD) Attentional Control (AC) Memory for Sound Patterns (UM) Word Fluency (FW) Verbal (print) lang. comp. (V) © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  21. 21.     WJ IV author classifications   Independent classifications Battery/Test name   Broad Narrow   Broad Narrow Tests of Cognitive Ability 01. Oral Vocabulary Gc VL,LD Gc VL 02. Number Series Gf RQ,I Gf RQ 03. Verbal Attention Gwm WM,AC 04. Letter-Pattern Matching Gs P 05. Phonological Processing Ga/Glr PC/LA,FW 06. Story Recall Glr/Gc MM/LS Glr MM 07. Visualization Gv Vz Gv Vz 08. General Information Gc K0 Gc K0 09. Concept Formation Gf I Gf I 10. Numbers Reversed Gwm WM Gwm WM 11. Number-Pattern Matching Gs P Gs P 12. Nonword Repetition Ga/Gwm PC,UM/MS 13. Visual-Auditory Learning Glr MA Glr MA 14. Picture Recognition Gv MV Gv MV 15. Analysis-Synthesis Gf RG Gf RG 16. Object-Number Sequencing Gwm WM Gwm WM 17. Pair Cancellation Gs/Gwm/Gv P/AC/SS Gs P 18. Memory for Words Gwm MS Gwm MS Expert Consensus Evidence: Comparison of WJ IV author and independent broad and narrow CHC classifications of WJ III tests retained in WJ IV Independent classifications based on Flanagan et al. (2013) and Flanagan et al. (2006). (Reported for all COG, OL and ACH tests)
  22. 22. Test construct, process and content analysis  presented for all COG, OL and ACH tests in  the technical manual (Reported for all COG, OL and ACH tests)
  23. 23. Figural-visual Speed-fluency Quantitative-numeric Reading-writing Auditory-linguistic -2 -1 0 1 2 -2 -1 0 1 2 ORLVOC NUMSER VRBATN LETPAT PHNPRO VAL VISUAL GENINF CONFRM NUMREV NUMPAT NWDREP STYREC PICREC ANLSYN OBJNUM PAIRCN MEMWRD PICVOC O R LCM P SEGMNT RPCNAM SENREP UNDDIR SNDBLN RETFLU SNDAWR LWIDNT APPRO B SPELL PSGCMP CALC WRTSMP WRDATK ORLRDG SNRDFL MTHFLU SNWFL RDGREC NUMMAT EDIT WRDFLU SPLSND RDGVOC SCI SOCHUM MEMNAM VRBANL VISCLO NUMSEN Gs-Cog Gs-Ach Glr-LA Ga Grw Gwm- MS Gv Glr-MM Gwm- WM Gc Gq/Gf-RQ Gf Use of MDS Guttman Radex analysis to evaluate content validity Figure is for WJ IV norm study participants from ages 9-13 (model development sample; n = 785). Results for 6 age groups presented in technical manual
  24. 24. Sources of validity evidence presented •Representativeness of the WJ IV Test Content, Process, and Construct Coverage •Developmental Patterns of WJ IV Ability Clusters •Internal Structure and Relations within the WJ IV •Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement •Performance of Clinical Samples on WJ IV Measures
  25. 25. Sources of validity evidence presented •Representativeness of the WJ IV Test Content, Process, and Construct Coverage •Developmental Patterns of WJ IV Ability Clusters •Internal Structure and Relations within the WJ IV •Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement •Performance of Clinical Samples on WJ IV Measures © Institute for Applied Psychometrics;  Kevin McGrew  6-19-14
  26. 26. Stage 3: Confirmatory structural model cross- validation phase Stage 2: Exploratory structural model generation (MG) and evaluation Ages 3-5 A (n = 209) Ages 3-5 B (n = 208) Ages 6-8 A (n =412) Ages 6-8 B (n = 411) Ages 9-13 A (n = 785) Ages 9-13 B (n = 787) Ages 14-19 A (n = 842) Ages 14-19 B (n = 843) Ages 20-39 A (n = 625) Ages 20-39 B (n = 626) Ages 40-90+ A (n = 571) Ages 40-90+ B (n = 575 ) Model Development (MD) samples (A) Model Cross-Validation (MCV) samples (B) Stage 1: Split-sample random sample generation Three-stage internal/ structural validity procedures for WJ IV battery + + Cluster analysis (CA) Multidimensional scaling analysis (MDS) Principal component analysis (PCA) Stage 2A Exploratory structural analysis in Sample A Review of prior WJ, WJ-R, WJ III structural validity research Specify initial confirmatory factor analysis model (CFA) Review of contemporary CHC and cognitive neuroscience research CFA of MG model result from MD (A) sample in MCV (B) sample(s) Yes Most plausible and best fitting odel? Model adjustment(s) No Stage 2B Specify initial CFA models Stage 2C Evaluate and adjust CFA model
  27. 27. Stage 3: Confirmatory structural model cross- validation phase Stage 2: Exploratory structural model generation (MG) and evaluation Ages 3-5 A (n = 209) Ages 3-5 B (n = 208) Ages 6-8 A (n =412) Ages 6-8 B (n = 411) Ages 9-13 A (n = 785) Ages 9-13 B (n = 787) Ages 14-19 A (n = 842) Ages 14-19 B (n = 843) Ages 20-39 A (n = 625) Ages 20-39 B (n = 626) Ages 40-90+ A (n = 571) Ages 40-90+ B (n = 575 ) Model Development (MD) samples (A) Model Cross-Validation (MCV) samples (B) Stage 1: Split-sample random sample generation Three-stage internal/ structural validity procedures for WJ IV battery + + Cluster analysis (CA) Multidimensional scaling analysis (MDS) Principal component analysis (PCA) Stage 2A Exploratory structural analysis in Sample A Review of prior WJ, WJ-R, WJ III structural validity research Specify initial confirmatory factor analysis model (CFA) Review of contemporary CHC and cognitive neuroscience research CFA of MG model result from MD (A) sample in MCV (B) sample(s) Yes Most plausible and best fitting odel? Model adjustment(s) No Stage 2B Specify initial CFA models Stage 2C Evaluate and adjust CFA model All Stage 2A (exploratory) results reported in detail in the technical manual
  28. 28. Wards cluster analysis of ages 9-13 year sample (model development sample A; n=785) NUMREV OBJNUM VRBATN MEMWRD SENREP NWDREP UNDDIR MEMNAM WRDATK SPLSND SNDAWR SEGMNT PHNPRO SNDBLN VAL VISUAL ANLSYN CONFRM VISCLO PICREC RPCNAM RETFLU ORLCMP VRBANL HUM PICVOC ORLVOC GENINF SOC SCI STYREC WRTSMP RDGREC NUMSEN APPROB CALC NUMSER NUMMAT SPELL LWIDNT PSGCMP RDGVOC EDIT ORLRDG MTHFLU WRDFLU SNRDFL SNWRFL NUMPAT LETPAT PAIRCN Gc Glr-MM/Grw Gq/Gf-RQ Grw Gs Gs-ACH Gs-COG Acquired knowledge systems Gv Gf Gf+Gv Glr-LA Gwm- WM Gwm- MS Gwm Ga-PC Ga-PC/Grw Ga Ga+Gwm (sequential/serial processing) Cognitive Operations:   Controlled/deliberate/system 2 General intelligence (g) Bold font = Broad or general CHC abilities Underlined font= Possible intermediate level stratum dimensions Groupings to the left of the vertical dashed lines represent the groupings interpreted as representing valid CHC abilities. Higher- order groupings to the left of the dashed line are potentially interesting hypothesized intermediate ability dimensions. Glr-NA Glr (Retrieval Fluency) Cognitive Operations:   Automatic/system 1
  29. 29. At most ages 7-10 principal component exploratory solutions were extracted and reviewed
  30. 30. At most ages 7-10 principal component exploratory solutions were extracted and reviewed
  31. 31. At most ages 7-10 principal component exploratory solutions were extracted and reviewed
  32. 32. -2 -1 0 1 2 -2 -1 0 1 2 ORLVOC NUMSER VRBATN LETPAT PHNPRO VAL VISUAL GENINF NUMREV NUMPAT NWDREP STYREC PICREC ANLSYN OBJNUM PAIRCN MEMWRD PICVOC O R LCM P SEGMNT RPCNAM SENREP UNDDIR SNDBLN RETFLU SNDAWR LWIDNT APPRO B SPELL PSGCMP CALC WRTSMP ORLRDG SNRDFL MTHFLU SNWRFL RDGREC NUMMAT EDIT WRDFLU SPLSND RDGVOC SCI SOCHUM MEMNAM VRBANL VISCLO NUMSEN Gs-Cog Gs-Ach Glr-LA Ga Grw Gv Glr-MM Gc Gq/Gf-RQ Gwm- WM Gwm- MS WRDATK MDS (Guttman Radex) of ages 9-13 year sample (model development sample A; n=785 ) Gf CONFRM
  33. 33. A Two final WJ IV cross- validated models reported across age groups A = Broad CHC top-down model. Results presented for all cross-validation samples
  34. 34. Two final WJ IV cross- validated models finalized across age groups B = Broad+narrow CHC bottom-up model. Results presented for all cross-validation samples B
  35. 35. Stage 3: Confirmatory structural model cross- validation phase Stage 2: Exploratory structural model generation (MG) and evaluation Ages 3-5 A (n = 209) Ages 3-5 B (n = 208) Ages 6-8 A (n =412) Ages 6-8 B (n = 411) Ages 9-13 A (n = 785) Ages 9-13 B (n = 787) Ages 14-19 A (n = 842) Ages 14-19 B (n = 843) Ages 20-39 A (n = 625) Ages 20-39 B (n = 626) Ages 40-90+ A (n = 571) Ages 40-90+ B (n = 575 ) Model Development (MD) samples (A) Model Cross-Validation (MCV) samples (B) Stage 1: Split-sample random sample generation Three-stage internal/ structural validity procedures for WJ IV battery + + Cluster analysis (CA) Multidimensional scaling analysis (MDS) Principal component analysis (PCA) Stage 2A Exploratory structural analysis in Sample A Review of prior WJ, WJ-R, WJ III structural validity research Specify initial confirmatory factor analysis model (CFA) Review of contemporary CHC and cognitive neuroscience research CFA of MG model result from MD (A) sample in MCV (B) sample(s) Yes Most plausible and best fitting odel? Model adjustment(s) No Stage 2B Specify initial CFA models Stage 2C Evaluate and adjust CFA model All final Stage 2C (exploratory) and Stage 3 (confirmatory cross- validation) results reported in detail in the technical manual
  36. 36. Test EFA Other exp. methods CFA Cross-validation WJ III No No Yes No SB5 No No Yes No DAS-II No No Yes No KABC-II No No Yes (in “exploratory manner”) No WAIS-IV No No Yes No WISC-IV Yes No Yes Yes: Factor score (7) congruence in random CV sample of 440 WJ IV Yes Yes: MDS and cluster analysis Yes: Exploratory model generating CFA (MD samples) and CV CFA (CV samples) Yes: Randomly split samples at six age groups. CV of final MD models in CV sample Structural validity method comparisons across 7 intelligence batteries © Institute for Applied Psychometrics; Kevin McGrew 6- 19-14
  37. 37. Two final WJ IV cross-validated models across age groups: A = Broad CHC top-down model; B = Broad+narrow CHC bottom-up model A B 99.5 % of test and latent factor loadings cross- validated (were significant in all age-differentiated samples cross- validation B samples) 98.6 % of test and latent factor loadings cross- validated (were significant in all age-differentiated samples cross- validation B samples)
  38. 38. Sources of validity evidence presented •Representativeness of the WJ IV Test Content, Process, and Construct Coverage •Developmental Patterns of WJ IV Ability Clusters •Internal Structure and Relations within the WJ IV •Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement •Performance of Clinical Samples on WJ IV Measures © Institute for Applied Psychometrics; Kevin McGrew 6-19-14
  39. 39. 15 independent WJ IV concurrent validity studies differentiated by age/grade WJIV COG compared to: • WISC-IV, WAIS-IV, WPPSI-III, KABC-II, DAS-II WJIV OL compared to: • CELF-F, PPVT-4, CASL, OWLS WJIV ACH compared to: • KTEA-II, WAIT-III, OWLS-WE
  40. 40. Sources of validity evidence presented •Representativeness of the WJ IV Test Content, Process, and Construct Coverage •Developmental Patterns of WJ IV Ability Clusters •Internal Structure and Relations within the WJ IV •Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement •Performance of Clinical Samples on WJ IV Measures

×