Your SlideShare is downloading. ×
0
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
WJ IV NASP 2014 workshop:  Cognitive and Oral Language batteries by Dr. Kevin McGrew
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

WJ IV NASP 2014 workshop: Cognitive and Oral Language batteries by Dr. Kevin McGrew

8,644

Published on

Part 2 of 3 part WJ IV introduction workshop at NASP 2014 in Washington DC.

Part 2 of 3 part WJ IV introduction workshop at NASP 2014 in Washington DC.

Published in: Education
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
8,644
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
9
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. WJ IV Tests of Cognitive Ability: Overview of GIA and CHC factor clusters, new and revised tests, and select data analysis results Dr. Kevin McGrew Institute for Applied Psychometrics (IAP) © Institute for Applied Psychometrics; Kevin McGrew 2-04-14
  • 2. Kevin McGrew WJ IV Presentation Outline • Overview of WJ IV COG/OL system • New or repurposed COG tests • Technical tidbits (through out) • The COG battery design principles • Featuring cognitive complexity • WJ IV COG tests and clusters with a dash of data © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 3. Note: technical information was included in sides during live presentation but have been removed since they are not yet officially published in technical manual © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 4. Gf Gc Gwm Gs Ga Glr Gv General Intellectual Ability – g (7) Brief Intellectual Ability (3) © Institute for Applied Psychometrics; Kevin McGrew 01-23-14 Gf+Gc Composite (4) Fld. Reas. (2) Cmp. Know. (2) ST Wk. Mem. (2) Clusters available from Standard Cognitive easel (10 tests) (#) = # tests Cognitive Efficiency (2) Fld. Reas. (3) Cmp. Know. (3) ST Wk. Mem. (3) Qnt. Reas. (2) Cog. Pr. Spd. (2) Aud. Proc. (2) LT Retrieval (2) Visual. Proc. (2) Perc. Spd. (2) Number Facility (2) Other clusters available by combining Standard Cognitive tests with select tests from Extended Cognitive easel (8 tests) Cognitive Efficiency (4) Reading, Math and Writing Scholastic Aptitudes (each comprised of a mix of 4 different CHC abilities) Vocab. (2) Aud. Mm. Sp. (2) (These two clusters require one test each from Cognitive and Oral Language easels) Broad Oral Lang (3)* Phon. Cod. (2) Spd. Lx. Acc. (2) Oral Lang. (2)* Oral Expression (2) Listening Comp. (2)* (#) = # tests (* English or Spanish) Organization of WJ IV Cognitive (COG) and Oral Language (OL) batteries Clusters available from Oral Language easel(12 tests) (#) = # tests
  • 5. Gf Gc Gwm Gs Ga Glr Gv General Intellectual Ability – g (7) Brief Intellectual Ability (3) Clusters available from Standard Cognitive easel (10 tests) Gf+Gc Composite (4) Fld. Reas. (2) Cmp. Knw.(2) (#) = # tests ST Work. Mem. (2) Cognitive Efficiency (2) Fld. Reas. (3) Qnt. Reas. (3) Cmp. Knw.(3) ST Work. Mem. (3) Cog. Proc. Speed(2) Perc. Speed(2) Number Facility (2) Aud. Proc. (2) LT Ret. (2) Visual Proc. (2) Other clusters available by combining Standard Cognitive tests with select tests from Extended Cognitive easel (8 tests) Cognitive Efficiency (4) Scholastic Aptitude Clusters (each a mix of 4 CHC abilities)
  • 6. Contemporary CHC broad and narrow ability content coverage by WJ IV Cognitive (as per WJ IV authors) General Intelligence (g) DomainSpecific Knw. (Gkn) Quantitative Knowledge (Gq) Reading & Writing (Grw) Comp Knowledge (Gc) Fluid Reasoning (Gf) General science info. (K1) Mathematical achievement (A3) Reading decoding (RD) Language development (LD) Induction (I) Knowledge of culture (K2) Reading comprehension (RC) General verbal information (K0) Geography ach. (A5) Verbal (print) lang. comp. (V) Lexical knowledge (VL) Reading speed (RS) Short-Term Listening ability (LS) Spelling ability (SG) Long-Term Retrieval (Glr) Visual Processing (Gv) Auditory Processing (Ga) Speed (Gs) Memory span (MS) Associative memory (MA) Visualization (Vz) Phonetic coding (PC) Perceptual speed (P) General sequential reasoning (RG) Working memory capacity (WM) Meaningful memory (MM) Visual Memory (MV) Memory for Sound Patterns (UM) Number Facility (N) Quantitative reasoning (RQ) Attentional Control (AC) Naming facility (NA) Spatial Scanning (SS) Work Mem (Gwm) Speed of Lexical access (LA) NEW Word Fluency (FW) English usage (EU) Writing ability (WA) Shading designates abilities measured by WJ IV COG battery Writing speed (WS) © Institute for Applied Psychometrics; Kevin McGrew 01-23-14 Processing
  • 7. WJ IV COG and OL tests by CHC factor domains CHC Factors Gc Gf Gwm Glr Gv Ga Gs Story Recall Visualization (MM) (Vz) Phonological Processing Letter-Pat. Matching (PC/Glr-LA) (P) Nonword Repetition Pair Cancellation (PC/UM-MS) (P/WM-AC) Oral Vocabulary Number Series Verbal Attention (VL) (RQ) (WM/AC) General Information Concept Formation Numbers Reversed Visual-Aud. Learning Picture Recognition (K0) (I) (WM) (MA) (MV) Picture Vocabulary AnalysisSynthesis Memory for Words Rapid Picture Naming (VL/LD) (RG) (MS) (NA/LA) Object-Num. Sequencing Retrieval Fluency Sound Blending (WM) (FI/LA) (PC) Oral Comp. (LS) Segmentation (PC) Sentence Repetition Sound Awareness (MS) (PC) Under. Directions (WM/Gc-LS) COG OL © Institute for Applied Psychometrics; Kevin McGrew 01-23-14 New or sign. change Number-Pat. Matching (P)
  • 8. Two primary WJ IV COG design decisions Gc Gf Gwm Oral Vocabulary Number Series Verbal Attention General Information Concept Formation Numbers Reversed AnalysisSynthesis Memory for Words Glr Gv Story Recall Visualization Visual-Aud Learning Picture Recognition Ga Phon. Gs Processing Letter-Pat. Match. Nonword Pair Repetition Cancellation Number-Pat Match. Obj-Num. Sequencing • Which 7 tests should be combined for the GIA (g) cluster? • Which 2 tests from each CHC factor domain should be combined for the 7 CHC factor clusters?
  • 9. Comparison of composition of primary WJ III and WJ IV COG CHC and GIA clusters Gc Gf Gwm Glr Gv Ga g Gs Primary WJ III and WJ IV COG tests and Clusters Verbal Oral Vocabulary Number Series Attention General Information Concept Formation Numbers Reversed Verbal Comp. AnalysisSynthesis Memory for Words Story Recall Visualization Picture Visual-Aud Learning Recognition Phon. Processing Letter-Pat. Match. Nonword Pair Repetition Canc. Retrieval Spatial Sound Visual Fluency Relations Blending Matching * Auditory Attention GIA Decision (WJ IV Oral Vocab. was part of WJ III Verbal Comp.) (WJ III Spatial Relations is half of WJ IV Visualization) Tests in WJ IV COG clusters WJ IV GIA Tests in WJ III COG Clusters GIA Standard Speed WJ III GIA-Standard * Visual Matching is renamed Number-Pattern Matching in WJ IV © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 10. © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 11. The WJ IV COG is not your fathers intelligence test! The WJ IV COG GIA is a much more cognitively complex (and high g) measure of intelligence How did we do this? What evidence do we have to support this conclusion? © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 12. WJ IV Tests of Cognitive Ability: Overview and select data analysis results How do the general intelligence (g) clusters scores from …………. Compare to the general intelligence (g) total scores from ………… © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 13. Select concurrent validity evidence: Correlations of WJ IV primary COG g-scores with external measures WISC-IV WAIS-IV WPPSI-III KABC-II FS IQ FS IQ FS IQ FCI (n =174) (n =177) (n = 99) (n=50) SB-5 FS IQ DAS-II GCA (n = 50) (n = 49) WJ IV g-measures (see prior note regarding technical info presented at NASP that can’t be released yet until tech. manual published) Conclusion: The WJ IV GIA, BIA and Gf-Gc composite clusters demonstrate strong validity evidence as measures of general intelligence when the criterion are the global composite/total scores from other major IQ batteries in the field
  • 14. Yet to be published preliminary cross-battery gloading and cognitive complexity analysis of primary WJ IV/external IQ battery special studies Spatial position from center of MDS Guttman radex analysis “maps” g-loadings based on factor/principal component analysis g T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 (1a) Spearman’s general Factor model © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 15. 1-factor (unrotated) common-factor solution for WJ IV COG / WISC-IV composite scores (n=173) Verbal Comprehension Index (VCI) Auditory Processing (Ga) Fluid Reasoning (Gf) Perceptual Reasoning Index (PR) Comprehension-Knowledge (Gc) Short-Term Work. Memory (Gwm) Working Memory Index (WMI) Long-Term Retrieval (Glr) Visual Processing (Gv) Processing Speed Index (PSI) Cog. Processing Speed (Gs)  Loadings on Communality first unrotated Estimates common factor 0.809 0.654 0.804 0.646 0.804 0.646 0.800 0.639 0.779 0.607 0.764 0.584 0.749 0.562 0.683 0.466 0.604 0.365 0.569 0.323 0.537 0.288
  • 16. •The WJ IV GIA score is as good (better?) a measure of general intelligence (g) as the WISC-IV FS IQ when defined by g-loadings and MDS cognitive complexity analysis. Measures closer to the center of the radex of more cognitively complex 2 1 GWM GV WMI PRI  FSIQ PSI  0 • The WJ IV Ga cluster is a measure of complex cognitive abilities; comparable to WJ IV & WISC-IV Gf/PRI composites. GF VCI GIA GA  GLR -1 GC GS -2 -2 -1 0 1 2 MDS solution for WJ IV / WISC-IV composite and g-scores (n=173) 2 •The WJ IV measures cognitive abilities not represented in the WISC-IV (Ga, Glr, and possibly Gv).
  • 17. The WJ IV Auditory Processing (Ga) cluster is not your fathers Ga measure. WJ IV still has the Oldsmobile Ga (Phonetic Coding) in OL: COG now has more cognitively complex Ga measures The WJ IV has taken a broader contemporary view of the domain of Ga © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 18. Auditory Processing (Ga) abilities should no longer be considered the Rodney Dangerfield of CHC abilities School psych. and SLD have had a myopic “lamp post-search ” blinder focus on only one part of the very broad domain of Ga There has been an explosion of research (since Carroll’s 1993 treatise) that has identified potentially new important and cognitively complex Ga narrow abilities © Institute for Applied Psychometrics; Kevin McGrew 02-01-14
  • 19. Auditory Processing (Ga) abilities, when properly measured, should have a prominent chair at the roundtable of cognitive CHC abilities © Institute for Applied Psychometrics; Kevin McGrew 02-01-14
  • 20. Five primary design principles for WJ IV COG GIA The WJ IV COG GIA cluster tests should: 1. Be the best factor indicators of each CHC broad domain 2. Be the best predictors of achievement from each CHC broad domain 3. Be the most cognitively complex indicators from each CHC broad domain 4. Be the best measures of g (general intelligence) from each CHC broad domain 5. Collectively should have a relatively equal balance of type of stimulus characteristics (verbal, numeric, figural) Related
  • 21. What is cognitive complexity? CHC factor breadth Cognitive complexity Factorial complexity (Does not necessarily equal) Degree of g-loading Complicated © Institute for Applied Psychometrics; Kevin McGrew 02-10-14
  • 22. Cognitively complex design characteristics (Lohman, 2011) • Larger number of cognitive component processes Parameters of cognitive efficiency in info. proc. models • Accumulation of speed component differences • Increased demands of attentional control (AC) & working memory • More important component processes (e.g., inference) • More demands on adaptive functions (assembly, control, and monitoring).
  • 23. One design objective in the WJ IV was to increase the cognitive complexity requirements for selected tests and clusters to provide greater ecological validity and interpretive relevance of the measures. Approach 1. Increasing the cognitive complexity of a test is often accomplished by making the test a mixed measure of more than one narrow CHC ability (factorially complex mixed CHC measures) Approach 2. A second approach is to increase the complexity of information processing demands of the tests within a specific narrow CHC domain (Lohman & Larkin, 2011; McGrew, in press b). This second form of cognitive complexity, not to be confused with factorial complexity, places greater demands on cognitive information processing (cognitive load), requires greater allocation of key cognitive resources (working memory or attentional control), and invokes the involvement of more cognitive control or executive functions (Arend, Colom, Botella, Contreras, Rubio, & Santacreu, 2003; Jensen, 2011; Lohman & Larkin, 2011; Marshalek, Lohman, & Snow; 1983). This second approach to increasing test cognitive complexity was a primary design principle for the WJ IV. © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 24. Analogy: Think of general intelligence (g) as a system of relatively independent cognitive abilities (relatively construct “pure” pulleys) working together to deal with a specific cognitive task load Gf © Institute for Applied Psychometrics; Kevin McGrew 01-23-14 Gwm Gc Ga Gs Gv Glr G? Most contemporary CHC designed individual tests have focused on developing relatively pure measures of each cognitive ability (mental pulley)
  • 25. Approach 1 to developing cognitively complex tests – Construct factorially complex measures (a system of pulleys from 2 or more domain functions working in combination). Gf + Gv Gf + Gv Gwm Gc Ga Glr Gv Gs © Institute for Applied Psychometrics; Kevin McGrew 01-23-14 In this approach a test is designed to be a mixed measure of two (or more) cognitive abilities (mental pulleys; Gf + Gv)
  • 26. Approach 1 example Gf + Gv Gf + Gwm + Gc + Gq Ga Glr Gv Gs © Institute for Applied Psychometrics; Kevin McGrew 01-23-14 Wechsler Arithmetic test
  • 27. Approach 2 is to increase the complexity of information processing demands of the tests within a specific CHC cognitive functional domain. Tasks are still as relatively pure a measure of the CHC domain as possible but there is a deliberate increase in the number of “mini-pulleys” (cognitive information component complexity) that work together within the CHC domain. This was the primary approach used for certain WJ IV tests. Ga Gv Gwm © Institute for Applied Psychometrics; Kevin McGrew 01-23-14 Gc Gf Glr Gv Gs
  • 28. Professor Butts and the “over engineered” self-operating napkin machine (aka., Rube Goldberg machines)
  • 29. COG Test 5: Phonological Processing • Ga (PC) / Glr (LA/FW) •3 subtests (Word Access; Word Fluency; Substitution • Measures three aspects of speech sound processing that requires the efficiency construction of soundbased lexical representations • High in cognitive complexity and g. Best single Ga test predictor of achievement. High loading on Ga and secondary low loading on Gc (accessing the lexicon). Also loaded on narrow LA factor in broard+narrow bottom-up CFA models. • In GIA, Ga, and all reading and writing scholastic aptitude clusters
  • 30. COG Test 5A: Phonological ProcessingWord Access • Word Access: Examinee provides a word that has a specific phonemic element in a specific location (Sample items deleted for test security reasons)
  • 31. COG Test 5B: Phonological ProcessingWord Fluency • Word Fluency: Examinee names as many words as possible that begin with a specified sound in 1 minute (Sample items deleted for test security reasons)
  • 32. COG Test 5B: Phonological ProcessingSubstitution • Substitution: Examinee substitutes part of a word to create a new word (Sample items deleted for test security reasons)
  • 33. Five primary design principles for WJ IV COG GIA The WJ IV COG GIA cluster tests should: 1. Be the best factor indicators of each CHC broad domain 2. Be the best predictors of achievement from each CHC broad domain 3. Be the most cognitively complex indicators from each CHC broad domain 4. Be the best measures of g (general intelligence) from each CHC broad domain 5. Collectively should have a relatively equal balance of type of stimulus characteristics (verbal, numeric, figural) (Note: Trade-offs between the four design objectives were sometimes necessary) © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 34. Three-stage internal/ structural validity procedures for WJ IV battery Stage 1: Split-sample random sample generation (total n = 7,416) Model Development (MD) samples (A) Ages 3-5 A (n = 209) Model Cross-Validation (MCV) samples (B) Ages 3-5 B (n = 208) Stage 2: Exploratory structural model generation (MG) and evaluation Stage 3: Confirmatory structural model crossvalidation phase Stage 2 a Exploratory structural analysis in Sample A Cluster analysis (CA) Model adjustment(s) No Ages 6-8 B (n = 411) + Ages 9-13 A (n = 785) Ages 9-13 B (n = 787) Principal component analysis (PCA) Ages 14-19 A (n = 842) Ages 14-19 B (n = 843) Ages 20-39 A (n = 625) Ages 20-39 B (n = 626) Ages 40-90+ A (n = 571) Ages 40-90+ B (n = 575 ) Ages 6-8 A (n =412) Specify initial confirmatory factor analysis model (CFA) + Multidimensional scaling analysis (MDS) Review of contemporary CHC and cognitive neuroscience research Review of prior WJ, WJ-R, WJ III structural validity research Copyright; Institute for Applied Psychometrics; K.McGrew 1-23-14 Most plausible and best fitting odel? CFA of MG model result from MD (A) sample in MCV (B) sample(s) Yes
  • 35. Structural validity method comparisons across 7 intelligence batteries EFA Other exp. methods CFA Cross-validation WJ III No No Yes No SB5 No No Yes No DAS-II No No Yes No KABC-II No No Yes (in “exploratory manner”) No WAIS-IV No No Yes No WISC-IV Yes No Yes Yes: Factor score (7) congruence in random CV sample of 440 WJ IV Yes Yes: MDS and cluster analysis Yes: Exploratory model generating CFA (MD samples) and CV CFA (CV samples) Yes: Randomly split samples at six age groups. CV of final MD models in CV sample Test
  • 36. A 99.5 % of test and latent factor loadings crossvalidated (were significant in all agedifferentiated samples crossvalidation B samples) !!!!!! 98.6 % of test and latent factor loadings crossvalidated (were significant in all agedifferentiated samples crossvalidation B samples) !!!!!! B Two final WJ IV cross-validated models across age groups: A = Broad CHC top-down model; B = Broad+narrow CHC bottom-up model
  • 37. Five primary design principles have resulted in the WJ IV COG GIA being significantly different in composition than the WJ III COG GIA-Std. The WJ IV COG GIA cluster tests should: 1. Be the best factor indicators of each CHC broad domain 2. Be the best predictors of achievement from each CHC broad domain 3. Be the most cognitively complex indicators from each CHC broad domain 4. Be the best measures of g (general intelligence) from each CHC broad domain 5. Collectively should have a relatively equal balance of type of stimulus characteristics (verbal, numeric, figural) (Note: Trade-offs between the four design objectives were sometimes necessary) © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 38. Two primary methods for measuring cognitive complexity of tests Spatial position from center of MDS Guttman radex analysis “maps” g-loadings based on factor/principal component analysis g T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 (1a) Spearman’s general Factor model © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 39. Guttman’s Radex Theory Ability tests can be classified by: • Degree of cognitive complexity • Differences in kind of content • Differences in type of processes Uses MDS (multidimensional scaling) © Institute for Applied Psychometrics; Kevin McGrew 02-05-14
  • 40. Example of MDS (Radex Model) © Institute for Applied Psychometrics; Kevin McGrew 02-05-14 The closer a test is to the center of the figure, the more it is related to other tests. The center represents the most cognitively complex tests
  • 41. 2 The MDS “distance metrics” were saved. The calculation of a distance-based “relative cognitive complexity” (rCC) metric for COG/OL tests GENINF SENREP 1 SNDBLN NWDREP COG test closest to center (Phonological Processing) identified. © Institute for Applied Psychometrics; Kevin McGrew 02-04514 PICVOC HUM SCI SOC ORLRDG WRDATK SEGMNT RDGVOC ORLCMP STYREC VRBANL MEMWRD VRBATN SNDAWR UNDDIR 0 EDIT LWIDNT WRTSMP RDGREC ORLVOC PSGCMP SPELL PHNPRO SPLSND NUMSEN NUMSER APPROB CALC SNRDFL NUMMAT MEMNAM MTHFLU SNWRFL WRDFLU CONFRM VAL VISUAL OBJNUM NUMREV NUMPAT LETPAT ANLSYN -1 RETFLU VISCLO COG OL ACH Res. PAIRCN PICREC RPCNAM -2 -2 -1 0 1 2 The distance of all other tests from Phonological Processing was calculated so that high values indicated more relative cognitive complexity (rCC) and lower values (PICREC) indicated less rCC. Guttman radex MDS of all WJ IV (and 4 research tests) in WJ IV norm sample (ages 6-90+)
  • 42. PHNPRO-Ga SNDAWR-Ga 2.10 MDS relative CC (rCC) Correlation = .93; but correspondence diverges as test become higher in g and cognitive complexity VRBATN-Gwm 1.55 NUMSER-Gf NUMREV-Gwm SEGMNT-Ga ORLVOC-Gc ORLCMP-Gc UNDDIR-Gwm CONFRM-Gf OBJNUM-Gwm PICVOC-Gc SENREP-Gwm STYREC-Glr 1.00 RETFLU-Glr LETPAT-Gs NUMPAT-Gs 0.45 VISUAL-Gf ANLSYN-GF MEMWRD-Gwm SNDBLN-Ga NWDREP-Ga PAIRCN-Gs RPCNAM-Glr PICREC-Gv -0.10 0.4 0.5 g-loading 0.6 0.7 Relationship between g-loadings and MDS-based relative cognitive complexity (rCC) for WJ IV COG and OL tests 0.8
  • 43. Five primary design principles for WJ IV COG GIA The WJ IV COG GIA cluster tests should: 1. Be the best factor indicators of each CHC broad domain 2. Be the best predictors of achievement from each CHC broad domain 3. Be the most cognitively complex indicators from each CHC broad domain 4. Be the best measures of g (general intelligence) from each CHC broad domain 5. Collectively should have a relatively equal balance of type of stimulus characteristics (verbal, numeric, figural) Related
  • 44. The Cognitive tests were evaluated on the basis of four (of five total) quantifiable COG design criteria Real data augmented by Siskel and Ebert informal rating system  Average CHC factor loadings  Average achievement correlation across domains   Average degree of g-loadings Average degree of relative cognitive complexity (Average across ages 6-90+) © Institute for Applied Psychometrics; Kevin McGrew 02-05-14
  • 45. Author/expert CHC narrow factor classifications Comprehension – Knowledge (Gc) Lexical Knowledge (VL) Language Development (LD) PICVOC (Voc. ClusterLD/VL) * Test in GIA ORLVOC*    © Institute for Applied Psychometrics; Kevin McGrew 2-04-14 General Information (K0) (Gc cluster) GENINF
  • 46. COG Test 2: Number Series • Was in WJ III Diagnostic Supplement • Gf-RQ (Quantitative Reasoning) • Not a “controlled learning” test as are Concept Formation (Gf-I) and Analysis-Synthesis (Gf-RG) • More Gf “in the wild” – without examiner provided scaffolding • Extensive history as a premier Gf test in the psychometric measurement of intelligence • High in cognitive complexity and g. Best single test predictor of achievement. Best indicator of Gf factor. • In GIA, BIA, Gf-Gc Composite, Gf, Gf3, Quantitative Reasoning (RQ), and one Math Aptitude clusters.
  • 47. COG Test 2: Number Series The examinee is presented with a series of numbers with one number missing in the series. The examinee must determine the missing number. (Sample items deleted for test security reasons)
  • 48. One of the most frequent Gf variables found in Carroll’s 1993 seminal review © Institute for Applied Psychometrics; Kevin McGrew 01-23-14 © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 49. Number Series: Extensive history as a premier Gf test in the psychometric measurement of intelligence © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 50. Extensive research literature that has investigated the nature of Number Series tests © Institute for Applied Psychometrics; Kevin McGrew 01-23-14
  • 51. Author/expert CHC narrow factor classifications Fluid Reasoning (Gf) Quantitative Reasoning (RQ) NUMSER*  Induction (I) (Gf cluster) CONFRM   General Sequential Reasoning (RG) (Gf3 cluster) (Quantitative Reasoning Cluster-RQ) * Test in GIA © Institute for Applied Psychometrics; Kevin McGrew 2-04-14 ANLSYN 
  • 52. COG Test 3: Verbal Attention • Measure of Gwm (working memory-WM; attentional control-AC) • More ecological “real world” valid measure of working memory • High in cognitive complexity and g. Within Gwm, the most cognitively complex, one of best indicators of Gwm factor, and best predictor of achievement • In GIA, BIA, Gwm, Gwm3, Cognitive Efficiency, and one Reading and 1 Written Language Aptitude clusters.
  • 53. COG Test 3: Verbal Attention (Sample items deleted for test security reasons) In addition to working memory (WM), task requires attentional control (AC: controlled executive function; aka, focus)
  • 54. Author/expert CHC narrow factor classifications Short-Term Working Memory (Gwm) Attentional Control (AC) VRBATN* Memory Span (MS) Working Memory (WM) NUMREV    cluster) (Gwm  OBJNUM  (Gwm3 cluster) * Test in GIA © Institute for Applied Psychometrics; Kevin McGrew 2-04-14  MEMWRD (Auditory Memory Span Cluster-MS) SENREP
  • 55. COG Test 4: Letter-Pattern Matching • Measure of Gs (perceptual speed) and orthographic processing • This speeded test (all WJ IV speeded tests) is based on a new rate-based method of scaling the scores that eliminates the need for bonus points • Within Gs, it matches Number Pattern Matching in g, Gs factor loading, and prediction of achievement. Is more cognitively complex than Number Pattern Matching • In GIA, Gs, Perceptual Speed (P), Cog. Eff. and Cog. clusters
  • 56. COG Test 4: Letter-Pattern Matching (Sample items deleted for test security reasons) Examinee locates and circles the two identical letter patterns in a row of six patterns in the Response Booklet
  • 57. Short-Term Working Memory (Gwm) Cognitive Processing Speed (Gs) Author/expert CHC narrow factor classifications Perceptual Speed (P) LETPAT*    Attentional Control NUMPAT  PAIRCN  (Gs cluster) (Perceptual Speed cluster-P) © Institute for Applied Psychometrics; Kevin McGrew 2-04-14 * Test in GIA
  • 58. Long-Term Retrieval (Glr) Meaningful Memory (MM) Associative Memory (MA) STREC* (Glr cluster)   VAL   Author/expert CHC narrow factor classifications Speed of Lexical Access (LA) RPCNAM (Speed RETFLU * Test in GIA © Institute for Applied Psychometrics; Kevin McGrew 2-04-14 of Lexical Access cluster-LA)
  • 59. OL Test 3: Segmentation • • • • • Ga (PC) Examinee listens to words and identifies word parts In OL Phonetic Coding (PC) cluster Highest loading test on Ga factor across all ages A moderate measure of g and predictor of ach. across all ages; much more so (and more cognitively complex) than Sound Blending. • Such tasks have been reported to be strong predictors of early reading (Bouwmeester et al, 2011; Geuden & Sandra, 2003)
  • 60. OL Test 3: Segmentation (Sample items deleted for test security reasons)
  • 61. Short-Term Working Memory (Gwm) Auditory Processing (Gs) Author/expert CHC narrow factor classifications PHNPRO*  Memory for Sound Patterns (UM) Phonetic Coding (PC) SEGMNT SNDBLN (Phonetic Coding cluster-PC) (Ga cluster) * Test in GIA © Institute for Applied Psychometrics; Kevin McGrew 2-04-14 NWDREP
  • 62. COG Test 7: Visualization • Measure of Gv-Visualization (Vz) •Visualization consists of two subtests that each measure Gv-Vz (visualization) via tasks that vary on task complexity and degree of “minds eye” (mental rotation) manipulations • Within Gv, highest on cognitive complexity, g, Gv factor, and prediction of achievement • In GIA, Gv and both Math Aptitude clusters
  • 63. COG Test 7A: Visualization-Spatial Relations (Sample items deleted for test security reasons) Examinee identifies the two or three pieces that form a complete target shape
  • 64. COG Test 7B: Visualization-Block Rotation • Was in WJ III Diagnostic Supplement • Gv-Vz (Visualization) • Based on classic “mental rotation” tasks (e.g., Shepard& Metzlar; Vandenberg) (Sample items deleted for test security reasons) Examinee identifies the two block patterns that match the target pattern, but have been rotated in space
  • 65. COG Test 7B: Visualization-Block Rotation • Was in WJ III Diagnostic Supplement • Gv-Vz (Visualization) • Based on classic “mental rotation” tasks (e.g., Shepard & Metzlar; Vandenberg)
  • 66. Author/expert CHC narrow factor classifications Visual Processing (Gv) Visualization (Vz) Visual Memory (MV) VISUAL*  (Gv cluster) * Test in GIA © Institute for Applied Psychometrics; Kevin McGrew 2-04-14 PICREC
  • 67. WJ IV COG Scholastic Aptitude (SAPT) Clusters: Primary design considerations Back to the future: The SAPT concept is not new. SAPT’s were included in the WJ and WJ-R; they were replaced with the Predicted Achievement option in the WJ III. Each WJ IV COG SAPT cluster score is based on a combination of four tests that together produce the most efficient and strongest prediction for the selected achievement area. The SAPTs are the combination of WJ IV COG tests that are the most highly correlated with each curricular area as identified statistically (multiple regression); consideration was also given to relevant theory and the extant research literature (e.g., McGrew & Wendling,2010). Some tests were eliminated as possible predictor tests (e.g., Number Series) if there was clear predictor-criterion contamination (e.g., Math Problem Solving cluster includes Number Matrices). Each SAPT must include four tests that are each from different CHC broad ability domains. Copyright, Institute for Applied Psychometrics, K.McGrew; 01-23-14
  • 68. Clarification of Ability Construct Terminology Copyright, Institute for Applied Psychometrics, K.McGrew; 01-23-14
  • 69. Ability “as used to describe an attribute of individuals, ability refers to the possible variations over individuals in the liminal levels of task difficulty (or in derived measurements based on such liminal levels) at which, on any given occasion in which all conditions appear favorable, individuals perform successfully on a defined class of tasks” (p. 8, italics in original). “every ability is defined in terms of some kind of performance, or potential for performance (p. 4).” Cognitive Abilities Abilities on tasks “in which correct or appropriate processing of mental information is critical to successful performance” (p. 10; italics in original). Achievement abilities “refers to the degree of learning in some procedure intended to produce learning, such as an informal or informal course of instruction, or a period of self study of a topic, or practice of a skill” (p. 17). As noted by Carroll (1993) Copyright, Institute for Applied Psychometrics, K.McGrew; 01-23-14
  • 70. Abilities Cognitive Abilities Achievement Abilities General Intelligence (g) Quantitative Knowledge (Gq) Comp Knowledge (Gc) Fluid Reasoning (Gf) Short-Term Memory (Gsm) Long-Term Storage & Retrieval (Glr) Visual Processing (Gv) Auditory Processing (Ga) Speed (Gs) Etc. Reading & Writing (Grw) Etc. Etc. Etc. Etc. Etc. Etc. Processing Rdg Apt Math Apt Etc. Etc. Ach. domaingeneral apt. Ach. domainspecific apt. Copyright, Institute for Applied Psychometrics, K.McGrew; 01-23-14 Vertical columns represent abilities, factors or latent traits (primarily factor-analysis derived internal structural validity constructs) Horizontal arrow rows represent aptitudes (primarily multiple regression derived external [predictive] validity constructs) Conceptual distinction between Abilities: Cognitive abilities, achievement abilities, and aptitudes (McGrew, in press b)
  • 71. WJ IV Scholastic Aptitude Cluster Organization Gf Concept Formation (I) Gc Gwm Oral Vocabulary (LD/VL) Oral Vocabulary (LV/VL) Number Series (RQ) Oral Vocabulary (LD/VL) Verbal Attention (WM) Num Pattern Matching (P) Num Pattern Matching (P) WJ IV Ach Clusters Reading Broad Reading Reading Comp Reading Comp-Ext Reading Fluency Reading Rate Phonological Processing (PC) Basic Rdg Skills Writing Broad Writing Written Expression Story Recall (MM) Basic Writing Skills Visualization (Vz) Math Broad Math Math Calc Skills Visualization (Vz) Pair Cancellation (P/EF) Numbers Reversed (WM) Gv Phonological Processing (PC) Phonological Processing (PC) Glr Phonological Processing (PC) Num Pattern Matching (P) Verbal Attention (WM) Oral Vocabulary (LD/VL) AnalysisSynthesis (RG/RQ) Ga Num Pattern Matching (P) Oral Vocabulary (LV/VL) Oral Vocabulary (LD/VL) Gs Math Prob Solving Grw/Gq domain general Grw domain specific Gq domain specific Copyright, Institute for Applied Psychometrics, K.McGrew; 01-23-14
  • 72. Scholastic Aptitude Clusters: Potential Uses • • • • • • Designed to predict near term academic performance Time efficient referral-focused selective testing Time efficient academic domain-specific screening Time efficient annual review evaluations Gifted and talented screening – domain-specific talents? May be useful in estimating quickness of response to intervention • Provide information regarding the concept of “expected underachievement” • Formulation of differential academic domain expectations • ?
  • 73. CHC achievement abilities and WJ IV clusters CHC cognitive abilities and WJ IV scholastic aptitude cognitive clusters Gf Gc Gwm Gs Ga Glr Gv Gq Grw Reading Rdg. Cmp. Rdg. Apt. A (Predictor Score) Regression-based prediction models that account for regression-to-the-mean Brd. Rdg. Rdg. Flu. Rdg Cmp. Ex Rdg. Rate Rdg. Apt. B (Predictor Score) Bas. Rdg. Sk. Wr. Lng. WL. Apt. A (Predictor Score) Brd. Wr. Lg. Wr. Exp. WL. Apt. B (Predictor Score) Bas. Wr. Sk. Math. Math Apt. A (Predictor Score) Brd. Math Math Cal.Sk. Math Apt. B (Predictor Score) Predicted Target Cluster Score Math Pr. Slv. (minus) (See Table 1-10 for the specific tests and cluster information) Scholastic Aptitude/Achievement comparison procedures Copyright, Institute for Applied Psychometrics, K.McGrew; 01-23-14 Actual Target Cluster Score Difference Score (equals) (Compare to distribution of difference scores in WJ IV norm sample to determine significant strength or weakness) SD and PR for specified SD cut-off score
  • 74. Additional select technical and psychometric information (McGrew, LaForte, Schrank, 2014) Dr. Kevin McGrew Institute for Applied Psychometrics (IAP) Copyright; Institute for Applied Psychometrics; K.McGrew 02-5-14
  • 75. National Norm Sample • 7,416 participants • Preschool (664) • K-12 (3,891) • College/University (775) • Adult (2,086) • Ages 2-90+ years, Grades K.0-18.0 • 100 geographically diverse communities from 46 states and the District of Columbia • The Standards for Educational and Psychological Testing (aka., the Joint Standards) (American Psychological Association, American Educational Research Association, & National Council on Measurement in Education, 1999; in press) guided the norming and technical analysesCopyright; Institute for Applied Psychometrics; K.McGrew 1-23-14
  • 76. Sources of validity evidence presented •Representativeness of the WJ IV Test Content, Process, and Construct Coverage •Developmental Patterns of WJ IV Ability Clusters •Internal Structure and Relations within the WJ IV •Relationship of WJ IV Scores to Other Measures of Cognitive Abilities, Oral Language, and Achievement •Performance of Clinical Samples on WJ IV Measures Copyright; Institute for Applied Psychometrics; K.McGrew 02-5-14
  • 77. Dr. Kevin McGrew internet resources (@iqmobile)

×