Implications of 20 Years of CHC Cognitive-Achievement Research: Back-to-the-Future and Beyond CHC

1,894 views
1,721 views

Published on

Much has been learned about CHC CHC COG-ACH relations during the past 20 years (McGrew & Wendling’s, 2010). This presentation, made at the First Richard Woodcock Institute on Cognitive Assessment (Sept 29, 2012), built on this extant research by first clarifying the definitions of abilities, cognitive abilities, achievement abilities, and aptitudes. Differences between domain-general and domain-specific CHC predictors of school achievement were defined. The promise of Kafuman’s “intelligent” intelligence testing approach was illustrated with two approaches to CHC-based selective referral-focused assessment (SRFA). Next, a number of new intelligent test design (ITD) principles were described and demonstrated via a series of exploratory data analyses that employed a variety of data analytic tools (multiple regression, SEM causal modeling, multidimensional scaling). The ITD principles and analyses resulted in the proposal to construct developmentally-sensitive CHC-consistent scholastic aptitude clusters, measures that can play an important role in contemporary third method (pattern of strength and weakness) approaches to SLD identification.
The need to move beyond simplistic conceptualizations of COG COG-ACH relations and SLD identification models was argued and demonstrated via the presentation and discussion of CHC COG-ACH causal SEM models. Another example was the proposal to identify and quantify cognitive-aptitude-achievement trait complexes (CAATCs). A revision in current PSW third-method SLD models was proposed that would integrate CAATCs. Finally, the need to incorporate the degree of cognitive complexity of tests and composite scores within CHC domains in the design and organization of intelligence test batteries (to improve the prediction of school achievement) was proposed. The various proposals presented in this paper represented a mixture of (a) a call to return to old ideas with new methods (Back-to-the-Future) or (b) the embracing of new ideas, concepts and methods that require psychologists to move beyond the confines of the dominant CHC taxonomy of human cognitive abilities (i.e., Beyond CHC).

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,894
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
46
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Implications of 20 Years of CHC Cognitive-Achievement Research: Back-to-the-Future and Beyond CHC

  1. 1. Implications of 20 Years of CHCCognitive-Achievement Research:Back-to-the-Future and Beyond CHC Kevin S. McGrew PhD Woodcock-Muñoz Foundation
  2. 2. Staying current with “IQ McGrew” (@iqmobile) ICDP Blog
  3. 3. Introduction and ContextDr. Woodcock’s legacy & impact on my career and this paper My WJ data sandbox The Journey (2002now) Back-to-the-future Beyond CHC
  4. 4. General General Intelligence (g) Quantitative Comp - Long-Term Processing Reading & Fluid Short-Term Visual Auditory Broad Knowledge Knowledge Storage & Writing (Grw) Reasoning (Gf) Memory (Gsm) Processing (Gv) Processing (Ga) Speed (Gs) (Gq) (Gc) Retrieval (Glr) Narrow Mathematical Reading General verbal Memory span Associative Visualization Phonetic coding Perceptual speed Induction (I) knowledge (KM) decoding (RD) information (K0) (MS) memory (MA) * (Vz) (PC) (P) Mathematical Reading Language General Speeded Speech sound Working memory Meaningful Rate of test- achievement comprehension development sequential discrimination capacity (MW) memory (MM) * rotation (SR) taking (R9) (A3) (RC) (LD) reasoning (RG) (US) Resistance to Reading speed Lexical Quantitative Free-recall Closure speed Number facility auditory stimulus (RS) knowledge (VL) reasoning (RQ) memory (M6) * (CS) (N) distortion (UR) Memory for Reading Spelling ability Listening ability Ideational Flexibility of sound patterns speed/fluency (SG) (LS) fluency (FI) ** closure (CF) (UM) (RS) Maintaining & Writing English usage Communication Associational Visual memory judging rhythm speed/fluency (EU) ability (CM) fluency (FA) ** (MV) (U8) (WS) General Musical discrim. Writing ability Grammatical Expressional Spatial scanning Speed + & judgment (U1 (WA) sensitivity (MY) fluency (FE) ** (SS) U9) Sens. to probs. Writing speed Serial perceptual Absolute pitch /altern. Sol. (WS) integration (PI) (UP) fluency (SP) ** Acquired Knowledge + Memory Originality Length Sound * Learning /creativity (FO) estimation (LE) localization (UL) Efficiency ** ** Retrieval Fluency Naming facility Perceptual (NA)** illusions (IL) Functional groupings Word Fluency Perceptual (FW) ** alternations (PN) Conceptual groupings Figural Fluency Imagery (IM) + = additional CHC abilities in groupings (FF) ** in Part 2 of model Domain-Independent General Sensory-Motor Domain Figural flexibilityFigure 1. CHC v2.0 model based on Schneider and McGrew (2012) Capacities + (FX) ** Specific Abilities (Sensory) +
  5. 5. GeneralGeneral Intelligence (g) Domain Reaction & Tactile Abilities Specific Know. Psychomotor Olfactory Kinesthetic PsychomotorBroad Decision Speed (Gkn) Speed (Gps) Abilities (Go) (Gh) Abilities (Gk) Abilities (Gp) (Gt)Narrow Simple reaction Speed of limb Olfactory Static strength ? ? ? time (R1) movement (R3) memory (OM) (P3) Choice reaction Writing speed Multilimb time (R2) (fluency) WS coordination (P6) Semantic Speed of Finger dexterity processing speed articulation (PT) (P2) (R4) Mental Movement time Manual comparison (MT) speed (R7) dexterity (P1) Inspection time Arm-hand (IT) steadiness (P7) General Speed + Control precision (P8) Aiming (A1) Acquired Knowledge + Gross body equilibrium (P4) Motor Functional groupings Sensory-Motor Domain Specific Abilities + Conceptual groupings + = additional CHC abilities in groupings in Part I of model Figure 1 (continued). CHC v2.0 model based on Schneider and McGrew (2012)
  6. 6. CHC COGACH Relations: What We Know Today•Almost all available CHC-designed COGACH research is limited to the WJBattery•The primary action in CHC COGACH relations is at the narrow ability level• There is a future for “intelligent” intelligence testing, even in the currentresponse-to-intervention (RTI) environment
  7. 7. General Intelligence (g) Comp - Long-Term Processing Fluid Short-Term Visual Auditory Knowledge Storage & Reasoning (Gf) Memory (Gsm) Processing (Gv) Processing (Ga) Speed (Gs) (Gc) Retrieval (Glr) Language Naming facility Working memory Perceptual speedM development capacity (MW (NA) (P) (LD)a R Associativet Listening ability (LS) memory (MA) dh g General verbal information (K0) Ach. Domain General Cognitive AbilitiesA Ac Lexical Memory span Meaningful Phonetic coding c knowledge (VL) (MS) memory (MM) (PC)h h Speech soundi discrimination i (US)e Resistance to e Rdg. Domain Specific auditory stimulusv Cognitive Abilities distortion (UR) ve em Visualization Number facility m Quantitative (Vz) (N)e reasoning (RQ) en General sequential Speeded n rotation (SR)t reasoning (RG) t Math. Domain Specific Visual memory Induction (I) Cognitive Abilities (MV) [Developmental (age-based) differences are not captured by this abridged summary. See McGrew & Wendling (2010) for this information] Established narrow CHCrdg./math ach. relations abridged summary
  8. 8. Clarification of Ability Construct Terminology
  9. 9. Ability“as used to describe an attribute of individuals, ability refers to thepossible variations over individuals in the liminal levels of task difficulty(or in derived measurements based on such liminal levels) at which, onany given occasion in which all conditions appear favorable, individualsperform successfully on a defined class of tasks” (p. 8, italics in original). “every ability is defined in terms of some kind of performance, or potential for performance (p. 4).” Cognitive AbilitiesAbilities on tasks “in which correct or appropriate processing of mentalinformation is critical to successful performance” (p. 10; italics in original). Achievement abilities“refers to the degree of learning in some procedure intended to producelearning, such as an informal or informal course of instruction, or aperiod of self study of a topic, or practice of a skill” (p. 17). As noted byCarroll (1993)
  10. 10. What is “aptitude” Aptitude (Defined in this paper—narrow sense, not broader Richard Snow definition) Aptitude is defined as the combination, amalgam or complex of specific cognitive abilities, that when combined, best predict a specific achievement domain
  11. 11. Abilities Achievement Abilities Cognitive Abilities General Intelligence (g)Quantitative Comp - Long-Term Processing Reading & Knowledge Fluid Short-Term Visual AuditoryKnowledge Storage & Writing (Grw) (Gc) Reasoning (Gf) Memory (Gsm) Processing (Gv) Processing (Ga) Speed (Gs) (Gq) Retrieval (Glr) Rdg Apt Math Apt Etc. Etc. Etc. Etc. Etc. Etc. Etc. Etc. Etc. Vertical columns represent abilities, factors or latent traits (primarily Ach. domain- factor-analysis derived internal structural validity constructs) general apt. Horizontal arrow rows represent aptitudes (primarily multiple Ach. domain- regression derived external [predictive] validity constructs) specific apt. Conceptual distinction between Abilities: Cognitive abilities, achievement abilities, and aptitudes
  12. 12. Selective Referral-Focused Assessment (RFSA) Kaufman’s “Intelligent” Intelligence testing Intelligent “RFSA” CHC Cog- Ach CHC-based Research batteries Synthesis CHCTheory
  13. 13. General Intelligence (g) Comp - Long-Term Processing Fluid Short-Term Visual Auditory Knowledge Storage & Reasoning (Gf) Memory (Gsm) Processing (Gv) Processing (Ga) Speed (Gs) (Gc) Retrieval (Glr) Language Naming facility Working memory Perceptual speedM development capacity (MW (NA) (P) (LD)a R Associativet Listening ability (LS) memory (MA) dh g General verbal information (K0) Ach. Domain General Cognitive AbilitiesA Ac Lexical Memory span Meaningful Phonetic coding c knowledge (VL) (MS) memory (MM) (PC)h h Speech soundi discrimination i (US)e Resistance to e Rdg. Domain Specific auditory stimulusv Cognitive Abilities distortion (UR) ve em Visualization Number facility m Quantitative (Vz) (N)e reasoning (RQ) en General sequential Speeded n rotation (SR)t reasoning (RG) t Math. Domain Specific Visual memory Induction (I) Cognitive Abilities (MV) [Developmental (age-based) differences are not captured by this abridged summary. See McGrew & Wendling (2010) for this information] Established narrow CHCrdg./math ach. relations abridged summary
  14. 14. Two illustrative CHC general selective referral-focused assessment (SRFA) scenarios: BRS problems for ages 6 to 8 yrs
  15. 15. The evolution of differential Scholastic Aptitude Clusters (SAPTs) Developmentally sensitive CHC- WJ III Pred. designed SAPTs Ach. GIA WJ-R option SAPTs WJSAPTs
  16. 16. Developmentally-Sensitive CHC-Consistent Scholastic Aptitude Clusters (McGrew, 1986, 1994)
  17. 17. ITD - Developmentally-Sensitive CHC- Consistent Scholastic Aptitude Clusters Run final MR Backward model at each Run MR deletion of age and smooth models tests from MR regression across entire model. Inspect coefficients by school-age each step age Select WJ III WJ III norm results noting tests based sample “bridesmaid” CHC on first step predictorsCOG>ACH for initial res. predictorsynthesis pool
  18. 18. Developmentally-Sensitive CHC- Consistent Scholastic Aptitude ClustersStandardized regression coefficient Vis-Aud Learning (Glr-MA) Verbal Comp. (Gc-LD/VL) Age group (in years)
  19. 19. Standardized regression coefficient Verbal Comp. (Gc-LD/VL) Visual Matching (Gs-P) Number Matrices (Gf-RQ) Verbal Comp. (Gc-LD/VL) Analysis-Synthesis (Gf-RG) Numbers Reversed(Gsm-WM) Analysis-Synthesis (Gf-RG) Number Matrices (Gf-RQ) Numbers Reversed(Gsm-WM) Visual Matching (Gs-P) Age group (in years) Age 5 6 7 8 9 10 11 12 13 14 15 16 17 18 GIA-Std. 32 39 44 46 53 56 50 60 64 56 53 65 53 47 MR-Apt. 46 42 47 53 56 61 62 63 71 72 64 77 64 66 Difference 6 3 3 7 3 5 12 3 13 12 11 12 11 19 Smoothed standardized regression coefficients of best set of WJ III cognitive test predictors of WJ III Math Reasoning(MR) cluster from ages 6 thru 18. Table is % of MR variance accounted for by GIA-Std and MR Aptitude as constructed and weighted per the figure.
  20. 20. Standardized regression coefficient Verbal Comp. (Gf-LD/VL) Visual Matching (Gs-P) Vis-Aud Learning (Glr-MA) Sound Awareness (Ga-PC/Gsm-WM) Sound Blending (Ga-PC) Numbers Reversed (Gsm-Wm) Visual Matching (Gs-P) Numbers Reversed (Gsm-Wm) Sound Blending (Ga-PC) Verbal Comp. (Gc-LD/VL) Sound Awareness (Ga-PC/Gsm-WM) Vis-Aud Learning (Glr-MA) Age group (in years) Age 5 6 7 8 9 10 11 12 13 14 15 16 17 18 GIA-Std. 33 40 42 39 41 50 43 35 43 48 48 48 59 45 BRS-Apt. 50 49 50 48 45 56 48 43 50 54 52 52 63 52 Difference 17 9 8 9 4 6 5 6 7 6 4 4 4 7 Smoothed standardized regression coefficients of best set of WJ III cognitive test predictors of WJ III Basic ReadingSkills (BRS) cluster from ages 6 thru 18. Table is percent of BRS variance accounted for by GIA-Std and BRS Aptitude as constructed and weighted per the figure.
  21. 21. Developmentally-Sensitive CHC- Consistent Scholastic Aptitude Clusters ITD: “Intelligent” Test Design PrinciplesITD: SAPTs are better predictors of achievement than g-based composites ITD: SAPTs require a mixture of domain-general and domain- specific CHC cognitive abilities • Test developers should utilize the extant CHC COGACH relations literature when selecting the initial pool of tests to include in the prediction models ITD: SRFA requires 3-way thinking. 3-way interaction of CHC abilities X achievement domains X age (developmental status).
  22. 22. Developmentally-Sensitive CHC- Consistent Scholastic Aptitude Clusters ITD: “Intelligent” Test Design PrinciplesITD: Developmental trends are critically important in aptitude-achievement comparisons • Test developers should provide age-based developmental weighting of the tests in the different CHC-consistent SAPTs •Those who implement an aptitude-achievement consistency/concordance SLD model must be cautious and not use a "one size fits all" approach when determining which CHC COG abilities should be examined for the aptitude portion of the consistency model
  23. 23. Developmentally-Sensitive CHC-Consistent Scholastic Aptitude Clusters Group vs individual centered focus (McGrew & Flanagan, 1998) • Group-based statistical results may not translate perfectly to all individuals • “Intelligent” testing is required • “We are the instrument”
  24. 24. CHC-Consistent Scholastic Aptitude Clusters SRFA Strategy WJ III example in basic reading skills (BRS) and math reasoning (MR) Optimal developmentally weighted linear combination of WJ III tests General Intelligence (g) Comp - Long-Term Processing Fluid Short-Term Visual Auditory Knowledge Storage & Reasoning (Gf) Memory (Gsm) Processing (Gv) Processing (Ga) Speed (Gs) (Gc) Retrieval (Glr) Snd Blending Verbal Numbers Vis.-Aud. (PC) Visual Matching Comprehension Reversed Learning (P) WJ III Basic Rdg. Skills Aptitude (LD) (MW) (MA) Snd Awareness (PC;Gsm-WM) Analysis- Verbal Synthesis Numbers Comprehension (RG) Reversed Visual Matching WJ III Math Reason. Aptitude (LD) (MW) (P) Number Matrices (RQ) Examine PSW within aptitude clusters (and as suggested by other testsadministered and other non-test information) to determine additional selective follow-up assessment in narrow ability domains
  25. 25. CHC COGACH relations research & SRFA provides opportunity to engage in “intelligent” testing (ala, A. Kaufman) “ Tests do not think for themselves, nor do they directly communicate with patients. Like a stethoscope, a blood pressure gauge, or an MRI scan, a psychological test is a dumb tool, and the worth of the tool cannot be separated from the sophistication of the clinician who draws inferences from it and then communicates with patients and professionals”Meyer et al. (2001). Psychological testing and psychological assessment. American Psychologist,
  26. 26. Beyond CHC
  27. 27. Beyond CHC # 2: WJ III Productive Exploratory Rabbit Hole (circa 2009-2010) Experience Data Sets •WJ III norm data •WJ III+ other batteries (WISC-R; WAIS-III/WMS-III/KAIT) •WAIS-IV subtest correlations Methods •Cluster analysis •Multidimensional scaling analysis (MDS) – 2D and 3D •Standard and Carroll EFA+CFA exploratory factor analysis •Model-generation CFA (SEM) •CHC cognitive causal SEM models
  28. 28. Beyond CHC: Linear minds living in a non-linear world“A fundamental limitation of any theory built on a rectilinear system offactors it that it is not of a form that well describes natural phenomena. Itis thus unlikely to be fully adequate. It is a system that can accuratelydescribe rectangular structures built by humans…but not the rounded andirregular structures of mother nature. The phenomena of nature are notusually well described by the linear equations of a Catesian coordinatesystem….The equations that describe the out structure and convolutions ofbrains must beparabolas, cycloids, cissoids, spirals, foliums, exponentials, hyperboles, andthe like (p. 84). (Horn & Noll, 1997)
  29. 29. Beyond CHC #1: CHC + Information Processing Causal SEM Models CHC g+ specific COG<>ACH SEM res. CHC COC- abilities COG>ACH (person ACH reg WJ-R SEM res. fit?) studies WJ III + IP/CPMGf-Gc/ models CHCtheory
  30. 30. Beyond CHC #1: CHC + Information Processing Causal SEM Models Independent Variables Dependent Variable (IV)) – Cog. (DV) – Ach.(Note: Residuals and significant correlations between residuals are omitted from the TeCog. Test 1 Cog TeAch. Test 5 TeCog. Test 2 LV1 Ach LV3 TeAch. Test 6 TeCog. Test 3 diagram for readability purposes TeCog. Test 4 TeAch. Test 3 Cog Ach TeCog. Test 5 LV2 g LV2 TeAch. Test 4 TeCog. Test 6 Cog LV3 TeCog. Test 7 TeAch. Test 1 Ach TeCog. Test 8 Cog LV1 LV nth TeAch. Test 2 TeCog. Test nth
  31. 31. Beyond CHC #1: CHC + Information Processing Causal SEM Models
  32. 32. Visual Matching Cognitive efficiency Mem.for Sent. Indirect effect Mem. Span Decision Speed Gs (MS) Direct effect Mem.for Words Cross Out Aud. Wrk. Mem. Wrk. Mem. (WM) Num. Reversed Block Rotation Spat. Relations Verbal Comp. Gv Pic. Recognition Oral Comp. Gc Mem.for Names Gen. Info. Ages 6-8 Ret. Fluency g Anal.-Synth. Glr DR Vis-Aud.Lrg. Conc. Form. Gf Vis.-Aud. Lrg. Numerical Reas. Sound Blending .27 Inc. Words Ga Word Word Attack Attack Sound Patterns Effects Direct Indirect TotalPlausible CHC/IP COGWord Attack causal model in WJ III norm data (ages 6-8) Gs 0.19 0.40 0.59 MS 0.00 0.34 0.34 Chi-square =1016.5. df=239 WM 0.00 0.54 0.54 GFI=.93; AGFI=.91; PGFI=.74 g 0.36 0.23 0.59 RMSEA=.055 (.051-.058) Ga 0.27 0.00 0.29
  33. 33. Stankov, Boyle and Cattell (1995) who stated, within the context of research on human intelligence““while we acknowledge the principle of parsimony and endorse itwhenever applicable, the evidence points to relative complexityrather than simplicity. Insistence on parsimony at all costs canlead to bad science” (p. 16).
  34. 34. Beyond CHC #1: Develop SEM “person fit” indices ? Indirect effect Visual Matching Cognitive efficiency Mem.for Sent. Direct effect Gs Mem. Span Decision Speed (MS) Mem.for Words Cross OutChi-square =1016.5. df=239GFI=.93; AGFI=.91; PGFI=.74 Aud. Wrk. Mem.RMSEA=.055 (.051-.058) Wrk. Mem. (WM) Num. Reversed Block Rotation Spat. Relations Verbal Comp. Gv Pic. Recognition Oral Comp. Gc Mem.for Names Gen. Info. Ages 6-8 Ret. Fluency g Anal.-Synth. Glr DR Vis-Aud.Lrg. Conc. Form. Gf Vis.-Aud. Lrg. Numerical Reas. Sound Blending Ga .27 Word Word Attack Inc. Words Attack Sound Patterns
  35. 35. A challenge to the LISRELites, AMOSites, MPLUSites in the room Build it an they shall come.
  36. 36. Beyond CHC #1: CHC + Information Processing Causal SEM ModelsExample:
  37. 37. Beyond CHC #1: CHC + Information Processing Causal SEM ModelsExample:
  38. 38. Beyond CHC #2: Cognitive-Aptitude- Achievement Trait Complexes (CAATC’s) Cog-Apt- Ach Trait Beyond Complexes Jöreskog (CAATC) Psych trait syndrome New SLD complex model ideas Third theory & method SLD research modelsWJ/ (apt-achWJ-R consistency)SAPTs
  39. 39. Beyond CHC #2: Cognitive-Aptitude-Achievement Trait Complexes (CAATC’s) Aptitude for Acd. Domain Academic Domain Cognitive Abilities Degree of cohesion Cognitive-Aptitude-Achievement Trait Complex
  40. 40. Beyond CHC: Jöreskog syndromeAmerican psychology, and mainstream quantitative schoolpsychology, have expressed little interest in non-confirmatory statisticalmethodological lens (e.g., exploratory cluster analysis; MDS) in favor ofwhat I call Jöreskog syndrome—an almost blind allegiance and belief instructural equation modeling confirmatory factor analysis (SEM-CFA)methods as the only way to see the “true light” of the structure ofintelligence and intelligence tests
  41. 41. Beyond CHC: Jöreskog syndrome The law of the instrument “Give a small boy a hammer, and he will findthat everything he encounters needs pounding”
  42. 42. Important Reminder: All statistical methods, suchas factor analysis (EFA or CFA) have limitations andconstraints.It only provides evidence of structural/internal validityand typically nothing aboutexternal, developmental, heritability, neurocognitivevalidity evidenceNeed to examine other sources of evidence and useother methods – looking/thinking outside the factoranalysis box
  43. 43. Beyond CHC #2: Cognitive-Aptitude- Achievement Trait Complexes (CAATC’s) Cog-Apt- Ach Trait Beyond Complexes Jöreskog (CAATC) Psych trait syndrome New SLD complex model ideas Third theory & method SLD research modelsWJ/ (apt-achWJ-R consistency)SAPTs
  44. 44. 2 Notes on WJ-R Derived Scholastic Aptitude Clusters (SAPTs) C GRWAPT = Gc(LD/VL) + Gs(P) + Ga(PC) + Glr(VAL) or Gsm-MS (RAPT and WLAPT nearly overlapped in figure. Given their high degree of overlap, they were GA (PC) 1 GLR (MA) combined into a single GRWAPT in the figure) GV (MV/CS) MAPT = Gc(LD/VL) + Gs(P) + Gf(I) + Gf(RG) -WJ-R SAPTs each comprised of 4 tests with equal GC (LD/VL) weightings (.25) BCA GSM (MS) -Bold font designates shared test CHC ability EXT GRWAPT content in GRWAPT and MAPT 0 A B WJ-R CHC factor clusters MAPT GF (I/RG) BRDG WJ-R broad achievement lcusters WJ-R Broad Cognitive Ability & BWLANG Scholastic Aptitude Clusters-1 BMATH Note: Measures closer to the center are more cognitively complex. The distance between points represents the inter- relations between variables. Highly-related GS (P) variables are spatially closer-have less distance between their circles. D-2 Figure 9. Guttman radex MDS -2 -1 0 1 2 analysis summary of WJ-R cognitive, aptitude, and achievement measures A  B = Visual-figural/numeric/quantitative  Auditory-linguistic/language dimension across all ages in WJ-R norm sample C  D = Cognitive operations/processes Acquired knowledge /product dimension
  45. 45. 2 C Math (Gq) cognitive-aptitude- achievement trait complex r =.55 GA (PC) 1 GLR (MA) Reading/Writing (Grw) GV (MV/CS) cognitive-aptitude- achievement trait complex GC (LD/VL) BCA GSM (MS) Notes on WJ-R Derived Scholastic EXT GRWAPT Aptitude Clusters (SAPTs) 0 A B GRWAPT = Gc(LD/VL) + Gs(P) + Ga(PC) + Glr(VAL) MAPT or Gsm-MS GF (I/RG) BRDG (RAPT and WLAPT nearly overlapped in figure. BWLANG Given their high degree of overlap, they were combined into a single GRWAPT in the figure)-1 BMATH MAPT = Gc(LD/VL) + Gs(P) + Gf(I) + Gf(RG) -WJ-R SAPTs each comprised of 4 tests with equal weightings (.25) GS (P) Angle = approximately 57o -Bold font designates shared test CHC ability r = approximately .55 content in GRWAPT and MAPT D-2 WJ-R CHC factor clusters -2 -1 0 1 2 WJ-R broad achievement lcusters AB = Visual-figural/numeric/quantitative Auditory-linguistic/language dimension WJ-R Broad Cognitive Ability & CD = Cognitive operations/processes Acquired knowledge /product dimension Scholastic Aptitude Clusters Figure 10. WJ III based reading and math cognitive-aptitude-achievement trait complexes (CAATC)
  46. 46. Cognitive-aptitude-achievement trait complexes Cognitive-aptitude-achievement trait complex (CAATC)A constellation or combination of related cognitive, aptitude, and achievement traitsthat, when combined together in a functional fashion, facilitate or impede theacquisition of academic learning
  47. 47. Cognitive-aptitude-achievement trait complexesCAATCs emphasize theconstellation or combination ofelements that are related andare combined together in afunctional fashionImply a form of a centrallyinward directed force that pullselements together much likemagnetism
  48. 48. Cohesion definedCohesion appears the most appropriate term forthis form of multiple element bonding. Cohesionis defined, as per the Shorter English OxfordDictionary (Brown, 2002), as “the action orcondition of sticking together or cohering; atendency to remain united” (Brown, 2002, p. 444).Element bonding and stickiness are also conveyedin the APA Dictionary of Psychology(VandenBos, 2007) definition of cohesion as “theunity or solidarity of a group, as indicated by thestrength of the bonds that link group members tothe group as a whole” (p. 192).
  49. 49. Beyond CHC: Comparison of current PSW and CAATC SLD models Cognitive / Academic Strengths Cognitive Strength Discrepant/ DiscordantDiscrepant/ Discrepant/Discordant Discordant Aptitude Academic for Acd. Domain Domain Cognitive Academic Cognitive Abilities weakness weakness Degree of cohesion Consistent/ Concordant Cognitive-Aptitude-Achievement Trait Complex Common Components of Third-Method Approaches to SLD Identification Dashed shapes designate academic domain related cognitive abilities. (adapted from Flanagan & Alfonso, 2011) Suggested re-conceptualization of academic and cognitive weaknesses (and possible SLD identification model) based on cognitive-aptitude- achievement trait complexes (CAATC)
  50. 50. 2 C Math (Gq) cognitive-aptitude- achievement trait complex r =.55 GA (PC) 1 GLR (MA) Reading/Writing (Grw) GV (MV/CS) cognitive-aptitude- achievement trait complex GC (LD/VL) BCA GSM (MS) EXT GRWAPT 0 A B MAPT GF (I/RG) BRDG Aptitude Academic for Acd. Domain BWLANG Domain Cognitive-1 BMATH Abilities Degree of cohesion GS (P) Cognitive-Aptitude-Achievement Angle = approximately 57o Trait Complex r = approximately .55 D-2 -2 -1 0 1
  51. 51. Beyond CHC: Potential benefit of CAATC based SLD modelsThe identification of CAATC taxon’s that betterapproximate “nature carved at the joints” (Meehl, 1973,as quoted and explained by Greenspan, 2006, in thecontext of MR/ID diagnosis).Such a development would be consistent with Reynoldsand Lakin’s (1987) plea, 25 years ago, for disabilityidentification methods that better represent dispositionaltaxon’s rather than classes or categories based on specificcutting scores which are grounded in “administrativeconveniences with boundaries created out of political andeconomic considerations” (p. 342).
  52. 52. Beyond CHC: Proposed CAATC based SLD model (early ideas) • Evaluating the degree of cohesion within a Cognitive / Academic CAATC is integral and critical first step Strengths • The stronger the within-CAATC cohesion, the more confidence one could place in the Discrepant/ identification of a CAATC as possibly indicative Discordant of a SLD Aptitude • If the CAATC demonstrates very weak Academic for Acd. Domain Domain cohesion, the hypothesis of a possible SLD Cognitive should receive less consideration Abilities Degree of cohesion • PSW-based SLD identification would be based Cognitive-Aptitude-Achievement first on the identification of a weakness in a Trait Complex cohesive specific CAATC which is then determined to be significantly discrepant fromDashed shapes designate academic domain related cognitive abilities. relative strengths in other cognitive and achievement domains
  53. 53. Beyond CHC: Proposed CAATC based SLD model (early ideas) Quantifying degree of cohesion is likely possible via use of Euclidean Geometry metrics For example, Mahalanobis distance measure which can quantify the cohesion between CAATC measures as well as distance from the centroid of a CAATC exist (see Schneider, 2012)
  54. 54. Beyond CHC #3: Optimizing Cognitive Complexity of CHC measures Optimizing cognitive MDS and complexity “cognitive of CHC complexity” measures CHC findings COG>ACH rels. “Narrow is better” First CHC IQ batteriesfocused on broad stratum
  55. 55. Beyond CHC #3: Optimizing Cognitive Complexity of CHC measuresCHC factor breadth Cognitive complexity
  56. 56. 2 SNDISC PHNAWR GIA-EXT and three-test broad clusters 1 GA Two-test broad clusters Two-test narrow clusters GV3 ASMEM AUDMS GV PHNAW3 GLR GC VISUAL GIA-EXT GSM RDGCMP 0 GF GF3 RDGBR WRKMEM MTHREA RDGBS NUMREA MTHBR-1 PERSPD MTHCAL GS-2 -2 -1 0 1 2 MDS radex model based cognitive complexity analysis of primary WJ III clusters
  57. 57. 2 SNDISC PHNAWR GIA-EXT and three-test broad clusters 1 GA Two-test broad clusters Two-test narrow clusters GV3 ASMEM AUDMS GV PHNAW3 GLR GC VISUAL GIA-EXT GSM RDGCMP 0 GF GF3 RDGBR WRKMEM MTHREA RDGBS NUMREA MTHBR-1 PERSPD MTHCAL GS-2 -2 -1 0 1 2 MDS radex model based cognitive complexity analysis of primary WJ III clusters
  58. 58. 2 SNDISC PHNAWR GIA-EXT and three-test broad clusters 1 GA Two-test broad clusters Two-test narrow clusters GV3 ASMEM AUDMS GV PHNAW3 GLR GC VISUAL GIA-EXT GSM RDGCMP 0 GF GF3 RDGBR WRKMEM MTHREA RDGBS NUMREA MTHBR-1 PERSPD MTHCAL GS-2 -2 -1 0 1 2 MDS radex model based cognitive complexity analysis of primary WJ III clusters
  59. 59. Beyond CHC #3: Optimizing Cognitive Complexity of CHC measuresAccording to Lohman (2011), those tests closer to thecenter of an MDS radex model are more cognitivelycomplex, and this is due to five possible factor:• Larger number of cognitive component processes• Accumulation of speed component differences• More important component processes (e.g., inference)• Increased demands of attentional control and workingmemory• More demands on adaptive functions (assembly, control,and monitoring).
  60. 60. Beyond CHC #3: Optimizing Cognitive Complexity—Implications for Test Battery Design and Assessment Strategies•The push to feature broad CHC clusters in contemporary IQbatteries (or in XBA assessments) fails to recognize theimportance of cognitive complexity•Developing factorially complex measures is one way toachieve cognitive complexity (e.g., KABC-II, DAS-II, Wechslers)•ITD: It is proposed that within-CHC domain cognitivecomplexity should be an important ITD
  61. 61. Beyond CHC #3: Optimizing Cognitive Complexity—Implications for Test Battery Design and Assessment StrategiesAs per Brunswick Symmetry and BIS Model: Need to pay more attentionto matching the predictor-criteria space on the dimension of cognitivecomplexity (e.g., levels of aggregation)
  62. 62. Beyond CHC #3: Cognitive Complexity and CHC COGACH relationsMcGrew & Wendling’s (2010) “narrow is better” may need revision to… “Within CHC-domain cognitively complexity is better”
  63. 63. Beyond CHC #3: Optimizing Cognitive Complexity—Implications for Test Battery Design and Assessment Strategies Possible implication for use of the WJ III Battery: ITD: Broad+narrow hybrid example to optimize ach. prediction “Front end” featured clusters• Fluid Reasoning (Gf)• Comprehension-Knowledge (Gc)• Long-term Retrieval (Glr)• Working Memory (Gsm-MW)• Phonemic Awareness 3 (Ga-PC)• Perceptual Speed (Gs-P)• Visualization (not clear winner)Then, if broad Gsm, Ga, Gs, Gv measures are desired..supplemental testing as per administration of• Gs (Decision Speed)• Gsm (Memory for Words)• Gv (Picture Recognition)
  64. 64. Beyond CHC #3: Optimizing Cognitive Complexity—Implications for Test Battery Design and Assessment Strategies ITD: IQ test batteries of the future might better be based on a hybrid (broad+narrow) partially inverted CHC model thatdeliberately incorporates within-CHC domain cognitive complexity into the test/cluster design process and battery configuration or suggested testing sequence
  65. 65. Concluding Comments Proximal Implications“Intelligent” selective-referral focused assessments (SRFA) • Types of Strategies • General SRFA • Scholastic Aptitude Cluster-based SRFA • Important considerations • Recognize domain-general and domain-specific CHC COG-ACH relations • Recognize 3-way COC x ACH x Age interaction • Recognize importance of cognitive complexity in SRFA • Narrow may not necessarily be better as a general rule • Use broad+narrow inverted CHC hybrid approach to assessment • Cautious use of CHC COG-ACH relations findings with non-WJ III batteries
  66. 66. Concluding Comments Proximal ImplicationsDevelop Developmentally-Sensitive CHC-based Scholastic AptitudeClusters (ITD) • The research knowledge and statistical and computer software technology exists • e.g., WJ III GIA; WJ III Predicted AchievementInvestigate and validate more “dynamic/interacting” CHCCOGACH SEM modelsUse more “Intelligent Test Design” (ITD) principles when revisingold test batteries or developing new test batteries
  67. 67. Concluding Comments More Distal ImplicationsDevelop SEM “person fit” statistics for possible diagnostic andinstructional purposesPursue research into the validity and utility of identifying cognitive-aptitude-achievement trait complexes (CAATCs) • Identify and validate CAATCs • Develop metrics for operationalizing CAATCs • Ability domain cohesion metrics • Investigate validity and utility of CAATC based SLD models for understanding learning and identifying learning problems
  68. 68. Concluding Comments More Distal ImplicationsUse more “Intelligent Test Design” (ITD) principles when revisingold test batteries or developing new test batteriesIncorporate suggested “Intelligent Test Design” (ITD) principles intocurrent “best practice” test development principles whendeveloping new test batteries • Broad+narrow inverted CHC hybrid approach (ITD)
  69. 69. Concluding Comments Enduring ImplicationsIntelligence researchers and test developers need to embrace awider diversity of validated theories, models, and data analyticmethodological lenses to counter Jöreskog syndrome. ”If I have seen farther, it is by standing on the shoulders of giants” As stated by Isaac Newton in a letter to Robert Hooke in 1676:
  70. 70. Concluding Comments Enduring ImplicationsExploratory research methods need to be used more frequently byintelligence researchers Many a scientific adventurer sails the uncharted seas and sets his course for a certain objective only to find unknown land and unsuspected ports in strange parts. To reach such harbors, he must ship and sail, do and dare; he must quest and question. These chance discoveries are called “accidental” but there is nothing fortuitous about them, for laggards drift by a haven that may be a heaven. They pass by ports of opportunity. Only the determined sailor, who is not afraid to seek, to work, to try, who is inquisitive and alert to find, will come back to his home port with discovery in his cargo (p. 177)

×