Part I: Beyond the CHC tipping point: Back to the future


Published on

An overview of the CHC (Cattell-Horn-Carroll) theory of intelligence within a historical and "waves of interpretation" context. Presents idea that CHC has reached the "tipping point" in school psychology..and...this is allowing assessment practitioners to realize past attempts to engage in individual strength and weakness interpretation of CHC based test profiles

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Part I: Beyond the CHC tipping point: Back to the future

  1. 1.   <ul><li>Kevin McGrew, Ph.D. </li></ul><ul><ul><li>Director, IAP </li></ul></ul><ul><ul><li>Institute for Applied Psychometrics </li></ul></ul><ul><ul><li>Visiting Professor, Ed. Psych. </li></ul></ul><ul><ul><li>University of Minnesota </li></ul></ul>Beyond the CHC “Tipping Point” – Back to the Future
  2. 2. Cattell-Horn-Carroll (CHC) Theory of Cognitive Abilities: Historical Context and Development Within “Waves Of Intelligence Test Interpretation ”
  3. 3. In sociology , a tipping point or angle of repose is the event of a previously rare phenomenon becoming rapidly and dramatically more common Tipping Point
  4. 4. There is only one proven “law” in psychology The law of individual differences
  5. 5. On Intelligence... (Neisser et al., 1996) <ul><li>Individual’s differ in their ability to understand complex ideas, to adapt effectively to the environment, to learn from experience, to engage in various forms of reasoning, etc. </li></ul><ul><li>Concepts of intelligence are attempts to clarify this complex set of phenomena (p. 77). </li></ul>
  6. 6. The Need For A Scientific Foundation For The Development And Interpretation Of Cognitive Ability Tests “ Useful technology is based on scientific understanding; the better the science, the more effective the technology can be. So it is with psychological tests. They should be based on the most dependable and current evidence of science ” (Horn, 1991)
  7. 7. Three Primary Research Traditions <ul><li>Psychometric </li></ul><ul><li>Information Processing </li></ul><ul><li>Dynamic </li></ul>
  8. 8. On Intelligence... (Neisser et al., 1996) <ul><li>The psychometric approach is the oldest and best established , but others also have much to contribute. </li></ul><ul><li>We should be open to the possibility that our understanding of intelligence in the future will be rather different from what it is today (p.80). </li></ul>
  9. 9. Importance Of Classification Taxonomies In All Sciences Classification is arguably one of the most central and generic of all our conceptual exercises…without classification, there could be no advanced conceptualization, reasoning, language, data analysis, or for that matter, social science research (K.D. Bailey, 1994). A specialized science of classification of empirical entities known as taxonomy (Bailey, 1994; Prentky, 1994) is ubiquitous in all fields of study because it guides our search for information or truth.
  10. 10. Importance Of Classification Taxonomies: Two Examples Linnean hierarchical taxonomy
  11. 11. In psychology, we have have been searching for decades for an empirically and theoretically grounded taxonomy of cognitive abilities for the interpretation of the reliable variance of tests (Rectangle represents the total variance of a single cognitive ability test) Error variance - individual/situational variables (e.g., distractibility) -item variables (e.g., item sampling and item gradients ; test floor and ceiling ) -examiner variables (e.g., rapport, scoring and administration errors) -testing environment variables (e.g., noise, comfort) Unique abilities (variance) not shared in common with other tests (specificity ) Error variance Reliable variance ( reliability)
  12. 12. Waves Of Intelligence Test Interpretation (Kamphaus et al., 1997, 2005) <ul><li>Wave 1 – Quantification of a General Level (g) </li></ul><ul><li>Wave 2 – Clinical Profile Analysis </li></ul><ul><li>Wave 3 - Psychometric Profile Analysis </li></ul><ul><li>Wave 4 – Applying Theory to Test Interpretation </li></ul><ul><li>Wave 5 – Greater emphasis on content validity </li></ul>
  13. 13. McGrew’s Wave 5 Speculation <ul><li>Wave 1 – Quantification of a General Level (g) </li></ul><ul><li>Wave 2 – Clinical Profile Analysis </li></ul><ul><li>Wave 3 - Psychometric Profile Analysis </li></ul><ul><li>Wave 4 – Applying Theory to Intelligence Test Interpretation </li></ul><ul><li>Wave 5 – Application of computer technology & new statistical </li></ul><ul><li>& measurement methods/software to </li></ul><ul><li>theory-based test methods (Wave 4) + integration of psychometric and information processing research </li></ul><ul><li>and theories may now allow us to realize the </li></ul><ul><li>promise(s) of prior waves </li></ul>
  14. 14. <ul><li>Classification of individuals into groups based on IQ (g) scores </li></ul><ul><ul><li>Classification schemes based on global IQ scores </li></ul></ul>Wave 1 - Quantification Of A General Level ( g )
  15. 16. Wave 1 - Quantification Of A General Level ( g ) Spearman’s g- theory was major theoretical influence
  16. 17. Wave 1 - Quantification Of A General Level ( g ) Binet’s intelligence test was primary measurement breakthrough (provided global g -type score)
  17. 18. Reliable variance ( reliability) s g Wave 1 - Quantification Of A General Level ( g ) Primary test interpretation was focused on variance associated with g (general intelligence)
  18. 19. Wave 2 - Clinical Profile Analyses Rapaport, D., Gill, M. M., & Schafer, R. (1945–1946). Diagnostic psychological testing. (2 vols.). Chicago: Year Book <ul><li>Introduced Clinical Subtest Profile and Test Item Level Analyses </li></ul><ul><ul><li>Was found to be seriously wanting based on empirical research </li></ul></ul>
  19. 21. Wave 2 to 3 – Advances in Structural Theories of Intelligence Had Little Influence on Test Interpretation <ul><li>Factor-analytic research resulted in significant advances in understanding the structure of intelligence </li></ul><ul><li>These studies did not immediately “cross-over” to influence the applied practice of intelligence test development and interpretation </li></ul><ul><li>The most influential structural advances during this time included the following: </li></ul>
  20. 22. Cattell’s Fluid (Gf) and Crystallized (Gc) Theory of Intelligence (circa 1941) Did not immediately result in the development of a clinical/applied measure of intelligence
  21. 23. <ul><li>The ETS Factor Reference Group Made Significant Gains in Delineating the Structure of Human Cognitive Abilities </li></ul><ul><ul><li>WERCOF Abilities (Well Replicated Common Factors) </li></ul></ul><ul><ul><li>Kit of Factor-Referenced Tests </li></ul></ul><ul><li>Goal was to identify/catalogue human cognitive abilities and to provide ability/factor “reference” or “marker” tests </li></ul><ul><ul><li>Cattell/Carroll APA 1957 communication (click here) </li></ul></ul>
  22. 24. Thurstone’s Primary Mental Abilities model was influential (circa 1955) Shifted focus away from g to primary (WERCOF) mental abilities
  23. 25. Structure-Of-Intellect (SOI Model)
  24. 26. <ul><li>The SOI model lacks solid empirical evidence (Carroll, 1993; Cronbach & Snow, 1977; Gustafsson & Undheim, 1996; Messick, 1992; Vernon, 1961). </li></ul><ul><li>The SOI model “ is fundamentally defective” (Carroll, 1993; p. 59) and should be “marked down as a somewhat eccentric aberration in the history of intelligence models ; that so much attention has been paid to it is disturbing, to the extent that textbooks and other treatments of it have given the impression that the model is valid and widely accepted, when clearly it is not” (Carroll, 1993; p. 60). </li></ul>
  25. 27. Wave 2 to 3 – Advances in Applied Measures of Intelligence <ul><li>David Wechsler developed and presented the </li></ul><ul><li>first practical dichotomous practical/clinical intelligence test battery (1939) </li></ul>
  26. 28. An Important Point Regarding Wechsler’s Original Dichotomous Battery Structure <ul><li>The Wechsler scales were designed to assess two different ways intelligence can be expressed (i.e., two different languages). Wechsler did not consider the Verbal and Performance scales to represent two different types of intelligence </li></ul><ul><ul><li>(Kamphaus; 1993; Reynolds & Kamphaus, 1990; Zachary, 1990) </li></ul></ul><ul><li>There is no such thing as nonverbal abilities , and no empirically supported verbal/nonverbal theories of intelligence (Kamphaus, 1993) </li></ul><ul><li>Although making a huge practical contribution, the Wechsler battery structure had the indirect and unfortunate effect of misdirecting our understanding and interpretation of intelligence test performance </li></ul>
  27. 29. Wave 3 – Psychometric Profile Analyses Cohen, J. (1959). The factorial structure of the WISC at ages 7-6, 10-6, and 13-6, Journal of Consulting Psychology, 23 , 285-299. Kaufman, A. S. (1979). Intelligent testing with the WISC-R . New York: Wiley-Interscience.
  28. 31. Kaufman’s Wechsler-based psychometric (“Intelligent Testing”) approach contributed to the movement (in test interpretation) to beyond g Wave 3 – Psychometric Profile Analyses VC = Verbal Comprehension PO = Perceptual Organization Unique abilities not shared in common with other CHC factor indicators (specificity ) Reliable variance ( reliability) g PO VC Secondary ability Primary ability Error variance
  29. 33. An unfortunate impediment to the narrowing the intelligence theory/measurement gap was the development and influence of a Wechsler-centric philosophy of test interpretation VCI/POI/FFD/PS Evaluating Existing Tests Test Interpretation Individual Differences Research (e.g., ATI) Designing New Tests Communication (standard nomenclature) Outcomes Research
  30. 34. <ul><li>Ipsative interpretation </li></ul><ul><li>……… the process of generating strength and weakness hypotheses about cognitive abilities based on an analysis of an individual’s subtest scores that deviate significantly (either in a positive or negative direction) from the average (mean) of all the subtest scores on the intelligence battery </li></ul>Wave 2 & 3 both focused on Ipsative Profile Analysis
  31. 35. Ipsative Profile Analysis Average (mean) + 1 SD - 1 SD Strength Weaknesses Generate hypotheses based on patterns of strengths and weaknesses among subtests
  32. 36. <ul><li>Despite its popularity and intuitive appeal, ipsative analysis has significant limitations which have led to considerable criticism of the approach </li></ul>“ Just Say No To Subtest Analysis” According to academic school psychology, you should feel guilty and ashamed if you generate hypotheses based on test profiles
  33. 37. Wave 2 & 3 Contributed to Attempts to Measure “Differential Aptitudes” 1 Apt 1 Apt 2 Apt 3 IQ (g)
  34. 38. IQ (g) Criterion After g was accounted for In the prediction…….. But, research found that………. 1 Apt 3 … .measures of specialized or differential abilities contributed little or nothing to the prediction
  35. 39. McNemar, Q. (1964 ). Lost: Our intelligence: Why? American Psychologist , 19 , 871-882 The search to develop differential predictors of academic performance was deemed a failure Therefore…long live full-scale g -scores !!
  36. 40. “ Just Say Maybe” (McGrew et al., 1997) <ul><li>Improved Theory </li></ul><ul><ul><li>Cattell-Horn-Carroll (aka . Gf-Gc ) Theory </li></ul></ul><ul><li>Improved Theory-Based Operational Measurement </li></ul><ul><ul><li>WJ-R and WJ-III </li></ul></ul><ul><ul><li>CHC Cross-Battery </li></ul></ul>
  37. 41. “ Just say no & maybe” research differs in the construct validity of the measures used <ul><li>CHC focus on clusters instead of individual tests: </li></ul><ul><ul><li>Increased reliability </li></ul></ul><ul><ul><li>Increased construct validity </li></ul></ul><ul><ul><ul><li>Reduction of construct irrelevant variance </li></ul></ul></ul><ul><ul><ul><li>Better construct representation </li></ul></ul></ul>“ Just say no” focused on individual tests “ Just say maybe” focuses on combinations (clusters) of tests
  38. 42. <ul><li>The failure to find that specific abilities add anything to the prediction of achievement beyond that already provided by a g score may be a correct interpretation for the set of constructs measured by the Wechsler batteries, but may be a premature generalization to apply to all intelligence batteries </li></ul>“ Just say maybe” (McGrew, Flanagan, Keith, Vanderwood, 1997)
  39. 43. So…..get ready, buckle up and……… Wave 4+5
  40. 44. Wave 4 – Applying Contemporary Intelligence Theories to Test Interpretation (and research & development)
  41. 45. Wave 4 – The placement of empirically-grounded theories of intelligence in the center of test development and interpretation – a Theory-centric philosophy and approach to test interpretation VCI/POI/FFD/PS Evaluating Existing Tests Test Interpretation Individual Differences Research (e.g., ATI) Designing New Tests Communication (standard nomenclature) Outcomes Research
  42. 46. CHC ( Gf-Gc )  WJ-R/III Wave 4 – Applying Contemporary Intelligence Theories to Test Interpretation: Two Primary Theories Luria (Sim-Succ) Cattell (Gf-Gc) Primary Theorists & Researchers Horn (Expanded Gf-Gc) Carroll ( Gf-Gc Three-Stratum) Das (PASS) CHC Theory Luria-Das Theory Resulting Assessment Batteries and Approaches <ul><li>K-ABC </li></ul><ul><li>CAS </li></ul><ul><li>K-ABC II </li></ul><ul><li>WJ-R </li></ul><ul><li>WJ-III </li></ul><ul><li>KAIT </li></ul><ul><li>CHC Cross- </li></ul><ul><li>Battery </li></ul><ul><li>SB5 </li></ul><ul><li>K-ABC II </li></ul><ul><li>DAS-II </li></ul>
  43. 47. In sociology , a tipping point or angle of repose is the event of a previously rare phenomenon becoming rapidly and dramatically more common Tipping Point
  44. 48. “ It is this author’s personal opinion, that this moment, a moment where the interests and wisdom of a leading applied test developer ( Woodcock ), the leading proponent of Cattell-Horn Gf-Gc theory ( Horn ), and one of the preeminent educational psychologists and scholars of the factor analysis of human abilities ( Carroll ) intersected (see section C in Table 1), was the flash point that resulted in all subsequent theory-to--practice bridging events that led to today’s CHC theory and related assessment developments .  A fortuitous set of events had resulted in the psychometric stars aligning themselves in perfect position to lead the way for most all subsequent CHC assessment related developments.” (McGrew, 2004, 2005) The fortuitous March, 1986 “meeting of the minds” – the CHC Intelligence-to-Theory “flash point”
  45. 49. 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Year 0 100 200 300 400 500 600 700 Freq. of terms in NASP listserv messages WJ CHC Gf-Gc CHC designed WJ III battery published Carroll/Horn informally agree to CHC umbrella term
  46. 50. Founded July 8, 2001 Current n (11-5-07) = 956
  47. 51. The CHC “Theory-to-Practice” Tipping Point has occurred – sometime between 2001 and 2003 CHC Tipping Point So……what!!!!! Why is this relevant to me???
  48. 52. Three effects of intelligence batteries being based on confluence of research and theory – Wave 4 (Kamphaus et al. 2005) <ul><li>Test-specific training is of less value </li></ul><ul><ul><li>After a psychologist knows the theories….he or she can interpret most modern intelligence tests with confidence </li></ul></ul><ul><ul><ul><li>“ CHC-as-a-second-language” </li></ul></ul></ul><ul><ul><ul><li>It is now more important for a clinician to understand the constructs of intelligence , as opposed to receiving specific “Wechsler” or “Binet” training </li></ul></ul></ul>
  49. 53. <ul><li>Pre- and post-professional training priority shifts to sufficient knowledge of theories of intelligence that inform modern test construction and interpretation </li></ul><ul><li>As intelligence tests seek to measure similar core constructs , they increasingly resemble commodities </li></ul>Three effects of intelligence batteries being based on confluence of research and theory – Wave 4 (Kamphaus et al. 2005)
  50. 54. Among contemporary intelligence tests, the Woodcock-Johnson III …is the instrument most closely aligned with the Cattell-Horn…and Carroll theories of intelligence . CHC theory … served as the blueprin t for the WJ III (Kamphaus et al., 2005)
  51. 55. Wave 5 – (stay tuned –Part II of today’s presentation)