Part I: Beyond the CHC tipping point: Back to the future

Kevin McGrew, Ph.D.
Director, IAP
Institute for Applied Psychometrics
Visiting Professor, Ed. Psych.
University of Minnesota
Beyond the CHC “Tipping Point” – Back to the Future

Cattell-Horn-Carroll (CHC) Theory of Cognitive Abilities: Historical Context and Development Within “Waves Of Intelligence Test Interpretation ”

In sociology , a tipping point or angle of repose is the event of a previously rare phenomenon becoming rapidly and dramatically more common Tipping Point

There is only one proven “law” in psychology The law of individual differences

On Intelligence... (Neisser et al., 1996)
Individual’s differ in their ability to understand complex ideas, to adapt effectively to the environment, to learn from experience, to engage in various forms of reasoning, etc.
Concepts of intelligence are attempts to clarify this complex set of phenomena (p. 77).

The Need For A Scientific Foundation For The Development And Interpretation Of Cognitive Ability Tests “ Useful technology is based on scientific understanding; the better the science, the more effective the technology can be. So it is with psychological tests. They should be based on the most dependable and current evidence of science ” (Horn, 1991)

Three Primary Research Traditions
Psychometric
Information Processing
Dynamic

On Intelligence... (Neisser et al., 1996)
The psychometric approach is the oldest and best established , but others also have much to contribute.
We should be open to the possibility that our understanding of intelligence in the future will be rather different from what it is today (p.80).

Importance Of Classification Taxonomies In All Sciences Classification is arguably one of the most central and generic of all our conceptual exercises…without classification, there could be no advanced conceptualization, reasoning, language, data analysis, or for that matter, social science research (K.D. Bailey, 1994). A specialized science of classification of empirical entities known as taxonomy (Bailey, 1994; Prentky, 1994) is ubiquitous in all fields of study because it guides our search for information or truth.

Importance Of Classification Taxonomies: Two Examples Linnean hierarchical taxonomy

In psychology, we have have been searching for decades for an empirically and theoretically grounded taxonomy of cognitive abilities for the interpretation of the reliable variance of tests (Rectangle represents the total variance of a single cognitive ability test) Error variance - individual/situational variables (e.g., distractibility) -item variables (e.g., item sampling and item gradients ; test floor and ceiling ) -examiner variables (e.g., rapport, scoring and administration errors) -testing environment variables (e.g., noise, comfort) Unique abilities (variance) not shared in common with other tests (specificity ) Error variance Reliable variance ( reliability)

Waves Of Intelligence Test Interpretation (Kamphaus et al., 1997, 2005)
Wave 1 – Quantification of a General Level (g)
Wave 2 – Clinical Profile Analysis
Wave 3 - Psychometric Profile Analysis
Wave 4 – Applying Theory to Test Interpretation
Wave 5 – Greater emphasis on content validity

McGrew’s Wave 5 Speculation
Wave 1 – Quantification of a General Level (g)
Wave 2 – Clinical Profile Analysis
Wave 3 - Psychometric Profile Analysis
Wave 4 – Applying Theory to Intelligence Test Interpretation
Wave 5 – Application of computer technology & new statistical
& measurement methods/software to
theory-based test methods (Wave 4) + integration of psychometric and information processing research
and theories may now allow us to realize the
promise(s) of prior waves

Classification of individuals into groups based on IQ (g) scores
Classification schemes based on global IQ scores
Wave 1 - Quantification Of A General Level ( g )

Wave 1 - Quantification Of A General Level ( g ) Spearman’s g- theory was major theoretical influence

Wave 1 - Quantification Of A General Level ( g ) Binet’s intelligence test was primary measurement breakthrough (provided global g -type score)

Reliable variance ( reliability) s g Wave 1 - Quantification Of A General Level ( g ) Primary test interpretation was focused on variance associated with g (general intelligence)

Wave 2 - Clinical Profile Analyses Rapaport, D., Gill, M. M., & Schafer, R. (1945–1946). Diagnostic psychological testing. (2 vols.). Chicago: Year Book
Introduced Clinical Subtest Profile and Test Item Level Analyses
Was found to be seriously wanting based on empirical research

Wave 2 to 3 – Advances in Structural Theories of Intelligence Had Little Influence on Test Interpretation
Factor-analytic research resulted in significant advances in understanding the structure of intelligence
These studies did not immediately “cross-over” to influence the applied practice of intelligence test development and interpretation
The most influential structural advances during this time included the following:

Cattell’s Fluid (Gf) and Crystallized (Gc) Theory of Intelligence (circa 1941) Did not immediately result in the development of a clinical/applied measure of intelligence

The ETS Factor Reference Group Made Significant Gains in Delineating the Structure of Human Cognitive Abilities
WERCOF Abilities (Well Replicated Common Factors)
Kit of Factor-Referenced Tests
Goal was to identify/catalogue human cognitive abilities and to provide ability/factor “reference” or “marker” tests
Cattell/Carroll APA 1957 communication (click here)

Thurstone’s Primary Mental Abilities model was influential (circa 1955) Shifted focus away from g to primary (WERCOF) mental abilities

Structure-Of-Intellect (SOI Model)

The SOI model lacks solid empirical evidence (Carroll, 1993; Cronbach & Snow, 1977; Gustafsson & Undheim, 1996; Messick, 1992; Vernon, 1961).
The SOI model “ is fundamentally defective” (Carroll, 1993; p. 59) and should be “marked down as a somewhat eccentric aberration in the history of intelligence models ; that so much attention has been paid to it is disturbing, to the extent that textbooks and other treatments of it have given the impression that the model is valid and widely accepted, when clearly it is not” (Carroll, 1993; p. 60).

Wave 2 to 3 – Advances in Applied Measures of Intelligence
David Wechsler developed and presented the
first practical dichotomous practical/clinical intelligence test battery (1939)

An Important Point Regarding Wechsler’s Original Dichotomous Battery Structure
The Wechsler scales were designed to assess two different ways intelligence can be expressed (i.e., two different languages). Wechsler did not consider the Verbal and Performance scales to represent two different types of intelligence
(Kamphaus; 1993; Reynolds & Kamphaus, 1990; Zachary, 1990)
There is no such thing as nonverbal abilities , and no empirically supported verbal/nonverbal theories of intelligence (Kamphaus, 1993)
Although making a huge practical contribution, the Wechsler battery structure had the indirect and unfortunate effect of misdirecting our understanding and interpretation of intelligence test performance

Wave 3 – Psychometric Profile Analyses Cohen, J. (1959). The factorial structure of the WISC at ages 7-6, 10-6, and 13-6, Journal of Consulting Psychology, 23 , 285-299. Kaufman, A. S. (1979). Intelligent testing with the WISC-R . New York: Wiley-Interscience.

Kaufman’s Wechsler-based psychometric (“Intelligent Testing”) approach contributed to the movement (in test interpretation) to beyond g Wave 3 – Psychometric Profile Analyses VC = Verbal Comprehension PO = Perceptual Organization Unique abilities not shared in common with other CHC factor indicators (specificity ) Reliable variance ( reliability) g PO VC Secondary ability Primary ability Error variance

An unfortunate impediment to the narrowing the intelligence theory/measurement gap was the development and influence of a Wechsler-centric philosophy of test interpretation VCI/POI/FFD/PS Evaluating Existing Tests Test Interpretation Individual Differences Research (e.g., ATI) Designing New Tests Communication (standard nomenclature) Outcomes Research

Ipsative interpretation
……… the process of generating strength and weakness hypotheses about cognitive abilities based on an analysis of an individual’s subtest scores that deviate significantly (either in a positive or negative direction) from the average (mean) of all the subtest scores on the intelligence battery
Wave 2 & 3 both focused on Ipsative Profile Analysis

Ipsative Profile Analysis Average (mean) + 1 SD - 1 SD Strength Weaknesses Generate hypotheses based on patterns of strengths and weaknesses among subtests

Despite its popularity and intuitive appeal, ipsative analysis has significant limitations which have led to considerable criticism of the approach
“ Just Say No To Subtest Analysis” According to academic school psychology, you should feel guilty and ashamed if you generate hypotheses based on test profiles

Wave 2 & 3 Contributed to Attempts to Measure “Differential Aptitudes” 1 Apt 1 Apt 2 Apt 3 IQ (g)

IQ (g) Criterion After g was accounted for In the prediction…….. But, research found that………. 1 Apt 3 … .measures of specialized or differential abilities contributed little or nothing to the prediction

McNemar, Q. (1964 ). Lost: Our intelligence: Why? American Psychologist , 19 , 871-882 The search to develop differential predictors of academic performance was deemed a failure Therefore…long live full-scale g -scores !!

“ Just Say Maybe” (McGrew et al., 1997)
Improved Theory
Cattell-Horn-Carroll (aka . Gf-Gc ) Theory
Improved Theory-Based Operational Measurement
WJ-R and WJ-III
CHC Cross-Battery

“ Just say no & maybe” research differs in the construct validity of the measures used
CHC focus on clusters instead of individual tests:
Increased reliability
Increased construct validity
Reduction of construct irrelevant variance
Better construct representation
“ Just say no” focused on individual tests “ Just say maybe” focuses on combinations (clusters) of tests

The failure to find that specific abilities add anything to the prediction of achievement beyond that already provided by a g score may be a correct interpretation for the set of constructs measured by the Wechsler batteries, but may be a premature generalization to apply to all intelligence batteries
“ Just say maybe” (McGrew, Flanagan, Keith, Vanderwood, 1997)

So…..get ready, buckle up and……… Wave 4+5

Wave 4 – Applying Contemporary Intelligence Theories to Test Interpretation (and research & development)

Wave 4 – The placement of empirically-grounded theories of intelligence in the center of test development and interpretation – a Theory-centric philosophy and approach to test interpretation VCI/POI/FFD/PS Evaluating Existing Tests Test Interpretation Individual Differences Research (e.g., ATI) Designing New Tests Communication (standard nomenclature) Outcomes Research

CHC ( Gf-Gc )  WJ-R/III Wave 4 – Applying Contemporary Intelligence Theories to Test Interpretation: Two Primary Theories Luria (Sim-Succ) Cattell (Gf-Gc) Primary Theorists & Researchers Horn (Expanded Gf-Gc) Carroll ( Gf-Gc Three-Stratum) Das (PASS) CHC Theory Luria-Das Theory Resulting Assessment Batteries and Approaches
K-ABC
CAS
K-ABC II
WJ-R
WJ-III
KAIT
CHC Cross-
Battery
SB5
K-ABC II
DAS-II

In sociology , a tipping point or angle of repose is the event of a previously rare phenomenon becoming rapidly and dramatically more common Tipping Point

“ It is this author’s personal opinion, that this moment, a moment where the interests and wisdom of a leading applied test developer ( Woodcock ), the leading proponent of Cattell-Horn Gf-Gc theory ( Horn ), and one of the preeminent educational psychologists and scholars of the factor analysis of human abilities ( Carroll ) intersected (see section C in Table 1), was the flash point that resulted in all subsequent theory-to--practice bridging events that led to today’s CHC theory and related assessment developments . A fortuitous set of events had resulted in the psychometric stars aligning themselves in perfect position to lead the way for most all subsequent CHC assessment related developments.” (McGrew, 2004, 2005) The fortuitous March, 1986 “meeting of the minds” – the CHC Intelligence-to-Theory “flash point”

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Year 0 100 200 300 400 500 600 700 Freq. of terms in NASP listserv messages WJ CHC Gf-Gc CHC designed WJ III battery published Carroll/Horn informally agree to CHC umbrella term

Founded July 8, 2001 Current n (11-5-07) = 956

The CHC “Theory-to-Practice” Tipping Point has occurred – sometime between 2001 and 2003 CHC Tipping Point So……what!!!!! Why is this relevant to me???

Three effects of intelligence batteries being based on confluence of research and theory – Wave 4 (Kamphaus et al. 2005)
Test-specific training is of less value
After a psychologist knows the theories….he or she can interpret most modern intelligence tests with confidence
“ CHC-as-a-second-language”
It is now more important for a clinician to understand the constructs of intelligence , as opposed to receiving specific “Wechsler” or “Binet” training

Pre- and post-professional training priority shifts to sufficient knowledge of theories of intelligence that inform modern test construction and interpretation
As intelligence tests seek to measure similar core constructs , they increasingly resemble commodities
Three effects of intelligence batteries being based on confluence of research and theory – Wave 4 (Kamphaus et al. 2005)

Among contemporary intelligence tests, the Woodcock-Johnson III …is the instrument most closely aligned with the Cattell-Horn…and Carroll theories of intelligence . CHC theory … served as the blueprin t for the WJ III (Kamphaus et al., 2005)

Wave 5 – (stay tuned –Part II of today’s presentation)

Part I: Beyond the CHC tipping point: Back to the future

by Kevin McGrew on Nov 28, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 13

Statistics

Part I: Beyond the CHC tipping point: Back to the future — Presentation Transcript

http://www.slideshare.net	8
http://blackboard.stthom.edu	4
https://blackboard.stthom.edu	1