Evolution of the Assessment of Cognitive Functions July 10, 2008 Richard W. Woodcock University of Southern California Third National School Neuropsychology Conference

Evolution of the Assessment of Cognitive Functions

July 10, 2008

Richard W. Woodcock

University of Southern California

Main Topics How has our conceptualization of cognitive ability evolved over the past century?

How has our conceptualization of cognitive ability evolved over the past century?

Main Topics How has our conceptualization of cognitive ability evolved over the past century? How well have intelligence batteries adapted to these changes?

How has our conceptualization of cognitive ability evolved over the past century?

How well have intelligence batteries adapted to these changes?

Main Topics How has our conceptualization of cognitive ability evolved over the past century? How well have intelligence batteries adapted to these changes? What are the next challenges for test developers and users?

How has our conceptualization of cognitive ability evolved over the past century?

How well have intelligence batteries adapted to these changes?

What are the next challenges for test developers and users?

Path of Progress Better Cognitive Theory Better Cognitive Tests Better Information Better Instructional and Clinical Planning

Single General Ability Conceptualizations of Intelligence

Evolution of Intelligence Tests Single General Ability SB SB L,M 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010

Single General Ability Pair of Abilities Conceptualizations of Intelligence

Evolution of Intelligence Tests 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 Pair of Abilities WB WISC WAIS WISC-R

Single General Ability Pair of Abilities Limited Set of Broad Abilities Conceptualizations of Intelligence

Evolution of Intelligence Tests 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 Incomplete Set of Abilities WJ SB IV

Single General Ability Pair of Abilities Limited Set of Broad Abilities “ Complete” Set of Broad Abilities Conceptualizations of Intelligence

Evolution of Intelligence Tests 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 “ Complete” Set of Broad Abilities WJ-R

Nine Broad CHC Factors Stores of Acquired Knowledge Comprehension-Knowledge ( Gc ) Reading-Writing ( Grw ) Mathematics ( Gq )

Stores of Acquired Knowledge

Comprehension-Knowledge ( Gc )

Reading-Writing ( Grw )

Mathematics ( Gq )

Comprehension-Knowledge ( Gc) The breadth and depth of knowledge including verbal communication, information, and reasoning when using previously learned procedures .

The breadth and depth of knowledge including verbal communication, information, and reasoning when using previously learned procedures .

Reading-Writing ( Grw) An ability in areas common to both reading and writing. Probably includes basic reading and writing skills, and the skills required for comprehension and expression .

An ability in areas common to both reading and writing. Probably includes basic reading and writing skills, and the skills required for comprehension and expression .

Quantitative Knowledge ( Gq) The ability to comprehend quantitative concepts and relationships, the facility to manipulate numerical symbols.

The ability to comprehend quantitative concepts and relationships, the facility to manipulate numerical symbols.

Nine Broad CHC Factors Stores of Acquired Knowledge Comprehension-Knowledge ( Gc ) Reading-Writing ( Grw ) Mathematics ( Gq ) Thinking Abilities Long-Term Retrieval ( Glr ) Visual-Spatial Thinking ( Gv ) Auditory Processing ( Ga ) Fluid Reasoning ( Gf )

Stores of Acquired Knowledge

Comprehension-Knowledge ( Gc )

Reading-Writing ( Grw )

Mathematics ( Gq )

Thinking Abilities

Long-Term Retrieval ( Glr )

Visual-Spatial Thinking ( Gv )

Auditory Processing ( Ga )

Fluid Reasoning ( Gf )

Long-Term Retrieval ( Glr) The ability to store information efficiently and retrieve it later.

The ability to store information efficiently and retrieve it later.

Visual-Spatial Thinking ( Gv) Spatial orientation, the ability to analyze and synthesize visual stimuli, and the ability to hold and manipulate mental images.

Spatial orientation, the ability to analyze and synthesize visual stimuli, and the ability to hold and manipulate mental images.

Auditory Processing ( Ga) The ability to discriminate, analyze, and synthesize auditory stimuli.

The ability to discriminate, analyze, and synthesize auditory stimuli.

Fluid Reasoning ( Gf) The ability to reason and solve problems that often involve unfamiliar information or procedures. Manifested in the reorganization, transformation, and extrapolation of information.

The ability to reason and solve problems that often involve unfamiliar information or procedures. Manifested in the reorganization, transformation, and extrapolation of information.

Nine Broad CHC Factors Stores of Acquired Knowledge Comprehension-Knowledge ( Gc ) Reading-Writing ( Grw ) Mathematics ( Gq ) Thinking Abilities Long-Term Retrieval ( Glr ) Visual-Spatial Thinking ( Gv ) Auditory Processing ( Ga ) Fluid Reasoning ( Gf ) Cognitive Efficiency Processing Speed ( Gs ) Short-Term Memory ( Gsm )

Stores of Acquired Knowledge

Comprehension-Knowledge ( Gc )

Reading-Writing ( Grw )

Mathematics ( Gq )

Thinking Abilities

Long-Term Retrieval ( Glr )

Visual-Spatial Thinking ( Gv )

Auditory Processing ( Ga )

Fluid Reasoning ( Gf )

Cognitive Efficiency

Processing Speed ( Gs )

Short-Term Memory ( Gsm )

Processing Speed ( Gs) Speed and efficiency in performing automatic or very simple cognitive tasks.

Speed and efficiency in performing automatic or very simple cognitive tasks.

Short-Term Memory ( Gsm) The ability to hold information in immediate awareness and then use it within a few seconds.

The ability to hold information in immediate awareness and then use it within a few seconds.

Single General Ability Pair of Abilities Limited Set of Broad Abilities “ Complete” Set of Broad Abilities Narrow Abilities Underlying Broad Abilities Conceptualizations of Intelligence

Evolution of Intelligence Tests Narrow Abilities Underlying Broad Abilities WJ III 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010

Comprehension-Knowledge ( Gc ) General information ( K0 ) Listening ability ( LS ) Language development ( LD ) Lexical knowledge (VL) Language comprehension (V) Oral production and fluency (OP) Information about culture (K2) Examples of Narrow Abilities

General (verbal) information ( K0 ) “ Where would you usually find a ___________?” “ What would people usually do with a ______________?” K0: A Narrow Gc Ability

General (verbal) information ( K0 )

“ Where would you usually find a ___________?”

“ What would people usually do with a ______________?”

Listening ability ( LS ) Task: Listen to a sentence or two and provide an appropriate final word. LS: A Narrow Gc Ability

Listening ability ( LS )

Task:

Listen to a sentence or two and provide an appropriate final word.

Reading-Writing ( Grw ) Spelling ability (SG) Cloze ability (CZ) Reading decoding (RD) Reading comprehension (RC) Reading speed (RS) Writing ability (WA) English usage knowledge (EU) Examples of Narrow Abilities

Quantitative Knowledge ( Gq ) Mathematical achievement (A3) Mathematical knowledge (KM) Examples of Narrow Abilities

Long-Term Retrieval ( Glr ) Associative memory (MA) Ideational fluency (FI) Naming facility (NA) Meaningful memory (MM) Figural fluency (FF) Examples of Narrow Abilities

Ideational fluency (FI) “ How many different vehicles can you name? You will have one minute. Say the names as fast as you can. FI: A Narrow Glr Ability (Not actual test item.)

Naming facility ( NA ) “ Tell me the names of these pictures as fast as you can.” NA: A Narrow Glr Ability (Not actual test items.)

Auditory Processing ( Ga ) Speech-sound discrimination (US) General sound discrimination (U3) Resistance to auditory stimulus distortion (UR) Phonetic coding (PC) Attention and concentration (AC) Maintaining and judging rhythm (U8) Memory for sound patterns (UM) Musical discrimination and judgment (U1, U9) Examples of Narrow Abilities

US: A Narrow ( Ga ) Ability Speech sound discrimination (US) (trees) (keys) (peas) “ Point to keys.” (Not actual test item.)

Speech sound discrimination (US)

PC: A Narrow Ga Ability Phonetic coding (PC) /d/ - /o/ - /g/ /b/ - /a/ - /s/ - /k/ - /i/ - /t/ (Not actual test items.)

Phonetic coding (PC)

/d/ - /o/ - /g/

/b/ - /a/ - /s/ - /k/ - /i/ - /t/

Visual-Spatial Thinking (Gv) Spatial relations (SR) Flexibility of closure (CF) Visualization (Vz) Closure speed (CS) Length estimation (LE) Visual memory (MV) Spatial scanning (SS) Examples of Narrow Abilities

Fluid Reasoning ( Gf ) General sequential reasoning (RG) Quantitative reasoning (RQ) Induction (I) Examples of Narrow Abilities

RG: A Narrow Gf Ability General sequential reasoning (RG) 4 3 2 ___ 36 27 18 ___ 14 14 10 6 6 ___ (Not actual test items.)

General sequential reasoning (RG)

4 3 2 ___

36 27 18 ___

14 14 10 6 6 ___

Processing Speed (Gs) Perceptual speed (P) Semantic processing speed (R4) Writing speed (WS) Rate of test taking (R9) Number facility (N) Examples of Narrow Abilities

P: A Narrow Gs Ability Perceptual speed (P) a e c a d f bc af st qr af al srv srw svr swx ssr svr (Not actual test items.)

Perceptual speed (P)

a e c a d f

bc af st qr af al

srv srw svr swx ssr svr

Short-Term Memory ( Gsm ) Memory span ( MS ) Working memory ( WM ) Examples of Narrow Abilities

Working memory ( WM ) “ I am going to name some things and some numbers. After I say them, you say the things in the same order that I said them. Then you tell me the numbers in the same order that I said them.” goat….5….chair….3…..7….rug WM: A Narrow Gsm Ability (Not actual test item.)

Working memory ( WM )

“ I am going to name some things and some numbers. After I say them, you say the things in the same order that I said them. Then you tell me the numbers in the same order that I said them.” goat….5….chair….3…..7….rug

Sources of Information Expert clinicians Expert theoreticians

Expert clinicians

Expert theoreticians

Some Cross-Battery References Alfonso, V., Flanagan, D. & Radwan, S. (2005). The impact of the Cattell-Horn-Carroll Theory on Test Development and Interpretation of Cognitive and Academic Abilities. In D. P. Flanagan and P. L. Harrison (Eds.) Contemporary Intellectual Assessment: Theories, Tests and Issues (pp. 185-202). New York: Guilford Press. Flanagan, D., Ortiz, S. & Alfonso, V. (2007). Essentials of Cross-Battery Assessment (Second Edition). Hoboken, NJ: John Wiley & Sons. Flanagan, D. P., Ortiz, S. O., Alfonso, V. C., & Mascolo, J. T. (2002). The achievement test desk reference (ATDR) . Boston: Allyn & Bacon.

Alfonso, V., Flanagan, D. & Radwan, S. (2005). The impact of the Cattell-Horn-Carroll Theory on Test Development and Interpretation of Cognitive and Academic Abilities. In D. P. Flanagan and P. L. Harrison (Eds.) Contemporary Intellectual Assessment: Theories, Tests and Issues (pp. 185-202). New York: Guilford Press. Flanagan, D., Ortiz, S. & Alfonso, V. (2007). Essentials of Cross-Battery Assessment (Second Edition). Hoboken, NJ: John Wiley & Sons. Flanagan, D. P., Ortiz, S. O., Alfonso, V. C., & Mascolo, J. T. (2002). The achievement test desk reference (ATDR) . Boston: Allyn & Bacon.

Some Cross-Battery References Flanagan, D. P., & McGrew, K. S. (1997). A cross-battery approach to assessing and interpreting cognitive abilities: Narrrowing the gap between practice and science. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 314-325). New York: Guilford Press. McGrew, K. S. (1997). Analysis of the major intelligence batteries according to a proposed comprehensive Gf-Gc framework. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 151-179). New York: Guilford Press.

Flanagan, D. P., & McGrew, K. S. (1997). A cross-battery approach to assessing and interpreting cognitive abilities: Narrrowing the gap between practice and science. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 314-325). New York: Guilford Press. McGrew, K. S. (1997). Analysis of the major intelligence batteries according to a proposed comprehensive Gf-Gc framework. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 151-179). New York: Guilford Press.

Sources of Information Expert clinicians Expert theoreticians Aided by information from factor analysis studies

Expert clinicians

Expert theoreticians

Aided by information from factor analysis studies

Factor Analysis Exploratory factor analysis Confirmatory factor analysis Joint factor analyses

Exploratory factor analysis

Confirmatory factor analysis

Joint factor analyses

Some Joint FA Studies Woodcock, R. W. (1990). Theoretical foundations of the WJ-R measures of cognitive ability. Journal of Psychoeducational Assessment, 8 , 231-258. Phelps, L., McGrew, K. S., Knopik, S. N., & Ford, L. (2005). The general ( g ), broad, and Narrow CHC stratum characteristics of the WJ III and WISC-III tests: A confirmatory cross-battery investigation. School Psychology Quarterly, 20(1), 66-88. Sanders, S., McIntosh, D. E., Dunham, M, Rothlisberg, B. A, & Finch, H. (2007). Joint confirmatory factor analysis of the Differential Ability Scales and the Woodcock-Johnson Tests of Cognitive Abilities—Third Edition. Psychology in the Schools, 44(2), 119-138. Hunt, M. (2007). A joint confirmatory factor analysis of the Kaufman Assessment Battery for Children, Second Edition, and the Woodcock-Johnson Tests of Cognitive Abilities, Third Edition, with preschool children. Unpublished doctoral dissertation, Ball State University, Muncie.

Woodcock, R. W. (1990). Theoretical foundations of the WJ-R measures of cognitive ability. Journal of Psychoeducational Assessment, 8 , 231-258.

Phelps, L., McGrew, K. S., Knopik, S. N., & Ford, L. (2005). The general ( g ), broad, and Narrow CHC stratum characteristics of the WJ III and WISC-III tests: A confirmatory cross-battery investigation. School Psychology Quarterly, 20(1), 66-88.

Sanders, S., McIntosh, D. E., Dunham, M, Rothlisberg, B. A, & Finch, H. (2007). Joint confirmatory factor analysis of the Differential Ability Scales and the Woodcock-Johnson Tests of Cognitive Abilities—Third Edition. Psychology in the Schools, 44(2), 119-138.

Hunt, M. (2007). A joint confirmatory factor analysis of the Kaufman Assessment Battery for Children, Second Edition, and the Woodcock-Johnson Tests of Cognitive Abilities, Third Edition, with preschool children. Unpublished doctoral dissertation, Ball State University, Muncie.

What might your next test battery do? Suggest tests based on the referral concern

Suggest tests based on the referral concern

What might your next test battery do? Suggest tests based on the referral concern Seek and integrate qualitative information to present along with test scores

Suggest tests based on the referral concern

Seek and integrate qualitative information to present along with test scores

What might your next test battery do? Suggest tests based on the referral concern Seek and integrate qualitative information to present along with test scores Suggest interventions and compensatory procedures

Suggest tests based on the referral concern

Seek and integrate qualitative information to present along with test scores

Suggest interventions and compensatory procedures

Summary

Summary

Path of Progress Better Better Better Cognitive Cognitive Better Instructional Theory Tests Information and Clinical Planning

Single General Ability Pair of Abilities Limited Set of Broad Abilities “ Complete” Set of Broad Abilities Narrow Abilities Underlying Broad Abilities Conceptualizations of Intelligence

Evolution of Intelligence Tests

Test batteries of the future will be more informative. The Good News …

Test batteries of the future will be more informative. Test batteries of the future will be more complex as test developers close the gap between cognitive theory and practice. … the Bad News