The Art and Science of Applied Test Development. This is the first in a series of PPT modules explicating the development of psychological tests in the domain of cognitive ability using contemporary methods (e.g., theory-driven test specification; IRT-Rasch scaling; etc.). The presentations are intended to be conceptual and not statistical in nature. Feedback is appreciated.

1. The Art and Science of Test Development—Part A Planning, development frameworks & domain/test specification blueprints The basic structure and content of this presentation is grounded extensively on the test development procedures developed by Dr. Richard Woodcock Kevin S. McGrew, PhD. Educational Psychologist Research Director Woodcock-Muñoz Foundation

5. “ Generic” Woodcock test development flowchart

6. Test/Battery Development: Practical “Broad Stroke” Framework (Woodcock)

7. Test/Battery Development: Common Conceptual Psychometric Validity Framework (Bensen, 1998 summary)

8. The bible of test development: The “Joint Standards”

14. Practical “Broad Stroke” Framework Common Conceptual Psychometric Validity Framework This presentation is an integration of the practical and psychometric test/battery frameworks

15. Substantive Stage Structural (Internal) & External Stages

20. Carroll and Cattell-Horn Model Comparison Fluid Intelligence Crystallized Intelligence Gen. Memory & Learning Broad Visual Perception Broad Auditory Perception Broad Retrieval Ability Broad Cognitive Speediness Dec/Reaction Time/Speed Gf Gq Gsm Gv Ga Gs CDS Grw Gc Glr Fluid Intelligence Crystallized Intelligence Quantitative Knowledge Short-Term Memory Visual Processing Auditory Processing Long-Term Retrieval Processing Speed Correct Decision Speed Reading/ Writing g Cattell-Horn Carroll Gf Gy Gv Gs Gt Gc Gr Gu

21. ... most disciplines have a common set of terms and definitions (i.e., a standard nomenclature) that facilitates communication among professionals and guards against misinterpretations. In chemistry, this standard nomenclature is reflected in the ‘Table of Periodic Elements’. Carroll (1993a) has provided an analogous table for intelligence….. (Flanagan & McGrew, 1998)

27. Substantive Stage of Test Development: Develop Test Design and Specification Blueprint Example domain to be used for illustration of process: Gv (Visual Processing) g Gf Gv Glr Gs Gc Gsm Ga Theoretical Domain = Cattell-Horn-Carroll (CHC) theory of cognitive abilities Cylinders = broad CHC abilities Circles = narrow CHC abilities

31. Narrow Gv ability definitions Spatial Relations (SR): Ability to rapidly perceive and manipulate relatively simple visual patterns or to maintain orientation with respect to objects in space. Visualization (Vz): The ability to apprehend a spatial form, object, or scene and match it with another spatial object, form, or scene with the requirement to rotate it (one or more times) in two or three dimensions. Requires the ability to mentally imagine, manipulate or transform objects or visual patterns (without regard to speed of responding). Visual Memory (MV): Ability to form and store a mental representation or image of a visual stimulus and then recognize or recall it later. We will focus on one: Visualization (Vz)

33. End of Part A. Additional steps in test development process will be presented in subsequent modules as they are developed