My favorite color is red. T FI don’t know how to swim. T FI have a dog named Fido. T FMy watch is real gold. T F
CONTENT-RELATED EVIDENCE OF VALIDITY Refers to the adequacy with which the content of a testrepresents the content of the curricular aim about which inferences are to be made.Two Approaches:1.Developmental Care2.External Reviews
DEVELOPMENTAL CAREEmploy a set of test-development procedures focused on assuring that the curricular aim’s content is properly reflected in the assessment procedure itself.
EXTERNAL REVIEWS Assembling of judges who rate the contentappropriateness of a given test in relationship to the curricular aim the test allegedly represents
THE ISSUE OF ALIGNMENTNorman Webb of the University of Wisconsin’s Method of Determining Alignment Categorical concurrence : Are the same or consistent categories used in both curricular expectations and assessments?Depth-of-knowledge consistency : To what extent are the cognitive demands of curricular aims and assessments the same? Range of knowledge correspondence : Is the span of knowledge reflected in curricular aims and assessments the same?Balance of Representation : To what degree are different curricular aims given equal emphasis on the assessments?
CRITERION-RELATED EVIDENCE OF VALIDITYCollected only in situations where educators are using an assessment procedure to predict how well students will perform on some subsequent criterion variable.
CONSTRUCT RELATED EVIDENCE Measuring what’s hidden Gathered through a series of studies Three Approaches to Collecting Construct Related Evidence of Validity1.Intervention Studies2.Differential-Population Studies3.Related-Measures Studies
INTERVENTION STUDIES We hypothesize that students will respond differently to the assessmentinstrument after having received some type of treatment (or intervention)
DIFFERENTIAL-POPULATION STUDIESWe hypothesize that individuals representing distinctly different populations will score differently on the assessment procedure under consideration.
RELATED-MEASURES STUDIES We hypothesize that a given kind of relationship will be present betweenstudents’ scores on the assessment device we’re scrutinizing and their scores on a related or unrelated assessment device. Convergent Validity (+ +) Discriminant Evidence (+ -)
SANCTIONED AND UNSANCTIONED FORMS OF VALIDITY EVIDENCEFace Validity• the appearance of a test seems to coincide with the use to which the test is being putConsequential Validity• refers to whether the uses of test results are valid Refer to Standards for Educational and Psychological Testing
RELIABILITY/VALIDITYValid score-based inferences almost certainly guarantee that consistent test results are present. Vs. Consistent test results almost certainly guarantee that valid score-based inferences are presentEvidence of valid score-based inferences almost certainly requires that consistency of measurement is present.
WHY DID I JUST SIT HERE AND LEARN ALL THIS? Give serious thought to the content of an assessment domain being represented by a test. There is value in having a colleague review your tests’ content. At least you know about the other forms of validity evidence. Validity does NOT reside on the test itself.