Roadmap Today: Begin Exam 2 material (Chapters 5, 6, 4) Scales of measurement Psychometric properties Reliability ValidityTuesday: Finish chapter 5 Discuss Exam 1
Zoom out: where are we? We have: A research question An idea for a research design A hypothesisBut how do we measure what we’re interested in?
Scales of Measurement to measure themWe study variables and need accurately4 scales of measurement Nominal Ordinal Interval Ratio
Nominal Scale symbols classify or categorize into GROUPS or TYPES Name, Categorize, Classify Caution: use of numbers to indicate groupExamples- gender, marital status, experimental condition
Ordinal Scale A rank order scale of measurementExamples- order of finish, Letter grade in class, social class (low, med., high)Allows you to determine which person is higher or lower but not how much higher or lower. Can’t make direct comparisons
Interval Scale Rank ordering PLUS equal intervals of distance between adjacent numbersExample- Celsius and Fahrenheit temperature, IQ scores, yearNow you can make comparisonsEqual distances but no absolute zero point
Ratio Scale rank ordering, equal intervals PLUS an absolute zero pointAbsolute zero = absence of variableExamples- Kelvin temperature, income, weight, height, response time.
Psychometric properties Reliability: Consistency/stability of scoresValidity: Are you measuring what you are trying to measure?Ideally, we want: Measures that are reliable Inferences that are validReliability is necessary but not sufficient in order to have validity
Example: Internal Consistency I feel hungryI feel happyI have green eyesBig Bird is scaryI like turtles http://www.youtube.com/watch?v=CMNry4PE93Y
Internal Consistency Measured using coefficient alpha (α) a.k.a. Cronbach’s alpha Should be .7 or higherHigh values mean the items are measuring the same constructIf your scale measures more than 1 thing, each construct gets its own coefficient α
Interrater Reliability of ratings madeInterrater reliability- consistency by different judges GRE writing section Expressive writing studies Correlation between ratings should be strong/positive
Interobserver Agreement observers agreepercentage of times different % of times raters agree- easy to calculate and understand
Validity Accuracy of inferences or interpretations made on the basis of scoresMeasuring schizophrenia, or love We can’t directly observe it! It’s the accuracy of the interpretation from the test
Validity ConstructOperationalizationImportant to consider: Does your operationalization truly reflect what you’re measuring?ValidationNever-ending process
Obtaining Validity: Based on Content Content validity: judgment of the degree to which items adequately represent a construct’s domain. Do items appear to represent the thing you’re trying to measure? (face validity) Does your measure exclude any important parts of what you’re trying to measure? Does your test measure something besides what you wanted? (i.e., include irrelevant items)
Obtaining Validity: Based on Internal Structure Some constructs are multidimensional and need measures that address all dimensionsHomogeneity—degree to which a set of items measure a single construct Item-to-total correlation Coefficient alpha
Obtaining Validity: Based on Relations to Other Variables Criterion-related validity: degree to which scores predict or relate to an already established testTwo types of criterion validity: Predictive: using your measure to predict future performance Concurrent: using your measure to predict current performance on the same construct, or a related one.
Obtaining Validity: Based on Relations to Other Variables Convergent validity: relationship between your measure and other measures of that same constructDiscriminant validity: evidence that scores from your measure are NOT similar to scores of tests on different constructs.
Appropriate Use of Reliability and Validity Info Reliability and validity info apply to the measure of interest in the reported sample Situation-specific, not broadStandardized tests: norming group If you want to use a test with a group not represented in the norming group, be cautiousReport R & V for your own sample, and be wary of articles that make blanket statements about a measure’s R & V