Measuring Variables and Sampling
Roadmap                        Today: Begin Exam 2 material (Chapters 5, 6, 4)   Scales of measurement   Psychometric ...
Zoom out: where are we?           We have:   A research question   An idea for a research design   A hypothesisBut h...
Scales of Measurement                       to measure themWe study variables and need accurately4 scales of measuremen...
Nominal Scale                symbols classify or categorize into GROUPS or TYPES   Name, Categorize, Classify   Cautio...
Ordinal Scale                  A rank order scale of measurementExamples- order of finish, Letter grade in class, socia...
Interval Scale                  Rank ordering PLUS equal intervals of distance between adjacent numbersExample- Celsius...
Ratio Scale                   rank ordering, equal intervals PLUS an absolute zero pointAbsolute zero = absence of vari...
Psychometric properties            Reliability: Consistency/stability of scoresValidity: Are you measuring what you are...
Think about a Target        



Measuring Reliability                 4 Primary types    Test-Retest Reliability    Equivalent- Forms Reliability    ...
Test- Retest                  Refers to consistency over timeSame measure administered twice (with a time interval betw...
Equivalent-Forms Reliability            Equivalent forms- two versions of the same measure   Administer to the same gro...
Internal Consistency               Consistency with which test items measure a single construct.More items increases re...
Example: Internal           Consistency               I feel sadI feel downI feel depressedI feel miserableI feel aw...
Example: Internal           Consistency               I feel hungryI feel happyI have green eyesBig Bird is scaryI l...
Internal Consistency               Measured using coefficient alpha (α)   a.k.a. Cronbach’s alpha   Should be .7 or hi...
Interrater Reliability                           of ratings madeInterrater reliability- consistency  by different judges...
Interobserver Agreement                       observers agreepercentage of times different   % of times raters agree- e...
Validity                       Accuracy of inferences or interpretations made on the basis of scoresMeasuring schizophr...
Validity                      ConstructOperationalizationImportant to consider:   Does your operationalization truly ...
Obtaining Validity:            Based on Content                          Content validity: judgment of the degree to whi...
Obtaining Validity:  Based on Internal Structure                         Some constructs are multidimensional and need m...
Obtaining Validity: Based on Relations to Other Variables                         Criterion-related validity: degree to ...
Obtaining Validity: Based on Relations to Other Variables                       Convergent validity: relationship betwee...
Appropriate Use of Reliability      and Validity Info                          Reliability and validity info apply to th...
Upcoming SlideShare
Loading in...5
×

Chapter 5

359

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
359
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Chapter 5

  1. 1. Measuring Variables and Sampling
  2. 2. Roadmap Today: Begin Exam 2 material (Chapters 5, 6, 4)  Scales of measurement  Psychometric properties  Reliability  ValidityTuesday:  Finish chapter 5  Discuss Exam 1
  3. 3. Zoom out: where are we? We have:  A research question  An idea for a research design  A hypothesisBut how do we measure what we’re interested in?
  4. 4. Scales of Measurement  to measure themWe study variables and need accurately4 scales of measurement  Nominal  Ordinal  Interval  Ratio
  5. 5. Nominal Scale symbols classify or categorize into GROUPS or TYPES  Name, Categorize, Classify  Caution: use of numbers to indicate groupExamples- gender, marital status, experimental condition
  6. 6. Ordinal Scale A rank order scale of measurementExamples- order of finish, Letter grade in class, social class (low, med., high)Allows you to determine which person is higher or lower but not how much higher or lower.  Can’t make direct comparisons
  7. 7. Interval Scale Rank ordering PLUS equal intervals of distance between adjacent numbersExample- Celsius and Fahrenheit temperature, IQ scores, yearNow you can make comparisonsEqual distances but no absolute zero point
  8. 8. Ratio Scale rank ordering, equal intervals PLUS an absolute zero pointAbsolute zero = absence of variableExamples- Kelvin temperature, income, weight, height, response time.
  9. 9. Psychometric properties Reliability: Consistency/stability of scoresValidity: Are you measuring what you are trying to measure?Ideally, we want:  Measures that are reliable  Inferences that are validReliability is necessary but not sufficient in order to have validity
  10. 10. Think about a Target 
  11. 11.
  12. 12.
  13. 13.
  14. 14. Measuring Reliability 4 Primary types  Test-Retest Reliability  Equivalent- Forms Reliability  Internal Consistency Reliability  Interrater ReliabilityIndicate level of reliability with a reliability coefficient  Correlation; should be positive and strong (> .70)
  15. 15. Test- Retest Refers to consistency over timeSame measure administered twice (with a time interval between)
  16. 16. Equivalent-Forms Reliability Equivalent forms- two versions of the same measure  Administer to the same group of peopleProblem- hard to develop equivalent measuresExample: SAT, GRE
  17. 17. Internal Consistency Consistency with which test items measure a single construct.More items increases reliability, but we use as few items as possible  Why?
  18. 18. Example: Internal Consistency I feel sadI feel downI feel depressedI feel miserableI feel awful
  19. 19. Example: Internal Consistency I feel hungryI feel happyI have green eyesBig Bird is scaryI like turtles  http://www.youtube.com/watch?v=CMNry4PE93Y
  20. 20. Internal Consistency Measured using coefficient alpha (α)  a.k.a. Cronbach’s alpha  Should be .7 or higherHigh values mean the items are measuring the same constructIf your scale measures more than 1 thing, each construct gets its own coefficient α
  21. 21. Interrater Reliability  of ratings madeInterrater reliability- consistency by different judges  GRE writing section  Expressive writing studies  Correlation between ratings should be strong/positive
  22. 22. Interobserver Agreement  observers agreepercentage of times different  % of times raters agree- easy to calculate and understand
  23. 23. Validity Accuracy of inferences or interpretations made on the basis of scoresMeasuring schizophrenia, or love  We can’t directly observe it!  It’s the accuracy of the interpretation from the test
  24. 24. Validity ConstructOperationalizationImportant to consider:  Does your operationalization truly reflect what you’re measuring?ValidationNever-ending process
  25. 25. Obtaining Validity: Based on Content Content validity: judgment of the degree to which items adequately represent a construct’s domain.  Do items appear to represent the thing you’re trying to measure? (face validity)  Does your measure exclude any important parts of what you’re trying to measure?  Does your test measure something besides what you wanted? (i.e., include irrelevant items)
  26. 26. Obtaining Validity: Based on Internal Structure Some constructs are multidimensional and need measures that address all dimensionsHomogeneity—degree to which a set of items measure a single construct  Item-to-total correlation  Coefficient alpha
  27. 27. Obtaining Validity: Based on Relations to Other Variables Criterion-related validity: degree to which scores predict or relate to an already established testTwo types of criterion validity:  Predictive: using your measure to predict future performance  Concurrent: using your measure to predict current performance on the same construct, or a related one.
  28. 28. Obtaining Validity: Based on Relations to Other Variables Convergent validity: relationship between your measure and other measures of that same constructDiscriminant validity: evidence that scores from your measure are NOT similar to scores of tests on different constructs.
  29. 29. Appropriate Use of Reliability and Validity Info Reliability and validity info apply to the measure of interest in the reported sample  Situation-specific, not broadStandardized tests: norming group  If you want to use a test with a group not represented in the norming group, be cautiousReport R & V for your own sample, and be wary of articles that make blanket statements about a measure’s R & V
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×