1. +
Measurement
From concepts to observations
By @drcarp | Serena Carpenter

2. +
Proposal Paper

3. +
Measurement

4. +
Measurement
The process of systematic observation and
assignment of numbers to phenomena
according to rules

5. +
Operationalization

6. +
Operationalization
Process of connecting concepts to
observations
Developing procedures to measure variation
in variables
 Process of specifying the operations that will
indicate the value of cases on a variable

7. +
Concepts Variables |Dimensions Indicators
Binge Drinking Frequency of heavy
episodic drinking
“How often within the
past two weeks did you
consume 5 or more
drinks containing
alcohol in a row?”
Poverty Subjective poverty
Absolute poverty
“Would you say you
are poor?”
Family income/Poverty
threshold
Socioeconomic
Status
Income
Education
Occupational prestige
Income + education +
prestige

8. +
Constructing questions
Asking people questions is often the most
common way to measure social variables
Closed-ended vs. Open-ended

9. +
Levels of measurement
Nominal
Ordinal
Interval
Ratio

10. +
Nominal
Variables have two or more categories
No numeric scales but assigned a number
Groups such as
Male-female
Study in quiet room vs noisy room

11. +
Attributes must be:
Mutually exclusive
Exhaustive

12. +
Ordinal
Rank order points on a scale
Intervals between items are not known or are
not equal
Examples:
2, 3, or 4-star restaurants
Ranking TV shows by popularity

13. +
Within
the last year about how
have used…..
Did
not
use
1X
per
year
6X per
year
1X per
month
2X per
month
1x
per
week
3X
per
week
5X
per
week
Every
day
Tobacco
Alcohol
Marijuana
Cocaine
Amphetamines
Sedatives
Other illegal drugs

14. +
Interval
Intervals between points on the scale are of
equal value
No true “zero” amount

15. +
How do you think your close
friends feel (or would feel) about
you……
Don’t
disapprove Disapprove
Strongly
disapprove
Trying marijuana once or twice
Smoking marijuana occasionally
Smoking marijuana regularly
Trying cocaine once or twice
Taking cocaine regularly
Trying LSD once or twice
Taking LSD regularly

16. +
Ratio scale
A true zero point – absence of the variable
Zero on weight means no weight
Can form ratios: 10 pounds is twice as heavy
as 5 pounds
Use more sophisticated statistical tests for
ratio and interval scales
You can always transform ratio-level
variables into lower-level variables

17. +
Having observed two youngsters take a nasty spill while fetching water,
Research Smith wondered if a warning message to “be careful while
climbing the hill” might help avert this sort of mishap.
He also wondered if males were more likely to fall first, causing their female
partners (hanging onto the bucket) to come tumbling after.
Smith recruited 60 pairs of 10-year-old children to participate in his
experiment. Each pair consisted of one boy and one girl. Each pair was
given a bucket and instructed to go up the hill and fetch a pail of water.
Half of the groups were told to “be careful while climbing the hill.” The other
half were simply told to fetch the water with no warning. The pairs were
randomly assigned to a “warning” and “no warning” group. The participants
performed each task separately.
Three observers watched all of the pairs. The observers noted if any of the
children fell down while fetching the water and which child (boy or girl) fell
first.
1. IVs and DVs
2. How are they operationally defined?
3. What type of measures for each variable (N, O, I, R)

18. +
Exercise
List the variables that will be in your study
and place a N, O, I, R to tell me at what level
you plan to measure your variables

19. +
Scales

20. +
Scale
Composite measure of a variable

21. +
Constructing scales
 Determine what you want to measure
 Generate a large pool of items
 Determine format structure
 Have a pool of experts to review it
 Administer it to sample
 Evaluate items
 Optimize scale length

22. +
Scale creation guidelines
 Start with twice as many items as you will need
 Negative worded items
 Every item should reflect the construct
 Construct short items (20 words or less)
 Avoid emotionally-loaded items
 Double-barreled
 Avoid using always, never
 Avoid double negatives/positives

23. +
Index
 Composite measure that summarizes and rank-orders specific
observations and represents some more general dimension
 Political Activism
 Wrote a letter to a public official
 Signed a political petition
 Gave money to a political cause
 Gave money to a political candidate
 Wrote a political letter to the editor
 Persuaded someone to change his or her voting plans

24. +
Likert
Strongly disagree, disagree, neither nor
disagree, agree, strongly agree
Variable created that cannot be directly
observed

25. +
Semantic Differential
Respondents rate their opinions on a linear
scale between two endpoints that have
opposite meanings
Good/Bad, Dirty/Clean, Moral/Immoral
Good 1 2 3 4 5 6 7 Bad

26. +
Measurement error
 Error is a combo of random error and measurement
error
 Random – error that cannot be predicted or
controlled
 Measurement – faulty measurement procedures

27. +
Evaluating levels of measurement:
Validity & reliability
 Validity
 Face validity
 Criterion validity
 Construct validity
 Reliability
 Test-retest reliability
 Interobserver reliability
 Interitem reliability vs split-half reliability

28. +

29. +
Unreliable measure
Good 1 2 3 4 5 6 7 Bad
Wrong 1 2 3 4 5 6 7 Right
Harmful 1 2 3 4 5 6 7 Beneficial
Fair 1 2 3 4 5 6 7 Unfair
Wise 1 2 3 4 5 6 7 Foolish
Negative 1 2 3 4 5 6 7 Positive

30. +
Validity
Does the measure cover the full range of the
concept’s meaning?

31. +
Face validity
The degree to which a measurement device
appears to accurately measure a variable

32. +
Construct or factorial validity
 Operational definition of a variable
 Statistical analysis used to see how items correlate with
one another and do not correlate with other items
 The degree to which the measurement or manipulation of
the variable accurately reflects the underlying theoretical
construct
 Grouped items are called factors (dimensions)

33. +
Criterion validity
The degree to which a measurement device
accurately predicts behavior on a criterion
measure
Predictive
Concurrent
Discriminant Validity

34. +
Reliability
Consistency indicates high reliability

35. +
Test – retest reliability
Take measure two times
Reliability established when the two scores
are very similar
Reliability coefficient – a correlation
coefficient that ranges from 0.00 to 1.00
Highly similar scores are close to 1.00

36. +
Split-half Reliability
One administration of the survey
Compare responses to odd- and even-
numbered items

37. +
Cronbach’s Alpha Reliability
 Better approach used to establish
 0.90+ Excellent
 0.80 - .0.90 Good
 0.70 - 0.80 Respectable
 0.65 - 0.70 Minimally acceptable
 0.60 - 0.65 Undesirable
 - .0.60 Unacceptable

38. +
Inter-rater reliability
Examines the agreement of observations
made by two or more raters

39. +
Intercoder reliability
Two or more observers/coders judge the
same phenomena
Nominal
Cohen’s kappa and Scott’s pi
Interval and ratio
Coefficient Alpha

40. +
Future
Exercise
Manuscript structure
Critique journal articles exercises
Exam Feb. 14th
Access to a computer?