This document discusses quantitative measurement methods, including different levels of measurement, reliability, validity, and how statistics can be used. It explains nominal, ordinal, and interval-ratio levels of measurement and provides examples. Reliability and validity are also defined, and strategies for assessing reliability like test-retest and inter-rater reliability are presented.

1. PAD 503-01 A -Quantitative Methods I
Module 1- Theory and Levels of Measurement

2. Using Statistics
▫ Statistics are useful tools for analyzing and understanding the world and
everyday life.
▫ Statistics can be used to:
- Demonstrate the connection between obesity and heart disease.
- Track attitudes towards health care reform.
- Compare commuting patterns across different cities.
- Can you think of a topic statistics might be useful for?

3. Theory of Measurement
▫ Measurement
- It is the assignment of numbers to some phenomenon that we
are interested in analyzing.
▫ Example:
- The effectiveness of army officers is measured by having
senior officers rate junior officers on various traits
- Educational attainment may be measured by how well a
student scores on standardized achievement tests

4. Theory of Measurement
▫ Operational Definition
- Is a statement that describes how a concept will be measured
▫ Indicator
- Is a variable, or set of observations, that results from applying
the operational definition.

5. Measurement validity
▫ Validity can either be convergent or discriminant
▫ Convergent Validity
- consists of providing evidence that two tests that are believed to
measure closely related skills or types of knowledge correlate
strongly. That is to say, the two different tests end up ranking
students similarly.
▫ Discriminant Validity
- consists of providing evidence that two tests that do not
measure closely related skills or types of knowledge do not
correlate strongly (i.e., dissimilar ranking of students).

6. Measurement validity
▫ Validity is not all or nothing
▫ Evidence can be less than completely convincing
▫ There may be no clear criterion of validity
▫ A measure can be valid for one purpose, but not another
▫ Often, validity is a matter of subjective judgment

7. Reliability
▫ Reliability of a measure
- Does the measure give consistent results when applied to the
same object or event?
- How large is the random error (noise)?

8. Why reliability matters
▫ Calculating averages
- Given a large enough sample, unreliability can be averaged out
▫ Estimating relationships
- Poor reliability attenuates relationships
- Potentially important effects may be missed
▫ Classifying individuals
- Random errors can affect individuals
- Example: Determining a family’s eligibility for welfare benefits
▫ Tracking changes over time
- In response to the growing emphasis on accountability and
performance measurement

9. Reliability assessment strategies
▫ Test-retest reliability
- Measuring the same thing twice
- When is this a good approach? When not?
- Example: giving a group of students a test at two different
periods of time to see if their knowledge of a topic has
improved
▫ Inter-rater reliability
- How consistent are the research workers?
- Example: asking street cleanliness raters to rate the same
section of a city

10. Scales and indexes
▫ Measures composed of multiple indicators
▫ Some examples:
- Consumer Price Index (CPI)
- Index from prices of many goods and services
- Measures overall price level
- Grade point average
- Index formed from grades in all classes
- Measures overall academic performance
- Self-esteem scale
- Indicators of self-esteem are answers given to various survey
questions

11. Types of Measures
▫ Objective Indicator
- Seeks to minimize discretion
▫ Subjective Indicator
- Requires some judgment to assign a value

12. Level of Measurement
- There are three levels of measurement:
- Nominal
- Ordinal
- Interval-ratio

13. Level of Measurement
- Nominal levels of measurement
- Nominal variables have ‘scores’ or categories that are non-
numerical.
- The lowest level of measurement available.
- Only the relative sizes of the categories can be compared.
- Examples of nominal variables include gender and
religious affiliation.
- Can you think of another nominal level variable?

14. Level of Measurement
- A Nominal Measure of Employee Gender
- An example of a Nominal Variable
1= female
0= male
Name Employee Gender
Jones 1
R. Smith 0
Franklin 1
Barnes 1
A. Smith 0

15. Level of Measurement
▫ Ordinal level of measurement
- Ordinal variables have ‘scores’ or categories that can be
ranked from high to low.
- The categories of ordinal variables can be described as being
“more or less” with respect to each other.
- A limitation of ordinal variables is that we do not know the exact
distance from one score or category to the next.
- Examples of ordinal variables include social class (lower,
working, middle, upper) and “Likert” items (strongly agree,
agree, disagree, strongly disagree).
- Can you think of another ordinal level variable?

16. Level of Measurement
▫ A Ordinal Measure of the concept “Satisfaction”
▫ An example of an ordinal Variable
1= Very Satisfied
2= satisfied
3= neutral
4= dissatisfied
5= very dissatisfied
Name Satisfaction
Jones 2
R. Smith 3
Franklin 1
Barnes 2
A. Smith 3

17. Level of Measurement
▫ Interval-ratio level of measurement
- Interval-ratio variables are measured as numbers.
- The ‘scores’ of an interval-ratio variable can be ranked from
high to low and we can quantify the distance between different
scores.
- Some interval-ratio variables have “true” zero points.
- All mathematical operations are possible with interval-ratio
variables.
- Examples of interval-ratio variables include income (measured
in dollars) and number of siblings.
Can you think of another interval-ratio level variable?

18. Level of Measurement
▫ Some variables can be measured at different levels.
- For example, educational attainment can be measured in the
number of years completed (0, 1, 2, 3, ..., 12, 13, etc. – an
interval-ratio variable) or it can be measured in the highest
degree received (less than high school, high school diploma,
associates degree, bachelors degree, etc. – an ordinal
variable).
- Can you think of another variable that can be measured at
different levels?

19. Research, policy and practice
 Performance measurement
 Measuring outcomes and managing for results
 Evaluation research
 Demonstrating effectiveness to sponsors and funding entities
 Evidence-based policy and programs
 Basing policy, programs, and practice on knowledge of what works
(causation)
 Q: What examples of each of these do you see in your area of work
(or interest)?

20. INPUTS Resources that an organization
uses to achieve its goals
Annual budget, employees,
volunteers
OUTPUTS Are tangible indicators that
show how an organization uses
its resources
Number of cases processed per
employee, number of housing
units built
OUTCOMES Are more precise indicators of
performance than outputs
because they focus more on
quality than on quantity
Participants of job training
participants who receive full
time jobs
Performance Measurement Techniques