PUBLIC HEALTH

1. Planning, Implementing & Evaluating
Health Promotion Programs
Sixth Edition
Chapter 5 Lecture
Measurement,
Measures,
Measurement
Instruments, and
Sampling

2. © 2013 Pearson Education, Inc.
Measurement
• Measurement – the process of applying numerical or
narrative data from an instrument (e.g., a
questionnaire) or other data-yielding tools to objects,
events, or people (Windsor, Clark, Boyd, & Goodman,
2004)
• The planner/evaluator needs to
– Identify the instrument or tool
– Determine how the data will be categorized using number
or words
– Decide how these categories of data will be classified
• Types
– Quantitative – numerical data collected
– Qualitative – data collected with the use of narrative and
observational approaches

3. © 2013 Pearson Education, Inc.
Importance of Measurement in Program
Planning & Evaluation
• Health education specialists need to be comfortable
and competent with measurement.
• Examples of when health education specialists will
need measurement knowledge & skills include
– Locating evidence for a rationale
– Conducting a needs assessment
– Planning an evaluation
– Generating evidence for a funding agency
– Interpreting program results

4. © 2013 Pearson Education, Inc.
Levels of Measurement
1. Nominal - data into categories; mutually exclusive;
categories are exhaustive
2. Ordinal - data into categories; mutually exclusive &
exhaustive; but also rank ordered; can't measure
distance between categories
3. Interval – (numerical data) data into categories;
mutually exclusive & exhaustive; rank orders; can
measure distance between categories; no absolute
zero
4. Ratio – (numerical data) data into categories;
mutually exclusive & exhaustive; rank orders; can
measure distance between categories; there is an
absolute zero
• The levels are hierarchical in nature, and will dictate
what statistical test can be used to analyze them.

5. © 2013 Pearson Education, Inc.
What level of data do each of these questions
generate? – 1
• What is your sex?
• In what state do you currently live?
• In what country were you born?
• Rank order the following health conditions from most
serious to least serious: Fractured arm, A cold,
Nonfatal heart attack
• I would feel comfortable planning a health promotion
program for a health agency SA A N D
SD

6. © 2013 Pearson Education, Inc.
What level of data do each of these questions
generate? – 2
• Place in order of least important to most important
the following environmental concerns - acid rain,
recycling, pollution
• What was the approximate temperature when you got
up this morning?
• How old are you? 0-20 years; 21-40 years; 41-60
years; 61+ years
• The last time it was checked, what was your blood
pressure?
• How old were you (in years) on your last birthday?

7. © 2013 Pearson Education, Inc.
Desirable Characteristics of Data – 1
• Psychometric Qualities – instrument shows
reliability, validity, and fairness
• Reliability – consistency in the measurement
process; the degree to which a measure is free from
errors of measurement; obtained score = true score +
error score
• Validity – measures what it is intended to measure;
correctly measuring concepts under investigation

8. © 2013 Pearson Education, Inc.
Desirable Characteristics of Data – 2
• Types of Reliability
– Internal consistency: checks if items on the instrument are
measuring the same research domain.
– Test-retest (stability reliability): the same instrument is used
to measure the same group of people under similar, or the
same, conditions, at two different points in time.
– Rater reliability: intrarater (1 person); interrater (2 or more),
focuses on the consistency of observations.
– Parallel forms reliability: focuses on whether different forms
of the same measurement instrument, when measuring the
same subjects, will produce similar results.

9. © 2013 Pearson Education, Inc.
Desirable Characteristics of Data – 3
• Types of Validity
– Face: it appears to measure what it is supposed to
measure.
– Content: items are a representative sample of the content
and/or behavior of the domain being addressed; typically
created by an expert panel or jury.
– Criterion-related: scores are correlated with some other
measure of an individual's behavior or performance;
predictive validity (future event), concurrent validity (new
and established).
– Construct: degree to which a measure correlates with other
measures it is theoretically expected to correlate with;
convergent validity (should correlate with), discriminant
validity (should not correlate with)
– Sensitivity (those with) and Specificity (those without)

10. © 2013 Pearson Education, Inc.
Desirable Characteristics of Data – 4
• Fairness – Measure ''is appropriate for the
individuals of various ethnic groups with different
backgrounds, gender, educational levels, etc.'' (Torabi,
1994, p. 56).
– To ensure fairness, planners/evaluators should have an
understanding of the culture of those in the priority
population.
– Cultural competence is a developmental process defined
as a set of values, principles, behaviors, attitudes, and
policies that enable health professionals to work effectively
across racial, ethnic, and linguistically diverse populations
(Joint Committee on Terminology, 2012).

11. © 2013 Pearson Education, Inc.
Desirable Characteristics of Data – 5
• Bias – biased data are distorted data; often caused
by the way they are collected; e.g., response bias.
– Can be introduced in the selection or sampling process of
study participants, in the study's design or measurement
process, or in the intervention phase.

12. © 2013 Pearson Education, Inc.
Measurement Instruments
• Measurement Instrument –
– the item used to measure the variables of interest
• Instrumentation –
– ''a collective term that describes all measurement
instruments used'' (Cottrell & McKenzie, 2011, p. 146)
• Tests –
– instrument used in educational measurement
• Questionnaire (survey instrument) –
– gathers information on a variety of factors (e.g.,
awareness, skills, behavior) related to one or more topics
• Scale –
– used to measure on one concept; e.g., attitudes

13. © 2013 Pearson Education, Inc.
Using an Existing Measurement Instrument
• Is there another instrument already available that
could be used?
– Can save time and costs, but may not be appropriate or
relevant.
• Steps in evaluating an existing instrument (Cottrell &
McKenzie, 2011)
1. Identifying measurement instruments
2. Getting your hands on the instrument
3. Is it the right instrument? Psychometric qualities? Used
with similar participants? Standard or normative scores?
Culturally appropriate? Reading level? Cost to use?
4. Final steps before proceeding. Permission? Other
conditions?

14. © 2013 Pearson Education, Inc.
Creating a Measurement Instrument – 1
• Create only if another cannot be found or
adapted.
• Components of an instrument
– Introduction – includes purpose and directions
– General questions
– Questions of interest
– Sensitive questions at the end
• Wording of questions is very important in gaining
needed information.
– Clear and bias free
– Avoid questions with a specific direction and two-part
questions
– Use terminology that will be understood; avoid offensive
language

15. © 2013 Pearson Education, Inc.
Steps in Creating a Measurement Instrument

16. © 2013 Pearson Education, Inc.
Creating a Measurement Instrument – 2
• Many different types of questions; consider the level
of measurement
• Freedom in responding
– Structured (closed) or selected response items have the
least freedom
• Matching, multiple choice
– Less structured (but still closed)
• Likert scale (SA A N D SD) and multiple choice questions
with an ''other'' option
– Unstructured (open) or constructed response items have
the most freedom
• Short answer and essay questions

17. © 2013 Pearson Education, Inc.
Sampling
• Census – all the participants (i.e., the U.S. Census)
• Sample – some of the participants
• Sampling unit – element or set of elements
considered to be part of the sample (Babbie, 1992); may
be an individual, organization, or geographical area
(Bowling, 2005)
• Sampling – a process by which a part is taken as
representative of the whole
• Universe - unspecified by time or place
• Population – specified by time or place
• Survey population – those who are accessible (the
sampling frame)

18. © 2013 Pearson Education, Inc.
Relationship of Study Populations

19. © 2013 Pearson Education, Inc.
Sampling Procedures
• Probability Sample –
– All elements of the survey population have an equal
chance or probability of being selected (Green & Lewis,
1986); sometimes referred to as a scientific sample or
random sample.
• Nonprobability Sample –
– All individuals in the survey population do not have an
equal chance or probability of being selected to participate
in the needs assessment or evaluation. Participants can be
included on the basis of convenience (because they have
volunteered, are available, or can be easily contacted) or
because they possess a certain characteristic.

20. © 2013 Pearson Education, Inc.
Probability Sampling Procedures
• Simple random sample (SRS) – Equal chance of
being selected; need frame; use TORN or computer
• Fishbowl (out of a hat) – Approximates SRS; not as
precise; w/ & w/o replacement
• Systematic – Uses list; selects at a constant interval
(N/n) after random start
• Cluster or area – SRS of groups, not individuals
• Matrix – Responses of several combined as
response of one
• Stratified
– Proportional stratified – Divides into sub-groups (strata);
selects SRS from strata in proportion to strength of strata in
population
– Nonproportional stratified – Divides into strata; selects
SRS from strata

21. © 2013 Pearson Education, Inc.
Scenario - select 100 students from the school population of 1000
Stratified Sampling
• Proportional
• Stratum pop. sample
– Fr. 400 40
– Soph. 300 30
– Jr. 200 20
– Sr. 100 10
• Nonproportional
• Stratum pop. sample
– Fr. 400 25
– Soph. 300 25
– Jr. 200 25
– Sr. 100 25

22. © 2013 Pearson Education, Inc.
Nonprobability Samples
• Used when a probability sample cannot be obtained
or is not needed.
– Can also be used when resources are limited and a
probability sample is too costly or time consuming.
• Participants can be included on the basis of
convenience (because they have volunteered, are
available, or can be easily contacted) or because
they possess a certain characteristic.
• Limitations:
– Results can be generalized to the total survey population.
– Bias may also occur since those who are not included in
the sample may differ in some way from those who are
included.

23. © 2013 Pearson Education, Inc.
Nonprobability Sampling Procedures
• Convenience – includes accessible subjects meeting
some criterion
• Volunteer – includes those motivated to self-select
• Grab – includes whomever planners can access
through direct contact
• Homogeneous – includes those selected because of
unique trait
• Judgmental – includes those whom planners judge
to be typical of possessing a given trait
• Snowball – includes those identified by planners &
others referred by initial subjects
• Quota – includes those chosen in approximate
proportion to the population traits they represent

24. © 2013 Pearson Education, Inc.
Sample Size
• How big is big enough?
– Practical & statistical considerations
• Three major theoretical considerations
– Central limit theorem (CLT); groups >30
– Power analysis (specific to effect size & statistic)
– Precision & reliability

25. © 2013 Pearson Education, Inc.
Sample Sizes for Studies Describing Population
Proportions When the Population Size is Known

26. © 2013 Pearson Education, Inc.
Pilot Testing
• Also called piloting or pilot study
• What is it?
– A set of procedures used by planners/ evaluators to try out
various processes during program development on a small
group of subjects prior to actual use/implementation
– A dress rehearsal (McDermott & Sarvela, 1999)
• Purpose – To identify and correct any problems
– Used to identify problems with measurement instruments,
data collection procedures, data analysis, materials,
strategies, and sometimes to establish validity & reliability

27. © 2013 Pearson Education, Inc.
Types of Pilot Testing
• Preliminary Review
– Colleagues, not priority population, review the instrument.
• Pre-pilot (or mini-pilots)
– Small number [5 to 6] pulled from priority population.
• Pilot Test
– A representative sample of the priority population is used to
determine the quality of the instrument.
• Field Study
– Combines all materials previously tested into a complete
program.

28. © 2013 Pearson Education, Inc.
Ethical Issues Associated with Measurement
• HIPAA
• Voluntary participation; should not be coerced or
deceived to participate
• Right to discontinue participation
• Private & sensitive data must be protected
• Illegal acts during data collection
• Appropriate data analysis
• Reporting of results